Cinder offers classes for image processing, matrix, quaternion, spline and vector math, but also more general stuff like XML, HTTP, IO, and 2D Graphics.

The more generic stuff is implemented via third-party libraries, such as TinyXML, Cairo, AntTweakBar (a simple GUI), Boost (smart pointers and threads) and system libraries (QuickTime, Cocoa, DirectAudio, OpenGL) – certainly an ambitious range of technologies and uses.

Their examples are impressive, especially some of the demos by Robert Hodgin (flight404):

Cymatic Ferrofluid by flight404 (be sure to watch the videos).

Robert Hodgin has also created a very nice Cinder tutorial, which guides you through the creation of a quite spectacular particle effect.

Finally, it should be noted the openFrameworks offers related functionality, also based on C++.

The Demoscene

It was only a matter of time, before a Mandelbox would show up on the Demoscene:

Hochenergiephysik by Still is a 4K demo, featuring the Mandelbox. If 4KB sounds bloated, Still has also created a 1K demo: Futurism by Still.

And while we are at it, may I suggest these as well: The Cube by Farbrausch, Rove by Farbraush, and Agenda Circling Forth by Fairlight & Cncd.

New software

NodeBox 2.0 is out in a beta version. The big news is that it is now available for Windows. It also sports a graph-based GUI for patching nodes together.

Tacitus is a GUI for creating per-pixel GPU effects. In concept it is similar to Pixel Bender. It has a very nice look and feel, but a big short-coming is that it is not possible to directly edit the GPU scripts in the GUI – you have to compile your script to a plugin via an included compiler. Another feature I miss, is the ability to directly navigate the camera using the mouse on the viewport, instead of using sliders (something Pixel Bender also doesn’t support). But Tacitus is still in beta, and it will be interesting to see where it is going. It comes with a single plugin, a port of Subblue’s Mandelbulb Pixel Bender plugin. Tacitus is Windows only.

NeuroSystems Substance is an ‘Evolutionary and Organic Art Creator’. Some interesting concepts here, including a real-time global illumination raytracer (video here). Unfortunately, the raytracer is not part of the free viewer. Surprisingly, NeuroSystems impressive visualization technology seems to originate from SIMPLANT, a real-time 3D breast implant simulator. Substance is Windows only, and the full (non-free) versions should be released very soon.

Gifts for Geeks

A Calabi-Yau Manifold Crystal sculpture.

“… no one, not even Benoit Mandelbrot himself [...] had any real preconception of the set’s extraordinary richness. The Mandelbrot set was certainly no invention of any human mind. The set is just objectively there in the mathematics itself. If it has meaning to assign an actual existence to the Mandelbrot set, then that existence is not within our mind, for no one can fully comprehend the set’s endless variety and unlimited complication.”

Roger Penrose (from The Road to Reality)

The recent proliferation of 3D fractals, in particular the Mandelbox and Mandelbulb, got me thinking about the reality of these systems. The million dollar question is whether we discover or construct these entities. Surely these systems give the impression of exploring uncharted territory, perhaps even looking into another world. But the same can be said for many traditional man-made works of art.

I started out by citing Roger Penrose. He is a mathematical Platonist, and believes that both the fractals worlds (such as the Mandelbrot set) and the mathematical truths (such as Fermat’s last theorem) are discovered. In his view, the mathematical truths have an eternal, unchanging, objective existence in some kind of Platonic ideal world, independent of human observers.

In Penrose’s model, there are three distinct worlds: the physical world, the mental world (our perception of the physical world), and the cryptic Platonic world. Even Penrose himself admits that the connections and interactions between these worlds are mysterious. And personally I cannot see any kind of evidence pointing in favor of this third, metaphysical world.

Designer World by David Makin

Roger Penrose is a highly renowned mathematician and physicist, and I value his opinions and works highly. In fact, it was one of his earlier books, The Emperors New Mind, that in part motivated me to become a physicist myself. But even though he is probably one of the most talented mathematicians living today, I am not convinced by his Platonist belief.

Personally, I subscribe to the less exotic formalist view: that the mathematical truths are the theorems we can derive by applying a set of deduction rules to a set of mathematical axioms. The axioms are not completely arbitrary, though. For instance, a classic mathematical discipline, such as Euclidean geometry, was clearly motivated by empirical observations of the physical world. The same does not necessarily apply to modern mathematical areas. For instance, Lobachevsky’s non-Euclidean geometry, was conceived by exploring the consequences of modifying one of Euclid’s fundamental postulates (interestingly non-Euclidean geometry later turned out to be useful in describing the physical world through Einstein’s general theory of relativity).

But if modern mathematics has become detached from its empirical roots, what governs the evolution of modern mathematics? Are all formal systems thus equally interesting to study? My guess is that most mathematicians gain some kind of intuition about what directions to pursue, based on a mixture of trends, historical research, and feedback from applied mathematics.

Mandelballs by Krzysztof Marczak [Mandelbox / Juliabulb mix]

Does my formalist position mean that I consider the Mandelbrot set to be a man-made creation, in the same category as a Picasso painting or a Bach concert? Not exactly. Because I do believe in a physical realism (in the sense that I believe in a objective, physical world independent of human existence), and since I do believe some parts of mathematics is inspired by this physical world and tries to model it, I believe some parts of mathematics can be attributed an objective status as well. But it is a weaker kind of objective existence: the mathematical models and structures used to describe reality are not persistent and ever-lasting, instead they may be refined and altered, as we progressively create models with greater predictive power. And I think this is the reason fractals often resemble natural structures and phenomena: because the mathematics used to produce the fractals was inspired by nature in the first place. Let me give another example:

Teeth by Jesse

Would a distant alien civilization come up with the same Mandelbrot images as we see? I think it is very likely. Any advanced civilization studying nature, would most likely have created models such as the natural numbers, the real numbers, and eventually the complex numbers. The complex numbers are extremely useful when modeling many physical phenomena, such as waves or electrodynamics, and complex numbers are essential in the description of quantum mechanics. And if this hypothetical civilization had computational power available, eventually someone would investigate the convergence of a simple, iterated system like z = z² + c. So there would probably be a lot of overlapping mathematical structures. But there would also be differences: for instance the construction of the slightly more complex Mandelbox set contains several human-made design decisions, making it less likely to be invented by our distant civilization.

I think there is a connection to other areas of generative art as well. In the opening quote Penrose claims that no-one could have any real preconception of the Mandelbrot sets extraordinary richness. And the same applies to many generative systems: they are impossible to predict and often surprisingly complex and detailed. But this does not imply that they have a meta-physical Platonic origin. Many biological and physical systems share the same properties. And many of the most interesting generative systems are inspired by these physical or biological systems (for instance using models such as genetic algorithms, flocking behavior, cellular automata, reaction-diffusion systems, and L-systems).

Another point to consider is, that creating beautiful and interesting fractal images as the ones above, requires much more than a simple formula. It requires aesthetic intuition and skills to choose a proper palette, find an interesting camera location, and it takes many hours of formula parameter tweaking. I know this from my experiments with 3D fractals – I’m very rarely satisfied with my own results.

But to sum it all up: Even though fractals (and generative systems) may posses endless variety and unlimited complication, there is no need to call upon metaphysical worlds in order to explain them.

Then I stumbled upon Shader Toy, by Inigo Quilez (whom I’ve mentioned several times on this blog). A couple of things make Shader Toy stand out:

It runs inside your browser. It uses the emerging WebGL standard, which is JavaScript bindings for OpenGL (ES) 2.0. OpenGL can be used directly inside a Canvas HTML element, including support for custom shaders. As Shader Toy demonstrates, this makes it possible to do some very impressive stuff, such as real-time GPU-accelerated raytracing inside an element on a web page.

The examples are great. While Shader Toy itself is mostly a thin wrapper around the WebGL functionality, the great thing about it is the example shaders: 2D fractals and Demo Scene effects, but also complex examples like the Slisesix 4K demo, and examples of raytracing, and complex fractals, like the Quaternion Julia set, and the Mandelbulb.

The only problem with WebGL is, that it is not supported by the current generation of browsers.

The good news is that the nightly builds of Firefox, Safari (WebKit), and Chromium (Google Chrome) all support it, and are quite easy to install: this is a good place for more information. If you use the Chromium builds, you don’t have to worry about messing up your existing browser configuration – the nightly builds are standalone versions and can be run without installation.

There are lots of complex shader tools out there: for instance, NVIDIAs FX Composer, AMDs Rendermonkey, TyphoonLabs OpenGL Shader Designer, and Lumina, but Shader Toy makes it very easy to get started with shaders. And it provides a rare insight into how those amazing 4K demos were made.

I do not think of myself as an artist, and neither do I work in the academia. So it was not at all obvious for me to attend the conference. But when I got a an email from Celestino Soddu (the chairman of the conference) asking me to consider participating in the conference, I became curious since the conference revolves around many of the concepts that interests me: genetic algorithms, swarms and flocking, multi-agent systems, sound synthesis, architecture, digital photography, etc…

So I went, and gave a short introduction to Structure Synth and its history (Chomsky’s formal grammars, Chris Coynes context-free design grammars, and the relation to Lindenmayer systems).

The paper is available here (PDF):
Structural Synthesis using a Context-Free Grammar Approach.

Structure Synth image.

I will start out by saying that I enjoyed the conference a lot. People were very friendly and interesting, and I had a lot of good discussions. And I think the diverse mixture of different cultures, nationalities, fields and practices is exciting – even though it also meant that some of the presentations became too tangential to my interests – and some were even nearly incomprehensible to me.

Some of my personal highlights in the conference were Arne Eigenfeldts “In Equilibrio”, a multi-agent music system, Daniel Bisig and Tatsuo Unemis “Swarms on Stage – Swarm Simulations for Dance Performance” and Philip Galanters theoretical essay on “Fitness and Complexification in Evolutionary Art” – even though I do not agree with Philip here: I think the idea of establishing an aesthetic fitness function, which could be used by genetic algorithms, is a futile endeavor. The AI community seems to have had little progress with mimicking human behavior the last forty years (e.g. see my conversation with last years Loebner prize contest winners), and surely aesthetic judgments require a lot beyond what is needed to pass a simple Turing test.

Sculpture (found somewhere in Milano).

Another highlight was Celestino Soddu’s own introduction – it contained a slideshow with an enormous amount of his own generated architectural works, and I think it demonstrated an impressive and consistent approach to generative architecture. But it also made me wonder if we will ever see a skyscraper created by a generative system.

As a final note, I also think the academic community should try to establish some sort of communication to the vibrant generative art internet community and demo scene practitioners. I am not sure exactly how this could be accomplished, but many interesting projects seems to emerge from these settings.

Adam M. Smith has begun working on cfml – a context-free music language. It is a Context-Free Design Grammar – for music. I’m very interested in how this develops.

A graphical representation of cfml output (original here)

Cfml is implemented as an Impromptu library. Impromptu is a live coding environment, based on the Scheme language, and has existed since 2005. Andrew Sorensen, the developer of Impromptu, has created some of the most impressive examples of live coding I have seen. In particular, the last example, inspired by Keith Jarrett’s Sun Bear Concerts, is really impressive. (I might be slightly biased here, since I believe that Jarrett’s solo piano concerts – especially the Köln Concert and the Sun Bear Concerts – rank among the best music ever made).

Finally, Supercollider 140 is a selection of audio pieces all created in Supercollider in 140 characters or less. An interesting example of using restrictions to spur creativity. Another example is the 200 char Processing sketch contest.

Free Indy Game Development

This month also saw the release of the Unreal Development Kit, basically a version of the Unreal Engine 3, that is free for non-commercial use. This is great news for amateur game developers, but for me, the big question was whether this could be used as a powerful platform for generative art or live demos. I downloaded the kit and played around with it for a while, but while the 3D engine is stunning, UDK seems very geared towards graphical development (I certainly do not want to do draw my programs, and the built-in Unrealscript does not impress me either).

In related news, that basic version of Unity 2.6 is now also free. The main focus of Unity is also game development, but from a generative art / live demo perspective it holds greater promise. Unity offers an advanced graphics engine with user-scriptable shaders, integrated PhysX physics engine, and 3D audio.

Unitys development architecture is also very solid: scripts are written in (JIT-compiled) JavaScript, and components can be written in C# (using Mono, the open-source .NET implementation). Using a dynamic scripting language such as JavaScript to control a more rigid body of classes written in a more strict, statically typed environment, such as C#, is a good way to manage complex software. All Mozilla software – including Firefox – is built using this model (JavaScript + XPCOM C++ components), and newer platforms, such as Microsoft’s Silverlight platform also use it (JavaScript + C# components).

I made a few tests with Unity, and it is simple to control and instance even pretty complex structures. I considered writing a simple Structure Synth viewer using Unity, but was unfortunately put a bit off, when I discovered that Screen Space Ambient Occlusion and Full Screen Post-Processing Effects are not part of the free basic edition. The iPhone version of the Unity engine is not free either, but that is probably as could be expected.

It will be interesting to see if Unity will be picked up by the Generative Art community.

SIGGRAPH Asia

Finally two papers presented at SIGGRAPH Asia 2009 should be noted:

Shadow Art creates objects which cast three different shadows.

Sketch2Photo creates realistic photo-montages from freehand sketches annotated with text labels.

GroBoto is a very polished product. The GUI is slick, and there are loads of advanced visualization customizations: textures, lightning and animation. When moving and rotating objects an OpenGL view is used, but the scene is always automatically rendered using an internal raytracer, which is really amazingly fast (typically less than a second).

My only complaint is that you are somewhat limited by the presets offered by GroBoto. It would be amazing to be able to completely script the objects. Yet again, that would make GroBoto a tough competitor to Structure Synth

GroBoto is available for $59 (using the coupon offer) for Windows and Mac OS X.

Be sure to a look at their gallery for more images or try the demo.

Unique aesthetics combined with a nice soundtrack, and randomly generated invaders!

The invader generator was inspired by Jared Tarbell’s Invader Fractal algorithm. Be sure to also check out Dave Bollinger’s Pixel Robots – also inspired by Tarbell’s work.

• Part I was about Context-Free Grammars and natural language parsing.

• Part II was about Lindenmeyer systems, Context Free design grammars, and Structure Synth.
• This third part is about generative poetry and Tree-Adjoining Grammars.
• Part IV will be about Style Grammars and Shape Grammars.

Issue 1: Fall 2008

(From Issue 1: Fall 2008)

In October 2008, Stephen McLaughlin and Jim Carpenter released ‘Issue 1: Fall 2008′, a 3785 page compilation of poems by more than 3000 contemporary American poets. (Download here).

Besides the stunning size of the book, this would probably have gone relatively unnoticed, if it weren’t for the fact that the 3000+ authors had not submitted any poems to the editors. Neither were they able to recognize any of the poems as their own. This of course created quite a stir.

It turns out that the book was generated by Erica T Carter, which actually is the name of a computer program written by Jim Carpenter.

Erica T Carter

Erica T Carter (ETC3) uses Tree-Adjoining Grammars (TAGs) for generating poems. TAGs are related to formal grammars, but instead of production rules operating on symbolic sequences, TAGs operate on trees.

Considering the example above, a tree may be inserted (substituted) into another tree, if its root node matches a terminal node (the a-2 and a-3 trees may be attached to the NP non-terminal). Besides the substitution operation, it is also possible to do adjoining operations, where a tree may be expanded by inserting another tree inside it (this paper has a short introduction to TAGs).

‘Erica T Carter’ is open source and may be downloaded (or tested) here.

Judging from the source and comments, the phrase structures in the TAGs were handpicked by Jim Carpenter after analyzing works by Frank O’Hara, Sylvia Plath, Gary Snyder, and Rachel Blau DuPlessis (not that I have read anything by them). In the online version of Erica, the default grammar choice is the 166-trees ‘Mimetic’ grammar, which is described as an amalgam of the rest of the grammars – I’m guessing that this is the one used for ‘Issue 1: Fall 2008′.

In addition to the trees, Erica uses a lot of extra information from a 30MB database (public available from the site too). The database contains lexicon tables with entries like ‘abashless = adj’ and ‘abate = v’ apparently based on Emily Dickinson Poems and Joseph Conrad’s Heart of Darkness (’Apocalypse Now’). The database also contains more exotic tables, like ‘pronunciation’ (’anachronism = AH0 N AE1 K R AH0 N IH2 Z AH0 M’) which may be used to construct rhymes.

The auto-generated poems seems very impressive, but after skimming through the text some patterns emerge, which seems unlikely if the poems were to be written by different (human) authors: for instance 2205 lines start with “Like a …” (that is 4% of the 55685 total lines!).

On the other hand the poems impress by containing cross serial dependencies as in this example:

Where both the title and content of the poem refer to ‘background’. This particular feature – where different parts of the generated sentences correlate – is difficult to achieve with context free grammars, since the substitutions of symbols does not depend on how other symbols are expanded.

Tree Adjoined Grammars and Context Sensitivity

Tree Adjoined Grammars fits in between the Context Free and Context Sensitive classes in the Chomsky Hierarchy: they describe Mildly Context-sensitive languages.

Where as Context Free grammars have the pleasant property of being parsable in polynomial space and time, they are not adequate for describing for example the natural languages (features such as cross serial dependencies and agreements are difficult to capture: for instance a formal grammar for a natural language may contain ‘noun’ and ‘verb’ non-terminals. But nouns and verbs depend on each other – something which is not easily expressed in a context free grammar, leading to output such as ‘Words is funny’).

So the Mildly context-sensitive languages provide a middle ground – they are better at capturing the structure of natural languages, but still not as difficult to handle as a true context-sensitive language.

Generative Art and the Freedom of Restrictions

Now, why even care about the complexity of the grammars? After all we are not interested in analyzing syntax or deriving mathematical properties of the grammars, we are interested in generating structure.

For instance, why restrict Context Free Art and Structure Synth to context free systems? The cost of generating structures does not increase by switching to a more powerful grammar, only the cost of analyzing the systems.

I think the answer relates to the very essence of ‘generative art’.

For me, generative art is all about exploring systems, without being too much in control. When generating structures, it should not be possible to anticipate how a given structure turns out by looking at the rules. There should be a sense of indeterminism and surprise in the result. The system needs not to necessarily be driven by random choices in order to achieve this – for instance the Mandelbrot set is a good example of this: nobody would have been able to imagine how complex images a simple system like “z -> z^2+c” could create, yet there is nothing stochastic in the generation of the set.

Choosing to work within a restricted rule system is a way to give up some control and force yourself to think differently. You have to explore and work within the limitations of the system, which may lead to interesting and surprising results.

More generic languages, for instance the popular Java-based Processing, have no limitations in expressiveness (Java, like all other general purpose programming languages, is Turing complete). Is Processing not suitable for generative art, because of its universal expressive power? Well, the answer is of course that Processing is very suitable and it is indeed widely used for producing generative art. In fact any Structure Synth or Context Free Art system could be emulated in Processing/Java because of this universal power. But Processing is also suitable for many other applications, like for instance Data Visualization and other non-generative tasks. Context Free Art and Structure Synth on the other hand force you to explore generative systems.