Syntopia

Generative Music Software

Adam M. Smith has begun working on cfml – a context-free music language. It is a Context-Free Design Grammar – for music. I’m very interested in how this develops.

A graphical representation of cfml output (original here)

Cfml is implemented as an Impromptu library. Impromptu is a live coding environment, based on the Scheme language, and has existed since 2005. Andrew Sorensen, the developer of Impromptu, has created some of the most impressive examples of live coding I have seen. In particular, the last example, inspired by Keith Jarrett’s Sun Bear Concerts, is really impressive. (I might be slightly biased here, since I believe that Jarrett’s solo piano concerts – especially the Köln Concert and the Sun Bear Concerts – rank among the best music ever made).

Finally, Supercollider 140 is a selection of audio pieces all created in Supercollider in 140 characters or less. An interesting example of using restrictions to spur creativity. Another example is the 200 char Processing sketch contest.

Free Indy Game Development

This month also saw the release of the Unreal Development Kit, basically a version of the Unreal Engine 3, that is free for non-commercial use. This is great news for amateur game developers, but for me, the big question was whether this could be used as a powerful platform for generative art or live demos. I downloaded the kit and played around with it for a while, but while the 3D engine is stunning, UDK seems very geared towards graphical development (I certainly do not want to do draw my programs, and the built-in Unrealscript does not impress me either).

In related news, that basic version of Unity 2.6 is now also free. The main focus of Unity is also game development, but from a generative art / live demo perspective it holds greater promise. Unity offers an advanced graphics engine with user-scriptable shaders, integrated PhysX physics engine, and 3D audio.

Unitys development architecture is also very solid: scripts are written in (JIT-compiled) JavaScript, and components can be written in C# (using Mono, the open-source .NET implementation). Using a dynamic scripting language such as JavaScript to control a more rigid body of classes written in a more strict, statically typed environment, such as C#, is a good way to manage complex software. All Mozilla software – including Firefox – is built using this model (JavaScript + XPCOM C++ components), and newer platforms, such as Microsoft’s Silverlight platform also use it (JavaScript + C# components).

I made a few tests with Unity, and it is simple to control and instance even pretty complex structures. I considered writing a simple Structure Synth viewer using Unity, but was unfortunately put a bit off, when I discovered that Screen Space Ambient Occlusion and Full Screen Post-Processing Effects are not part of the free basic edition. The iPhone version of the Unity engine is not free either, but that is probably as could be expected.

It will be interesting to see if Unity will be picked up by the Generative Art community.

SIGGRAPH Asia

Finally two papers presented at SIGGRAPH Asia 2009 should be noted:

Shadow Art creates objects which cast three different shadows.

Sketch2Photo creates realistic photo-montages from freehand sketches annotated with text labels.

I’ve implemented a random color scheme in Structure Synth, using a new ‘color random’ specifier.

But what exactly is a random color? My first attempt was to use the HSV color model and choose a random hue, with full brightness and saturation.

This produces colors like this:

Most of my Nabla pictures used this color scheme. It produces some very strong colors.

Then I tried the RGB model using 3 random numbers, one for each color-channel, which creates this kind of colors:

But what about greyscale colors:

I decided that it was necessary to be able to switch between different color schemes.

So I created a new ’set colorpool’ command. Besides the color schemes above (’set colorpool randomhue’, ’set colorpool randomrgb’, and ’set colorpool greyscale’) I created two additional color schemes:

One where you specify a list of colors:

(For this image the command was: “set colorpool list:orange,white,white,white,white,white,white,grey”. As is evident it is possible to repeat a given color, to emphasize its occurrence in the image.)

And on where you specify an image which is used to sample colors from:

The command used for the above image was: “set colorpool image:001.PNG”. Whenever a random color is requested (by the ‘random color’ operator), the program will sample a random point from the specified image and use the color of this pixel. This is a quite powerful command, making it possible to imitate the color tonality of another picture.

Now this is all good. But I realized that there are some problems with this approach.

The problem is that geometry and the colors draw numbers from the same random number generator (the C-standard library ‘rand()’ function).

This means that changing the color scheme changes the geometry (since the color schemes use a different number of random numbers for each color – randomhue uses 1 random number per color, the image sampling uses two (X and Y) random numbers per color, the randomrgb uses three).

This is not acceptable, since you’ll want to change the color schemes without changing the geometry. Another problem is that C-standard library ‘rand’ function is not platform independent – so even if you specify a EisenScript together with an initial random seed, you will not get the same structure on different platforms.

I solved this by implementing new random generators in Structure Synth. I now use two independent Mersenne Twister random number generators, so that I have two random streams – one for geometry and one for colors.

Some months ago, John Resig created processing.js – an impressive JavaScript port of processing, which draws its output on a ‘canvas’ element entirely client-side inside your browser (at least if your web-browser is Firefox 3 or a recent nightly build of WebKit, that is).

Now Context Free (the original inspiration for Structure Synth) has been ported to JavaScript too: Aza Raskin has created ContextFree.js (Source here).

JavaScript has undergone a tremendous evolution. From creating cheesy ‘onMouseOver’ effects for buttons on web pages to being the ‘glue’ binding together complex applications like Firefox or Songbird (the Mozilla application frameworks works by stringing together C++ components with JavaScript). Likewise Microsoft chose to build their Silverlight technology on .NET components which can be controlled by JavaScript in the browser.

And of course the ActionScript in Adobe Flash is also JavaScript. Adobe (and/or Macromedia) has put a lot of effort into creating fast JavaScript implementations – most notably their Tamarin virtual machine and Just-In-Time compiler, which in theory should make JavaScript almost as fast as native code – or at least comparable to other JIT compiled languages such as Java and the .NET languages. Tamarin is open-sourced, and will eventually make it into Firefox 4.

Finally, while the Tamarin virtual machine was built to execute (and JIT) bytecode originating from JavaScript, other languages may target Tamarin as well. Adobe has demonstrated the possibility of compiling standard C programs into Tamarin parseable byte-code (their demo included Quake, a Nintendo emulator, and several languages like Python and Ruby).

So perhaps a future version of Structure Synth could be running as C++ compiled into Tamarin bytecode in a Flash application…

The Demo Scene never cease to amaze me. The technical quality of these demos is amazing – complex 3D scenes rendered real-time, procedural textures, real-time sound synthesis, and incredible low foot-prints.

Recently I stumbled upon demoscene.tv which features recorded videos (flash video) of many of the best demos. Of course part of the fun is actually running these demos, to be amazed that they are indeed real-time, but sadly my laptop is not geared towards neither CPU or GPU intensive activities.

A few selected demos:

fr-041: debris by Farbrausch

Lifeforce by andromeda software development

Kindernoiser (yep, weird name) is a a 4096 byte demo of 3D julia sets. For comparison the HTML for this page is close to 30 KB.

If you do not have a powerful graphics card, try the video linked to below.

KinderNoiser

By coincidence I came across Jeff Minter’s company Llamasoft and surprisingly discovered that it was still going strong.

Jeff Minter, probably most famous for his somewhat… surreal C64 games (”Attack of the Mutant Camels”, “Revenge of the Mutant Camels” and even “Metagalactic Llamas Battle at the Edge of Time”), apparently has been hacking away on light synthesizers for the past twenty years.

His light synthesizers are complex visualization modules either music-controlled or driven by human interaction. And his latest incarnation, Neon, is actually used in the Xbox 360’s dashboard.

Who would have guessed that Llamasoft code would end up in the Xbox 360 firmware?

Experimental Gameplay started out as a student project at Carnegie Mellon University in 2005.

Given three basic rules (”Each game must be made in less than seven days / by only one person / be based around a central theme (e.g. gravity or swarms)”) some very interesting projects were created. Since then the website has opened up, and everyone is allowed to post their experimental games.

The experiences gained from the project are summed up in: How to Prototype a Game in Under 7 Days (Gamasutra Feature).

Mechanical programming of robots in the year 60 AD:

First seen on abstractmachine (now that is a site with a design to remember).

Digital Art Software showdown:

Processing is a Java-based environment for creating digital art. ProcessingBlogs often showcases examples of Processing creations. This Flock of birds demo and the Webcam controlled video pong are nice examples.

VVVV is a realtime video synthesis framework. It is a Windows only flowgraph based system, which seems to be built on top of DirectX and DirectShow. As seen on the Eno Henze image above, VVVV is also very suitable for creating static pictures.

For Mac OS X users, NodeBox looks very impressive (short summary: “NodeBox does less, prettier, in Python. Processing does more, uglier, in Java“) Their gallery page has lots of good examples.

ChucK is an audio programming language for real-time performance. It is possible to add and modify code without stopping the audio stream, which makes live coding sessions possible. I played a bit around with ChucK in the miniAudicle editor, which can be recommended – the real Audicle (a spectacular OpenGL interfaced IDE) on the other hand proved to be quite unstable.

For a more extreme version of live coding check out Quoth (now this is weird – a Zork-inspired user interface).