The LilyPond Report #10

I recently realized that many people were involved or simply interested in interfacing LilyPond with diverse experimental music representation languages:

• Bernardo Barros is trying to translate SuperCollider output into LilyPond scores,
• Karim Haddad is pursuing a similar goal with OpenMusic,
• Jaime Oliver is working on a Pure Data export,
• José Padovani translates music from Common Music to LilyPond,
• David Psenicka’s Fomus has a built-in LilyPond export, which allows non-free languages such as PWGL to be converted into LilyPond scores,
• Graham Percival uses the Marsyas sound analysis software to produce native LilyPond code,
• Torsten Anders has developed his own constraint system, Strasheela,
• Andrea Valle has been working on a complex algorithmic system that integrates (using a Python wrapper?) LilyPond, SuperCollider and TeX,
• Víctor Adán wrote a set of Python modules, named Cuepatlahto, for creating Lilypond scores in a conceptual approach,
• Trevor Bača has a similar project named Lascaux; these two projects have been merged into the Abjad project, that is still in active development.

Quite an impressive list — and yet I may very well be missing somebody or something. Anyway, I have to investigate this promising stuff.

All I want is something to write in next week’s Lilypond Report…

What is this strange thing called algorithmic composition?

As far as I can see, it is a way to create music using a series of mathematical operations.

Today, Trevor Bača told me something I had never realized: "Algorithmic" composition doesn’t necessarily imply composition with computers: composition could be plenty algorithmic centuries before computers showed up: it’s customary, in fact, to see references to Mozart’s dice games, the canonic transformations of the twelve-tone row, and even French isorhythm à la Machaut at the beginning of a great many articles, books and dissertations on algorithmic composition".

When I try to think about it, I realize that, de facto, mathematics are a vital part of any music language (and particularly Western music from the past five centuries, where this relationship has been getting stronger and stronger — as the music was more and more precisely written? The art of polyphony, from gregorian counterpoint to baroque fugue and contemporary serialism, relies on mathematical principles. It’s even more visible when it comes to rhythmic structures: even the crappiest techno music is basically made of 4^n measures.

I have also read a 2003 essay by Torsten Anders; this made me understand the distinction between automated composition and computer-aided composition:

• Automated composition "is usually expected to compose music as fully as possible […] to allow scientific verification by comparing the original and the model-composed music.".

• On the contrary, computer-aided composition is still an artistic act. I remember Trevor also tried to explain the same idea: When graphic designers sit around and use Photoshop, are they engaged in "algorithmic design" or "algorithmic photo manipulation"? Really stop and think about this question for a moment. This helps us imagine how the compositional can be somehow computer-assisted without being fully algorithmic.

OK. Somehow, this makes sense.

Miller Puckette

Today, thanks to José, I’ve learned about someone important named Miller Puckette. I have to show you his official picture. Apparently, this man has written one of the most important piece of software for contemporary music.

I have begun to read his book The theory and Technique of Electronic Music. At the beginning, he says "electronic techniques to

• record,
• synthesize,
• process, and
• analyze musical sounds […] came into [their] modern form in the years 1948-1952, but [their] technological means and artistic uses have undergone several revolutions since then".

It’s interesting to note that according to him, sound recording, synthesis, processing and analysis share a common historical ground (that merely corresponds to the beginning of modern computer science).

A 1956 advertisement

Indeed, the first approach to algorithmic composition seems to have been based on statistical analysis of existing music. I have just found the following glorious pamphlet, proudly published in 1956 and entitled A Machine to Compose Music:

Since the functioning of the human mind may seem to be beyond analysis, the thought that a machine could compose music might seem absurd at first. However, by the use of statistical analysis and the application of information theory, such a machine could be built.

This first orientation was clearly what Torsten referred to as automated composition. This approach, as far as I can tell, originated alongside with the Artifical Intelligence utopia [1]; it was this era where humans wanted computers to compete with them, either by writing fake Bach scores or by playing chess against them.

The 1956 advertisement mentions the use of statistics applied to music. This is particularly interesting, since this kind of analysis seems to be the common ground of most automated composition experiments since then. In a 1996 essay on Grammar-Based Music Composition, John Mc Cormack reports that "A diverse variety of stochastic processes have been applied to music composition. Most involve two stages:

• First, some existing musical expression or quantity is analysed.
• Following the analysis, re–synthesis is applied using the selected stochastic technique".

Statistics and randomness: that’s about all a computer can do to pretend becoming an "intelligent" composer [2]. Quoting McCormack again, for instance, "one popular approach to computer composition is the use of Markov chains […] Most attempts using this form of modelling first require some existing composition to be analysed".

Interestingly, none of the people I’ve interviewed so far ever referred to the concept of "intelligence" any more. Visions seem to have changed during the past 15 years; a paper by Bruce L. Jacob, written in the early 90s, expresses this progressive slide, by referring to the computer as both an intelligence and a tool:

The question what is algorithmic composition is analogous to the question what is artificial intelligence. Both are hotly debated. Neither are answered easily. Both ask fundamental questions about the authorship of ideas.

[…] Algorithmic composition is as old as music composition. It is often considered a cheat, a way out when the composer needs material and/or inspiration. It can also be thought of as a compositional tool that simply makes the composer’s work go faster.

A tool.

In other words, let’s not try any longer to teach the computer how to write music on its own, but instead teach it how to write what we want it to write. No more, no less.

Honestly, I’m confused.

Randomness. Is that the real name of "creativity"?

I remember Mozart’s dices. Computers are good at generating (apparently) random numbers. Does it make them good at composing too?

Today I’ve discovered this german guy named Karlheinz Essl. I tried to read his thesis in German, and was struck by a formula: "die labyrinthische Dialektik von Chaos und Ordnung" (the labyrinthine dialectic of chaos and order).

As far as I understand it, "pure" chaos (if there’s any) couldn’t be qualified as art: the composer has to put some order in it. The artistic process is precisely this delicate balance between randomness and rules.

No matter how, a computer has to be taught what it has to do, and how to do it. The purpose of a composition program is exactly to allow the user (the composer) to easily define such rules.

This implies, at some point, that the way the program is designed can influence the way you compose; according to Miller Puckette, "The design of the software cannot help but affect what computer music will sound like, but we software writers must try not to project our own musical ideas through the software".

For instance, LilyPond itself can compose music; have a look at the snippet entitled "Generating random notes".

\score { #(let ((random-state (seed->random-state 128))) (ly:export (make-music 'SequentialMusic 'elements (map (lambda x (let ((idx (random 12 random-state))) (make-music 'EventChord 'elements (list (make-music 'NoteEvent 'duration (ly:make-duration 2 0 1 1) 'pitch (ly:make-pitch (quotient idx 7) (remainder idx 7) 0)))))) (make-list 24))))) }

The code behind it tells Lily

• to produce exclusively single notes (make-music 'NoteEvent)
• with a fixed duration (ly:make-duration 2 0 1 1, that is quarter notes)
• not above the upper g ((quotient idx 7))
• choosing among all notes in the scale ((remainder idx 7))
• without any accidental (0)
• and to stop after having produced 24 notes.

GNU Solfege

Another similar example is the GNU Solfege application, created in 1999 by Tom Cato Amundsen, a former Swedish LilyPonder. Solfege is written in Python, and includes about four hundred pre-defined rules (and one hundred ready-to-use modules) to randomly generate scales, chords, harmonic progressions etc. Nothing LilyPond couldn’t do… but in a much simpler way.

A twelve-notes serie in GNU Solfege

However, both LilyPond and Solfege are only suitable for relatively simple composing rules. As Jaime noted, "I suppose one of the things LilyPond is not suited for is to make algorithmic operations on musical material except for the usual transposition, doubling durations, etc..

Most programming environments are perfect for this because they use number representations. Therefore the need to be able to go from each representation to the other. I imagine that LilyPond users that are interested in algorithmic procedures would benefit of being able to create numbers and converting them into LilyPond code."

I had to ask Trevor.

Trevor Bača

There’s one thing I noticed about Trevor Bača (besides the fact that he is a distinguished LilyPond contributor and sponsor): he often has a lot of things to say.

I remembered having read a long and interesting mail of his on the list in 2007; therefore a few days ago I asked him his opinion about algorithmic composition. Here’s his first answer: "Hi Valentin, excellent topic! I have a huge amount to say about this … probably too much …" — see what I’m talking about?

OK, so, Trevor, what do you think?

— What matters to me is the model of the score — the "formalization", if we want to borrow Xenakis’s term. Lilypond has an incredible model… but one that isn’t iterable or traversable or subject to the other composerly operations one expects of an environment for computer-assisted composition. LilyPond files are for input, not manipulation.

— That meets what José told me yesterday. Ok, so are you saying that you need specific tools?

— Yes. Five or four years ago, Víctor Adán and myself independently developed two different systems in Python, to give ourselves formal(ized) control over musical score. His was named Cuepatlahto, mine was named Lascaux.

— Interesting! That reminds me of the beginning of LilyPond…

— I flew to New York so Víctor and I could compare our codebases directly and we decided to unify out projects. The result is Abjad, a new system which takes the best of both Lascaux and Cuepatlahto and then cleans up the model and adds more.

— OK. I understand this system is still in development…

— Yes. Basically, it can be seen as an extension of the interactive Python interpreter, designed to let you build up larger and larger instances of notational elements like tuplets and staves, all of which know how to format themselves as Lilypond input.

It can be compared to some existing systems: for example, Chris Ariza’s athenaCL models events in time and focuses on the creation of event-list stye list-of-floats output; Abjad models music score and focuses on the creation of hierarchically nested Lilypond output..

— Why creating such a system? What does it allow you to do?

— What’s really interesting to me are not what computational *tools* composers use to compose but what composers are actually *doing* when we use computational tools. In my music I’m interested in building up massively composite textures and working out gestures with so many points of local change as to become difficult to manage by hand. I also (re)analyze my own musical material to extract fully or partially formed patterns to seed, grow, expand or conflict with new material in the same piece (or even in newer, later pieces). So I use computers.

I think what matters most is that some stuff is waaay to difficult to do by hand and that computers help a damn lot. It’s much easier to manage, for example, a palette of 18 different types of attack in the flute, all floating around and interacting with each other in patterned ways if you’ve got a symbolic model hanging around somewhere. And it’s also much easier to go back and ask questions of your own material such as "how many occurrences are there of consecutive quintuplet thirty-second notes between measures 31 and 90 that feature interval adjacencies not greater than a minor third?" when you have an instance of a data structure of your own material that you can iterate and interrogate.

— Hm. Let me rephrase: what does it allow you to do, concretely?

— In 1993 I started work on a piece that I would later call "Poème récursif". The original was a short 13-page score for 16 percussionists running 64 + 1 measures. Here’s a page from that original version of the score.

1993 version

Twelve years later, in 2005, I wrote a second ’verse’ of "Poème récursif". The dimensions were expanded to 64 parts deep and 256 measures long.

2005 version

You can immediately tell what role LilyPond played in the compositional process. The sketch from 1993 is completely hand-written, it took me weeks to write it out, even after the structure of the piece was determined almost completely. I remember wanting to go much deeper at that time, but doing so by hand was completely prohibitive: it meant working with many thousands of notes per page.

Enter LilyPond. Working with code on the front end and LilyPond on the back end, I was able to work through dozens and dozens of different configurations of all sorts of parameters, trying and trying and trying again until I was able to map, frame, fracture and recombine exactly the right parts of exactly the right structures to get what I wanted.

— I think I’m starting to understand… (am I?)

— The truth is that I’ve always been deeply uncomfortable with this process. What exactly do we convey when we mine algorithms or patterns? Searching for variations — whether by hand or machine — is one thing. But it seems to me that there is some sort of line we can cross over in passing to certain algorithms that are so … massive, perhaps … that they can generate entire pieces and also produce monolithic, even closed results. There’s nothing wrong with these results, of course. But there’s a certain type of sorcery needed to communicate effectively within the "wind it up and let it go" way of working. And I think that magic may yet be too black for me.

— "Wind it up and let it go"?

— I’ve stolen this bit from Andrea’s way of talking about his own process. And I believe he’s even referred to himself as "radically of the wind it up and let it go" mentality. Which goes to show that there are those among us that can cast that particular type of black magic …

---

After this Report was first published, I received a mail from Victor Adan, adding some comments to what Trevor had already said:

I would just like to add a comment about algorithmic composition in general. The definitions I know always revolve around the *method* of composing, and so debates arise as to which methods are valid as algorithmic and which are not. Does using random processes make one an algorithmic composer? Does mapping cellular automata to pitches? Does using 12 tone rows? Constraint programming?

It seems to me that algorithmic composition is as much an aesthetic paradigm, a way of thinking about composition, as it is a methodology. In algorithmic composition it’s not so much the specific sequence of sounds or notes that matters as it is the algorithm that produces the sequence. i.e. it is as much (or more) about how a thing is generated as it is about what is generated. Music is a byproduct—a measure, even—of the algorithm, which becomes the artifact of main interest.

Thanks to both Trevor and Victor; good luck on your common project!

---

After having discussed with Trevor, there was one little thing I had to investigate. He had mentioned Christopher Ariza, that some of the other guys had already mentioned.

I opened his website, thinking I might found some other cool composition software I wasn’t aware of…

And then I saw it.

This list of composition systems.

Er…

Let me count…

118 items!!!

Gasp…

So many programs; so many systems; so many people…

One of the first things Torsten told me, a few days ago, is that "there exists an amazingly large number of algorithmic composition systems". I could never have imagined how much this statement was true…

However, Torsten did also provide me with a good angle to start: "Various programming languages are used, but Common Lisp is kind of the lingua franca of algorithmic composition (I assume for historic and cultural reasons)".

Indeed. A few weeks ago the Report mentioned Igor Engraver software, that is written in Common Lisp; there’s also quite a few music-dedicated languages that are based on Common Lisp, such as:

• Fomus, a translator of algorithmic composition output into different notation formats (Lilypond, MusicXML…)

• Common Lisp Music, a sound synthesis system

• Common Music Notation, a notation system, like Lilypond though less advanced in default notation quality

• Common Music (and its father, Schottstaedt’s Pla), a composition system

• Compo, a composition system

• Patchwork, a visual programming composition system (user programs graphically by connecting boxes with patch chords) — and its two derivatives:

• OpenMusic, or the
• non-free [3] PWGL.

Hard not to get lost, isn’t it?! [4]

What makes Common Lisp so attractive? According to Torsten, "Common Lisp is a compromise which brings together the various features of several now obsolete Lisp dialects, it is a *huge* language" [5].

There’s more: Common Lisp is a high-level [6], multi-paradigm [7] language.

For instance, José Padovani told me what Common Lisp (and particularly Common Lisp Music) allows him to do:

José Padovani

I generate some codes that write down a lilypond code. Then, I run LilyPond to see my "experiments". These codes are not organized in a library like Fomus, as I am not a programmer, but a composer.

In a composition of mine, I’m using Lisp code to quantize some onsets from a spoken text. The quantizer is very particular, since I want the measures to begin according with the accented pulses, etc.

In another project, I have written a lisp code that creates some "textures" with genetic algorithms and writes down the result for 18 string instruments, with some quantizing rules. What is nice is that, firstly, I had initially used these functions to generate senoid grains; then since I liked the result, I decided to do something similar for string orchestra.

Well, that made me want to get my hands dirty. I decided to try the first program in the above list: Fomus.

— Testing Fomus —

Fomus was developed by the former LilyPonder David Psenicka; as I said, it’s written in Common Lisp (though José told me its author is recoding it in C++). It’s name stands for FOrmat MUSic, and according to its description "The purpose of it is to facilitate the conversion of "raw" algorithmic output data into readable music notation, a process that can be frustrating; […] FOMUS attempts to remove this frustration by making a reasonably intelligent attempt at spelling notes, quantizing offsets and durations into readable rhythms, laying out the information into voices and staves". It supports the following output formats:

• LilyPond
• MusicXML
• Common Music Notation
• MIDI

So, many export formats. This choice, as far as I understand, is what Torsten refers to as an "output-format-neutral music representation" [8]

Fomus is released as a compressed archive; after uncompression, one can either load Fomus inside a Common Lisp interpreter, using
(load "load.lisp") and then (use-package :fm), or install it as a standalone application through an ./install.sh script (unlike LilyPond, it does not contain its own interpreter, so one might want to install, for instance, the Gnu Common Lisp implementation).

As a matter of fact, I couldn’t make it work (there seemed to be a problem with the AutoAccidentals code). However, I did find some interesting snippets in the documentation:

Most of Fomus remains yet to be documented. However, it looks very interesting, and its syntax looks so familiar that I wish it could be entirely implemented in Scheme, and therefore provide a nice superset to LilyPond. This way, one could use it as a more user-friendly standalone and cross-platform application. (I was running Windows when I tried it, and it was a bit painful — not to mention it kept assuming I should have Adobe Acrobat installed).

Moreover, it seems to support only LilyPond 2.4 export (when we were still using the TeX backend). A nice project, that needs interesting ideas less than resources to implement these… [9].

Today I received a mail from Andrea Valle.

I’m actually using a very eclectic approach, sometimes using Python, sometimes SuperCollider for "gluing" stuff together, sometimes my output is LilyPond, sometimes graphical notation.

So, I’m a bit skeptical about a general system, as I change my solutions on a piece basis.

Andrea Valle

Browsing Andrea’s MySpace and Youtube page, I understood that he’s interested in very different music languages, including acousmatic systems.

I had almost forgotten that many (if not most) of the contemporary music created in the past half century is actually electroacoustic music, made of sounds rather than notes [10]. "Many algorithmic composition systems are also genuinely sound synthesis systems", as Torsten made me realize.

In an article that is to be published soon at Nime2008, Andrea makes some interesting assumptions about the relationship between elctroacoustic and algorithmic music:

Algorithmic musical composition has been proposed and practiced widely starting from the ’50s. […] An interesting shift in perspective has occurred roughly from the ’60s up to present day.

The first approaches to algorithmic composition were driven by instrumental scoring. But […] the idea of a purely algorithmic approach, in which a strict formalization rules the whole composition process, is no more pursued in its integrity and has migrated from the instrumental domain to the electroacoustic one.

And why is that?

In fact, considering the final output of the composition process, while in the electroacoustic domain the synthesis of the audio signal is a trivial task per se, in the instrumental domain the generation of musical notation still remains a very difficult task.

This notational issue has prevented the diffusion of real algorithmic practice for instrumental composition. Such an approach, in which the composition process is turned into a completely algorithmic workflow –from the first idea to the final score–, can be defined as Integrated Algorithmic Composition.

You would have guessed that this "Integrated Composition" thing is what Andrea Valle (the "wind it up and let it go" kind of guy, remember?) is most interested in.

How does he achieve that?

Instead of starting from a solid framework where to insert modules…

(That would be the Common Lisp approach we’ve already mentioned.)

[… it is possible to start from an indefinite variety of available modules to be plunged –when necessary– into an open environment. Such an environment is fluid because it is intended as a glue capable of attaching different modules together.

In other words, it’s no more about implementing modules inside a single program, but about writing a "meta-program" (a so-called "wrapper") around several completely different programs.

A Python wrapper for graphical notation

This method can be used for dealing with randomly-generated notes (stochastic process), or for reacting in real time to some events, for instance an audio input:

A composition project involving parameter extraction from audio signals. […] was to use as starting material an excerpt from Sophocles’ Antigon, which was read by a philologist so to respect as possible the reconstructed Greek classic pronunciation. Three voices sing melodies generated from data resulting from the analysis of the original audio file, in particular from the fundamental frequency and the first two formants.

The Praat software has been chosen for the analysis task. The SuperCollider application has been chosen both as system glue and as an audio module: as a language, SuperCollider is rich in data structure, highly expressive, provides an interface to the OS environment, allows for string manipulation; as an audio server, it provides state-of-the-art sound processing.

And for the music score output? Let me give you a clue — its name is Pond…

Click here to listen to a sample.

I’m not sure if electroacoustic music is really more "purely algorithmic" than music involving notes and scores. Though I haven’t studied the question as Andrea has, I have the feeling that most electroacoustic musics, at a some level, rely on composer’s arbitrary choices as much as any other music (the structure, the gestures for instance).

What is true, however, is that most facilities dedicated to contemporary music around the world, deal with sound processing and synthesis as well as they deal with algorithms and programming. Some of the most famous of these are:

• the IRCAM in Paris, France (where Miller Puckette used to work, and where the Lilyponder Karim Haddad works too, as far as I can see),

• the CCRMA at Stanford’s University, USA and

• the ICCMR in Plymouth, England, where Torsten Anders is involved.

Remember this guy named Miller Puckette? A few days ago, José told me to read a paper of his, entitled "Max at seventeen". Who’s Max? It’s actually a software developed by Puckette at the IRCAM in the 80s. It was named after one of the most important programmers in the past 50 years: Max Mathews.

Back to the 50s — you know, when some people wanted machines to write music like Bach? Well, some other people actually began to regard the computer, not as a brain, not as a tool, but as a… music instrument.

As soon as 1949, the CSIRAC (Australia’s very first computer) was able to play digital sound. However, it’s in the USA that Max Mathews developed the MUSIC programming language, that is, according to José Padovani, the "father’ of all modern audio programs (for instance, the popular Free Software CSound, created by Barry Vercoe, one of Mathews’ students).

In the 80s, Miller Puckette (one of Vercoe’s students, again), at the IRCAM, began to imagine a new style of programs, designed for "real-time computer music performance".

How do you define a music instrument? Do you see a piano as a 88-keys keyboard and three pedals? Well, to Puckette, "in a multi-tasking environment we could describe a piano as 91 tasks, one for each key and pedal. The performer—the piano user—chooses in which order to trigger the 91, and how many times they will be triggered".

Do you see where this is going? Max is mainly about defining some events, and then triggering them while performing. It’s a real-time oriented application, that can deal with pre-recorded sound samples, but also with notes, as you will see.

Many music software are oriented towards real-time use. These includes SuperCollider, which the LilyPond Report already mentioned in a previous issue, and ChucK. As far as I can see, however, both are more designed to deal with sound processing and synthesis than notes.

— Testing PureData —

Therefore, I decided to install Pure Data (pd), which is the Miller’s open-source implementation of Max.

Since I was running Windows (again), it allowed me to notice how cross-platform is pd (whereas the original Max was only developed for Mac) — as far as I can see, it relies on the Tk library. When you launch it, and start a new project, you see nothing but a blank window. According to Puckette, this is a deliberate choice:

On starting Max, the user sees nothing but a blank page—no staves, time or key signatures, not even a notion of ``note," and certainly none of instrumental ``voice" or ``sequence." Even this blank page carries stylistic and cultural freight in at least one interesting respect: the whole idea of incorporating paper in the music-making endeavor was an innovation of Western Art Music, for which many other musics have no use at all.

So say we all…

OK, let’s try to put something on this page:

My first attempt at pd

As you can see, some various objects are available but it is mainly about boxes. A box can contain a musical event, or trigger another event, or both. In fact, you build your own interface, the way you want. I couldn’t go any further, but let’s ask YouTube to do it for me:

Please have also a look at this video to see how the same principle can be extended.

The computer as an instrument. That could be… Well, that could be actually fun, to quote Miller Puckette:

Computer science has never found a metric for determining whether or not a computer program is fun to use.

[…] We speak of ``playing" a violin, not ``working" it. If using a computer program feels like working in a bank or a hamburger chain restaurant, musicians won’t (and shouldn’t be asked to) do it.

[…] The computer should ideally feel in the musician’s hands like a musical instrument, needing only to be tuned up and then played.

Yesterday’s examples demonstrated sound sampling from real recordings. However PureData/Max can also deal with MIDI format, and therefore generate its own notes — which brings us back to algorithmic composition.

Quoting Miller Puckette, who seems quite frustrated with MIDI (but who isn’t?):

because of the limitations of 1987 (you could have a GUI or a programmable DSP engine but not both in the same address space) Max became a ``MIDI program.’’ There was nothing fundamental in Max that was derived from MIDI, however; as far as Max was concerned, MIDI was an I/O interface and nothing more.

— The Lexikon Sonata—

The other day I told you about Karlheinz Essl. I had to show you a composition of his: the Lexicon sonata, that has been composed (and performed) using Max.

I am impressed by the overall expressiveness of this work. You can distinctly hear (maybe feel) dramatic progressions, musical gestures, tension etc.

The output, as far as I understand, is entirely in MIDI, though the sound is actually not so bad. There’s nothing pre-recorded here, everything is generated on the fly, and it clearly demonstrates what real-time musical gestures can be made of. Please have a look at the web page where he explains it all: the way music events are named, for instance, is particularly interesting. Esprit, Joyce, Clouds, …

— Real time—

This ability to create events while performing, like you would do when improvising with an actual music instrument, is what is commonly referred to as a "real time" process.

What does real-time change, in terms of user experience?

Jaime Oliver

Jaime — Well, changing parameters while listening makes a great difference for me.

The fact that it is a real-time application doesn’t contradict that it is a quite complete programming environment able to save information in different representations. So being real-time doesn’t imply that it can’t produce information that could then be used ’offline’ or use ’offline’ information in real time.

As a composer/computer musician, my personal interest is to be able to switch representations, and of course one of them is the musical score as a mean to communicate with other musicians, especially performers with traditional occidental instruments. This means to use algorithmic techniques to generate melodic, harmonic or rhythmic materials and output them into LilyPond.

José — Although it is very powerful for designing real-time systems, PD is not suited to non real-time symbolic processing of scores, or symbolic music structures of the western tradition.. What I mean is: a "quarter" or a "Staff"doesn’t mean much in PD.

I use PD very much, but not for non-real-time music purposes: it’s convenient for real-time sound experiments and for interactive music projects (like compositions for instrument and live electronics).

— Can you really create music scores using a real-time program?

Jaime — Pd allows for both sound processing and music scores creation: athough it doesn’t have a score format as such, you could represent a ’musical score’ in many ways, the most simple ones being lists, arrays and text files. I’ve seen Miller’s attempts to make music scores and they are not even close to LilyPond’s functionalities; his attempts are more a temporary display.

Therefore I am currently developing an external that takes pitch-duration pairs and converts them into a LilyPond score.

Torsten — As José mentioned, PD was not intended originally for creating notated scores, but many other composition systems are, and most of these output into Lilypond.

By the way, creating Lilypond scores this way is much more easy than, say, creating a Finale file — the textual format of Lilypond is well documented and relatively easy to create automatically.

One other aspect I like about creating Lilypond files automatically for algorithmically created music is the ability to annotate the score with analytical information which was part of the algorithmically created result.

— Is there a strong, fundamental design difference between "real time" and non-real-time music software?

Torsten — Is there a strong difference between improvisation and composition?

There are systems which focus on sound synthesis, systems which focus on interactivity with various means, systems which focus on notated score output, etc. Some are addressing multiple concerns, but nevertheless these concerns differ greatly in their needs.

José — These past years, the two trends are getting closer. PWGL, for example, is capable to process and synthesize sound in real-time and has created a particular own notation app.

Marcos Alessi Bittencourt managed to make a score box inside PureData, using a C translation of Common Music Notation.

It would also be very nice to visually represent Common Lisp functions in PureData (possibly using Open Sound Control), like PatchWork and its derivative do. There’s a Lisp interface object for Max, and now a Java implementation as well…

— Interfaces—

— José, you were talking about visual representations; there’s indeed a fundamental difference between text-based programs, and graphical programs such as Pure Data or OpenMusic… Don’t you guys like to have a visual output?

José — I am interested in visualizations of all kinds. For instance, I have used some of Common Music Markov functions to create Computer Assisted Poetry, based on poems of Carlos Drummond de Andrade (a well-known Brazilian poet), and have generated some visualizations of spoken texts in Processing, a Java based language for visual arts (though it is more visual oriented, it has some librarys to synthesize sound and can be linked with PD, SuperCollider or Max/MSP with the Open Sound Control protocol).

Trevor — I’m extremely dependent on actual notation when I work, and I know for a fact that I would never have been able to write my most recent pieces without some sort of computational front end (the code) and some sort of music notation back end — and that back end is LilyPond.

Torsten — The overall goal is similar in various systems: to provide a rich toolbox for defining computational music theory models.

But whereas some put particular emphasis on making these tools accessible for a non-programmer, code-based programs address a different audience: users which have either a bit programming experience or are willing to learn that. (Though they still don’t need to be professional programmers: I am a musician too.)

I fully understand that visual programming is beneficial for many users which have often little programming background. However, for my own purposes a visual programming language does not "scale up" — complex music pieces simply end up in spaghetti

My journey was coming to its end, but I couldn’t leave without paying Torsten Anders a visit.

As I already told you, Torsten works at the ICCMR in Plymouth, England. On the ICCMR’s website, one can read:

"We are chiefly interested in computational modelling of Music. We are developing new intelligent musical systems that will be able to evolve their own rules for musical composition and ability to interact with musicians and listeners in much more sophisticated ways than the present ones can do. We predict the emergence of new kinds of music content, most of which will be generated on the fly, requiring new modes of representation, access and interaction."

Looks promising, doesn’t it?

Torsten Anders

Please have a look at the list of publications by Torsten, that are all freely available [11] I’ve already quoted his 2003 essay:
Composing Music by Composing Rules: Computer Aided Composition employing Constraint Logic Programming;
the following quotes will be taken from there, as well as from his thesis
Composing Music by Composing Rules: Design and Usage of a Generic Music Constraint System.

As you can see through these titles, Torsten aims to deal with music "rules", and program "constraints".

Basically, the latter is nothing but the implementation of the former. Music is based on rules? Let’s teach a computer how to apply these rules!

For centuries, compositional rules were an important device for expressing compositional knowledge in a music theory.

The attitude of constraint based computer aided composition is particularly close to the way many musicians and music textbooks are thinking or talking: musicians often describe, for instance, a certain compositional style by a set of compositional rules. The concept of constraint based computer aided composition is hence intuitively understood by many musicians.

Many musicians thus feel comfortable with a computational model based on the notion of rules. For example, rule-based approaches attracted much attention among composers, because by defining rules composers can formalise virtually any explicitly available compositional knowledge as a task which the computer can solve automatically.

What is a compositional rule, precisely?

For a composer, it is particularly intuitive to describe a score or a musical style by modular compositional rules.

A rule restricts score objects (e.g. sets of notes) and their parameters (e.g. durations or pitches). Frequently, a rule controls mutual dependencies between multiple score objects and parameters.

Compositional rules are particularly well suited to describe music, because rules describe the multi-dimensional nature of music in a modular way. When listening to music, we perceive various aspects such as rhythm, harmony, voice leading or instrumentation simultaneously.

Similarly, the formalisation of a task as complex as composition is greatly simplified when the task is stated in a modular way. When the task description is broken down into rhythmic rules, melodic rules, rules on the harmony etc. then the various musical dimensions are formalised one by one.

The programmer’s goal, therefore, is to allow the composer to define his own compositional rules, as "constraint satisfaction problems" (CSP).

Basically, we could see this program as a filter or a sieve for randomly generated events. Some of these events will be left random (as unknown variables), other will be constrained.

Since the program does not want to affect the composer’s style, this randomness has to be the most generic possible: in other words, if you do not feed the system with any rule, it should be able to "invent" any music possible — his search space would be infinite.

A most generic music constraint system would allow its user to implement any music theory model conceivable. From the perspective of the present research, such a system is an ideal system.

The set of solutions for this extreme CSP (which is only theoretical) contains any conceivable score. The user can freely constrain any unknown information in order to reduce the set of solutions as desired.

So, it’s not about starting from scratch and making music, but on the contrary, starting from an infinity of possible scores, and narrowing down these possible solutions to fit what you have in mind.

These rules can be applied in different ways:

Composers often prefer a way of working which is situated somewhere between composing ‘by hand’ and formalising the composition process such that it can be delegated to the computer.

For example, the composer can determine some aspects of the music (e.g. certain pitches) by hand and constrain other aspects by rules. Alternatively, the composer may specify the high-level structure (e.g. the formal structure) manually and let the computer fill in the details.

[Deterministic constraints] can be used together with other compositional rules. […] A traditional musical example which uses a similar technique can be found at the beginning of the first movement in the fourth symphony of Johannes Brahms. All intervals between the first eight pitches of the starting theme are falling thirds (or rising sixth) – thus forming a simple pattern of the pitch classes, which is further controlled by the underlying harmony.

This approach was already implemented in PatchWork and OpenMusic (through OMClouds, PWConstraints and Situation). Though I do not know anything (nor could I find any easily accessible piece of information) about these, it’s time for me to introduce you to the result of Torsten’s research: time to meet.. Strasheela!

— Testing Strasheela—

Strasheela is the name of the scarecrow in the Russian story The Wizard of the Emerald City. This software is actually written in Oz, so you might get the allusion.

Strasheela is one of the best documented projects I have seen and read about during this investigation. Not only are Torsten papers fully accessible, as we’ve already seen, but Strasheela’s documentation itself is nice and looks very promising — LilyPond’s Documentation Editor himself, Graham Percival, is credited on the site; could that be a coincidence?

Launching Strasheela through Emacs

Strasheela needs an implementation of Oz to work; the only one available is Mozart, easily usable through the GNU Emacs editor. This solution seems to be more or less cross-platform, but I only tested it under GNU/Linux.

I earlier referred to Common Lisp as a multi-paradigm language; so is Oz, though it was originally conceived following a very specific paradigm: "concurrent constraint programming, which subsumes concurrent, logic, and functional programming", which is the reason why Torsten chose it in the first place: it makes Oz/Strasheela the first "special constraint programming platform for music composition".

There would be many things to say, for instance, on the way random notes are generated and filtered (what Torsten calls the "search process"). This requires such a huge amount of computing that Strasheela gives the ability to define a so-called "search-strategy" that can make the process quicker and more efficient.

Another thing I’d like to briefly mention is the internal representation of music in Strasheela. From what I understand, its nature is actually double: precise textual representation vs abstract data typology (ADT). Here are some examples.

In the following snippet, you can see a simple melody and its textual representation.

On a more abstract level, here’s a motif declaration from Beethoven’s 5th Symphony:

motif (durations : [2, 2, 2, 8] pitchContour : [=,=,-])

Now for some Constraint Satisfaction Problems.

In the following example, we want to obtain a symmetric twelve-tone series:

Torsten kindly provided me with an example demonstrating a microtonal chord progression in 31 equal temperament. "Besides demonstrating a formal idea on creating such chord progressions, it shows how analytic information can be added automatically to the LilyPond score".

Harmonic progression - 31ET

You can also see the code behind it, and read this discussion about it.

Additionally, I just have to show you, in bulk, some cool functions I’ve noticed in Strasheela examples:

Like Fomus, Strasheela is output format neutral. It can export in MIDI, CSound, LilyPond; it can also be interfaced with Fomus, to obtain MusicXML files.

I can’t, obviously, sum up all of its functions here (that would imply that I’ve fully understand it, which I have not). But I have to say I have been seduced by its promises.

Having installed it, I unfortunately couldn’t make it work at all. I kept getting a "broken pipe" Operating System Error, which was very frustrating.

However, please have a look at these examples on the website, and maybe you’ll be seduced as I’ve been:

• This example implements the classic music theory, as defined by Fux in 1725;

• A very impressive Counterpoint example involving rhythm parameters as well;

• An example of Strasheela’s Real-Time capabilities: when interfaced with SuperCollider, Strasheela can compose a counterpoint or an accompaniment to a melody on the fly.

This latest feature is really impressive. As far as I can see, Torsten wrote about it with the scientist/composer Eduardo Miranda, who works at Plymouth too.

Making Strasheela capable for real-time performances makes it an extremely valuable and plastic tool; as we’ve seen, many composers like to instantly hear or see the result of their work, even if they’re not performing per se.

And what about the user-experience? After having seen how convinced Torsten was about the utility of text-based interfaces, and moreover in a very specific language, I didn’t expect him to consider other possible implementations… But here’s what he has to say about Strasheela’s future:

Porting Strasheela to a language which is more widely-used in the computer-aided composition community or which offers special support for unexperienced programmers makes the system more easily accessible.

An example of a widely-used language is Common Lisp, because there exist a considerable number of Lisp-based composition systems Visual languages enjoy much popularity among musicians (especially data-flow language such as Max and its relatives).

It would be interesting to investigate how visual programming can facilitate the use of a generic music constraint system like Strasheela in a way that scales well even for complex constraint problems.

If this were to happen, Strasheela could play a major role in tomorrow’s computer-aided music creation!