Greetings everybody, and welcome to this twenty-third issue of the LilyPond Report!
Another year, another Report. This month we’re welcoming new LilyPond Report editor David Kastrup, who (in addition to being a talented developer) has been busy writing about some of the new, awesome features recently added to LilyPond... And speaking of awesomeness, don’t miss his interview with composer/contributor Mike Solomon, whose work never ceases to amaze!
As always, you can post your comments at the bottom of the page, or send your contributions to the LilyPond Report’s next issues.
Editorial
by David Kastrup & Valentin Villenave.
Greetings everybody, and a happy new year to all!
Yes, it’s that time again, when the Report raises back from the dead and its editors make plans to keep it alive a bit further. This year we’re back with a slightly larger team, since GNU veteran programmer David Kastrup has joined us to make the Report even deeper and more interesting!
That, however, will obviously not be enough on the long run. Thus, consider this editorial a call to arms: more than ever, the LilyPond Report needs more contributions from its readers! User testimonies, programmer complaints, nice stories (no matter how fluffy or useless), benchmarks and tests, anything that can make our community a more pleasant or a more efficient place. We’re particularly interested, of course, in hearing from the various team involved: translators, bug squad, programmers, documentation writers, this Report is yours!
Not only is this (sort of) monthly newsletter a fun way to report the overall shape of the LilyPond project to the general public; it’s also, for our community itself, a place of choice to call attention to a particular point or problem, that otherwise might get lost in the endlessly-flowing mailing lists.
For example, there may be a need to regain some interest in our reviewing procedures: many pretty good LilyPond programmers don’t find the time to take part in reviewing patches. We need to make sure that what ends up in LilyPond can be used and understood by more than just its original author — which may not happen when things are left underdocumented, or when the contributor, coming from a music background, does not obey basic programming principles. Another matter in need of work and attention is our infrastructure: Graham Percival has recently posted a call for more tasks to be automated, in order to make the "routine maintenance" less unpleasant and frustrating.
Release news
by Valentin Villenave
We are now at version 2.15.26 (the previous unstable version 2.15.25 having been skipped due to build problems). This development release contains the usual number of bugfixes, and should in particular bring happiness to Microsoft Windows users, who have been previously unable to run lilypond-book due to a particularly annoying bug.
Please note, however, that a few critical bugs remain; this is but a development version — not even a release candidate — and ordinary users should preferably use the stable 2.14 version.
What’s up with LilyPond?
by Valentin Villenave, David Kastup & Graham Percival
It has been a long time since the last Report has been published, so it is not surprising that a number of things have happened. In the free software tradition of scratch-your-itch, a few new contributors have appeared and worked on their favorite problems, but there has been less development activity these past few months than in the summer. Part of the reason might be that new procedures for committing changes have been tested and put into place, and occasional contributors might feel hesitant about what this means for them. In a nutshell, direct commits to master are discouraged; instead you should push to the staging branch. After automated testing, the staging branch will get committed to master automatically. The benefit of this procedure is that the little accidents that tended to make master unusable several times a month can be avoided.
Seasoned developers might feel insecure about what procedures to follow. But those are actually simple. Check out the summary for experienced developers, a very short but important document that should, alongside with the longer Contributor’s Guide, tell you all there is to know with regard to our project development and current organization.
An Interview With Mike Solomon
by David Kastrup
Our special guest this month is Mike Solomon, a valuable LilyPond contributor and a rather unpredictable composer; after having studied and worked in California and Florida, he has been living in France for a couple of years. Our editor David Kastrup had this interview with him via email in December 2011.
The LilyPond Report — Hello Mike, thanks for agreeing to do this interview with the LilyPond Report. You have become one of the more prolific contributors to LilyPond recently, and that of course piques our readers’ interest. Of course, we can hardly avoid talking LilyPond, but let’s begin with the big picture: How would you describe yourself when music is not a topic?
- A portrait of Mike Solomon
- "I’m the one seated on the cellist, facing backwards."
Mike Solomon — That limits me to the 3% of my life that is not musical, which is mostly devoted to comedy writing and New York Jets football.
L.R. — Interesting; any links we could share?
Mike — There’s not that much... But otherwise, comedy is a big part of my work, all of which is downloadable on my personal website.
As for the Jets, there’s not much to link to - I am a die hard fan and I try to watch the games whenever I can.
L.R. — A quick back-of-the-envelope calculation makes me guess that you spent about 9 months on those endeavors. At what time of your life would you say that music did become a topic?
Mike — Rumor has it that my father gave me pre-natal piano lessons.
L.R. — Wouldn’t your mother have preferred a smaller instrument for your first home teaching lessons? What would such a lesson have looked like?
Mike — Well, that’s a picture you can’t post on this website...
L.R. — Speaking of which, would you say that the photograph you sent me (thereabove) is typical for a gig of yours?
Mike — It’s tame.
L.R. — What instruments do you play yourself, and when did you start with them? What was it that made each instrument important for you to learn?
Mike — I have played piano for 29 years and have sung for 23, in various ensembles, shows and solo gigs in America and Europe.
- Das Zauberbuch (excerpt)
- M. Solomon, 2011 (Original file.) CC by-sa.
L.R. — Looking at the information so far, that would have made you start singing at the age of 5 or 6...
Mike — Piano was never really a choice - it was just always kinda around. I am not a very good player, but it is tough for me to conceive of my life without the piano - it’s where my hands feel most at home.
Singing was thanks to the American public school system: the best for music education in the world.
L.R. — Considering the music education I received in Germany (my "Early Musical Education" classes started before primary school so that I actually learnt sight reading earlier than reading), the assertiveness of this statement rather piques my curiosity.
In what respects would you consider the American school system to have given you a better musical education than that you see pupils in France getting?
Mike — I can’t speak for how music education is done in 2011, but the extent of most French twenty-somethings’ musical education was learning to play the flute for a couple years unless their parents enrolled them in a conservatory. Several localities have gone well beyond this, and I have the good fortune of working for one of them (Dunkirk) that is great about bringing music into the public schools.
L.R. — Oh, and of course a question that every artist (and programmer of free software) can’t avoid hearing occasionally: is that how you make a living?
Mike — Yes, and every score that has been part of the making of this living has been typeset using LilyPond.
L.R. — Looking at examples of your scores, one gets the impression that you focus on different things than one would usually expect from composers (I’d actually say that your code exhibits a similar style). Putting this focus to paper would, for that reason, appear to favor tools that have not specialized too much on standard tasks.
Can you tell us when LilyPond caught your interest, and what it was that made it appeal to you?
Mike — It’s not that I found LilyPond to be the ideal tool suited to my ideas, but rather that my manner of creating work changed as I discovered LilyPond. When I was a graduate student at the University of Florida, the bottom 2/5 of my G4 PowerBook’s monitor started to randomly black out. One day, while waiting for the bus, a cyclist riding on the sidewalk hit me in the back in full stride. I had my PowerBook in my backpack and this sealed the deal for the lower 2/5 of my monitor. So, not able to use a GUI (Finale) anymore, I turned to LilyPond (with which I had until then only experimented).
L.R. — It seems like the best prospects for recruiting high-profile LilyPond developers should then be in the Netherlands, the motherland of bicycling. Given your rather special requirements, what is your preferred editor for creating LilyPond scores?
Mike — Joe’s Own Editor. I’ve tried them all and got speediest fastest with joe. I’ve since had no need to switch.
Using the tool helped me realize that the true value of free software
is not that it lacks a purchase price, but that it gives you the
freedom to do whatever you want. So, the "focus" of which you speak
is something that, to a large extent, exists because of LilyPond.
L.R. — That’s quite an inspiring endorsement for the founding principles of Free Software.
What made you start working on LilyPond rather than with it in earnest?
Mike — I had a gig coming up where I needed woodwind fingering charts and LilyPond didn’t have them, so I made them.
L.R. — Now I am curious: what was it that you could express using a fingering chart that was not possible to convey musically using notes?
Mike — Multiphonics and bisbigliandi.
- granini di luce beccucciati da uccelli di silenzio (excerpt)
- M. Solomon, 2009. (Original file) CC by-sa
L.R. — And you did not mention playing woodwind instruments yourself: so how have you acquired instrument-specific knowledge that is so special that its notation requires fingering charts?
Mike — Lots of reading - there are lots of great texts and websites out there with audio examples that provide solid information to start you off. But multiphonics vary a great deal from player to player and from instrument to instrument and I needed a tool that could make these diagrams fast as I got feedback from performers.
L.R. — I have to add that as a violinist I found it quite liberating to work with Urtext editions of Bach’s solo partitas and sonatas: while Bach did not write down fingerings, they were inherent in the notes, and reading the mind of the composer over wide passages by playing the singularly anatomically reasonable fingering was actually quite an elating experience.
Do you think that exercising this kind of tight control over the performer and his instrument leads to results that better match your musical vision?
Mike — Absolutely. That’s not to say that all forms of specificity lead to better results, but anything that gives more information about the mechanics of how to play an instrument in notationally-ambiguous situations seems almost universally welcome. Sometimes, the tight control doesn’t line up with a musical vision as much as a "what would happen if I..." thought experiment. For example, the work six cercles for six players (takes a while to download - best viewed via Opera) is meant to garner a response from the performers that less precision would have a difficult time conjuring up. This work, like all of my recent work, is heavily indebted to LilyPond and other open source software.
The same holds true for footnotes and a few other features. Doing
these things helped me to get to know certain corners of the LilyPond
code base well, so each subsequent change is not that taxing. If I
see a request for a feature, I can usually estimate how much time
it’ll take to implement and, if I have the time, I implement it.
L.R. — Would you say that the availability of LilyPond has had an influence on how you compose music? Or do create your compositions mostly independently from how you end up typesetting them? Have you changed decisions in your compositions because of typesetting problems?
- Lucky Wok (excerpt)
- Mike Solomon, 2012. This is a short preview of a yet-to-be-published score.
Mike — I only ever conceive of (non-video) works in terms of how they will sound. I record myself singing mock-ups of the entire work before a single note is committed to the page and then transcribe it through LilyPond. The idea that something I do is not transcribable does not enter my head: I have confidence that if I want a sound, the only limitation in my getting it is my own failure to notate it, not LilyPond’s failure to provide a means of notating it. I have never run into a typesetting problem that was not solvable through LilyPond, InkScape, or JavaScript. The only time I ever change my mind in terms of typesetting is a question of expediency: sometimes, I’ll need to use a crude hack to get something done instead of finding an elegant and extensible solution. I try not to do this too often.
L.R. — Where would you want to see LilyPond heading? What areas in LilyPond are you comfortable working on? What areas would you like to see more work on? Are there areas where you would have wanted to do work on, but have found too hard to navigate?
Mike — There are certainly areas of LilyPond that I do not know at all. MetaFont, for example, is a magical black box out of which LilyPond’s fonts appear. The build system is a black forest of commands that somehow assemble LilyPond. At a certain time, all of LilyPond appeared like this to me, but the code is well written and, if you follow the order of what things are called when, it can’t not make sense.
I’d love to see more work on certain basic structural features of
LilyPond that, by design, limit how fast it can evolve. For example,
the way that non-musical objects (like markups) are laid out in
LilyPond is underdeveloped compared to musical layout. I’d like to
see this streamlined into a unified system that creates graphical
objects and puts them on a given medium via various layout managers.
I would also like to see LilyPond adopt linear programming so that
certain constraints do not have to be solved one-after-the-other but
instead can be solved in concord based on a given goal.
L.R. — Now being able to interface the inner linear programming blackboxes with user-defined typesetting tasks actually has been one pet peeve of mine with TeX, so if you figure out how to do this reasonably well and transparently, you’ll certainly have my blessings.
Mike — Lastly, I’d like to represent graphical objects’ boundaries with more precision so that collision avoidance can be more fine-tuned and uniform.
I see LilyPond becoming like SCORE with easier input syntax and more
extensibility. I think that, if the program gets more modular and can
drum up development teams around certain issues whose progress does
not impede work on other issues, it can get there in the next five
years. I, as well as several other LilyPond users and developers
working in France, have been trying to integrate LilyPond into music
pedagogy in hopes that students find it useful and, if so, that they
contribute to the development of the program.
L.R. — Well, it certainly sounds like something that LilyPond could make use of, and perhaps this task will become easier as LilyPond progresses.
Thank you for this rather inspiring interview, and all the best for your work with and without LilyPond!
Mike — Thanks!
Feature story: Prelude #1 in Scheme
by David Kastrup
Now as an illustration of the impact of the recent work on LilyPond programming, I want to pick up an example that Nicolas Sceaux has made the topic of a LilyPond programming article on his website.
First thing to note is that his analysis and solution remain fully
valid for now: after running convert-ly on his input file to
account for some comparatively minor syntax changes since when it was
written, the file runs and compiles.
Nicolas starts his analysis by showing the results of an evaluating several expressions at the Guile prompt in order to figure out the structure of his input expressions, as well as the structure of the output expressions he is going to create from them:
guile> (display-scheme-music #{ \notemode { c' e' g' c'' e'' } #})
The article does not actually mention how to arrive at such a prompt but the current version of LilyPond offers in its Scheme tutorial the advice to start
lilypond scheme-sandbox
for that purpose, and if it had been available at that time, it would have worked similarly. A noteworthy difference, however, is that
guile> (display-scheme-music #{ c' e' g' c'' e'' #})
is now sufficient (and does not produce unwanted additional output).
As the next step in his tutorial, Nicolas defines a macro ->
that is used as a syntactical shortcut to traverse through the layers
of the
SequentialMusic → EventChord → NoteEvent
hierarchy. Now this actually touches a sore point in LilyPond: there
are no established standards for conveniently deconstructing a music
expression. LilyPond has ad hoc functions in several modules for
doing this kind of operation. Some of these ways are more robust to
changes of LilyPond (like if the intermediate layer of
EventChord were to become optional,
Issue
2070), some less so. But there is no published interface.
Addressing this deficiency should be a future target, but in the mean time, things have become quite more relaxed because music functions can recognize a lot more argument types, including pitches. Since pitches are readily available as music function arguments, there is no more need to deconstruct whole music expressions in order to have access to the actually required information, the pitch. As a consequence, the main task of converting five pitches to a one-measure phrase (distributed across three voices) can be done with the simple function
ph = #(define-music-function (parser location p1 p2 p3 p4 p5)
(ly:pitch? ly:pitch? ly:pitch? ly:pitch? ly:pitch?)
#{
\repeat unfold 2 { $p1 2 } |
\repeat unfold 2 { r16 $p2 8. ~ $p2 4 } |
\repeat unfold 2 { r8 $p3 16 $p4 $p5 $p3 $p4 $p5 } |
#})
The other trick we are employing here is using "\parallelMusic"
for distributing material across three voices. To be fair,
\parallelMusic received a few tweaks upstream to cooperate
better with music functions in the course of this article, but nothing
that would not be equally desirable for a number of other things.
The last design decision leading to simple code is not to use an iterating construct over a single long music sequence, but rather call the above macro manually for each five-note pattern. While the automatic repetition did not add significant complexity to Nicolas’ rendition of the problem, it does not really fit well with the simplicity we can now achieve.
All in all, the meat of this piece can now be presented as
\parallelMusic #'(low middle high)
{
\ph c' e' g' c'' e''
[...]
\ph a c' e' g' c''
\voiceTwo | \change Staff = "down" \voiceOne | \oneVoice |
\ph d a d' fis' c''
[...]
\ph c, c g bes e'
c,2~ c, | r16 c8. ~ c4 ~ c2
| r8 f16 a c' f' c' a c' a f a f d f d |
c,2~ c, | r16 b,8. ~ b,4 ~ b,2
| r8 g'16 b' d'' f'' d'' b' d'' b' g' b' d' f' e' d' |
c,1\fermata | c1 | <e' g' c''>1\fermata \bar "|." |
}
This sets up the three music variables low, middle, and
high with the appropriate material for the respective voices.
One advantage of using an explicit macro \ph for each occurence
of the default phrase is that we can just fudge in stuff like the
staff change and the ending directly.
The spine of the piece can now be put together as
\score {
\new PianoStaff <<
\new Staff = "up" {
<< \high \\ \middle >>
}
\new Staff = "down" {
\clef bass
\low
}
>>
\midi {
\context {
\Score
\with \settingsFrom { \tempo 4 = 80 }
}
}
\layout {
\context {
\Staff
\with \settingsFrom { \accidentalStyle modern }
}
}
}
The context definitions introduce us to \settingsFrom, another
addition. While the Midi definition looks like a step backwards from
the original
\midi { \tempo 4 = 80 }
it turns out that this syntax has been deprecated since version 2.9.16 [UPDATE: see David’s Post-Scriptum below] and would have required rewriting as:
\midi {
\context {
\Score
tempoWholesPerMinute = #(ly:make-moment 80 4)
}
}
- Præludium BWV 846
- Download the complete LilyPond example.
Nicolas’ original version did not set an accidental style, something
we achieve score-wide with another use of \settingsFrom.
Instead he manually forced the natural in measure 29. We cannot do
this in input to \ph since forced accidentals are not a
property of the pitch but rather of the note event, and thus are no
longer accepted. But accepting a forced natural is not the same as
actually interpreting it, and the original code failed to extract this
information in addition to the pitch itself, and thus did not work as
intended anyway.
So if we recapitulate in what respects the current version of LilyPond has made it easier to tackle this particular challenge, we can note the following points:
- The actual information we wanted to process had been a sequence of pitches. Not having to extract them from single notes has been the major simplification of the original code.
\parallelMusichas been improved to a degree where we could employ it for this task. We have, however, been lucky that our basic phrase length has been a single measure: if it would have been half a measure, changing voices with a bar check would have resulted in warnings. Probably\parallelMusicshould be changed to just remove the bar checks, and instead check that every set of fragments is consistent in its length.\settingsFromis a convenient tool for creating context modifications.- The addition of a simplistic (and documented) Scheme sandbox actually provides an environment where one can carry out the experiments that Nicolas showcased.
Post-Scriptum: Prelude #1 in Scheme
Update from March 1., 2012:
In one of the examples above, we wrote that some syntax has been deprecated for several years; well it turns out that the upcoming version 2.15.32 will now allow writing the above \midi and \layout blocks just as
\midi { \tempo 4 = 80 }
\layout { \accidentalStyle modern }
thus no longer requiring you to know which contexts actually require
context modifications for the given functionality. This works with most
property-setting music commands and makes it much easier to set global styles for music.
D.K.
Bug Report of the Report
Valentin Villenave
From its very first issue nearly four years ago, the LilyPond Report has always had a "favorite quote" or "favorite bug report" section. Of course, it’s been a while since I’ve looked at our bug tracker; nevertheless, I’ve been keeping some quotes for a rainy day.
One of these doesn’t come from an actual bug report, but from a patch which Mike Solomon (our special guest this month) proposed in May 2011: Allow LilyPond to ignore certain note-heads in a stem. Seeing Mike (then a rather new contributor) tackle such a useful work made our Project Manager Graham Percival... well, quite inspired:
I see trees of green, red roses too,
I watch them bloom for me and you and the broken input/regression/drums.ly
And I think to myself, what a wonderful world.I hear babies cry, I watch them grow,
They’ll learn much more, than I’ll ever know, like a ton of programmingerror: Grob direction requested while calculation in progress.
continuing, cross fingers errors.And I think to myself, ohh what a wonderful world
Got any public email, patch, request or bug report worth quoting? Then please do chip in!
That concludes the twenty-third issue of The LilyPond Report. Although we have no plans for a predictable publishing schedule yet, Our next installment is to be expected somewhere in February 2012. Lots of things are happening currently, so this Report will be outdated by then — which is a Good Thing™.
In the meantime, feel free to send us your contributions !
Cheers,
Graham Percival, David Kastrup & Valentin Villenave
