Monday, December 10, 2012

Software entropy

At work someone recently claimed that  "entropy always increases, and the same is often true for software systems", and was there anything we could do about that.

This is of course a very interesting question so I started thinking.  When I started   to  formulate an email in reply to him  it turned out to be way too long and slightly off topic, so instead of replying by mail  I morphed my  reply into a couple of blogposts (this and another in our internal wiki).

But getting back to the topic; actually it's only true that entropy always increases if you study  closed systems   (it says so right in the wikipedia  definition linked to above).  In an open system you can move entropy out of the system and thus  decrease entropy,   that's why refrigerators work.

In statistical mechanics entropy is basically defined as the number of states a  system can be in that is compatible with what you observe.  Technically it's usually the logarithm of the  number of possible states that's used, but that's just a  technicality to avoid  dealing with really large numbers all the time.   So a refrigerator will move entropy out of the refrigerator, and by  doing so will make the uncertainty related to the exact location  of the atoms inside the refrigerator less, so the number of  possible combinations of  locations for  the atoms will be smaller.  See  how  it all fits together? ;-)

How do we bring the that metaphor back  the domain of software?  Well,  if we define "program entropy" as the number of different things  a human perceives a program to  possibly being capable of doing, then that number of things defines  that program's entropy.  If you think the program  can only do one thing it has an entropy of zero (log(1) = 0).   If instead  you have some code that you can't  wrap your head round and it could do a gazillion things for all you know, then log(gazillion) is the entropy of that piece of code.    So then  if you have two implementations, functionally equivalent but one  that has but has fewer possible interpretations  as interpreted by  a human reader than the other,  then the less ambiguous code has lower entropy by this definition.   Note that since this is about perception of the programs by humans, not by computer it makes sense to talk about a program having different entropies for different people. Also it does make a difference if the program is short enough to wrap a human mind around it.  If you can't wrap your mind about it you can't know what it does so its entropy will be high since it in theory could do many things.  Entropy jumps in large leaps when things get incomprehensible but conversely also drops quickly when incomprehensible pieces becomes comprehensible.

The same kind of argument can be applied to configurations of systems (essentially graphs and not necessarily  not lines of code), so  anything that brings us clearer code or configurations will lower our entropy.

Does this make sense to you?

One topic I just thought of is what this definition of entropy does wrt.  abstractions.   By  the definition above abstractions must have high entropy, or do they?  Well, abstractions must have high entropy  since they conceivably can do their things in many different ways, that's the very essence of abstraction.   Let's take an ordered collection as an example:   Ordered collections can   can be implemented in a  gazillion ways, but that doesn't matter since you're treating it only as an ordered collection and all those  gazillion way are essentially equivalent.  So perhaps it works the other way, an abstraction makes it unnecessary to think of all the different ways the program implements the abstraction, so instead of thinking of the many different implementation one things about only the abstraction, so the entropy is in fact lowered by using good abstractions.

Actually I think both these perspectives are valid, but if we let an individual reader's subjective perspective be the most central one, then I think it makes the most sense to claim that good abstractions will lower a program's entropy.   That is,  it will be true if the reader is of the kind of person that thinks that one abstraction is less complex than a set of direct implementations, and I know by experience that not everyone is like that, so opinions may differ systematically depending on how well the reader is able to internalize abstractions.

Sunday, April 8, 2012

The Aria from Bachs Goldberg variations

The aria from the Goldberg variations is a beautiful piece of music. Perhaps one of the most beautiful pieces ever written.   When Glenn Gould plays it (video clip above) is of course beautiful, it also sounds very simple.  Like something anyone can play.   Well.  It even feels that way when you play the first few bars, but that's a deception!   When playing this piece I feel like I'm trapped inside (a very beautiful) swiss watch:  There are cogwheels everywhere, and they have to match up exactly or the whole thing just  stops working.  It's not that hard to put the fingers on the right place at almost the right time, but getting it exactly right is surprisingly difficult.  But of course also rewarding.   This is the kind of music it's worth spending time on.  Every little bit of improved skill on my part  pays itself back handsomely in musical experience (for myself, I don't dare let anyone else listen to this until I'm considerably more skilled :-)

Sunday, February 26, 2012

What to play when learning to play

So I'm learning to play the piano. It's a joy. It's fun. It's demanding. It's as engaging as playing video games, only better since it's a (somewhat) useful skill I'm learning. But it'a also challenging in many ways. On of the many ways is to chose what to play? I've spent a couple of months dialing in the selection of music and I think I have a mechanism that works now.

The mathematician Hardy said "There is no permanent place in the world for ugly mathematics". That is so, but it's even more so for music. I've looked in a bunch of piano schools and unfortunately a lot of the pieces in them are just boring. The sad fact is that even if I had been able to play them perfectly (which I'm not), they would not sound good, because the pieces are bad. These schools are probably written with the best intentions e.g. being pedagogical or making money for the composer, but that simply doesn't matter. If I had spent time playing this music what I had been practicing most would be to be bored playing boring music that even when perfected would be boring . There is no permanent place in this world for ugly music and in particular there is not even a temporary place for it anywhere near my piano.

So then, what do I play? The answer is: I play the good stuff. Bach, Chopin, Beethoven, Sæverud,..... whatever. But it has to be good music. I've found that for me it's better to play good music one or even several notches above my current skill level and then struggle hard with it until it's (more) within my abilities than to suffer even a minute of something that's probably within my abilities, but intrinsically boring.

Right now I'm working on the second prelude in the well tempered clavier by J.S. Bach. (BVW 847). It's hard. I'm not ready for it. However, It's also fun. I can get some of it to sound good. I can think concretely about how to improve my playing. I can listen to an abundance of recordings of this piece and be inspired by them. I'm getting better every day and all every little improvement is so incredibly rewarding because the payback of even a little progress is immediate: Get those two bars in succession without missing a note and without having to pause between the bars to read the music to figure out where to put my fingers ... and *wham*: New cool "Bachy" progression unlocked, only several hundred more to go ;-) I know that there are at least two or three full layers of mechanical skill and musical interpretation I have to get through before I'm playing this piece with anywhere near the skill it deserves but that's ok :-) I'll happily play this piece for however long it takes and enjoy every second of it, even when I don't play it very well ;)

So that's this blogpost's key insight: Don't play shitty music. It's a waste of time.

Friday, January 13, 2012

I couldn't resist sharing this little gem :-)

A comment found in a piece of software I work on (with others, names have been removed to protect the innocent):

// This class is like Jerry Seinfeldt in the TV series "Seinfeldt".
// It's fairly central, and at the core there is something, but
// almost everything you can see is just mindless maneuvering
// and observation of form in order to avoid utter embarresment,
// not always successful. In our case this is of course a consequence
// of using Java as a medium of expression, in Seinfeldts case
// I'm less sure of the root cause.


Sunday, January 1, 2012

My new piano - some first impressions

I'm an adult. I've got four kids and I haven't played the piano since my early teens, when I quit because it was to boring. Since then I've played various percussion instruments for some decades, but lately I haven't had much time for that either. Kids and work take too much time. About a year ago I visited a friend, and he had a Yamaha digital piano. I sat down and tried to play some of the pieces he had lying around and found I could, kinda, sorta play the piano: Some of the motoric skills were still there and my music-reading skills were much better now than when I quit playing piano. I spent an afternoon playing on his piano, and I was hooked.

I'm a somewhat obsessive and impulsive personality, so to avoid veering away in every direction I usually let my impulses mature until they are irresistible, or as some would say: Obsessive :). In this case I let the impulse to buy a piano mature for almost six months before starting to looking for a piano. I asked friends, looked at reviews, read Graham's booklet and went to the local music shops. Before christmas I had narrowed my search down to two pianos: The Kawai CN-33 and the Yamaha CLP-430. I can't claim that the final decision was wholly rational. Both of these pianos are as far as I can tell really good ones so I guess it came down to the shop offering a much better deal on the CN-33. I ended up getting some 30% off relative to the sticker price and that was pretty much that.

After that, the matters have been mostly practical, so I'll run down how the first few days as a piano owner has been:

  • Transport: The piano weights 53 Kg, the box it came in weighs in at 20Kg, so the whole thing weighs 73 Kgs. It's 150cm x 65 cm x something, and fits nicely in a station wagon, but you do need to be two fairly strong individuals to transport it safely.
  • Unpacking: No instructions for assembling the piano was included. At least none that Icould find. There was a small sheet of paper on the top of the stack of pieces saying thatassembly instructions could be found in the owner's manual. Well, first of all the owner's manual was at the bottom, under 53 kilos of piano pieces wrapped in plastic, and I didn't find that until after the entire piano was assembled. Secondly the owner's manual didn't say much of interest except the pretty much obvious things about not pooring liquids into an electrical piano etc. In particular, no assembly instructions could be found.
  • Assembly: Luckily assembly wasn't that hard: There are sixteen screws in a small bag. The four shiny ones are used to fasten the pedalboard to the sideboards. The four long black ones are used to fasten the back-panel to the side-panels. The four shorter black ones are used to fasten the back-panel to the pedalboard, and the four machine-screws are used to fasten the piano to the stand. Plug in the wire coming out of the pedals into the hole labeled "pedals", and the power cable into the hole saying "power", and you're done. The whole thing took about an hour and some thinking. It wasn't hard, it was just that in a situation like this you really really don't want to do something wrong, so I had to stop and think a lot to make sure I didn't make any mistakes, and I think I didn't.
  • Using it: It is wonderful. It sounds ridiculously good. Really. It's unnatural :-) The CN-33 easily sounds better than any piano I've ever played on myself and most pianos I've heard other people play. Also It's just so much fun to play. Playing on the simulated ebony keyboard is just a sensuous experience. I won't compare it to touching human skin, since someone might take it the wrong way and be offended, but that's almost what it feels like. My kids have clicked on all the buttons and played with the piano for several days. They find it great fun. Myself, I've just used the piano sounds (no organs, no bass, no midi, no usb, no weirdness) and the recording feature. The recording feature and the metronome is actually extremely useful. The metronome for keeping pulse (as a percussionist I've played more duets with metronomes than anything and anyone else :-), and the recorder for immediately reviewing something, sometimes a full tune, sometimes just a few bars. I haven't done that with any instrument before, but I've found it extremely useful.

So that's that. So far I'm a happy CN-33 owner. Let's just hope I also one day can become a somewhat proficient pianist, but as of today I'm a happy one, and that that's also something ;-)