Reinventing Discovery - Michael Nielsen [31]
What makes the MathWorks competition special is that every time someone submits an entry, the code for their program is immediately made available for other people to download and reuse. That is, anyone can come in, “steal” someone else’s code, change it to get an improvement, and then resubmit it as their own, possibly vaulting over the other person on the leaderboard. This ability to reuse other people’s code has spectacular consequences. The leading programs are constantly being tweaked by very minor changes, often changing just a single line of code in an earlier entry. Changes come fast and furious, and some contestants become addicted, driven by the instant feedback and the feeling they are just a single idea away from the top of the leaderboard. One contestant has written:
I started to become “obsessed.” At home, although I am a father of three children, my full-time job was working on the contest. I worked maybe 10 hours after work each day. On Thursday it was clear that I wasn’t going to be able to work seriously (for my job), so I took a day off on Friday.
It’s similar to the rapid cycle of feedback that makes computer games addictive. You can always have one more shot at making a tiny improvement. It’s arguable whether that’s always a good thing—the contestant quoted sounds like he needs to take a holiday from his computer—but this relentless focus also produces amazing results.
The progress of the contest is vividly illustrated by the graph in figure 4.2. The horizontal axis is time, while the vertical axis is the score: for the CD packing problem, lower scores are better. Each dot on the graph represents a competition entry. The scores dropped so dramatically during the contest that the vertical axis has been rescaled—scores at the top are hundreds of times higher than scores at the bottom. The solid line marks the best score at any given time. As you can see, there are occasional big steps in the line, indicating breakthrough ideas that substantially improve the best score. After such a breakthrough, there is usually a period in which people make many minor tweaks to the leading entry, finding small improvements that further optimize the program, and leave them with the best score.
Figure 4.2. The progress of scores in the MathWorks programming competition. Lower scores are better. Credit: Copyright 2011 The MathWorks, Inc. Used by permission. Thanks to Ned Gulley for providing the figure.
The difference between the best early entries and the final winner is dramatic. In the CD packing contest, the best early submissions ran quickly, but left six minutes of space on the CD unused. The winning program ran approximately as fast, but left just 20 seconds unused, a nearly 20-fold improvement. It made use of contributions from at least nine people, over dozens of separate submissions. Although it’s a competition, the MathWorks contest thus functions in many ways as a large-scale collaboration. The organizer of the contest, Ned Gulley, said of the winning program: “no single person on the planet could have written such an oimize thalgorithm. Yet it appeared at the end of the contest, sculpted out of thin air by people from around the