• Choose a category
• All
• Classic-Fiction

By Root 357 0

how well it fills up the entire CD on each of the three test inputs. Of course, there’s no need for this to be done manually by an organizer. It can all be done automatically as soon as entries are submitted, so the score can be computed immediately. The only caveat is that for this to work the organizers need to keep the test inputs secret—if competitors knew, for example, that their program would be used on the Beatles catalog, they could tailor it specifically to the Beatles catalog, defeating the point of the competition. But provided the organizers are careful to keep the test inputs secret, they can automatically score entries as soon as they’re submitted.

Automated scoring is important because the scores help participants focus their attention where it will do the most good. If someone changes a program and causes a big jump (or even a small improvement) in the score, other people notice and check to find out what’s been changed: maybe that person has a great new idea. The automated scoring thus makes it easy for programmers to keep tabs on each others’ best ideas—even if the number of participants is very large—and to spot opportunities to use their own expertise to make further improvements, and so leapfrog over one another. Some of the programmers, for example, are experts on the detailed ins-and-outs of the programming language (called MATLAB) used in the competition. They watch other people’s programs carefully, and use their knowledge of MATLAB to make tiny optimizations, often changing just one or two lines of MATLAB code to be more efficient, and so shaving a fraction of a millisecond off the running time. Other competitors specialize in other ways. Some scour the scientific literature looking for inspiration. Others brainstorm completely new approaches. And some work on hybridizing existing approaches. Amidst all these differing approaches, the automated scoring plays a role similar to prices in a market, providing information that can be used to inform decision making by contest participants. While it’s impractical to conduct a conversation involving the more than 100 people who entered in the MathWorks competition—no one has time to pay attention to more than 100 separate voices—the score helps people make good decisions about where to focus their attention, and so fuels rapid improvement.

The MathWorks score is not perfect as a way of coordinating attention. Because the same scoring information is provided to everyone, it leads competitors to concentrate their attention in similar ways. For example, if someone jumps to the top of the leaderboard, then many participants will immediately shift their attention to that entry. Of course, some concentration of attention is good, but if everyone follows the same lead, then the group as a whole may neglect promising directions for exploration. You could imagine more complex signaling mechanisms that would spread attention more widely, and lead to a better allocation of expertise. For instance, people wh expertise in optimizing MATLAB code might be directed to programs whose gross structure was changing rapidly, but whose fine detail had not yet been optimized. Or perhaps there could be some way of detecting clusters of programs that make use of similar ideas. Contestants who enjoyed hybridizing different approaches could use this information to help them pick out the best programs in each cluster, and attempt to hybridize those.

These limitations aside, the MathWorks score does a great job of helping coordinate attention, and thus of helping the MathWorks collaboration scale. As a way of directing attention it works much more effectively than, for example, any mechanism available in the Polymath Project, which relied on the acumen of individuals to assess which contributions were worth following up on. It could take hours or days for the polymaths to identify the best new ideas. That’s fast, especially when compared to the usual pace of scientific research, but slow compared to the immediacy of the MathWorks score. The situation in Kasparov versus the World