I've never liked 'speed' as a programming metric because it's very deceptive. In this case we're talking about students going through some very basic training, and who will do things like blow 3 hours on a trivial-to-me syntax error... but it's "trivial-to-me" only because I, too, spent three hours on trivial syntax errors. I would expect these essentially "constant factors" to be dominating here.
I really don't like speed because there's a quality difference, too. I may not necessarily beat a fresh-out-of-college kid on some task on straight clock-time, but where theirs is bodged together, barely functional, and already extended to the maximum limit, mine will have a strong unit testing suite, be efficient, extensible, scalable, fewer bugs, probably even fewer features in places I know they shouldn't be, less code overall (despite the unit test suite), etc. And there will be tasks I can complete that they can't even get close to; what's the multiplicative difference between "I built a scalable system that serves millions" vs "I couldn't get the system to do anything useful" in the same time period? The productivity differences are really more about quality than raw speed. There are speed differences, certainly, but I think it's dominated by the quality difference.
Isn't that the whole point? If we can find a way to get rid of the "constant factor"-type problems, the students will have more time to concentrate on the overall quality rather than just handing in whatever they've been able to bodge into a working shape. Not only that, but it would get rid of the single most frustrating part of learning at a stroke.
I really don't like speed because there's a quality difference, too. I may not necessarily beat a fresh-out-of-college kid on some task on straight clock-time, but where theirs is bodged together, barely functional, and already extended to the maximum limit, mine will have a strong unit testing suite, be efficient, extensible, scalable, fewer bugs, probably even fewer features in places I know they shouldn't be, less code overall (despite the unit test suite), etc. And there will be tasks I can complete that they can't even get close to; what's the multiplicative difference between "I built a scalable system that serves millions" vs "I couldn't get the system to do anything useful" in the same time period? The productivity differences are really more about quality than raw speed. There are speed differences, certainly, but I think it's dominated by the quality difference.