Thursday, December 13, 2007

Is a Carl Doomed To Be C Student?

A study reported widely last month found that "students whose names start with the letters C or D, which denote mediocre marks in some grading systems, did not perform as well as other pupils with different initials."

As Carl Bialik points out in the Wall Street Journal, while the relationship that the researchers found was statistically significant, it doesn't mean it is important:

"University of California, Irvine, statistician Hal Stern points out something most media missed. The effect is tiny: 0.02 of a grade-point average point lower for the initials C and D (and this columnist isn't including that because of his first initial). Therein lies a lesson in the difference between statistical significance -- the confidence that there is some association between two factors -- and the strength of that association.

"In very large samples like the ones here, even small differences will be judged statistically significant," Prof. Stern says. "This means that we're confident the difference is not zero. It does not mean the difference we see is important." Prof. Nelson agrees that this effect is "so small that you shouldn't worry about it" when naming a child, though he does say the study exposes an example of how the unconscious mind can undermine conscious motivation.

But Bowling Green statistician Jim Albert warns: "You can prove any silly hypothesis ... by running a statistical test on tons of data.""