Via Kyle Deas' Fitful Murmurs, a link to an article in the Financial Times reporting on a study of celebrity "naked chef" Jamie Oliver's "Feed Me Better" pilot program of serving fresh-cooked, nutritious lunches in British schools. (The FT article reports that it involved a special "boot camp" training session for the dinner ladies.) I think I first heard about this program, and perhaps the research study, from my wife. There's a strong suggestion that it yields a few percentage points improvement in the number of students reaching particular levels in Science and English testing, and a more detailed examination of the regressions in the original article looks broadly consistent with that (even the "non-statistically significant" coefficients, including ones in Maths, were almost all in the right direction and mostly of about the same magnitude). The study compared the two-academic-year periods of 2002-2004, to 2005-2007, in five school districts ("local educational areas", LEAs) in the southeast of London---Greenwich, in which Oliver's fresh-cooked meal program was implemented beginning in the 2004-2005 school year, with four others in which it wasn't. The change in absenteeism in Greenwich, relative to the change in other districts, was estimated to be -1.2 percentage points, with a standard deviation of estimate of 0.365%, significant at p<0.01. This is a pretty significant drop--not a 1.2 percent drop in amount of absenteeism, but a change of 1.2 percentage points in the absenteeism rate---translating to about a 22.5% drop in the number of absentee students. (In the text, the authors cite only the 0.8 percent drop in authorized absences, since the change in unauthorized ones wasn't significant on its own; but their data shows the total change as significant too, which is what I described.) Even more interestingly, the estimated change, relative to non-treated schools, in the fractions of students reaching Levels 4 and 5 on the "Key Stage 2" standardized testing, broken down into English, Maths, and Science (for four coefficients) ranged from 2-6%, with standard deviations of these estimates around 2.5-3%, in a "school level data" regression analysis. (This means other factors that might have affected results were included in the analysis by including variables for school characteristics in the regression.) Only a couple of these six coefficients were significant (at the 0.10 level), but I think the overall pattern for the fractions of students attaining Level 4 and Level 5, with those coefficients that were not significant at 0.10 still large and in the right direction and roughly comparable to the standard deviation of estimate, adds persuasiveness to the results. Level 3 fraction was also included, but the effects were negligible compared to the standard deviation of estimates. (I haven't looked at the methodology carefully enough to determine whether Level 3 meant "exactly level 3" or "at least level 3... in the former case, the rough constancy at level 3 is perhaps good news as it would suggest students who scored at level 4 or higher rather than level 3, were at least roughly counterbalanced by ones who reached 3 rather than a lower level.) Pupil-level regressions were also done, with the pupil's percentile score as dependent variable, and pupil-level data to help control for socioeconomic status and other confounding variables; it gave broadly similar results: the relative effect on English results was large at 4.7 percent, and highly significant at p<0.01, while the Science doesn't look too bad either at 3.6 percent (apparently not significant even at 0.1 though) and even the maths is in the right direction.
I would have liked to see more explicit statements on the school and pupil-level characteristics included in the regressions, but I take the school-level characteristics to have been those summarized in Table 2. Actually, if we interpret "controls include" to mean that this is the full list of controls, then the information is in the note at the bottom of Table 3.
I'm not a practicing statistician or econometrician, so I won't pretend to give a definitive assessment of the methodology here (I also haven't spent that long thinking about it), but it all looks pretty decent. I'd love to hear comments here on the methodology from competent working statisticians or econometricians.
They did a control for the idea that the change in educational outcomes was due to the change in absenteeism, and concluded it wasn't, and a similar, but I think much more tenuous, attempt to somehow proxy for a "placebo effect", which they interpret as suggesting the placebo effect wasn't the thing. I'm not so sure it matters, if it continues to work.
Nice stuff; let's hope for expanded programs and more studies of them. Meanwhile I think it'll influence me to eat a little healthier myself.
Whilst it is great to show that healthy eating improves school grades, it is absolutely not required in order to be a strong supporter of this type of scheme. The main point is that healthy eating improves *health*, regardless or grades, and that the type of foods we are introduced to when we are young are habit forming. Thus, providing healthy food in school ought to eventually lead to a healthier adult population and reduced medical costs. It is nice if people are smart, but the health benefits are likely to vastly outweigh any improvement in grades you might find.
Personally, I don't think we need any scientific studies to deduce that a canteen where the only edible food available is chips (unless you are prepared to risk dayglow green "chicken" curry), as was the case in my secondary school, is not a good idea. I still have nightmares about the "home baked" bread rolls we had to eat, which were extremely undercooked apart from a few black burn marks.
Matt, good point. I realized after this post I hadn't made it clear that a likely interpretation of the decrease in absenteeism, which includes authorized absences for health reasons, is precisely improved health. The improved test performance might well also be an indirect sign of better health. The authors do point out that "one drawback of our analysis is that we have little information on the health outcomes of children, as well as whether children actually ate the meals or not." And they point out at the beginning of their "costs and benefits" section, that the health benefits they haven't estimated are likely significant. I think it was a main motivation of the project to begin with, and as you also point out, if patterns of healthier eating are set early there are likely to be lifelong benefits that aren't captured in a study like this.
I do suspect studies like this (or more importantly ones dealing directly with health effecs) might be definitely helpful in getting such programs, which do have significant cost, into effect in the face of people who say "well of course it's desirable, but can we really afford it?"