Early Ed Watch - logo

Debunking Bad Analysis on Oklahoma Pre-K

The Cato Institute's Adam Schaffer is much too smart to believe that his latest argument that pre-k is ineffective holds water. Schaeffer argues that Oklahoma's pre-k program isn't improving student achievement, because Oklahoma's performance on the National Assessment of Educational Progress (NAEP) has declined, relative to the national average, since the early 1990s.

This argument is seriously flawed. For starters, the comparisons Shaeffer is making are invalid because he's comparing NAEP results for different cohorts of students--and there are real differences between those cohorts. Since the early 1990s, the proportion of Oklahoma children who are Hispanic and who are English language learners has risen dramatically, a difference that needs to be taken account when comparing performance of current and past cohorts of Oklahoma students. Similarly, Schaeffer's analysis doesn't consider other factors besides pre-k--such as changes in standards, curriculum, or funding--that may or may not have occurred in Oklahoma's public schools during this time period, potentially affecting student performance. Schaeffer has training in research methods, so he has to know that using NAEP score changes to draw conclusions about the effects of Oklahoma's pre-k program, without controlling for other factors in play, is totally meaningless.

Moreover, the data Schaeffer is using don't even support his own argument. Oklahoma's performance on the 4th grade reading NAEP--the most relevant data point here--declined substantially between 1998 and 2002, but has been improving since 2002. Oklahoma didn't begin offering free voluntary pre-k to all four-year-olds until 1998--and any impacts from that program wouldn't have shown up in NAEP data until 5 years later, when those children were 9. While it's true that Oklahoma funded a targeted pre-k program for low-income children starting in 1990, the greatest expansion in pre-k occurred after 1998, so any results wouldn't be reflected in NAEP results until after 2003.

Schaeffer is correct that the overall evidence of long-term academic benefits from pre-k is much weaker than the evidence for short-term educational effects, which is strong. But that doesn't mean there's no evidence of long-term effects from high-quality early education programs. The now famous Perry Preschool found study, a randomized controlled trial of pre-k programs, found significant positive impacts on pre-k participants' educational attainment, earnings, and other indicators into middle age. Similarly, longitudinal research on participants in Chicago's Child Parent Centers program, a large scale, federally funded program that provided low-income children with high-quality pre-k and additional services through grade 3, also found similar long-term benefits. Both of these programs focused on poor children, however, not middle income students. Recent studies by Georgetown University researchers indicate that Oklahoma's quality pre-k program does improve the skills and knowledge of both low-income and middle-class children entering kindergarten (and does so more than Head Start centers do). But these studies don't include long-term follow-up. Previously released studies by the same researchers looked at children who attended Tulsa pre-k programs for the 2003-04 school year. Those children should be entering 4th grade this fall, so a follow-up study on how they're doing now would be quite informative.

More broadly, no one should look at pre-k as an "innoculation" that, administered once at age four, delivers improved academic performance without further follow-up through children's schooling. That's not how learning works. High-quality elementary schools must build on the base of improved skills and knowledge children bring with them from pre-k. If elementary schools are poor quality or otherwise unable to provide supports and curricula that build on pre-k learning gains, those gains will be squandered. That's why efforts to improve early education can't stop at pre-k, but need to continue through the early elementary years and indeed throughout a student's K-12 education. But that doesn't mean quality pre-k doesn't make an important contribution in getting that process off on the right foot. Oklahoma has done a good job in putting a high-quality pre-k program in place for the vast majority of its students. Now it needs to work to improve its kindergarten and elementary programs in order to sustain the documented gains children are making in pre-k.