Checking Assumptions About School Readiness
This is the third in a seven-part series on the future of Head Start. Please join us for a web chat on this topic on Tuesday, Sept. 22 at 12:30 p.m. EDT here at EarlyEdWatch.org. We invite you to email us questions to get the chat rolling.
Head Start children have been the subject of hundreds of studies over the program's 44 years in existence. So you might expect policymakers to have a solid understanding of whether the program is good at preparing kids for school. Not so.
Lately, when one asks about school readiness, the answer depends on who is doing the answering. In general, most people assume that Head Start helps poor kids get ready for school. After all, the program has survived for decades, so they figure it must be doing something right. But conventional wisdom among conservatives and school reformers is altogether different. They question the program's effectiveness and wonder if money is being well spent.
Dig deeper with questions on how Head Start compares to state-funded pre-K programs, and the situation gets worse. Misinformation is repeated in opinion pieces and policy forums. In one recent forum in Washington, D.C., Head Start was disparaged for not being a "pre-K" program, when in fact many states (Ohio and Delaware, among others) consider it a model of what a pre-K program should look like. A February op-ed said that Head Start was "failing" children despite the author's acknowledgement that children's scores on several pre-literacy tests went up after enrolling.
Tuesday: Competing, Collaborating and Evolving
Wednesday: Seeking Signs of Change Since 2007
Today: Checking Assumptions on School Readiness
Sept. 15: A Tilt Toward Literacy
Sept. 17: The Case for Comprehensive Services
Sept. 18: The Benjamin Buttonization of Head Start
Sept. 21: Future Tracks
Sept. 22: Web chat (email us your questions)
It's time to check our assumptions, zoom in on exactly what the data says, and isolate what we still need to learn. In the process, we'll take a look at some new research hot off the presses that should help to untangle the confusion.
First, the Cliffs Notes version: Nationally, Head Start is making a positive, though modest, impact on children's ability to do well in school. Neither state-funded pre-K programs nor Head Start are doing enough to match the life-changing gains that come from the oft-cited programs like the Perry Preschool Project, which was smaller, more comprehensive, and of higher quality than most of today's preschool programs. But in some states and communities (such as Tulsa, Oklahoma, and New Jersey's Abbott districts), Head Start programs have conformed to such high standards that the gains made by their participants are more comparable to those of Perry Preschool and other high-quality model programs.
Let's wade into the national data for more context. There are two main sources of nationwide data on what, and how much, children learn from Head Start. One is the Family and Child Experiences Survey -- FACES for short -- that tracks a sample of 2,400 children through their first year of Head Start up to kindergarten. It provides a snapshot of the gains children have made in learning to identify letters, show awareness of their sounds, acquire new vocabulary and develop their social skills. Its most recent findings showed that kids are making progress each of these fronts.
The other is called the Impact Study, which tracks about 5,000 kids. It is a more valid source for evaluating Head Start because it compares the outcomes of Head Start children to non-Head Start children. In fact, to conduct a truly scientific and controlled experiment, the researchers randomly assigned these children to either a Head Start or a non-Head Start program. In either case, the children's parents had qualified for and expressed an interest in Head Start.
First-year results, released in June 2005, showed that Head Start children were making more progress than the other group. The study's authors described the size of Head Start's impact as "small to moderate" - a couple of words that, as we'll discuss below, have special meaning in the language of educational statistics.
Specifically, the Impact Study has shown so far that children who entered Head Start at age 4 made significant gains compared to the control group on 5 of 15 indicators of cognitive and social-emotional development after a year of Head Start. Their parents also reported reading to their children more often. Participating children were better than their non-Head Start counterparts at identifying letters and naming letters, and they performed better on one of two tests of pre-writing skills. No evidence emerged of the program causing any negative outcomes, such as behavior problems or worsening literacy skills.
But Head Start children didn't differ significantly from the others on one of their two pre-writing assessments, two tests of vocabulary, an assessment of oral comprehension and a test of early math skills. No differences emerged on measures of problem behavior or socialization either.
Three-year-olds felt the first-year effects of Head Start more broadly. According to the study, they made significant improvement on 11 of 15 tests, including a few of the social-emotional assessments. And their parents, too, reported more instances of reading to them.
In some cases, Head Start
propelled children to a
moderately higher level than
children who didn't enroll.
In other cases, children
took smaller steps forward.
Statistical significance isn't all we're looking for here, however. We need to consider how big the differences are between Head Start students and their peers. That's where the descriptors "small" and "moderate" come in. In some cases, Head Start propelled children to a moderately higher level than children who didn't enroll. In other cases, children took smaller steps forward. The magnitude of these differences - known as the "effect size" -- is important because it can help researchers determine whether the benefits of a program are worth its costs in the long run
Does Head Start pass such a cost-benefit analysis? Are dollars recouped because of these small to modest effects? In 2007, two researchers -- Jens Ludwig at the University of Chicago and Deborah Phillips at Georgetown -- studied that question. They analyzed the Impact Study data in light of Head Start's estimated cost of about $9,000 per child per year. (The official cost per child in 2006-07 was $7,087 per year but that includes only federal funding, not the 20-percent match of in-kind or cash donations that Head Start centers are required to pull in.)
They concluded that the benefits in the long run -- such as school districts expending less on special education -- could outweigh the costs. But the long-term benefits wouldn't tip the scales anywhere near as dramatically as has been shown with programs like Perry Preschool, Abecedarian or the Child-Parent Centers of Chicago, which have much larger effect sizes than those of Head Start. This is a disparity that we've highlighted before. There is a big difference between the $10-to-$1 cost-benefit ratios of the small and high-quality early childhood programs touted by the Obama administration and the much smaller ratios associated with Head Start. Ludwig and Philip didn't publish a dollar-to-dollar ratio, but their study implies that for every $1 invested just a little more than a $1 is recouped.
More analysis is surely to come. The next installment of the Impact Study -- one that examines what happens after Head Start children finish first grade -- is due out on September 30.
Now, what about the second part of the Head Start question -- is Head Start any better or worse at preparing children for school than state-funded pre-K? Until recently we didn't have much to go on, primarily due to the wide variation in state pre-K programs. It has been nearly impossible to truly compare apples to apples. One exception is a small study from Georgia in 2006 that showed that children in the state's pre-K program achieved similar, and in many cases greater, gains on academic tests than those who attended Head Start.
A different and more up-to-date way to look at Head Start comes in a soon-to-be-released study on the pre-K and Head Start programs in Tulsa, Okla. In Oklahoma, state-funded pre-K is available free to every 4-year-old, and ever since the program began, researchers at Georgetown University -- Deborah Phillips, William T. Gormley and Amy E. Lowenstein -- have been studying its impact. Tulsa is a particularly interesting city to study because its Head Start program decided to ramp up its standards, making them equal to the unusually high standards required by Oklahoma's state program. Its teachers, for example, must have bachelor's degrees and, just as in the state's pre-K program, must be paid at the same rate as public school teachers.
The latest study out of Georgetown, which appears in the Early Childhood Research Quarterly, shows that this equalization of standards made a difference. No matter which program the children attended -- whether it was the state's pre-K program or Head Start -- Tulsa's 4-year-olds experienced better instruction and more appropriate classroom activities than their national counterparts in either pre-K or Head Start.
Interestingly, some common indicators of quality -- staff ratios, length of program or whether the staff held bachelor's degrees -- didn't appear to matter in determining children's learning gains. What mattered was that teachers were adept at engaging children in academic concepts and had a close relationship with them.
What mattered was that teachers
were adept at engaging children
in academic concepts and had a
close relationship with them.
Were there any distinctions within Tulsa, between programs? Yes and no. The children in Tulsa Head Start spent less time on math and more time on social studies than the children in state-funded pre-K. But other than that, no significant differences emerged in evaluations of teaching quality.
The new research, mind you, can't provide a perfect comparison of Head Start to state-funded pre-K because, no matter how many characteristics the two Tulsa programs may share, it is still likely that Head Start children are coming with hard-to-measure traits that derive from their level of poverty and may require different teaching techniques. But the study is important because previous research by the Georgetown team showed that 4-year-olds in both the Tulsa pre-K program and the Head Start programs make very large academic gains by the end of the year - the kind of steep inclines that are associated with Perry Preschool.
It's not clear yet what has spurred such achievement. But Phillips, the lead author of the study, has some theories. Public school teachers in Tulsa, she said, make a very good salary -- on par with the average salary of city residents -- compared to teachers elsewhere. And as noted earlier, pre-K and Head Start teachers also receive that high level of pay.
"If you are in Tulsa, there is no reason not to be a teacher," Phillips said. "You will earn a very good wage. And if want to be a pre-K teacher, you don't have to cut your salary in half in order to teach 4 year olds and not 5 year olds."
The Tulsa findings suggest that it's possible for Head Start and state-funded pre-K programs around the country to pull themselves up to relatively high levels of quality. But they may need to raise teacher salaries to match what is offered by the public school system -- a tall order in the midst of recession and at a time when waiting lists can, understandably, lead policymakers to want to open more slots instead of bumping up paychecks. The predicament calls to mind an earlier study by economist Janet Currie and health policy expert Matthew Neidell. They scoured a larger national dataset to determine if children's school performance was significantly associated with attendance in Head Start. They found that the answer was yes -- if the child went to a Head Start that was funded at a higher level than average.
How to address this funding quandary is the puzzle of the decade. At the very least, smart policymakers will want to make sure that funds are used wisely and services are distributed without redundancy. Given the similarities between Head Start and pre-K systems in many places, it's worth taking a long look at where they come together. Two potential areas of overlap -- pre-literacy instruction and comprehensive services -- will be the subject of our next two posts.