Head Start must hold its feet to the quality fire

Lack of sustained impact where early intervention programs are concerned is usually taken to signal one of three things: wrong "dosage," ineffective design or shaky evaluation. There may be many shades of disappointment in between, but findings just published by the US Head Start impact study are likely to divide opinion broadly in those directions.

The Congressionally-mandated randomized controlled study of the 2002-3 Head Start cohort reported last week. It measured the cognitive and social/emotional development, health status and behavior of 5,000 three- and four-year-olds assigned to programs or a control group.

The goal of the study, originating – remarkably enough – long ago in 1998, has been to determine improvements to school readiness and parenting practices, and to identify where impact might be most telling and for which children.

The message that clear improvements measured at the end of the first program year largely dissipated by the end of kindergarten brought an immediate announcement from the US Department of Health and Human Services of plans to strengthen the program as part of the Obama administration's focus on early learning through the first eight years of life.

It brought a sharper-tongued response from the likes of the Kansas City Star: "Head Start is a touchstone … Democracts are expected to defend it against its dubious Republican opponents … never mind the fact that $100bn has been spent on it since 1965 to little effect".

Aspects of the surrounding argument will be familiar to followers of the debate about the progress of Head Start's UK nephew or niece, Sure Start.

Writing in Education Week [1], for example, Mary Ann Zehr registered concern about inconsistencies in implementation and evaluation. The study authors themselves acknowledged that programs varied from site to site in terms of instruction quality in the key areas being measured. "They also stressed that children in the control group participated in a mixture of alternative child-care settings, including care by their parents. So the study is reporting only how Head Start children benefited above and beyond children in other kinds of early-childhood settings," Zehr said.

She went on to quote Craig T. Ramey, professor of health studies and psychiatry at Georgetown University in Washington, who said in an interview that researchers in the field of early-childhood education saw Head Start as a "great idea," but added that it must “hold its feet to the quality fire".

Ramey questioned the quality not only of some Head Start programs, but of the impact study as well, Zehr wrote. "He characterized it as an example of ‘poor scholarship and reporting of data’ in only recording effect sizes and neglecting to provide information about the performance of children on average.

It was not possible to tell whether Head Start students were "humming along at the national average" in terms of their cognitive learning, or if they were "at the 10th percentile" on standardized measures of cognitive learning.

Commentary from the New America Foundation [2]'s Early Education Initiative was in similar vein. Lisa Guernsey blogged:

The study doesn't tell us anything about the quality of the kindergarten or 1st grade programs that Head Start children attended after their Head Start years. That's not because of an oversight on the part of researchers – it's because that kind of data just doesn't exist. But it should.

The study looked at children who entered Head Start in the 2002-2003 year. Variability is inevitable, but we need more information on how Head Start as a whole has improved since then.

The question of "fade out" has come up again and again in early education circles. What we don't ask enough, however, is whether kindergarten and early elementary programs are structured to enable teachers and schools to actually act on those gains. Perhaps the question should be reversed: do elementary schools help to maintain the momentum? And if not, what can we do to help them do so?

The official response from the US Department of Health and Human Services [3] came from Education Secretary Arne Duncan. “The results make it clear that we need to build a more coordinated system of early care and education, and to focus on key improvements to teaching and learning in the early grades,” he said.

The study showed that at the end of one program year, access to Head Start positively influenced children’s school readiness, but when measured again at the end of kindergarten and first grade, Head Start children and the control group children were at the same level on many of the measures studied.

It was all a matter of increasing effectiveness through providing continuous support for vulnerable children from birth to eight.

• For more about the performance of early years programs, see The pros and cons of early years programs – where to start! [4]; for more about Sure Start in the UK, see Sure Start evaluator says “job done” – almost [5] and follow the links.