Proving that answers don’t come out of the blue

The 1990s were an unhappy decade for the US education system. In 1996, around the time of fatal shootings in schools in Moses Lake, Washington, and Lynville, Tennessee, one in ten adolescents told a national poll that they were afraid of being hurt or killed by classmates carrying weapons to school.

The same year, four years after the opening of the Center for the Study and Prevention of Violence (CSPV), at the University of Colorado at Boulder, a number of influential partners began a national initiative to identify effective violence prevention programs.

Called Blueprints for Violence Prevention, the initiative was marked apart by a desire to apply independently agreed standards of evidence and a readiness to help States and communities implement and evaluate the programs they selected.

Other organizations have since followed suit, but Center Director Delbert Elliott's Blueprints model was in the vanguard.

Since then and with the help of an independent panel of seven experts his team has sifted through over 600 programs. Just 11 have so far achieved the highest designation they offer of “model program”. Another 18 are in a secondary category of 'promising'. The rest – all 575-plus of them – languish somewhere behind.

In the absence of any internationally-agreed set of criteria (no wonder policy makers have made such a hash of prevention!) how is the selection made?

First, Blueprints screens for programs that claim to be effective in preventing violent crime, aggression, delinquency and substance misuse.

Then they bring three criteria into play. First there must be evidence of impact discovered using a rigorous evaluation design. This generally means a randomized controlled trial or an experimental evaluation with a good sample size, low attrition and robust measures. Where RCTs are not possible, panel members are prepared to consider quasi-experimental designs and to rely on good use of the necessary control group.

Secondly, the panel looks for sustained effects. And thirdly, bearing in mind that many successful interventions depend all too heavily on proximity to those originating them, Blueprints asks if the program has been replicated across several sites, and whether the results have sustained.

To be a “model program”, all three criteria must be met. The “promising” 18 have only made it as far as first base.

Were it just a matter of separating the wheat from the chaff, the stringency of the Blueprints test could by itself claim to have made a lasting contribution to better outcomes for children. But Elliott and his team had the foresight to realize even a decade ago, in the midst of what was being called a ”violence epidemic” and when the political pressure for solutions was on, that identifying what works was only half the battle.

Policy makers are often reluctant to acknowledge the value of effective prevention programs, but as soon as they see the light, another problem threatens: in their impatience for action and keenness to implement, corners all too often get cut.

As Elliott and his panel knew only too well, putting in place one third of an effective program does not reap one third of the results. The return is more likely be nil.

So Blueprints offers US States, communities and other sites technical assistance to help ensure “model” and “promising” programs alike are implemented with a high degree of integrity.

The preferred arrangement is a partnership between the program designers, who provide training and consultation for each site, the CSPV, which monitors the quality of implementation by way of a comprehensive process evaluation, and the sites, which contract to guarantee standards of implementation.

This collaborative approach not only benefits the sites supported by Blueprints, but it also helps to build the knowledge base for what is becoming known as “Type 2 translational research”.

Model programs start in the laboratory. Type 1 translation takes those ideas into practice, but the implementation is often very close to the scientist’s own world – sometimes startling so: many a prevention program has been tested on the children of university staff. Type 2 translation goes the extra mile. It examines what is needed to implant knowledge into everyday life.

In the case of Blueprints, Type 2 work has included developing methods to screen sites for their readiness to implement prevention models properly. Do they have the organizational capacity? Is there sufficient funding stability? Is the enough commitment among managers, delivery staff and program recipients?

This year’s Blueprints conference has brought around a thousand delegates to Denver from nine nations outside the US: Norway, Canada, Ireland, New Zealand, Australia, Sweden, United Kingdom, Denmark, Iceland.

Explainers

Blueprints for Violence Prevention

Established in 1996 at the Center for the Study and Prevention of Violence (CSPV) at the University of Colorado at Boulder, the Blueprints for Violence Prevention program monitors the effectiveness of prevention, early intervention and treatment programs in reducing adolescent violent crime, aggression, delinquency, and substance abuse.

proven model

A combination of an experimental evaluation – or randomised controlled trial – replicated in several locations and with sustained effects beyond the period of the intervention is generally viewed as a necessary precondition for a program to be designated as ‘proven’.

randomized controlled trials

Sometimes referred to as experimental evaluations, randomized controlled trials or RCTs randomly allocate potential beneficiaries of an intervention to a program or treatment group (who receive the intervention) or a control group (who do not). Outcomes for the two groups are then compared.

quasi-experimental evaluations

An evaluation method in which children referred to a program or other intervention are compared with a group of matched children who do not receive the program.

type 1 translation research

Type 1 translation research is concerned with getting ideas and evidence from the laboratory into policy and practice.

type 2 translation research

Type 2 translation research examines what is needed to apply learning from experiments in real life settings to everyday life.

Delbert Elliott

Delbert S. Elliott is Director of the Center for the Study of Prevention and Violence (CSPV) and Professor of Sociology at the Univerisity of Colorado, Boulder.

Center for the Study and Prevention of Violence (CSPV)

In 1996, with a number of influential partners, it began a national initiative that sought out effective violence prevention programs.

Login or register to post comments