“I have not failed 5000 times. I have successfully discovered 5,000 ways that do not work and I do not need to try them again”, Thomas Edison famously remarked. Like many in the field of science and engineering, Edison did not see failure as a bad thing per se, rather as a necessary consequence of inventing and testing.

But this belief is not widely shared by many in the social sciences, and as Berman and Fox, authors of a new book, Trial and Error in Criminal Justice Reform, argue, it is certainly not the commonly held view in their field. Indeed, so taken are they with the concept of failure that they have devoted their entire book to the subject. In it, they reflect on their own experiences at the Center for Court Innovation, as well as detailing five examples of US criminal justice innovations, each well intentioned and, in some cases, involving millions of dollars, that have not delivered the anticipated results.

While all the examples are interesting, it is the story of the DARE (Drug Abuse Resistance Education) program that is particularly compelling.

If success is judged by awareness and take-up, then DARE has been very successful. Within 10 years of its launch in 1983, it had reached five million students and was receiving $10 million of government money annually. The program uses police officers - 15,000 at the last count - to provide educational sessions about drugs and drug abuse to elementary school students. It is currently used in 75 per cent of US school districts and remains the nation's best-known drug education program.

But if success is judged by its impact on reducing drug use by young people, then DARE has failed. In fact, while early research suggested negligible long-term impact on drug use, a 1998 report by Rosenbaum and Hanson not only confirmed the findings of earlier studies, but suggested that the program was associated with increased drug use. This finding raised the possibility that the program was harmful.

This left DARE America, the program developers, in a difficult position. Daryl Gates DARE America's founder and leader, mounted an aggressive defense involving lawyers and adopted a hostile and critical stance towards the research community. The negative findings received considerable media attention, which was somewhat exacerbated by DARE's belligerent position.

DARE was saved, however, by two unexpected developments. First, a large foundation, the Robert Wood Johnson Foundation, stepped in with the offer of funds to revise and rigorously evaluate the curriculum. In DARE it saw an impressive infrastructure, excellent training and a well-established network of supporters. Instead of dismantling it, it wanted to help the organisation exploit the resources it had amassed and relaunch itself, which occurred in 2000.

Second, although political and financial support for DARE was weakened by the negative findings, many school districts continued to use the program. Carol Weiss, a researcher who studies the relationship between research, policy and practice, was curious to find out why.

She found that schools had relatively sophisticated reasons for keeping the program. On the whole, they were not ignorant of the negative findings, nor did they dismiss them. In most cases, those places that kept the program valued it for other reasons: the positive relationship between the students and police officers, as well as good links between the school and police department. Many, in fact, had never believed that the program could reduce drug misuse in the first place.

The jury is, though, still out on DARE. The ambitious evaluation has been undermined by circumstances beyond the research team's control and, after seven years and considerable investment, the research may well not determine whether or not DARE actually works.

Berman and Fox draw four themes from the case studies in the book, including the example of DARE.

First, they suggest that there are two main explanations for failure: a bad idea or poor implementation. They also argue that well-implemented good ideas can still fail if political support and resources are not secured. Similarly, a weak idea can be widely implemented with considerable political backing and financial support, but an undue focus on marketing and promotion can, as in the case of DARE, distract attention from whether or not the program is having the desired effect.

Second, failure is rarely black and white. Although we like to ask the question “does it work?”, with its implication that a clear-cut verdict can be reached, the reality is that the answer is that some programs work for some people for some of the time. It also depends where you stand and what you deem a success. There is no evidence that DARE reduces drug misuse but it was valued and deemed a success for other reasons by many of those who continued to support it even after Rosenbaum and Hanson’s negative findings.

Third, politics (with a big and small “p”) cannot be ignored or underestimated. It is common for a new leader to jettison the programs and ideas of their predecessor regardless of the evidence. A program may be successful in the eyes of the research community, but if it has been oversold - portrayed as a silver bullet or panacea - then sooner or later funders or lead officials will be disappointed and withdraw their support.

Finally, implementation is as important as the idea. Not only should a program be delivered as designed, but the ability of a program to be implemented in different places will be affected by context. Leaders, politicians and staff on the frontline are all part of the context that might undermine an idea that has worked well in another setting.

There is a danger that a book focused on failure could leave the impression that nothing works. Instead, though, this book serves to remind us that failure is normal - far more businesses fail than succeed but that does not inhibit entrepreneurship. Failure is a necessary part of the process of trying and testing. Instead of hiding them, failures should be scrutinized so that lessons can be learned and future success attained.

References:
Berman, G and Fox, A. (2010) Trial and Error in Criminal Justice Reform, Washington: Urban Institute Press

Weiss, C., Murphy-Graham, E. & Birkeland, S. (2005) 'An Alternative Route to Policy Influence: How Evaluations Affect D.A.R.E.' American Journal of Evaluation 26(1): 12-30

Rosenbaum, D & Hanson, G (1998) 'Assessing the Effects of School-based Drug Education: A Six Year Multi-level Analysis of Project D.A.R.E.' Journal of Research in Crime and Delinquency, 35 (4): 381-412.