What is it? Does it work? Is it worth the money? These are the first three questions a funder or policy maker is likely to ask someone claiming that they have something that will improve children’s lives.

On the first two there has been enormous progress. Developers have become quite sophisticated at explaining what they have to offer and robust evaluation, using experimental methods, is becoming more common and acceptable. But on the third question – is it worth the money? - progress has been slower. Basic economic analysis of interventions and programs is often absent from even the most sophisticated evaluations.

This dearth of this kind of information makes it difficult for those who are charged with purchasing services for children. For many programs it is hard to find out the basic unit cost, let alone know how the value of one program compares with another, or whether spending money on a particular program now does, in fact, save money in the long run.

In a recently published book, Michael Foster considers this matter by looking at interventions designed to reduce conduct problems. Of the many difficulties that young people encounter, the case for prevention and early treatment of conduct disorder is strong. Children with severe conduct difficulties evident at a young age have a greater risk of criminal activity, drug misuse and the like when they get to adolescence and adulthood. The costs of these problems to the tax payer and the individual can be very high. Common sense suggests that reducing conduct problems would lead to great financial savings.

Unfortunately, as Foster notes, it is not quite that simple. When we spend money on prevention, there is often a long gap between making the investment and seeing the benefit. In the case of conduct disorder, it is good to intervene early, perhaps when children are five or six years old. However, the serious problems will not be apparent until adolescence. Also, our targeting is imperfect. Since we don’t know which children with conduct problems will go on to have difficulties later (more will not than will) we have to offer interventions to some who would have been fine without help.

Foster illustrates some of the practical and technical challenges of economic evaluation by running a cost effectiveness analysis on two well-known programs: Incredible Years and Fast Track.

Incredible Years seeks to reduce conduct disorder in high-risk children, is targeted and has components for teachers, children and parents that can be offered on their own or combined. The program focuses on parental competence, positive discipline and classroom management.

Foster combined data from a number of studies and was able to determine that the parent program combined with the teacher program was the most cost effective application.

Fast Track [see: Long march shows Fast Track's the way] was an experiment to see if it was possible to reduce conduct disorder by offering a range of interventions (targeted and universal) over a long period. It was established in 1991 at a time when nothing appeared to work to prevent these types of problems.

Foster ran a cost-effectiveness analysis and found the program to be a good use of money in terms of potential savings to mental health and youth justice services. But despite the significant investment, it did not prove to be beneficial in terms of costs for substance use, special education services, school dropout and grade retention.

For an economist there are several ways of thinking about cost. First, as would expected, there is a distinction between fixed and variable costs. Variable costs for Incredible Years include staff pay, materials and meals for participants. A license for program is an example of a fixed cost, since the cost does not vary with the number of participants. Second, is the distinction between direct costs (those listed above) and indirect costs (those that are harder to link to the intervention), for example, the office costs for the staff when they are preparing for the program. The third type of costs are morbidity related and these are the values placed on the services that will not be used by children and families because their need has been reduced by the intervention.

In the case of Incredible Years, the comparison was between different versions of the same program. The analysis answered the question ‘Which combination of Incredible Years programs gets the most improvement for the least money?’ In the case of Fast Track, the analysis considered the costs to provide the program and the potential savings from a reduced use of services.

It is this issue of comparability that makes economic analysis so problematic.

Foster compares two analyses of the multi-systemic therapy program. In one economic analysis the authors considered the incremental costs of the program relative to the observed reduction in days in incarceration, hospitalization and residential treatment. After one year the costs of the program had nearly been offset by the reduced use of out-of-home placements.

The same program has subjected to a cost benefit analysis that indicates a cost of $4,264 generates net benefits of $18,213 for each young person who was treated.

On first glance the figures seem confusing; one appears to suggest a breakeven point after one year and the other showing a considerable return on investment. The difference between the two reflects technical differences in the way outcomes are valued and the time frame considered.

Foster argues that the potential of economic analysis is not being fully realized. Policy makers find arguments backed by this type of data to be very persuasive, yet there are only a handful of interventions for young people with conduct problems for which these types of calculations are made. This problem will only be overcome by economic analysis being incorporated within evaluations from the outset with some consensus about how to do these analyses to make across study comparison possible.

As well as practical and technical issues, these types of analyses expose a deeper concern about the interventions themselves. If a program is crafted solely with efficacy in mind, it is possible to implement something that would have no chance of being widely used in the real world, even if it was found to work. If the costs and potential benefits of a program were an integral part of the evaluation many more programs might not make that first step off the drawing board.

References:
Foster, M.E. (2010) Evidence-based treatment programs for conduct problems: are they cost effective? In Murrihy, R.C. (eds.) Clinical Handbook of Assessing and Treating Conduct Problems in Youth.