Randomized controlled trials – time for more system?

With those responsible for public policy warming to the value of experimental methods in the social sciences after decades of scepticism, this week’s York University conference on randomized controlled trials will turn down the heat a little by showing that you need more than one RCT to make solid evidence.

The focus if this year’s meeting is methods and synthesis and the virtues of meta-analysis.

Among the week’s speakers are Cambridge Professor of Criminology Lawrence Sherman who is expected to argue that placing criminals in custody “may deprive too many people of their liberty and many taxpayers of their money – simply for lack of evidence”.

His view that one-at-a-time trials are not enough, and that basing an evaluation strategy on a pre-planned synthesis of evidence is the surest path to more effective policy is shared by Alex Sutton a Reader in Medical Statistics from the University of Leicester.

In his keynote address, Sutton will explain that more must be done to ensure that future trials are designed with meta-analysis in mind. He will offer practical advice on how to put this into practice.

Another academic offering practical advice is Peter Tymms, Professor of Education from the University of Durham, who will reflect on the differential benefits of small and large scale trials. He will also point to the value of drawing on data being generated by other research or monitoring projects.

Michael Little, Director of the Dartington Social Research Unit, will describe attempts to take experimentally evaluated programs to the general population in Ireland and in Birmingham UK. Little will also outline some of the obstacles to the use of experiments in children’s services and propose methods for overcoming them.

The York conference is organized jointly by Dr Carole Torgerson (Institute for Effective Education), Professor David Torgerson (York Trials Unit) and Professor Stephen Gorard (University of Birmingham).

It was first convened in 2006 to highlight the resistance to experimental methods among social scientists and policy makers in a period when RCTs had revolutionized standards of evidence in the medical sciences.

The widespread suspicion was always ironic given that the method was first tested in the 1930s by a social researcher interested in educational outcomes.

Explainers

randomized controlled trials

Sometimes referred to as experimental evaluations, randomized controlled trials or RCTs randomly allocate potential beneficiaries of an intervention to a program or treatment group (who receive the intervention) or a control group (who do not). Outcomes for the two groups are then compared.

meta-analysis

Meta-analysis combines the results of several studies that use similar methods to explore similar research questions.