Trials, (myths) and tribulations

Randomized controlled trials in schools are very difficult to get off the ground. Teachers are reluctant to give up stretched teaching time; trials are costly and difficult to implement. At least, that’s what most people think, says Professor Peter Tymms. Thirty years ago it might have been true, but not any more.

Once a schoolteacher himself, now Director of the largest educational research group in the UK, the Centre for Evaluation and Monitoring (CEM), he told York conference delegates that a wealth of useful data was routinely being collected longitudinally on all students in UK schools.

If children were randomly allocated to a trial or control condition as part of an experiment - researchers did not need not collect much, if any, extra. It was often possible to test the effect of any intervention simply by examining changes in the monitoring data collected before and after the trial.

Projects such as Performance Indicators in Primary Schools (PIPS) and Assessment On-Entry for Children and Toddlers (ASPECTS) readily provided information about academic attainment, student relationships, attitudes, safety or learning.

The example he gave was the “naming and informing” evaluation. Roughly one child in every class of 30 had ADHD symptoms that caused him or her to fall back significantly on academic attainment, lose popularity and display behavioral difficulties.

With that average in mind, he had decided to test the effects of identifying children likely to receive a diagnosis of ADHD to their teachers and of circulating an information booklet about the disorder.

The measures used to assess the impact of the program related to scores already being collected. It emerged that identifying the children to their teachers and distributing information was associated with a negative impact on maths and reading. If the ADHD children were not identified but the booklet was still distributed, there was a beneficial impact on attitudes – but not on reading or maths scores.

The results of that study were interesting enough, but the fact that so little effort was required had wider implications. All the researchers really had to do was organize the random allocation and analyze the results.

As for signs of resistance at the front line, Tymms said that in his experience it was more often not teachers but education experts who resisted RCTs and evidence-based practice. He used the national literacy strategy as an example: the British government had spent £500 million developing it as a compulsory element in all schools, but to date no impact on reading levels had been detected. There was nothing to say whether it was effective.

“If someone tells you what to do, ask them, Where is the evidence? If they don’t have any, your judgment is as good as anybody’s!” he said, urging teachers to try out new strategies to improve outcomes for children and to evaluate them themselves.

“It is simply not enough to know something about the causes of problems; we need to try to change the world in some way to make any progress.”

Explainers

Peter Tymms

Peter Tymms is Director of the Centre for Evaluation and Monitoring (CEM) and Professor of Education at Durham University

Centre for Evaluation and Monitoring (CEM)

The Centre for Evaluation and Monitoring provides indicator systems to schools and colleges in the UK and elsewhere.

Login or register to post comments