It’s recognized that randomized controlled trials tend to find smaller effects than other evaluation methods. Now there’s the intriguing suggestion that the reporting of methodological quality in RCTs has an impact on trial outcomes.

This has been established in health but lately it’s been studied in criminal justice by Glyn Hallam and Amanda Perry from the University of York's Centre for Criminal Justice, Economics and Psychology. They assessed a series of trials against the CONSORT guidelines - an internationally recognized checklist of 22 items to be covered when reporting RCTs.

The York team analyzed 48 studies, coding them for how clearly they reported (1) the random allocation sequence, (2) whether the allocation sequence was concealed from those enrolling participants and (3) whether participants, researchers and providers were 'blind' as to which condition (treatment or control) participants were assigned to.

In contrast to findings in relation to clinical (health) research, they did not find significantly higher effect sizes in the less-clearly reported criminal justice studies. Tantalizingly, the presentation did not include descriptive results of how these studies measured up against the CONSORT guidelines, although in response to a question they said that it was 'a mixed bag'.

Nor was it explained why the quality of reporting might be associated with the size of effect detected. In children's services, where we are only just starting out as far as RCTs are concerned, these considerations might seem a stretch too far.

On the other hand, as an audience member commented, the people in the room represented a good proportion of researchers undertaking experimental studies in the UK, so it could have a significant impact if we made sure our studies were written-up to the highest standards. For further information see The Consort Statement