So what’s a policy maker and eminence grise like Andrew Adonis (UK Parliamentary Under-Secretary of State for Schools) doing at a conference for researchers? Is it a case of a politician really engaging with the evidence?
The good news is that he knows what a longitudinal study is. He cites nine government funded studies of that ilk in his speech and tells us that one of them, Early Provision of Pre-school Education (EPPE), is to be extended to follow children up to 14 years.
The bad news he brings – well, old news really – is that while research is one factor that influences policy decisions, it isn’t always the most significant. A wide range of others come into play: financial considerations, opinion polls – even what a minister’s child talks about back home in the kitchen after school. (and don’t we know it!)
The sensible consensus is that for research to play more of a role it needs to be more rigorous. Longitudinal studies go a long way, but they can’t tease out issues of causality. Alas there’s not been any mention of the magic words ‘randomised controlled trial’ (RCT) in any of the opening addresses. Later maybe…
Prof. Pring offers Popper
The special interest group (SIG) ‘Educational Research and Policy: Epistemological considerations turns out to be an interesting one. ‘Epistemology’ refers to the methods, validity and sources of knowledge, in this case to how educational research may influence policy.
Richard Pring from the University of Oxford goes for the philosophical jugular. He says he’s sceptical about the US ‘What Works clearing house’, and he cautions us against the McKenzian view that “what is real can be measured, and what is measured can be controlled”. He also quotes Karl Popper: evidence doesn’t equal proof, only support, and that this support may be weak or strong.
David Bridges from the Von Hügel Institute at Cambridge echoes Andrew Adonis’s opening address. The sources of evidence are many and varied: what we really need to decide is what can and what should be considered sources of evidence.
The problem with these philosophical musings is that they become so complex. The arguments are tightly reasoned, but the papers they come in are very long – and so very turgid! If the purpose of educational research is to influence policy, is this the way to go?
Too much data. So little time?
The sessions about local authority research to improve school improvement include some success stories (Lambeth bucking the trend in ethnic minority academic attainment), also worrying trends (Greenwich working class white boys showing poorer attainment) and old and new methods to measure attainment (the Pupil Achievement Tracker and RAISEonline software respectively).
They all focus purely on academic achievement in the form of key stage examination grades and are geared up to give very detailed analysis of these outputs. Nothing on anything other academic outcomes, though. No mention of the fact that schools are meant to be focusing on the development of the whole child or that OfSTed will soon be asking schools to collect data on behavior. I begin to think schools and local authorities are being required to collect too much data and not given enough time to do anything with it.
On a more positive, long-term note. The stellar cast of a startlingly good session on the Friday afternoon called ‘The impact of research on policy’ includes Ted Melhuish from Birkbeck College on the political and social conditions surrounding the development and evaluation of Sure Start, and the EPPE team on how this particular piece of longitudinal research has been particularly influential (and why).
A key question is not whether experiencing an intervention such as Sure Start or some form of pre-school education results in better outcomes, but why some variations are so much more effective than others. In the current culture of resistance to RCTs, particularly within the education field, longitudinal studies have the potential to offer such insights.
So while Lord Adonis’s opening address made no mention of RCTs in relation to educational research, and no-one else mentioned them either, contributions from longitudinal research appear to be making a real impact on educational policy.