Would any research scientist be so wantonly unscientific as to make a dodgy claim about the effectiveness of a program – considering the damage likely to be done to the cause of knowledge-based practice, worse the potential damage to children and families?
The truth is that shortcomings and disparities in the methods used to gauge effectiveness mean that unfounded claims of service effectiveness abound.
They aren’t quite of the same blatant order as Nestlé, for example, making the vacuous claim that its Ski Activ8 yogurt contains vitamins proven to optimize the release of energy from your diet (combined with healthy diet and lifestyle) or the Champneys (of health spa fame) claim that their (vinegar) foot patches extracts toxins from the body while you sleep.
But a press release I picked up at a London conference recently proudly proclaimed that a certain initiative had “succeeded in reducing truancy, exclusions and offending” when it was plain from the evaluation reports on the display stand that despite much good work, nothing on offer had the rigor needed to support such a statement with confidence.
So the audacity and intent to deceive are perhaps not replicated in children’s services, but the effect is the same: to mislead through the use of pseudo-science.
Notwithstanding recent interest in this area and with the exception of several popular “proven models”, it’s still the case that services are not really designed; they emerge. They are rarely the product of systematic development and testing underpinned by scientific research. Unless they are clinicians as well, researchers have tended not to take much interest in how services are formulated.
As such, services can be as much subject to fads as the lifestyle industries. In my slot at the London conference I used a painting of two canoeists to remind the audience of the fashion a few years ago for viewing outward-bound courses as the solution for youth disaffection. I asked how such an intervention could conceivably interrupt the chain of risk that leads from overcrowded housing to anti-social behavior via inconsistent parenting from a depressed mother. The answer, of course, is that it cannot; the suggestion is plainly ludicrous.
Homeopathic remedies provide a useful analogy. They are claimed to cure everything from eczema to HIV Aids. They are formulated by the repeated dilution of an ingredient until there is not a single molecule of it left in the final dose (the technical term is ‘30C’ – meaning that the original substance has been diluted by 1 drop in 100, 30 times). It is impossible reliably to distinguish these solutions from water. They contain no active ingredient, and as such there is no logical reason why they should achieve the ends they seek. The twist, of course, is that some people swear by them; they take a homeopathic pill and they feel better. Unfortunately, sugar pills and saltwater injections produce a similar experience. It’s