Coverage on the US Science magazine website of the Health Care Reform bill's safe passage burrows into the small print for the latest word on measuring "effectiveness".

"Most directly tied to health care," Jocelyn Kaiser writes, "is language creating a new, independent, non-profit institute for comparative effectiveness research (CER) – evidence-based studies that compare the value of medical treatments, such as two different drugs or a specific drug versus surgery."

Last year, the US Institute of Medicine announced that $1.1million for CER had been appropriated by Congress in the American Recovery and Re-investment Act. The Institute was asked to establish a working definition, develop priorities and identify necessary requirements for a CER enterprise – all of which it duly did, in the process launching a new growth industry.

Henceforth CER will refer to "the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care".

Bring on meta-analysis and systematic reviews; bring on effect size data; bring on, too, aspersions against the conventional wisdom concerning the design and unbeatable value of the randomized controlled trial. No control group? Two or more intervention groups? No stable ground?

Among commentators on the upsurge of critical interest in CER is On Social Marketing and Social Change, the website of Craig Lefebvre at The George Washington University School of Public Health and Health Services.

Responding to the Institute of Medicine (IOM) announcements and their publication of a 100 CER priority areas, Levebre quotes a 2009 paper from the Annals of Internal Medicine by a team of university medics who argued that as they were currently designed and conducted,

many RCTs are ill suited to meet the evidentiary needs implicit in the IOM definition of CER: comparison of effective interventions among patients in typical patient care settings, with decisions tailored to individual patient needs. Without major changes in how we conceive, design, conduct, and analyze RCTs, the nation risks spending large sums of money inefficiently to answer the wrong questions – or the right questions too late.

The focus of such argument tends to be on medical trials, but comparisons between preventive strategies nevertheless figure prominently in the IOM top 100.

Among the ones Lefebvre picks out, for example:

• compare the effectiveness of dissemination and translation techniques to facilitate the use of CER by patients, clinicians, payers, and others.
• compare the effectiveness of school-based interventions involving meal programs, vending machines, and physical education, at different levels of intensity, in preventing and treating overweight and obesity in children and adolescents
• compare the effectiveness of various strategies (eg, clinical interventions, selected social interventions [such as improving the built environment in communities and making healthy foods more available], combined clinical and social interventions) to prevent obesity, hypertension, diabetes, and heart disease in at-risk populations such as the urban poor and American Indians
• compare the effectiveness of wraparound home and community-based services and residential treatment in managing serious emotional disorders in children and adults
• compare the effectiveness of clinical interventions (eg, prenatal care, nutritional counseling, smoking cessation, substance abuse treatment, and combinations of these interventions) to reduce incidences of infant mortality, pre-term births, and low birth rates, especially among African American women.

Lefebvre argues, naturally enough, that social marketing is highly relevant in all these contexts and deserves to be evaluated as part of any wider comparison of services and implementation strategies.

At The Huffington Post D Brad Wright offers another perspective: pragmatic, comparative effectiveness studies run the danger of being swamped by too much data too difficult to compare – unless one throws into the equation an x factor: the money.

"If there are two treatments for a particular condition, which one works better? CER is designed to answer that question," he writes.

"After all, if the purpose of health care is to restore health, we certainly want to be using the most effective treatments. This is equally true if we're talking about a 50% efficacy rate compared to an 80% efficacy rate or if we're talking about a 90% efficacy rate compared to a 90.5% efficacy rate. Why settle for less than the best?

"Well, economists have the answer to that question: because the gains in efficacy may cost too much. If the procedure with 50% efficacy costs $1,500 we might well be willing to pay $6,000 for the procedure with 80% efficacy. However, if the procedure with 90% efficacy cost $1,500 it is doubtful that we'd be willing to pay another $4,500 for the 0.5% bump in efficacy. When the calculations include not only efficacy but also cost, we are no longer conducting CER but cost effectiveness analysis (CEA).

"A number of questions are raised by CEA that don't arise with CER. How much are we willing to pay for an additional gain in efficacy? For an additional year of life? For an additional year of high-quality life?"

See also: Luce B R, Kramer J M, Goodman S N, Connor J T, Tunis S, Whicher D and Sanford Schwartz J, "Rethinking Randomized Clinical Trials for Comparative Effectiveness Research: The Need for Transformational Change", Annals of Internal Medicine