Corporate Objective Analysis and Other Oxymorons

Drug trials: Stacking the deck
by Jim Giles
Nature 440, 270-272 (16 March 2006) | doi:10.1038/440270a

They answer only the questions they want to answer. They ignore evidence that does not fit with their story. They set up and knock down straw men. Levelled at politicians, such accusations would come as no surprise. But what if the target were the researchers who test drugs? And what if the allegations came not from the tabloid press, but from studies published in prestigious medical journals?

...Although outright deception is rare, there is now ample evidence to show that our view of drugs' effectiveness is being subtly distorted. And the motivation, say the researchers, is financial gain and personal ambition.

"Patients volunteer for trials, but finances and career motives decide what gets published," says Peter Gøtzsche, an expert in clinical trials and director of the Nordic Cochrane Centre in Copenhagen. "This is ethically indefensible. Change is not easy, but we must get there."

It is a dramatic conclusion to come from a field of study with no proper name, staffed by part-time volunteers. Most are journal editors, medical statisticians or public-health experts, united by fears for the integrity of clinical trials. For the devotees of 'journalology' or 'research into research', the literature on clinical trials is their raw data and patterns of bias are their results.

Some of these researchers are using their findings to change medical journals and make it harder for authors to misrepresent results. Others are working on what could become the biggest reform of clinical-trial reporting for decades: the creation of a comprehensive international registry of all clinical trials. It is a powerful idea, which could one day make all trial information public. It is also an idea that has pitched pharmaceutical companies against advocates for reform, in a tussle over whether transparency or commercial confidentiality best serves medical science.

One of the biggest problems with clinical-trial reporting, the suppression of negative results, shows the importance of such debates. Because clinical researchers are not obliged to publish their findings, ambiguous or negative results can languish in filing cabinets, resulting in what Christine Laine, an editor at the Annals of Internal Medicine in Philadelphia, Pennsylvania, calls "phantom papers". If that happens, the journal record will give an over-optimistic impression of the treatments studied, with consequences for peer reviewers, government regulators and patients.

One alleged example hit the headlines in 2004. At that time, the antidepressant Paxil (paroxetine), made by London-based drug giant GlaxoSmithKline, was a popular treatment for adolescents in the United States. But doctors have now been warned off prescribing Paxil to youngsters, after evidence emerged that it increases the risk of suicidal behaviour. It was claimed in a court case brought in the United States that GlaxoSmithKline had suppressed data showing this since 1998. Rick Koenig, a spokesman for GlaxoSmithKline, says the company thought the charges unfounded, but agreed to pay $2.5 million to avoid the costs and time of litigation.

Phantom papers can be tracked down through trial protocols — the document describing how a trial will be run and what outcomes will be measured — which have to be registered with local ethics committees. By matching papers with protocols, several groups have shown that many trials are completed but not published. And that, notes Laine, makes it impossible for journals and health agencies to assess potential drugs. "You never quite know if other data are out there that would influence your conclusions," she says.

Last year, for example, a French team showed that only 40% of trials registered with its country's ethics committees in 1984 had been published by 2002, despite more than twice as many having been completed [Decullier, E. , Lhéritier, V. & Chapuis, F. Br. Med. J. 331, 19–22 (2005)]. Crucially, papers with inconclusive results not only took longer to publish (see graph), they were less likely to see the light of day at all. Researchers in any field can sit on negative or inconclusive results. But critics say that clinical researchers carry a greater ethical burden, as their findings inform decisions about the licensing of drugs.

Nor do the problems end when a trial hits an editor's desk. Results from a trial of the arthritis drug Celebrex (celecoxib) looked good when they were published in 2000, for example, but less so when physicians scrutinized the full data set. The original paper, which appeared in the Journal of the American Medical Association (JAMA), dismissed fears that Celebrex could cause ulcers. But that was based on data collected over six months. When other physicians analysed a full year's worth — which the authors already had at the time of their JAMA submission — they claimed that Celebrex seemed to cause ulcers just as often as other treatments [Hrachovec, J. B. & Mora, M. J. Am. Med. Assoc. 286, 2398 (2001)]. The original study's authors say that the later data were too unreliable to be included, but acknowledge that they could have "avoided confusion" by explaining to editors why they had omitted them.

But even with all the data, journal editors face another challenge: hype...

The hype shows up in a paper's conclusions. In 2003, epidemiologist Bodil Als-Nielsen and her colleagues at the University of Copenhagen looked at factors that might influence researchers' conclusions about a drug's efficacy or safety [Als-Nielsen, B. , Chen, W. , Gluud, C. & Kjaergard, L. L. J. Am. Med. Assoc. 290, 921–928 (2003)]. Their analysis of 370 trials showed that the strongest predictor of the authors' conclusions was not the nature of the data, but the type of sponsor.

Trials funded by for-profit organizations were significantly more likely to reach a favourable verdict than those sponsored by charities or governments. Critically, the association was not explained by the papers having more positive results. In a study under review, Gøtzsche and his colleagues show that industry-funded meta-analyses — studies that combine results from several clinical trials of a drug — are similarly prone to draw positive conclusions that are not supported by the data.

For many clinical-trials experts, these funding biases explain all the others. For each act, be it the suppression of results or the omission of outcomes, there is a financial motive for the company whose drug is being tested. In many cases, the company funding the study also employs one or more of the authors. Given the combination of motive and opportunity, many see drug-company influence as an inevitably distorting factor...