The British Medical Journal

Naming Names: Is There an (Unbiased) Doctor in the House?

Cecille Isidro — Wed, 23 Jul 2008 01:29:00 -0400

Journalists often forget that conflicts of interest may biasthe opinions of their expert sources. Jeanne Lenzer and ShannonBrownlee explain how, in an attempt to disentangle commercialmessages from science, they have compiled a list of nearly 100independent medical experts to whom reporters can turn.

Ho hum, another medical scandal in the news. Earlier this monthUS Senator Chuck Grassley announced his intention to investigateAlan Schatzberg, chairman of the psychiatry department at StanfordUniversity and the incoming president of the American PsychiatricAssociation, about his multimillion dollar interest in CorceptTherapeutics, a company that is seeking to market a drug thatDr Schatzberg is researching with federal funding, and the extentto which he disclosed and was required to disclose that interestto Stanford.¹ In June the New York Times broke a front pagestory about the alleged failure of three top research psychiatristsat Harvard, each of them a proponent of drug treatment for psychiatricconditions in children, to report that since 2000 they had collectivelyreceived more than $4.2m (£2.1m; 2.6m) from various drugcompanies.²

After ignoring the growing controversy over conflict of interestfor years, journalists now seem only too happy to expose wrongdoingin medicine. Yet when it comes to reporting medical news, thoseself same reporters often seem to forget that conflicts of interestmight also bias the opinions of their expert sources. The mediaare filled with happy talk about "medical breakthroughs" thatis based on information gathered from sources with ties to theindustry. Yet simply knowing that conflicts of interest cancreate bias doesn’t answer the question of which studieswe ought to believe. Because journalists fail to seek out sourceswho can offer an independent perspective, many medical storiesin the popular media are either unbalanced or simply wrong.

In an attempt to disentangle commercial messages from scienceand to contribute to better reporting we took a drastic stepthat we believe can go to the heart of the problem: we decidedto name names. We created a list of nearly 100 internationalmedical experts in a wide variety of disciplines. But contraryto the "gotcha" tradition of journalism, the list’s membersare not physicians on the take but rather the reverse: theyare leading independent experts, many of them sources we havecultivated over years of reporting. It includes, from journalpublishing, two former editors of the New England Journal ofMedicine, the former editor of the Western Journal of Medicine,and a senior editor of PLoS Medicine; former advisers to theUS Food and Drug Administration; physician educators; researchers;bioethicists; epidemiologists, methodologists, geneticists,and clinicians from various specialties; medical whistleblowers;and several medical journalists.

Those applying to be on the list fill out a form affirming thatthey have not received "any financial support in any form frompharmaceutical or medical device manufacturers during the pastfive years" and that they don’t have other affiliationsor financial involvements that would present a conflict of interest.A three member board decides whether to accept applicants. Wealso maintain a "page 2" list of experts who willingly disclosetheir conflicts of interest or have ended their industry tiesbut only within the past five years. Despite their recent commercialties, these experts are included in the list because they haveprovided key insights into the inner workings of partnershipsbetween physicians and the industry—and thus have bittenthe hand that feeds them, in effect.

The reaction to the list, which has been embraced enthusiasticallyby our fellow reporters and roundly condemned by several alliesof the drug industry, suggests that the effect of simply gatheringthese names together could well go beyond making life a littleeasier for our fellow journalists.

Seeking Unbiased Sources

The need for such a resource is evident from studies showingthat bias and poor reporting on medical topics are widespreadin the popular media. Gary Schwitzer, a professor of journalismat the University of Minnesota, publishes HealthNewsReview.org,a website that reviews healthcare news for balance, accuracy,and completeness. Schwitzer and a team of academic researchersanalysed 500 stories published in top outlets between April2006 and April 2008 for two key criteria: did the journalistquote an independent expert, someone not involved in the relevantresearch; and did they make some attempt to report potentialconflicts of interest. The result? Half the stories failed tomeet these two very basic requirements.³

In another study Alan Cassels, a pharmaceutical policy researcherat the University of British Columbia, and his colleagues analysedmedia coverage of five prescription drugs published in 193 Canadiannewspapers in 2000.⁴ Cassels, who is on our list, found thatthe stories were overwhelmingly positive towards the drugs:all 193 articles included at least one drug benefit, while 68%(132/193) failed to mention any potential harm. Two thirds ofthe stories quoted a source by name, but only a scant 3% (5/164)included information about conflicts of interest for sourceswho were not government or industry officials.

In the view of one list member, Arnold Relman, former editorin chief of the New England Journal of Medicine and professoremeritus of medicine and of social medicine at Harvard MedicalSchool, such bias fails to serve the public good. "The publichas a lot at stake, and the media has a responsibility alwaysto be aware of the source of information and the conflicts thosesources might have when they report the results of clinicalresearch," he said. "People who have a financial stake in theresults of clinical research can well be biased in the way researchis conducted, in the way they report it, and what they say aboutit when interviewed by the media."

Changing the Status Quo

The question is why reporters seem unable to grasp the connectionbetween the large body of evidence showing that financial conflictsof interest create bias in medical research and the need forthe media to seek out independent sources. To be fair, journalistsface a daunting task when trying to sift through medical research,and many are as yet unaware of the profound influence the drugindustry has over research results and the ways in which theindustry shapes medical "truths." Many reporters also fail torealise that the individuals and organisations they turn tofor expert commentary, such as professional groups and charities,professional guideline authors, federal advisory panellists,and patients’ groups, often depend financially on theindustry. Thus there is a self reinforcing process in whichcommercially sponsored researchers, whose prominence is enhancedby the industry’s public relations machine, are dubbed"experts," while independent sources are cited less often.

From informal conversations with colleagues we also know thatother factors are at work when reporters fail to take conflictsof interest into account. Some confess that they hesitate toask sources about any potential conflicts for fear that thesource will take umbrage and refuse to be interviewed. Othersassume that if a study appears in a peer reviewed journal itmust be valid.

One of the solutions to the problem of biased news reporting,in the view of Michael Wilkes, professor of medicine and directorof global health at the University of California, Davis, isgreater transparency.⁵ We think the list is a step in that direction.The chief requirement for membership, besides a recognised areaof expertise, is that the expert must not have taken any industryfunding for at least the past five years. Beliefs about certaindrugs or treatments were not criteria for inclusion or exclusion.Indeed, the list includes experts who sit at opposite polesof the spectrum of beliefs on certain issues.

Backlash and Honor

Within days of our announcing that we would make our list availableto reporters the requests began pouring in. Thus far we havesent a copy of the list to 105 reporters, authors, and editorsfrom such media outlets as the New York Times, Newsweek, Forbes,Fortune, Bloomberg News, the Washington Post, US News &World Report, the Canadian Broadcasting Corporation, Medscape,and many other publications across the US and several othernations. Senators and a state attorney general have also requestedit.

Surprisingly, we are also receiving requests from recognisedexperts who wanted to be onthe list. Being a member, it seems,is a badge of honour, say several of the list members we interviewedfor the BMJ. Others, like list member Barnett Kramer, want toimprove the quality of medical reporting. Kramer, a medicaloncologist and associate director for disease prevention atthe US National Institutes of Health, said, "Working journalistscan be overwhelmed by PR, and they are often looking for expertswho can make comments." It’s useful, he said, to haveexperts "who are not involved directly with the research andhave no potential conflicts of interest relevant to the research."

The other surprise came after the publication of a story wewrote in the online magazine Slate that mentioned the list.⁶Within days bloggers were furiously accusing us of everythingfrom biased, sloppy reporting to being members of the Churchof Scientology (which is opposed to psychiatric drugs). Manyof our critics—virtually all of them backed by the industry—opinedthat our list was undoubtedly filled with experts who were onthe payrolls of plaintiffs’ attorneys. (A few have testifiedin court cases, and those who have been paid for their testimonyhave disclosed it for the list.) This venom was unexpected,as we imagined that the list would be viewed as a positive steptowards helping reporters identify doctors and other expertswho can address thorny and complex medical issues without havingcompeting financial interests. Now we think we understand thebacklash a little better.

One of the problems recognised by Schwitzer is that many journalistsrely for story ideas on news releases from the industry’spublic relations departments, and some even use releases asthe sole source of information on experts to interview. By offeringan alternative list of highly credible, independent experts,the industry may fear that its paid key opinion leaders⁷ andthe professional societies whose favour they cultivate willno longer be the first source of medical news.

Peter Gøtzsche, director of the Nordic Cochrane Centreand a member of the Danish group Doctors Without Sponsors, describedwhy he joined the list: "Industry knows that buying doctorsis an effective marketing tool . . . far more effective thanthe dollars they spend on drug representatives. This leads toless than optimal health care for patients."

Beyond the list’s usefulness to journalists, we hope thatit will also be used by government agencies, medical journaleditors, and professional societies as they seek out expertsto serve as editorialists and members of clinical guidelineand advisory panels. The FDA, for example, has a copy of thelist. We would be pleased to send it to other agencies and professionalsocieties.

It is widely claimed that genuine experts in medicine who aren’tfunded by the industry are nearly impossible to find. Indeed,one expert who declined our invitation to be listed said, "Ifyou eliminate people to whom industry turns for advice, you’reeliminating people who are more likely to have something worthsaying."

Readers can decide for themselves whether our list of independentexperts includes any experts with "something worth saying."

The list of independent medical experts

John Abramson, clinicalinstructor, Harvard Medical School

Marcia Angell, former editorin chief, New England Journal of Medicine

David Antonuccio,professor, Department of Psychiatry and Behavioral Sciences,University of Nevada

Michael J Barry, chief of general medicineunit, Massachusetts General Hospital, Harvard Medical School

KenBassett, professor of family practice, pharmacology, and therapeutics,University of British Columbia

Lisa Bero, professor, Universityof California, San Francisco

Stephen Bezruchka, Departmentof Health Services and Department of Global Health, School ofPublic Health and Community Medicine, University of Washington,Seattle

Laura Boylan, assistant professor, Department of Neurology,New York University

Phil Brewer, university medical director,Quinnipiac University, Connecticut; and past medical safetyfellow, US National Highway Traffic Safety Administration

HowardBrody, director, US Institute for the Medical Humanities

StevenR Brown, Banner Good Samaritan family medicine residency, Universityof Arizona College of Medicine

Daniel Carlat, assistant clinicalprofessor of psychiatry, Tufts University School of Medicine,and editor in chief, The Carlat Psychiatry Report

Alan Cassels,pharmaceutical policy researcher, University of Victoria, BritishColumbia

Robert Cook-Deegan, director, Center for Genome Ethics,Law and Policy, Duke Institute for Genome Sciences and Policy

SamS Dahr, Retina Center of Oklahoma

John M Davis, Gilman professorof psychiatry, University of Ilinois at Chicago

Raymond DeVries, professor, bioethics programme, University of MichiganMedical School

Richard Deyo, Kaiser Permanente professor ofevidence based family medicine, Department of Family Medicine,Oregon Health and Science University

Kay Dickersin, director,US Cochrane Center

Mark Ebell, deputy editor, American FamilyPhysician, and professor, University of Georgia

Carl Elliott,University of Minnesota Center for Bioethics

David J Elpern

MargaretEwen, Health Action International, Netherlands

Anne RochonFord, coordinator, Women and Health Protection, Canada

AdrianeFugh-Berman, professor, Department of Physiology and Biophysics,Georgetown University Medical Center, and director, PharmedOut.org

JosephGlenmullen, clinical instructor in psychiatry, Harvard MedicalSchool

Robert Goodman, founder and director of No Free Lunchand general internist at Montefiore Medical Center, New York

MerrillGoozner, director, Integrity in Science, US Center for Sciencein the Public Interest

Peter Gøtzsche, director, NordicCochrane Centre, Denmark

Mark E Helm, medical director, EBRx,Arkansas Evidence-Based Prescription Drug Program, and assistantprofessor, College of Pharmacy, University of Arkansas for MedicalSciences

David Himmelstein, associate professor of medicine,Harvard University

Jerome Hoffman, professor of medicine andemergency medicine, University of California, Los Angeles

JohnP A Ioaniddis, professor and chairman, Department of Hygieneand Epidemiology, University of Ioannina School of Medicine,Ioannina, Greece, and Institute for Clinical Research and HealthPolicy Studies, Department of Medicine, Tufts-New England MedicalCenter, Tufts University School of Medicine

Peter Juni, headof division, Institute of Social and Preventive Medicine, Universityof Bern, and director, Clinical Trials Unit, Bern UniversityHospital

Jon Jureidini, head, Department of Psychological Medicine,Children’s Youth and Women’s Health Service, Adelaide,and associate professor, disciplines of psychiatry and paediatrics,University of Adelaide

Scott Kim, assistant professor of psychiatry

PeterD Kramer, clinical professor of psychiatry and human behaviour,Brown University, Providence, Rhode Island

Barnett Kramer,associate director for disease prevention, US National Institutesof Health

Sheldon Krimsky, Tufts University, and Council forResponsible Genetics

Stefan Kruszewski, Stefan P Kruszewskiand Associates

Richard A Lange, professor of medicine, JohnsHopkins Hospital, Baltimore

Jeffrey Lacasse, assistant professor,Department of Social Work, College of Human Services, ArizonaState University at West Campus

Dara K Lee, staff cardiologist,Presbyterian Heart Group, Albuquerque, and vice president, MedicalStaff Affairs, Presbyterian Hospital, Albuquerque

GretchenLeFever, director of patient safety and performance excellence,Sentara, US

Trudo Lemmens, associate professor, Canada

JonathanLeo, associate professor of neuroanatomy, US

Joe Lex, emergencyphysician, US

Joel Lexchin, professor, School of Health Policyand Management, York University, Toronto

Abby Lippman, professor,McGill University, Montreal

Peter Lurie, Health Research Groupat Public Citizen, United States

William K Mallon, associateprofessor of clinical emergency medicine, Keck School of Medicineat the University of Southern California, and director, Divisionof International, LAC+USC Medical Center, Los Angeles

PeterR Mansfield, director, Healthy Skepticism, Australia

LindaMarsa, freelance journalist, US

Charlea Massion, Center forEducation in Family and Community Medicine, Stanford UniversitySchool of Medicine, and member of board of directors, AmericanCollege of Women’s Health Physicians

Charles Medawar,Social Audit, UK

Steven Miles, professor of medicine, Centerfor Bioethics, University of Minnesota

Barbara Mintzes, assistantprofessor, Department of Anesthesiology, Pharmacology and Therapeutics,University of British Columbia

Steven Morgan, associate professorand associate director, Centre for Health Services and PolicyResearch, School of Population and Public Health, Universityof British Columbia

Ray Moynihan, journalist, Australia

VijayaMusini, assistant professor, Department of Anesthesiology, Pharmacologyand Therapeutics, University of British Columbia, and TherapeuticsInitiative, Canada

Thomas L Perry, clinical assistant professor,Department of Anesthesiology, Pharmacology and Therapeuticsand Department of Medicine, University of British Columbia

BrucePsaty, professor of medicine and epidemiology, University ofWashington Cardiovascular Health Research Unit

Arnold Relman,former editor in chief, New England Journal of Medicine

DavidRind, senior deputy editor, UpToDate, and assistant clinicalprofessor of medicine, Harvard Medical School

Charles Rosen,clinical professor of surgery, University of California, Irvine,and founding director, US Association for Ethics in Spine Surgery

HayaRubin, director, research and evaluation, Palo Alto MedicalResearch Institute, California, and adjunct professor of medicine,Johns Hopkins University, Baltimore

Larry Sasich

John Schumann,assistant professor of medicine, University of Chicago, andMacLean Center for Clinical Medical Ethics, Chicago

Lisa Schwartz,Dartmouth Institute for Health Policy and Clinical Practice,Lebanon, New Hampshire

Gary Schwitzer, director, health journalism,MA programme, University of Minnesota School of Journalism andMass Communication

Vera Hassner Sharav, Alliance for HumanResearch Protection, US

Allen Shaughnessy, professor, TuftsUniversity School of Medicine, Boston, Massachusetts

AnthonySo, programme on global health and technology access, TerrySanford Institute of Public Policy, Duke University, Durham,North Carolina

Robert C Solomon, American College of EmergencyPhysicians, and medical editor in chief, ACEP News, US

DesSpence, general practitioner, Glasgow, and UK spokesman of NoFree Lunch

Sydney Z Spiesel, clinical professor of paediatrics,Yale University School of Medicine, and regular commentatorfor Slate and US National Public Radio

Alex Sugerman, attorney,Prescription Access Litigation, US

Leonore Tiefer, New ViewCampaign, and New York University School of Medicine

AlexanderTsai, residency training programme, Department of Psychiatry,University of California at San Francisco

Jennifer Washburn,journalist, US

H Gilbert Welch, Dartmouth Institute for HealthPolicy and Clinical Practice, Lebanon, New Hampshire

MichaelWilkes, professor of medicine and director of global health,University of California, and former vice dean of educationand former editor in chief, Western Journal of Medicine, Universityof California, Davis

Sidney Wolfe, director, Health ResearchGroup of Public Citizen, US

Steven Woloshin, Veterans AffairsOutcomes Group

Alastair Wood, US

Steffie Woolhandler, associateprofessor of medicine, Harvard University

James Wright, managingdirector, Therapeutics Initiative, Canada

Gavin Yamey, senioreditor, PLoS Medicine, US

References

Garber K. Committee questions a top psychiatrist. US News & World Report. www.usnews.com/articles/news/national/2008/06/26/committee-questions-a-top-psychiatrist.html.
Lenzer J. Review launched after Harvard psychiatrist failed to disclose industry funding. BMJ 2008;336:1327.[Free Full Text]
Schwitzer G. How do US journalists cover treatments, tests, products, and procedures? An evaluation of 500 stories. PLoS Med 2008;5:e95.[CrossRef][Medline]
Cassels A, Hughes MA, Cole C, Mintzes B, Lexchin J, McCormack JP. Drugs in the news: an analysis of Canadian newspaper coverage of new prescription drugs. CMAJ 2003;168:1133-7.[Abstract/Free Full Text]
Shuchman M, Wilkes MS. Medical scientists and health news reporting: a case of miscommunication. Ann Intern Med 1997;126:976-82.[Abstract/Free Full Text]
Brownlee S, Lenzer J. Stealth marketers: are doctors shilling for drug companies on public radio? Slate, www.slate.com/id/2190775/.
Moynihan R. Key opinion leaders: independent experts or drug representatives in disguise? BMJ 2008;336:1402-3.[Free Full Text]

Knowing Me, Knowing You

Ron Tang — Sun, 20 Apr 2008 02:09:00 -0400

Do you want to Google your genes or peer into your future risks of heart disease or cancer? Now you can, according to direct to consumer testing companies. Gone are the days when genetic testing was limited to doctors ordering tests for rare, but prognostically potent, single gene disorders such as Huntington’s disease, Duchenne’s muscular dystrophy, or cystic fibrosis. Thanks to an explosion of newly discovered single nucleotide polymorphisms, or SNPs (pronounced snips), companies are marketing genetic tests for traits ranging from the mundane -- eye colour and wet ear wax -- to serious conditions such as Crohn’s disease and Alzheimer’s disease.

While the global market for these tests is growing rapidly -- estimated at $730m (£366m; €463m) last year and growing by 20% annually[1 ] -- evidence that they can provide patients with clinically useful information is lagging far behind. There is little regulatory oversight of the tests, and even less in the way of clinical data to help doctors guide patients who go to them carrying printouts of their genetic details. Genetic tests and “personalised medicine” are supposed to enable doctors to customise each patient’s care, yet there is a paucity of studies on interventions for patients with genetic variants.

The promise being made to consumers is clear: forewarned is forearmed. The website for deCODE genetics, based in Reykjavik, says its tests will “help to empower individuals and their doctors.” The Californian company 23andMe, which has backing from Google and the biotech company Genentech, provides an “odds calculator” that the company says will allow customers to see which “health concerns are most likely to affect a person with your genetic profile.” Navigenics, also based in California, claims its tests can provide a “roadmap to optimal health” that can enable customers to “take action before a disorder strikes to delay or even prevent the illness altogether.” And all for a fee ranging from $1000 to $2500.

A more likely scenario is that these tests will raise more questions than they answer. It is unclear what consumers think they are learning from their “genetic blueprints.” Some screening tests, though non-invasive and seemingly harmless, have been shown to trigger a cascade of further evaluations and interventions that result in measurable harms while providing no benefit.[2-5 ]

Although widespread genetic testing might eventually lead to well defined risk profiles and the ability to tailor drugs to the individual, such results may not be available for many years. In the meantime, genetic testing poses important and largely unacknowledged risks. As well as clinical concerns, there are questions about the effect of the corporate partnerships that link genomic data mining companies to electronic medical records, hospitals, and drug companies.

Genomics revolution

Direct to consumer genetic testing has its roots in the human genome project, which was launched in 1984 with the promise of opening frontiers in medicine. Novel, personalised treatments would flow from an understanding of genetic underpinnings of disease, and some experts predict that the genomics revolution is poised to deliver information that will allow people to make personalised lifestyle changes or decide whether to have a child, and allow doctors to prescribe correct drugs at correct doses.[6 ] Linda Avey, cofounder of 23andMe, is an advocate of personalised medicine. “We’re still using very antiquated systems for diagnosis and for prescribing therapies. Whenever you take a drug it’s kind of a gamble whether your body will respond appropriately to it,” she said.

Thus far, however, only a handful of genetic tests can indicate how a patient will react to a drug, and even then, the test may lack clinical value. For example, a working group that reviewed studies of dosing of serotonin reuptake inhibitor drugs based on the CYP450 polymorphism concluded that there is no evidence that the test is “useful in medical, personal, or public health decision-making.”[7 ] Even the discovery of the genes for many single gene conditions has failed to lead to the expected cures. For some diseases, like Huntington’s disease and sickle cell anaemia, the prospects for treatment seem as remote as they were before the genes were discovered.

Along with genetic information, some companies offer to calculate a customer’s risk of developing conditions such as type 2 diabetes, cardiovascular disease, and prostate and breast cancer. 23andMe, for example, offers a scan of some 580,000 SNPs and a report on roughly 14 conditions for which customers might be at risk. These risk calculations are exceedingly rough, however, as most SNPs have been only loosely connected to any particular disease, and there are few hard numbers that can accurately predict the contribution of a particular polymorphism to an individual’s phenotypic risk.

A lack of data has not stopped genomics companies from capitalising on the appealing concept of personalised medicine, starting with their names, which include 23andMe (23 chromosomes), deCODEme, and Knome (pronounced know me). Knome is founded by George Church, professor of genetics at Harvard, who helped develop the first direct genomic sequencing method. For upwards of $350,000, Knome will sequence a customer’s entire genome. 23andMe also provides genetic information for “entertainment” and “education” purposes, such as whether one has genetic markers for an enhanced ability to taste bitterness or for athleticism.

Companies are careful to acknowledge in the fine print the lack of meaningful data. In a special message “To the Medical Community,” 23andMe acknowledges that the information it provides “is tailored to genotypes not to individuals” -- an acknowledgment that belies the claim of personalised, clinically relevant health information. But 23andMe cofounder Anne Wojcicki argues that people have a right to know their genetic information and says that the company is not providing “actionable” health information but information that is largely intended to educate and lead to better research.

But what is it that customers are being educated about? When asked if customers of 23andMe were under the impression that they could obtain health information that would be useful to them, Professor Church, who is an adviser to 23andMe, responded, “I hope not.” He added, “Education is useful, but distinct from clinically accepted diagnoses.”

Estimating risk

Nevertheless, doctors are likely to be seeing more and more patients arriving at their doors, genomic results in hand, requesting treatment for diseases they do not yet have or more screening tests. Most doctors, according to a recent systematic review by RAND Health, are “woefully underprepared” to counsel patients about genetic tests.[8 ] According to one study cited in the review, only 5% of doctors said they felt “confident in their ability to interpret test results.”

Rather than improving health, widespread genetic testing is likely to result in widespread anxiety. In a commentary published in JAMA, Gilbert Welch and Wylie Burke caution that there is substantial confusion about estimates of genetic risk.[9 ] Selection bias, they say, can lead to “overestimates of both risks and consequences of disease.” For example, the oft cited 87% risk of cancer by the age of 70 years among women who test positive for the BRCA1 gene was derived from tests of exceptionally high risk women who had at least four family members with ovarian or breast cancer diagnosed before age 60 years. Far lower estimates were obtained when the inclusion criteria for testing were broadened.[9] Another form of bias, surveillance bias, can lead to similar exaggeration of risk when people with a certain genetic trait choose to have more frequent testing. “The quickest way to develop breast cancer,” said Dr Welch, “is to be tested for it.”

Framing risk can have a powerful effect -- an effect that genomics companies are putting to use when marketing their tests. The website of 23andMe warns of a “1 in 8” chance of developing breast cancer, a risk that can be expected to alarm many women. Yet a woman’s risk is that high only once she reaches the age of 70. In the US, such risk estimates have led many women to overestimate their chances of dying of breast cancer, which accounts for only 3% of female deaths annually.[10]

Similarly, telling men (accurately) that more than half of them will have prostate cancer by the age of 60 years, and that men with certain genetic variants have a four times greater risk of developing prostate cancer than men with none of those variants (also accurate) could understandably persuade some men to have prostate specific antigen testing.[10 ] But if the information is framed differently men might not be so enthusiastic. “Only 3% of men die of prostate cancer,” Dr Welch says. Instead of telling men that they are at an 80% increased risk of developing prostate cancer, he says, doctors should tell men that their lifetime risk of dying from prostate cancer is 3%. “And an 80% increase of that risk doesn’t even increase the risk to 6%.”

Research benefits

The potential pay-off, for both patients and companies, lies with pharmacogenomics, the development of new, more personalised drugs. This requires access to huge numbers of research subjects, their genomes, and their phenotypic (clinical) records -- an extraordinarily expensive undertaking. In a clever reversal, companies are getting research subjects to pay -- rather than be paid -- to become research subjects. Genetic testing companies’ primary source of income is fees paid by customers. In future, they can expect to earn even more by selling the genomic information they gather to researchers and to biotechnology and drug companies. Customers of 23andMe sign a consent document indicating that the company may share their genomic data (anonymously) and that the company may sell the data to researchers. Customers of 23andMe are asked if they would like to make their health records available (free) to researchers to advance medical science.

Companies are planning ways to integrate genomic data with phenotypic information from patients’ electronic records. Google, which invested $3.9m in 23andMe, is poised to launch its online personal health records. (The cofounder of Google, Sergey Brin, is married to Ann Wojcicki, cofounder of 23andMe.) Ms Avey says 23andMe plans to work with personal health record companies to enable data merger.

Although 23andMe insists it is in the business of advancing research, its commercial appeal to the public suggests a different interest. Dr Welch says that telling customers about their risk factors could result in surveillance bias. “They might be conducting research,” he said, “but it won’t be good research.”

The potential gold mine in combining genomic and phenotypic information is evident from the fact that at least one company, the Coriell Institute for Medical Research in New Jersey, is offering genetic testing to 10 000 people. The institute has partnered with several healthcare organisations, including Cooper University Hospital, which is offering Coriell’s free genomic screening to the first 2000 of its 5500 employees who sign up. The response has been “overwhelmingly positive,” according to a hospital spokeswoman. Cooper’s chief medical officer, Simon Samaha, says that the hospital is planning training sessions for doctors, who expect an influx of patients after the screening.

While genomic databases may lead to better drugs, integrating genomic and phenotypic data has raised concerns about potential discrimination by insurance companies and employers. In the US, antidiscrimination laws vary from state to state, and there is little agreement among payers, physicians, researchers, and patients about whether insurance companies should have access to genomic information. Dr Samaha says that he’s not sure that it’s wrong for insurance companies to have access to such genomic data: “It raises a question about how open the economy should be.”

Failed regulation

Concerned with the potential for harm, advisory bodies in the US and UK have recommended regulatory oversight of direct to consumer genetic testing -- yet national agencies in both countries have failed to act. According to Public Citizen’s Health Research Group, a public interest group based in Washington, DC, the 2004 draft report by the US secretary’s advisory committee on genetics, health and society was “notable” for its “accurate diagnosis of the manifold problems in the oversight of genetic testing and... for its complete failure to identify an appropriate treatment for these problems.”[11 ]

Critics say that regulatory oversight is uneven to non-existent. There is little oversight of clinical validity (is the genetic variant reliably associated with the phenotypic change observed?) and clinical utility (do enhanced surveillance, prophylactic treatment, or lifestyle changes improve outcome?).

Even accurate detection of genetic variants is not necessarily assured. In 1997 and 2001, US governmental groups and an earlier advisory panel recommended proficiency testing for all laboratories doing genetic tests -- but those recommendations were simply ignored by federal agencies.[12 ] In an almost comical response, the Center for Medicare and Medicaid Services, the agency responsible for ensuring analytical validity under the Clinical Laboratory Improvement Amendments, argued that setting laboratory standards for genetic testing was too difficult. To prove its point, it cited the agency’s previous difficulty in ensuring the accuracy of cervical smear testing, which took 17 years to complete. The reason the amendments were passed (and proficiency testing instituted) was because of massive malpractice suits resulting from the deaths of women whose cervical smears had been misread.[12]

Peter Lurie, deputy director of Public Citizen’s Health Research Group, is disturbed by the lack of oversight of genetic testing. He worries that the public may “assume there is the same degree of oversight” for genetic tests as there is for tests such as a red blood cell count. “But,” he said, “they are not even remotely the same.” Similar problems plague the UK. Sir John Sulston, acting chair of the Human Genetics Commission, which lacks regulatory authority, says that nothing has changed since December 2007, when he wrote that direct to consumer genetic testing is “for now, largely in the hands of commercial test providers: the pharmaceutical companies, their marketing departments and PR agents.”[13 ]

What all this means is that testing companies are operating in a regulatory vacuum. The steps needed to protect the public have been clearly laid out by the various public interest groups and advisory panels. Until national agencies act, it will be up to doctors to handle the expected influx of questions and problems that arise from direct to consumer genetic testing. The first step towards reducing the chances of harm posed by widespread genetic testing is to educate doctors and patients about the limited value and potential harms of testing.[11-13]

Perhaps the most powerful argument for regulation comes from a surprising source. Dietrich Stephan, cofounder and chief science officer of Navigenics, says that his company has had discussions with the Food and Drug Administration and determined that “right now we are exempt from FDA regulation.” But he says his company would welcome FDA oversight because there are “a lot of charlatans and pseudoscience occupying this space and we are ready to be regulated.”

Notes

[1]. Herper M, Langreth R. Will you get cancer? Forbes.com 2007 Jun 18. www.forbes.com/free_forbes/2007/0618/052.html.

[2]. Bach PB, Jett JR, Pastorino U, Tockman MS, Swensen SJ, Begg CB. Computed tomography screening and lung cancer outcomes. JAMA 2007;297:953-61.

[3]. Barrette S, Bernstein ML, Leclerc JM, Champagne MA, Samson Y, Brossard J, et al. Treatment complications in children diagnosed with neuroblastoma during a screening program. J Clin Oncol 2006;24:1542-5.

[4]. Woods WG, Gao RN, Shuster JJ, Robison LL, Bernstein M, Weitzman S, et al. Screening of infants and mortality due to neuroblastoma. N Engl J Med 2002;346:1041-6.

[5]. Goldstein NM, Kollef MH, Ward S, Gage BF. The impact of the introduction of a rapid D-dimer assay on the diagnostic evaluation of suspected pulmonary embolism. Arch Intern Med 2001;161:567-71.

[6]. Feero WG, Guttmacher AE, Collins FS. The genome gets personal -- almost. JAMA 2008;299:1351-2.

[7]. Berg AO, et al. Recommendations from the EGAPP Working Group: testing for cytochrome P450 polymorphisms in adults with nonpsychotic depression treated with selective serotonin reuptake inhibitors. Gen Med 2007;91:819-25.

[8]. Scheuner MT, Sieverding P, Shekelle PG. Delivery of genomic medicine for common chronic adult diseases: a systematic review. JAMA 2008;299:1320-34.

[9]. Welch HG, Burke W. Uncertainties in genetic testing for chronic disease. JAMA 1998;280:1525-7.

[10]. National Cancer Institute. United States cancer statistics: 2004 incidence and mortality. Surveillance Epidemiology and End Results Database. http://seer.cancer.gov/faststats.

[11]. Lu E, Lurie P. Comment on the draft report on the US system of oversight of genetic testing (HRG publication #1832). Public Citizen 2007 Dec 21. www.citizen.org/publications/release.cfm?ID=7557.

[12]. Hudson K, Lurie P, Terry S. Petition requesting a genetic testing specialty and standards for proficiency testing (HRG publication #1787). Public Citizen 2006 Sep 26. www.citizen.org/publications/release.cfm?ID=7463.

[13]. Human Genetics Commission. More genes direct. London: Human Genetics Commission, 2007. www.hgc.gov.uk/UploadDocs/DocPub/Document/More%20Genes%20Direct%20-%20final.pdf.

An Untold Story?

Ron Tang — Wed, 27 Feb 2008 00:00:00 -0500

New generation antidepressants aren’t all they’re cracked up to be. That seems to be the central message in the meta-analysis published this week by Irving Kirsch and colleagues in PLoS Medicine,[1] and it was this message that made the headlines. Kirsch’s conclusion follows on the heels of similar studies showing that statins are useful in only a small subset of patients taking the drugs[2] and earlier studies finding that the safety and performance of cyclo-oxygenase-2 (COX 2) inhibitors were worse than they first seemed.[3] All of which further reinforces previous criticisms that regulators in the United Kingdom and the United States are not doing their duty to protect the public from useless or dangerous drugs. But there’s another, deeper problem here -- a problem that, ironically enough, was highlighted by GlaxoSmithKline’s news release stating that Kirsch’s conclusions are "incorrect" because he evaluated only a "small subset of the total data available." How can regulators, the public, and doctors know how useful (or how potentially dangerous) drugs really are unless outside researchers have access to all the data?

The gist of Kirsch’s analysis of published and unpublished data from studies of antidepressants in adults is that only a very small subset of patients seemed to benefit to a clinically significant degree. But even now, do we really know the truth about antidepressants? Or statins? Or any other drug on the market? Even when researchers make exhaustive searches and file requests with regulatory authorities under freedom of information legislation, the datasets they need to validate results can remain tantalisingly out of reach.

Ending publication bias: necessary but not enough

Lack of access to data is an ongoing problem in the United States, despite passage of the Food and Drug Administration Amendments Act (FDAAA) of 2007, which was intended to make clinical trial data more transparent. The act requires most clinical trials to be registered and their results to be posted at ClinicalTrials.gov, a clinical trials registry of the National Library of Medicine. It’s an admirable first step,[4] but the FDAAA may not reduce the likelihood of dangerous or ineffective drugs remaining on the market as much as some people might have hoped. For one thing, it fails to fully overcome the problem of publication bias, in part because some studies do not have to be registered or have their results posted (such as preclinical or toxicity trials or trials for drugs and devices that fail to win approval for any indication). But access to full data is also constrained by other policies and laws, such as trade secrecy laws, that prohibit the release of some important clinical data -- data not carried by ClinicalTrials.gov.

A 2008 study by Erick Turner and colleagues in the New England Journal of Medicine highlights the problem of publication bias.[5] They found that 8% of antidepressant trials with negative findings were reported as negative, while positive trials were reported as such 97% of the time. Publication bias is not limited to antidepressants. A former clinical reviewer for the FDA, Turner told the BMJ that it is critical for researchers to be able to obtain complete study protocols and full datasets to be able to determine whether a study’s conclusions are valid. His concerns were highlighted by a 1999 study that looked at papers published in five top medical journals.[6] It found that in 18% to 68% of articles (depending on journal) the authors’ conclusions as stated in journal abstracts either were not supported or were contradicted by data given in the body of the article.

To overcome publication bias, researchers often request data from the FDA. That’s what Kirsch and his colleagues did. They filed a request under the Freedom of Information Act with the agency for all study data for the six most widely prescribed of the new generation antidepressants on the market. The FDA identified 47 relevant trials, but for reasons known only to itself the agency failed to release data from nine of the trials. The FDA told the BMJ that it would look into the reasons for the failure to release the nine studies -- which, as it happens, all yielded negative results. Although Kirsch was able to get some data from drug makers, he was ultimately unable to obtain data for 38% of test participants in trials of sertraline and 23% in trials of citalopram -- making it impossible to analyse two of the six antidepressants for overall efficacy.

Part of the problem here, of course, is that to be able to ask for study data researchers must know that a study has been done. Until now this information was often not easy to come by. Companies could legally refuse to reveal that they were even conducting certain studies of drugs already on the market. The FDAAA remedies this problem by requiring researchers and companies to post the primary and secondary outcome measures of their studies at the time of registration (generally within 21 days of enrolment of the first patient) and the results within one year (with extensions up to two years) of the time that the FDA approves the drug or some other action is taken or the trial is concluded.

This is all well and good. But the law does not require the registration of studies that were performed in the past for drugs that are currently on the market. Should researchers wish to perform studies like that of Kirsch and his colleagues, ClinicalTrials.gov won’t make it much easier for them. Nor does the FDAAA mandate the release of underlying data. Instead, only the key summary results must be posted. Deborah Zarin, director of ClinicalTrials.gov, says that the registry will require summary results for primary and secondary outcome measures in tabular form, though details are still being worked out.

In some circumstances, that may be enough. But there are a number of cases in which it isn’t. Without access to underlying data being fully available to researchers, patients are at risk of serious harm. Fred Geisler, a neurosurgeon at the Illinois Neuro-Spine Center, points to the use of high dose steroids in patients with spinal injury on the basis of a single, potentially flawed study funded by the National Institutes of Health (NIH).[7] Geisler believes that several thousand patients have died as the result of high dose steroids used to treat acute spinal cord injury. Two recent surveys show that most neurosurgeons share his concerns.[8] They think that steroids are either useless or dangerous; yet when asked why most of them continue to give the drug, they cite fears of malpractice on the basis of the standard of practice set by the NIH study. Several researchers have lobbied unsuccessfully for the release of the underlying data, without which they cannot verify their concerns -- or lay them to rest.

When should a clinical outcome be a trade secret?

Perhaps the most daunting obstacle to full access to data -- and one that the FDAAA doesn’t deal with -- is the trade secrecy rules that allow the FDA and industry to prevent disclosure of critical data, regardless of what they are now required to post on ClinicalTrials.gov. This aspect of drug regulation surfaced in 2005, with the death of 19 year old Traci Johnson, who committed suicide while serving as a healthy volunteer in a trial of duloxetine for a new indication, urinary incontinence. After requesting the data on duloxetine from the FDA, one of us (Lenzer) found that Johnson’s death, in addition to those of at least four other volunteers, was not included. When questioned, the FDA cited trade secrecy laws,[9] which permit companies to withhold all information, even deaths, about drugs that do not win approval for a new indication, even when the drug is already on the market for other indications.

The potential risk to patients should be obvious. Take valdecoxib, a COX 2 inhibitor that was withdrawn from the market because it posed a serious risk of heart attacks.[10] In 2004 the health research group of Public Citizen, a non-profit, public interest organisation in Washington, DC, tried to assess the drug’s safety profile. In 2001 the manufacturer applied for approval to market valdecoxib for four indications: osteoarthritis, dysmenorrhoea, adult rheumatoid arthritis, and acute pain. The FDA approved the drug for the first three indications but not for acute pain, and some of the information about the acute pain trials was withdrawn from the FDA website and a statement given that the information contained "trade secret and/or confidential information that is not disclosable."[11] That left the researchers and the public in the dark. Did the drug fail to relieve acute pain? Did it have serious side effects? It is precisely these failed trials that should be made public.

Similarly, when Peter Jüni and colleagues requested data from the FDA on valdecoxib, they too received a page of material so heavily censored that it looked more like a military intelligence document than a medical study.[3] Redaction, the censorship of certain information in a document that is released, is carried out under exemption 4 of the Freedom of Information Act, which allows trade secrets or "protected commercial information" to be concealed. But what commercial interest is there in censoring the clinical outcomes of participants in clinical trials? Should a death be considered a trade secret? Should lack of efficacy be a trade secret? Sidney Wolfe, director of Public Citizen’s health research group, said, "I’ve never been able to get any kind of protocol for what [FDA staff] are instructed to redact, but in general they redact way more than they should." He added, "Of course, it’s a catch 22, because if you don’t know what they are redacting you can’t argue that it should not have been redacted."

Alastair Wood, the head of the FDA’s advisory committee on the safety of COX 2 inhibitors, insists that there is no reason ever to redact clinical trial data. "There are some things like manufacturing data that might be commercially sensitive information," he said, "but that’s not in clinical trials." So, how can withholding statistical reviews and clinical data be justified? Without knowing what information is being withheld or the rules guiding redaction, the interpretation of what constitutes a trade secret seems itself to be a trade secret.

The universe of data

Turner, author of the study on publication bias and antidepressants, says that although the ClinicalTrials.gov database is a positive step towards greater transparency he believes that an excellent database already exists. The FDA database, he says, should be made available to researchers. Unlike the ClinicalTrials.gov database, which includes information only from studies that either started or had not concluded by the end of 2007, the FDA database can reach back to provide data on the overwhelming number of drugs already on the market. Furthermore, the FDA database contains a vast array of scientific reviews, while the results posted at ClinicalTrials.gov will be far more rudimentary. The FDA says that it is far too onerous to put all its material online, especially as it would have to comb through all the data for trade secrets. But, as a number of experts have pointed out, the burden on the FDA from future requests made under the Freedom of Information Act would be lessened if it posted all its data.

Beyond studies of drugs and devices already on the market -- or that will come onto the market -- there are data from failed trials of drugs that never win. These data should also be preserved for researchers. Currently, when a drug isn’t approved, all information about it is protected as a trade secret. At first blush it might seem unnecessary to learn about drugs that aren’t on the market at all. But, Wood says, "Suppose someone develops a drug for disease X and it either causes toxicity or doesn’t work. That’s something people should know."

But data from failed drug applications are protected as trade secrets so that drug companies aren’t put at a "competitive disadvantage" when other companies, learning of the initial studies, aren’t forced to expend the same "wasted efforts." Reproducing wasted efforts can mean wasted lives -- and wasted money. It also violates the covenant between human participants and researchers by allowing them to be exposed to unnecessary risks. This is a question of ethics. Should a second company be allowed to launch a trial of a similar drug without the benefit of knowing it may pose a danger to participants? There should be a centralised database for such preclinical and failed trails, Wood says. "We shouldn’t be walking down paths that lead to nowhere."

Ultimately, forcing companies to expend "wasted efforts" so as to protect commercial interests, redacting clinical information from studies of drugs and medical devices, and failing to insist that data derived from the sacrifices of human volunteers be placed in the public domain simply can’t be reconciled with what is in the public interest. Trial participants, as well as patients who take drugs and doctors who prescribe them, deserve nothing less than the assurance that all the news -- not just the good news -- has been carefully assessed.

References

1. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 2008;5(2):e45 doi: 10.1371/journal.pmed.0050045.[CrossRef][Medline]

2. Abramson J, Wright JM. Are lipid-lowering guidelines evidence-based? Lancet 2007;369:168-9.[CrossRef][ISI][Medline]

3. Jüni P, Reichenbach S, Egger M. COX 2 inhibitors, traditional NSAIDs, and the heart. BMJ 2005;330:1342-3.[Free Full Text]

4. Groves T. Mandatory disclosure of trial results for drugs and devices. BMJ 2008;336:170.[Free Full Text]

5. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252-60.[Abstract/Free Full Text]

6. Pitkin RM, Branagan MA, Burmeister LF. Accuracy of data in abstracts of published research articles. JAMA 1999;281:1110-1.[Abstract/Free Full Text]

7. Bracken MB, Shepard MJ, Holford TR, Leo-Summers L, Aldrich EF, Fazl M, et al. Administration of methylprednisolone for 24 or 48 hours or tirilazad mesylate for 48 hours in the treatment of acute spinal cord injury. Results of the third national acute spinal cord injury randomized controlled trial. National acute spinal cord injury study. JAMA 1997;277:1597-1604.[Abstract]

8. Lenzer J. NIH secrets: study break. New Republic 2006 Oct 30. www.ahrp.org/cms/index2.php?option=com_content&do_pdf=1&id=398.

9. Lenzer J. Drug secrets: what the FDA isn’t telling. Slate 2005 Sep 27. www.slate.com/id/2126918/.

10. Waknine Y. Bextra withdrawn from market. Medscape Today 2005 Apr 7. www.medscape.com/viewarticle/502642?rss.

11. Public Citizen Health Research Group versus FDA—complaint (26 Feb 2004). www.citizen.org/documents/ACF5CF.pdf.