More than 80 million adults in the United States are estimated to use some form of alternative medicine, from herbs and megavitamins to yoga and acupuncture. But while sweeping claims are made for these treatments, the scientific evidence for them often lags far behind: studies and clinical trials, when they exist at all, can be shoddy in design and too small to yield reliable insights.
Now the federal government is working hard to raise the standards of evidence, seeking to distinguish between what is effective, useless and harmful or even dangerous.
“The research has been making steady progress,” said Dr. Josephine P. Briggs, director of the National Center for Complementary and Alternative Medicine, a division of the National Institutes of Health. “It’s reasonably new that rigorous methods are being used to study these health practices.”
The need for rigor can be striking. For instance, a 2004 Harvard study identified 181 research papers on yoga therapy reporting that it could be used to treat an impressive array of ailments — including asthma, heart disease, hypertension, depression, back pain, bronchitis, diabetes, cancer, arthritis, insomnia, lung disease and high blood pressure.
It turned out that only 40 percent of the studies used randomized controlled trials — the usual way of establishing reliable knowledge about whether a drug, diet or other intervention is really safe and effective. In such trials, scientists randomly assign patients to treatment or control groups with the aim of eliminating bias from clinician and patient decisions.
Sat Bir S. Khalsa, the study’s author and a sleep researcher at the Harvard Medical School, said an added complication was that “the vast majority of these studies have been small,” averaging 30 or fewer subjects per arm of the randomized trial. The smaller the sample size, he warned, the greater the risk of error, including false positives and false negatives.
Critics of alternative medicine have seized on that weakness. R. Barker Bausell, a senior research methodologist at the University of Maryland and the author of “Snake Oil Science” (Oxford, 2007), says small studies often have a built-in conflict of interest: they need to show positive results to win grants for larger investigations.
“All these things conspire to produce false positives,” Dr. Bausell said in an interview. “They make the results extremely questionable.”
That kind of fog is what Dr. Briggs and the National Center for Complementary and Alternative Medicine, with a budget of $122 million this year, are trying to eliminate. Their trials tend to be longer and larger. And if a treatment shows promise, the center extends the trials to many centers, further lowering the odds of false positives and investigator bias.
For instance, the center is conducting a large study to see if extracts from the ginkgo biloba tree can slow the progression of Alzheimer’s disease. The clinical trials involve centers in California, Maryland, North Carolina and Pennsylvania and recruited more than 3,000 patients, all of them over 75. The study is to end next year.
Another large study enrolled 570 participants to see if acupuncture provided pain relief and improved function for people with osteoarthritis of the knee. In 2004, it reported positive results. Dr. Brian M. Berman, the study’s director and a professor of medicine at the University of Maryland, said the inquiry “establishes that acupuncture is an effective complement to conventional arthritis treatment.”
In an interview, Dr. Briggs said another good way to improve clinical trials was to ensure product uniformity, especially on herbal treatments. “We feel we have really influenced the standards,” she said.
Over the years, laboratories have found that up to 75 percent of the samples of ginkgo biloba failed to show the claimed levels of the active ingredient. Scientists doing a clinical trial have a large incentive to fix that kind of inconsistency.
Dr. Briggs said such investments would be likely to pay off in the future by documenting real benefits from at least some of the unorthodox treatments. “I believe that as the sensitivities of our measures improve, we’ll do a better job at detecting these modest but important effects” for disease prevention and healing, she said.
An open question is how far the new wave will go. The high costs of good clinical trials, which can run to millions of dollars, means relatively few are done in the field of alternative therapies and relatively few of the extravagant claims are closely examined.
“In tight funding times, that’s going to get worse,” said Dr. Khalsa of Harvard, who is doing a clinical trial on whether yoga can fight insomnia. “It’s a big problem. These grants are still very hard to get and the emphasis is still on conventional medicine, on the magic pill or procedure that’s going to take away all these diseases.”