Skip to main content
Hard to Study: The Difficulty in Measuring Marijuana’s Value
(David McNew/Getty Images)

Hard to Study: The Difficulty in Measuring Marijuana’s Value

David W. Murray

(This essay is a response to a bevy of news accounts on claims about the efficacy of “medical marijuana” in treatment of post-traumatic stress disorder, the strange tale of researcher Dr. Sue Sisley, and charges of government suppression of access to research-grade marijuana. It posits that there are dangers in “science by press release,” while charting the difficulty in validating smoked marijuana’s potential as a therapeutic treatment. After examining the medical complexities, methodological challenges, and misleading medical claims made by drug legalization advocates, the essay concludes with a warning about “false positives” in medical research and argues against “down-scheduling” marijuana from its current status as a controlled substance.)

An effort to study marijuana’s potential in treating veterans for post-traumatic stress disorder (PTSD) has garnered attention from both the Washington Post and New York Times. There are strong similarities between the Post article from March 21, 2014 (Ariana Eunjung Cha, “Marijuana Research Hampered by Access from Government and Politics, Scientists Say”) and the Times story of August 8, 2014 (Serge F. Kovaleski, “Medical Marijuana Research Hits Wall of U.S. Law”). Both feature Dr. Sue Sisley, a clinical assistant professor at the University of Arizona, who sought to lead the study. And both support the assertion by legalization advocates that proof of marijuana’s value is lacking because the government stands in the way of the necessary research.

The Times termed such impediments “a major reason” for the lack of research, while Dr. Sisley claimed that the involvement in marijuana distribution of the Drug Enforcement Administration (DEA), which is charged with overseeing controlled substances, was unsettling: “Nobody could explain it—it’s indefensible. The only thing we can assume is that it is politics trumping science” (Rob Hotakainen, “Feds Accused of Steering Funding to Anti-Pot Researchers,” McClatchy, March 19, 2014).

Finally, both the Post and the Times support “rescheduling” marijuana (that is, changing the drug to a less stringent level of control under the Controlled Substances Act) in hopes of making it more readily available.

The Times cites the evaluation of the Department of Health and Human Services (HHS) that “if there is any future for marijuana as a medicine, it lies in its isolated components, the cannabinoids and their synthetic derivatives,” and not in smoked marijuana. According to the Times, this is a position taken by a political agency, whereas “scientists say this position has had a chilling effect on marijuana research.”

The Times neglects the fact that the HHS statement is derived from an analysis of marijuana’s potential issued by the Institute of Medicine (IOM) of the National Academy of, well, Sciences. (Surely the Times does not mean to imply that in order to be considered a scientist, a person must agree with its editorial stance on marijuana?)

It is further interesting that both the Post and the Times mention the financial support for the study by the Multidisciplinary Association for Psychedelic Studies (MAPS) but fail to point out that MAPS was for years funded by Peter Lewis, the billionaire drug legalizer, who died last year. MAPS is involved in numerous projects supporting the use of controlled substances as PTSD medicines. Perhaps not all the politics is on the government side?

PTSD aside, we face a serious scientific question: Why is it so hard to determine whether marijuana has legitimate medical value? It has been fifteen years since the Institute of Medicine gave a cautious endorsement for developing medicines from compounds found in marijuana (in S. J. Watson, J. A. Benson Jr., and J. E. Joy, “Marijuana and Medicine: Assessing the Science Base: A Summary of the 1999 Institute of Medicine Report,” Archives of General Psychiatry 57, no. 6 [June 2000]: 547–52).

While the IOM asserted that “marijuana is not a modern medicine” and professed doubt that smoking the raw weed would ever become an approved therapeutic intervention, it nonetheless described several conditions for which the medical potential could become a reality.

Of course, there are already approved medicines, such as Marinol, which are based on synthetic versions of cannabinoids found in marijuana. However, their availability does not seem to quiet marijuana activists.

In 2006, the Food and Drug Administration (FDA) reiterated the federal government’s position that marijuana (particularly as smoked leaves) should remain a Schedule I controlled substance, which is defined as a substance having no proven therapeutic value while presenting considerable risk of abuse.

Fundamental safety and efficacy remain unproven by FDA standards, yet reports, anecdotes, and even some proper studies in both animals and humans have kept the prospect of additional medical value alive.

Why has there not been a more definitive answer?

Marijuana advocates scoff at marijuana’s placement in Schedule I. They argue that drugs widely regarded as more dangerous (such as methamphetamine or cocaine) are Schedule II substances, meaning that they can be prescribed for certain medical conditions and are more readily available for research.

This argument overlooks the fact that scheduling decisions are based on two dimensions, and a demonstrated value in treatment can lessen concerns over abuse and even questions of safety, as we find for both methamphetamine and cocaine. As yet, marijuana fails on this second criterion.

All agree that more research is necessary, but few outside the research community recognize the complexity of the undertaking. Is there, as some argue, government complicity in denying marijuana’s true worth, or is the problem rather something unusual about marijuana that accounts for the lack of a clear resolution of the question?

Some researchers note the difficulty in acquiring research-grade marijuana, which must be secured from one official government source. The University of Mississippi holds the exclusive license to produce research marijuana in accord with international treaty requirements and is supported by the National Institute of Drug Abuse (NIDA). In addition, both the departments of Health and Human Services (through NIDA and the FDA) and Justice (through DEA) must approve research applications in order for the drug to be released.

While the potential for politicization is a concern, government critics should acknowledge that there are inherent difficulties in carrying out marijuana trials and that they account for much of the trouble researchers encounter in acquiring the drug.

First, there is the problem of the multiplicity of medical conditions for which marijuana advocates claim therapeutic benefit. The list of conditions for which one may acquire a “medical marijuana” card, for instance, is remarkably long and diverse and includes spasticity, pain, Crohn’s disease, HIV/AIDS wasting, Alzheimer’s disease, PTSD, vomiting, anxiety, sleep disorders, and glaucoma.

As Yale University psychiatrists noted in an article in the June 2014 Journal of the American Medical Association (JAMA) warning about marijuana as medicine, “the many conditions for which medical marijuana is approved have no common etiology, pathophysiology, or phenomenology, raising skepticism about a common mechanism of action” (Samuel T. Wilkinson and Deepak Cyril D’Souza, “Problems with the Medicalization of Marijuana,” Journal of the American Medical Association 311, no. 23 [June 18, 2014]: 2377–78).

To be fair, receptors for natural endocannabinoid neurotransmitters (such as anandamide) are widely distributed in several systems in mammals, including the brain and the immune system. Hence, widespread therapeutic action is not inconceivable, though it is unlikely for a single substance to address simultaneously and by the same mode of administration biological systems as diverse as neuropathic pain, the immune system, and memory formation. But this model of “one drug, many cures” is more reminiscent of faith healing or folk remedies than of targeted medical action with an understood biological mechanism.

A further problem for clinical trials is that there is no clearly optimal dose of marijuana for the various conditions for which it is approved. As the Yale psychiatrists note, “The concentration of THC (delta-9 tetrahydrocannabinol) and other cannabinoids in each marijuana cigarette, the size of cigarettes, and the quantity of smoke inhaled by users can vary considerably.”

This problem holds for studies of both potential benefits and risks. As cardiologist Dr. John Erwin told Medpage Today in response to a study linking marijuana to adverse cardiac events, “Marijuana use, in the form of smoking, is extremely difficult to study given lack of strong control groups with extreme variation in THC concentrations and the concomitant use of other drugs” (Todd Neale, “Cannabis and the Heart: Is There a Risk?” Medpage Today, updated April 24, 2014).

The variation in composition, purity, and concentration of the active constituents of marijuana makes it unlike most drugs produced for research purposes. As the Yale doctors note, “This is especially problematic because unlike most other prescription medications that are single active compounds, marijuana contains more than 100 cannabinoids, terpenoids, and flavonoids that produce individual, interactive, and entourage effects. Although THC is believed to be the principal psychoactive constituent of marijuana, other cannabinoids present in marijuana may have important effects that may offset THC’s negative effects.”

Another issue is the problem of recruiting patients for proposed clinical trials. Even the best studies, well designed and led by recognized experts, have difficulty securing enough patients who have serious medical conditions, in some measure because their own physicians are reluctant to see them take the risk.

Patients would have to suspend their current medications and gamble on an unknown. In addition, they would likely be subjected to smoke containing multiple compounds suspected of being carcinogens and to the psychological impact of intoxication, which many find disturbing. Furthermore, they would be taking a substance associated with immune suppression.

For cancer patients or those with HIV/AIDS, this risk is substantial, and their physicians often recommend against participation. This means that trials are commonly limited to a small number of subjects, which limits their statistical power.

Moreover, it is difficult to maintain the double-blind feature for a placebo-controlled protocol. Many patients will be experienced with marijuana’s effects on their thoughts, and hence they will know if they are being given a genuine marijuana cigarette or a placebo. The situation has been compared to a study providing vodka as a potential treatment, with distilled water being the placebo. They may look the same, but the experienced patient is well aware of the difference.

It is a bit of a double bind, but researchers also worry that patients who are marijuana naïve, and hence unaware of whether they are getting the real thing or the placebo, might not know how to self-administer a smoked “dose” in a manner that is regular and effective. All in all, offering a smoked drug for a clinical trial is fraught with problems.

These limitations help explain why NIDA-funded research on potential benefits from marijuana compounds most frequently uses formulations that are not smoked plant material.

There are currently twenty-eight separate NIDA-funded active grants studying marijuana as medicine. They are divided into studies of the plant itself, of purified chemicals extracted from the plant, of synthetic compounds comparable to the organic plant’s components, or of the naturally occurring neurotransmitters, endocannabinoids. Thirteen of these grants involve human subjects, while the rest are animal studies.

Only two grants involve the raw plant and smoked marijuana cigarettes. Some people interpret the relative lack of research on smoked marijuana as evidence of “government suppression” of marijuana research. This interpretation is misplaced. Most research is done on purified, vaporized, or synthetic forms of the drug not because NIDA is threatened by cannabinoids; rather, it reflects the inherent requirements of a sound research protocol.

In response to pressure by activists, some in Congress are determined to change marijuana’s status and to “down-schedule” it into the less restrictive Schedule II, which is for drugs with high abuse potential offset by some known therapeutic effectiveness, as demonstrated by clinical trials. As we have seen, however, this apparent solution is misdirected. Treating marijuana as a Schedule II drug may open up access, as intended, allowing more research. But it also may allow other developments.

According to the Times, “moving the drug to a less restrictive category could do more than reduce some obstacles to research, proponents say. It would be a significant step toward allowing doctors around the country to prescribe the drug” (Kovaleski, “Medical Marijuana Research Hits Wall”).

Perhaps a further effect of such down-scheduling would be that medical marijuana would be covered under the Affordable Care Act (ACA). That could be positive news for those with the highest rates of marijuana use, Americans aged 18-25. Since the ACA provides adult children up to age 26 with coverage on a parent’s plan, if marijuana were available through prescription, “medicinal” use of the drug might be covered by that plan.

The Times adds that “federal lawmakers say [down-scheduling] could also permit medical marijuana operations that are legal at the state level to take business deductions on their federal taxes” (ibid.).

But rescheduling does not respond to the actual difficulty, which lies in the nature of the marijuana trials themselves, especially those using smoked leaves. Moreover, rescheduling creates multiple unintended problems.

First, it would amplify discord between the United States and the rest of the international community, which by treaty regards marijuana as properly belonging in the restrictive Schedule I. Given that proponents have offered no new medical facts in support of rescheduling, a move by Congress to reschedule would be revealed to those charged with overseeing the integrity of the treaties as stemming from openly political, not scientific motives.

Second, if marijuana were placed in Schedule II, the increased access might abet agenda-driven research and publications by advocacy organizations that conduct “science by press release.” Those seeking to sway public opinion in favor of marijuana’s supposed benefits would likely capitalize on the opportunity, generating careless studies that would not withstand rigorous scientific review but would suffice to create positive headlines, softening up public opinion in favor of legalization.

Some of this complexity is already on display in connection with the study of veterans with PTSD. From the perspective of research methodology, this is a difficult study. PTSD is a psychological disorder without a clear etiology or resolution. It is subjective in the sense that it depends on the personal accounts of those who suffer from it.

This means that a medicine’s efficacy cannot be shown by a simple readout of a number, as with a blood-pressure medication. Further, the exact defining features of PTSD are ambiguous and shifting.

Can marijuana be helpful to sufferers? Anecdotes suggest that it might, but there are also clear risks in using a substance associated with depression, psychosis, sleep disruption, and even hallucination. Some physicians treating this disorder express concern that using marijuana would be counterproductive and might do more harm than good.

The protocol for PTSD researchers seeking access to marijuana has gone back and forth for several years with the HHS review committees, whose members point out deficiencies, and MAPS, the funder, which then revises it and tries again for approval. Recently, the HHS panel relented, granting Dr. Sisley permission to proceed with the marijuana trials, though concerns remain. (Now, in what might be a connected development, Dr. Sisley has found her one-year contract with the University of Arizona terminated, though the reasons are in dispute.)

The involvement of MAPS shows the complexity of the issue. The group has been central to several efforts to widen access to marijuana and other psychedelic drugs. In this context, it has been a party to an ongoing lawsuit against NIDA and the DEA that seeks to break NIDA’s “monopoly” over research-grade marijuana.

MAPS also funded previous studies of drugs other than marijuana for veterans with PTSD. It has been instrumental in supporting the use of MDMA (Ecstasy), also a Schedule I controlled substance, in PTSD research.

Skeptics note that the circle of researchers involved in these studies is small but apparently united by two common factors: MAPS funding and a quest for illicit psychedelic substances said to hold the potential for healing wounded veterans.

Hence, again there is reason for concern that political agendas might gain priority over dispassionate scientific study and that any glimmer of positive outcomes, even prior to peer-review publication, might be used to produce positive news stories, which all too often prove to be premature when the actual results become known.

The Washington Post recently ran a story on yet another member of the MAPS circle, a post-doctoral researcher at Imperial College London, who expressed his excitement over illicit mushrooms (Rachel Feltman, “Psychedelic Mushrooms Put Your Brain in a ‘Waking Dream,’ Study Finds,” July 3, 2014).

The MDMA research, the mushroom investigation, and the marijuana study are all supported by funding from MAPS, but regardless of the drug, the subjects, claims, and objectives seem strikingly similar.

It is hard not to see this “multiple drugs, one cure” set of undertakings as the obverse counterpart to the claims noted above regarding the multiplicity of conditions for which marijuana is the “answer.” The worry is that PTSD is a favored condition for political reasons as much as purely medical ones and that the several psychotropic drugs that are said to offer benefit are currently controlled substances in need of positive press.

Is there any real scientific downside to postulating near-miraculous efficacy for a drug addressing so many difficult conditions? In general, there are two threats between which any legitimate researcher must steer: the risk of a “false negative” and that of a “false positive.”

If research were to be so tightly constrained that a genuinely positive result was overlooked or prematurely dismissed, this would be a false negative. A chance to relieve suffering would be missed and a drug of value passed over. This outcome was apparently in Dr. Sisley’s mind when she told Arizona Public Media that “the bigger tragedy” of her dismissal is that “they’ve dashed the hopes and dreams of half a million veterans” (Maria Inés Taracena, “Fired Medical Pot Researcher: ‘We Were on Cusp of Implementing UA Study,’” July 2, 2014).

But the assumption that a false negative outcome is a tragedy depends on a bit of question begging. The hopes and dreams of veterans were “dashed” only if marijuana would clearly have provided therapeutic benefit. Alternatively, since marijuana is known to pose considerable psychological risk to users, some veterans may well have been spared exposure to a dangerous substance that could have done them harm.

False negatives, when they are real, are a missed opportunity, but in the long run, science will likely arrive at the right answer.

False positives, however, present a more serious risk. With a false positive, researchers wrongly conclude that a therapeutic answer has been found. “Marijuana cures PTSD” might be an example. This would be a false positive discovery if in reality the marijuana provided no therapeutic effect.

The danger in this case is larger than just the risk that suffering patients might be given something of no value (or a substance that might pose a risk to them physically). The threat of false positives is that they harm entire pathways of research by offering what seems to be an understanding of disease etiology and mechanism of action that is, in fact, a dead end. It can take years of research before this fundamental misdiagnosis is identified and corrected, and much research money and time is effectively wasted on a mirage. That is the genuine cost of “advocacy research” when it gains dominance.

With controlled substance drugs, that impact can be coupled with a further cost: the undermining of deterrence and prevention. When an advocacy group’s press release makes a drug appear to be safe and even therapeutic, this sends a message regarding the acceptability of use, even when the drug remains dangerous. Surely the widespread public opinion that marijuana is a valid medicine has played a role in undermining deterrence and perceptions of risk among youth.

Marijuana is an intoxicant. It can make people feel better, but that may not be the same as actually making a patient healthier. Alcohol also makes people feel better for a while. Perversely, some users think of alcohol as a stimulant because they become more active and happy when they drink it, but physicians know that over longer periods of time, alcohol acts as a depressant, and intoxication is only masking a condition, not healing it.

For all of marijuana’s promise, it poses the same risk: its THC intoxicant may mask for a time the impact of a disease, in which case, patients have been misled. Alternatively, marijuana may hold genuine promise as a valuable medication, able to relieve suffering and provide therapeutic value for disease conditions.

At this point, the latter has not been proven. More research is assuredly needed. But that research must be held to the standards of science and not subordinated to wish fulfillment or political agendas. Continuing Schedule I status for marijuana remains sound policy, unless and until medical science demonstrates the contrary.

Related Articles

Should Drive-Thru Windows Be Banned?

Hank Cardello

In an interview with Rob Ferrett on Wisconsin Public Radio, Hank Cardello discusses cities banning new drive-through windows....

Listen Now

The Realignment - Ep. 8: Bhaskar Sunkara, Debating the Socialist Manifesto

Saagar Enjeti & Marshall Kosloff

Bhaskar Sunkara, founding editor and publisher of the socialist magazine Jacobin, joins The Realignment to debate Marshall and Saagar on the vices and...

Listen Now

The Realignment - Ep. 5: Megan McArdle Defends the Market

Saagar Enjeti & Marshall Kosloff

Washington Post columnist Megan McArdle likens capitalism to Winston Churchill’s famous saying about democracy, saying it’s the “worst system except a...

Listen Now