Ketamine - Magic Antidepressant, or Expensive Illusion?

Not one but two new papers have appeared from the Carlos Zarate group at NIMH reporting that a single injection of the drug ketamine has rapid, powerful antidepressant effects.

One placebo-controlled study found a benefit in depressed bipolar patients who were already on mood stabilizers. The other found benefits in treatment-resistant major depression, though ketamine wasn't compared to placebo that time. Here's the bipolar trial:

There have now been several studies finding dramatic antidepressant effects of ketamine, a compound that all journalists seem contractually bound to call either a or a "club drug" or a "horse-tranquilizer". Great news?

If you believe it. But hold your, er, horses... there's a problem. As I said almost 3 years ago about one of the earlier ketamine trials:

In theory, the trial was double blind - neither the patients nor the doctors knew whether they were getting ketamine or placebo. But you'll know when you've been injected with 0.5mg/kg ketamine. You get high. That's why people take it [recreationally]. The study can't really be called double blind.

To their credit, Zarate et al did acknowledge this, and suggested that in future ketamine could be compared to another drug which produces noticeable effects. But they really should have done that to begin with.

It's now 2012, and there have still not been any published studies comparing ketamine to an active comparator i.e. a different drug that produces noticable psychoactive effects, to avoid unblinding. This means it's 12 years since the initial pilot report on ketamine in depression, and 6 years since the first large trial appeared.

The authors of the 2006 paper themselves wrote that "limitations in preserving study blind may have biased patient reporting... One potential study design in future studies with ketamine might be to include an active comparator" and suggested amphetamine for the big role.

Good idea. But six years later, we're still waiting. Which is really a bit silly. There have been dozens of papers written about the possible antidepressant effects of ketamine, from human trials to mouse work. That's a lot of research dollars (and dead mice) on something that might just be an active placebo.

Looking at the registered ketamine research on clinicaltrials.gov, I found that four active-comparator ketamine trials are in the pipeline (1,2,3,4), plus one cancelled (5). Only one is for depression though. The others being for OCD, cocaine dependence and suicidal ideation.

In all of these trials a benzodiazepine is the active comparator. Is that a good idea? Well, it's certainly better than nothing, but I wonder.

An active comparator has to "make an impression" on the patient equal to that produced by the real drug.  The null hypothesis, remember, is that ketamine has no specific antidepressant effect. That means it produces improvement through a combination of a) the placebo effect (expectation) and b) non-specific psychoactive changes.

More on that second one: any psychoactive drug might relieve depression by "taking your mind off it" and a change in mental state, as provided by a drug, also provides a demonstration that "I won't always feel this way". By showing that states of consciousness are products of brain chemistry, almost any drug could therefore offer a "glimmer of hope" to the depressed. If all this sounds very subjective, it is, but that's the point. Psychiatry is.

Would a benzo make as big an impression as 0.5 mg/kg ketamine IV? It's impossible to predict, really; so we'd need to ask people about the subjective strength of the drug effect. Personally, I worry that a lot of people just get sleepy on benzos and don't really feel much, so I'd prefer they used something a bit more hard-hitting like amphetamine, but maybe that's just me.

There's a deeper problem though. Suppose our ketamine-benzo trial finds no difference between ketamine and benzo. A critic could say, ah, but maybe it was just a "failed trial", so it doesn't overturn the positive studies. The patients weren't properly diagnosed, or weren't depressed enough, or were too depressed, etc.

Nitpicking such differences between studies is a well-practiced art.

Critics could complain in other ways if the study did find a benefit of ketamine. As I see it, the only way to settle this once and for all is to do a three-way randomized controlled trial - inactive placebo vs. active comparator vs. ketamine.

That way, if it's a failed trial, we'd know: there'd be no difference between ketamine and the inactive placebo. If there was a difference, but the active comparator was just as good as ketamine, that means it was all about nonspecific effets. Finally, if ketamine was better than the other two conditions, we could be pretty confident it was really working.

Also important is the question of volunteer expertise; subjects shouldn't be able to tell what drug they're on, but people who'd taken ketamine and/or the comparator drug before might be able to do that, so you'd want naive volunteers.

In conclusion: It's possible that ketamine has no specific antidepressant effects. To find out we ideally need a three-way trial, with both active and inactive comparators, careful monitoring of subjective drug effects and patient knowledge and expectations. Until that happens, I will be skeptical of ketamine in depression.

This is not because I just think it's impossible. Ketamine profoundly affects the brain in ways that we don't understand. I've suffered depression and I know it can come and go in a matter of minutes. So I think it's entirely possible that it works - but it's also possible that it's a nonspecific effect.

Look. I really want to know the answer to this. Both as a neuroscientist, and as a depression sufferer, this is very important to me. That's why we urgently need a good trial.

Link: See also the discussion and the comments over at The Neurocritic and this Scientific American piece which is pretty good except that it doesn't cover the active placebo issue.


Zarate CA Jr, Brutsche NE, Ibrahim L, Franco-Chaves J, Diazgranados N, Cravchik A, Selter J, Marquardt CA, Liberty V, and Luckenbaugh DA (2012). Replication of Ketamine's Antidepressant Efficacy in Bipolar Depression: A Randomized Controlled Add-On Trial. Biological psychiatry PMID: 22297150

Ibrahim, L., et al. (2012). Course of Improvement in Depressive Symptoms to a Single Intravenous Infusion of Ketamine vs Add-on Riluzole: Results from a 4-Week, Double-Blind, Placebo-Controlled Study Neuropsychopharmacology DOI: 10.1038/npp.2011.338

Take Your Placebos, Or Die

People who take their medication as directed are less likely to die - even when that "medication" is just a sugar pill.

This is the surprising finding of a paper just published, Adherence to placebo and mortality in the Beta Blocker Evaluation of Survival Trial (BEST). 

BEST was a clinical trial of beta blockers, drugs used in certain kinds of heart disease. The patients were aged about 60 and they all suffered from heart failure. Everyone was randomly assigned to get a beta blocker or placebo, then followed up for 3 years to see how they did.

Here's the big finding: in the placebo group of 1174 patients, the people who took all of their placebo pills on time (the good adherers), were significantly less likely to die than the patients who missed lots of doses. People who took over 75% as directed were 40% less likely to die than those with less than 75% adherence:

That's pretty interesting. The pills were placebos - they can't have had any benefit. So what's going on?

It gets even better. You might be tempted to write off these results as obvious: "Clearly, people who follow the study instructions are just 'healthy' people in other ways - maybe they take more exercise, eat better, etc. and that's what protects them."

Certainly, that's what I'd have said.

But what's remarkable is that when the authors corrected the statistics for all the confounding variables they measured - including things like age, gender, ethnicity, smoking, body mass index and blood pressure - it barely changed the effect. Some of the factors did correlate with adherence, but not in a way that it could explain the adherence effect on mortality.

This isn't the first study to find this effect. The authors themselves have already reported it, as have other researchers going back decades (many of which also tried, and failed, to explain it through confounding factors.) They say that it's unlikely to be a case of publication bias.

So what we have is a large effect, which cannot be causal, yet which can't be explained by any obvious confounds. Logically then, it must be the result of a confound (or more than one) that aren't obvious.

This is an important lesson. It's common for someone to do a study and find an interesting / scary / controversial correlation between two things. Often one is some kind of lifestyle factor, diet, environmental exposure, or whatever, and the other is some nasty disease. "And it wasn't explained by confounds!", such studies often conclude.

What the placebo adherence effect demonstrates is that there may be confounds no-one has thought of. They might even be impossible to measure. And if these mystery confounds can literally kill you, they can probably cause all kinds of other effects too.

In other words this illustrates the truism that correlation is not causation - not even when you're really sure it is...

Pressman, A., Avins, A., Neuhaus, J., Ackerson, L., and Rudd, P. (2012). Adherence to placebo and mortality in the Beta Blocker Evaluation of Survival Trial (BEST) Contemporary Clinical Trials DOI: 10.1016/j.cct.2011.12.003

Challenging the Antidepressant Severity Dogma?

Regular readers will be familiar with the idea that "antidepressants only work in severe depression".

A number of recent studies have shown this. I've noted some important questions over how we ought to define "severe" in this context, and see the comments here for some other caveats, but I'm not aware of any studies that directly contradict this idea.

Until now. A new paper has just come out which seeks to challenge this dogma - not the author's term, but I think it's fair to say that the severity theory is becoming a dogma, even if it's an evidence-based one (but then, all dogmas start out seeming reasonable).

However, while the new paper is interesting, I think the dogma survives intact.

The authors went through the archives of all of the trials of antidepressants for depressive disorders conducted at the famous New York State Psychiatric Institute for the past 30 years. They excluded any patients who were severely depressed, and just looked at the milder cases. The drugs were mostly the older tricyclic antidepressants.

With a mean HAMD17 score of about 14, the patients they looked at were certainly mild. By comparison, most trials today have a mean of well over 20, and according to the main studies supporting the severity dogma, you need a score of about 25ish to benefit substantially:

So what happened? They reanalyzed 6 trials with over 800 patients. Overall there was a highly significant effect of antidepressants over placebo in mild depression, with an effect size d=0.52, or about 3.5 HAMD points. This is actually better than most other studies have found in "severe" depression. If valid, these results would torpedo the severity theory.

This seems very interesting... but. There's a big but (I cannot lie). Although the authors say they wanted to include all the relevant trials from the NYSPI, they only had access to the data from 6. There were another 6 projects, but they were "pharmaceutical company studies from which data were not released to the investigators."

This pretty much wrecks the whole deal. If those 6 studies all found no benefit of the drug, the overall average results would be much less impressive. We have no way of knowing what those studies found, but I'd wager that most of them were negative, because of publication bias - we know that drug companies tend to publish positive studies and bury negative ones. Or at least they did, at the time these studies took place (there are better regulations now).

By contrast, severity dogma classic Kirsch et al (2008) avoided publication bias by looking at unpublished data. Fournier et al (2010), the other major severity study, didn't but the data were very similar to Kirsch et al so it's not hard to believe them.

So in my view, until we know what happened in the other 6 trials, we can't really interpret these results, and the severity theory stands.

Stewart, J., Deliyannides, D., Hellerstein, D., McGrath, P., and Stewart, J. (2011). Can People With Nonsevere Major Depression Benefit From Antidepressant Medication? The Journal of Clinical Psychiatry DOI: 10.4088/JCP.10m06760

Do Antidepressants Make Some People Worse?

Antidepressants may help depression in some people but make it worse for others, according to a new paper.

This is a tough one so bear with me.

Gueorguieva, Mallinckrodt and Krystal re-analysed the data from a number of trials of duloxetine (Cymbalta) vs placebo. Most of the trials also had another antidepressant (an SSRI) as well. And the SSRIs and duloxetine seemed to be indistinguishable so from now on I'll just call it antidepressants vs. placebo as the authors did.

People on placebo got, on average, moderately better over 8 weeks.

People on antidepressants fell into two classes. The largest class got, on average, a lot better. But about 25% did poorly, staying just as depressed as before. This "nonresponder" group did much worse than the placebo group - again on average. Here you can see the mean "trajectories" of depression symptoms (HAMD scores) in the three groups:

This raises the scary possibility that while antidepressants are helping some people, they're harming others. But hang on. It's complicated.

First off, maybe this is all a statistical illusion. When the authors say that the people on drug fell into two classes, what they mean is that when you try to model the data according to a certain mathematical model, assuming either 1, 2, 3 or 4 underlying classes, the 2 class solution was the best fit. While for placebo a 1 class solution was best.

We considered linear, quadratic, and cubic trends over time, with between 1 and 4 trajectory classes. We also considered piecewise models with a change point at 2 weeks, linear change before week 2, and quadratic change after week 2. The selection of the best model was based on the Schwartz-Bayesian information criterion and on the Lo-Mendell-Rubin (LMR) likelihood ratio test...

That's nice... but they don't present the raw data. They don't tell us whether, looking at the individual trajectories of people on antidepressants, you'd actually see two classes. What I want is a graph of how likely people are to get better by a certain amount. If Gueorguieva et al are right, I want it to look like this i.e. bimodal -

We're not shown this graph. I'll eat my hat if it does look like that, frankly, because if it did people would have noticed the bimodality in antidepressant trials ages ago.

True, statistical models can tell us things that aren't obvious by inspection, so even if this isn't what the data look like, they might still be right. It could be that the two "peaks" are so broad, and there's so much random noise, that they blur into one.

However, it's also true that you can fit an infinite number of models to any set of data and at some point you have to step back and say - am I making this more complicated than it needs to be?

It could be that a 2-class model is better than a 1-class model for the people on antidepressants, but only because they're both crap, and really, every patient has a different, unpredictable trajectory which is poorly captured by such models.

Let's assume however that this is true. What would it mean?

Firstly, the fact that one class of people on antidepressants does worse than people on placebo doesn't mean that antidepressants are harming them. The authors miss this point, when they say

there are 2 trajectories for patients treated with antidepressants and 1 trajectory for patients treated with placebo [so] some patients would seem to be more effectively treated with placebo than with a serotonergic antidepressant.

But that's fallacious. It treats a purely statistical entity as representing individual people. Suppose that what antidepressants do is to take people who, on placebo, would have improved a bit, and make them improve a bit more than they otherwise would have. You'd then end up with more people doing well, but also fewer people doing moderately because they'd have been "moved up" out of the middle ground.

That "nudging people off the fence" could lead to a bimodal distribution and two distinct classes. But in this case the people doing badly would have done badly either way. The drug didn't make them do badly, it just made doing-badly into a class. On the other hand it's consistent with antidepressants doing real harm. We can't tell.

We do know that other randomized controlled trials show very convincingly that in a small minority of people, mostly but not exclusively young people, antidepressants do worsen suicidal thoughts and behaviours. So it's plausible. But we just don't know yet.

What worries me is that this paper is the latest in a series of attempts  to use, well, creative statistical approaches to antidepressant trial data. This one is nowhere near as dodgy as the Cherrypicker's Manifesto I discussed last year, but it cites that paper and others by the same group. The first sentence of the Abstract of this paper makes the intention clear:

The high percentage of failed clinical trials in depression may be due to high placebo response rates and the failure of standard statistical approaches to capture heterogeneity in treatment response.

In other words, the reason clinical trials of new antidepressants often fail to show a benefit over placebo is not because the drugs are crap but because the statistics aren't subtle enough. And you can see where this is going: if only we could use statistical models to find the people who do benefit from antidepressants, and compare them to placebo, there'd be no problem...

Gueorguieva R, Mallinckrodt C, and Krystal JH (2011). Trajectories of depression severity in clinical trials of duloxetine: insights into antidepressant and placebo responses. Archives of General Psychiatry, 68 (12), 1227-37 PMID: 22147842

A Dangerous Truth about Antidepressants

An opinion piece by veteran psychiatrist and antidepressant drug researcher Sheldon Preskorn contains a remarkable historical note -

“A dangerous idea!” That was the response after a presentation I gave to a small group of academic leaders with an interest in psychopharmacology [over 15 years ago].

What evoked such a response? The acknowledgment that most currently available antidepressants specifically treat only one out of four patients with major depression based on the bulk of clinical trials data.

There was no argument about the accuracy of this statement, but...some claim it is “dangerous” to admit that the specific response rate to most antidepressants is 20%–30% because such an acknowledgment might undermine the value of antidepressant treatment.

By the "specific" response rate Preskorn means the number of depressed people who'll get better on antidepressants and who wouldn't have done so well on placebo. This rate is fairly low because, while most people get better on antidepressants, most of those improve on placebo as well.

Preskorn rejects the view that it's dangerous to acknowledge this:

...there are several problems with this reaction. First, it is hard to deny reality. The “placebo” response rate in antidepressant trials is arguably the most reproducible finding in psychiatry. Moreover, if available antidepressants were magic bullets, then polypharmacy would not be so common. Second, this reaction ignores the fact that antidepressants are tremendously valuable to the patients who specifically benefit from them...

Every treatment in every area of medicine has limitations. Acknowledging that fact should galvanize us to action. Denial on the other hand perpetuates the status quo.

Unfortunately, we're not told who these academic leaders were. I wonder if they included amongst their ranks some of the "key opinion leaders" in the field whose leadership proved rather less than ideal. The column is actually adapted from a 1996 article by Preskorn.

Preskorn is right, of course, that denying the fact that antidepressants are only substantially better than placebo in a fraction of people who get diagnosed with "depression" is wrong, and also misses the point: because hundreds of millions of Americans have diagnosable depression (due to the loose definition of "depression"), even if they only helped 1% of them, they'd still help over a million people.

But he doesn't mention that this approach was ultimately self-defeating. As a result of the failure to acknowledge that antidepressants are only helpful in some cases of depression (namely "severe" depression), these drugs became very widely used and - oh dear - people started saying that the drugs are being overused, and don't work in most people who take them.

Whoever could have seen that coming.

This has "devalued" antidepressants - and psychiatry itself - more than anything else has.

Preskorn SH (2011). What Do the Terms "Drug-Specific Response/Remission Rate" and "Placebo" Really Mean? Journal of psychiatric practice, 17 (6), 420-424 PMID: 22108399

Does MRI Make You Happy?

A startling new paper from Tehran claims Antidepressant effects of magnetic resonance imaging-based stimulation on major depressive disorder.

Yes, this study says that having an MRI scan has a powerful antidepressant effect.

They took 51 depressed patients, and gave them all either an MRI scan or a placebo sham scan. The sham was a "scan" in a decommissioned scanner. The magnet was off but they played recorded scannerish sounds to make it believable. Patients were blinded to group.

They found that people in the scanner group improved much more than those in the sham group over two weeks. Actually there were two different kinds of scans, T1 structural MRI and EPI functional MRI, but they were the same:

Now, if this is true, it's huge. Obviously. For one thing, it would undermine the whole premise of functional MRI, which is that it's a method of recording brain activity. If it's also stimulating the brain in some way at the same time, then it would make it hard to interpret those activations. In particular it would cast all the studies using fMRI in depression into doubt.

So is it true? I can't see any obvious flaws in the design. Assuming that the authors are right when they say that "patients could not distinguish the difference between the actual and sham MRI scan", i.e. assuming that the blind was truly blind, then the methodology was sound.

But let's look at the statistics. The paper is full of very impressive p values less than 0.001 but those turn out to all be referring to the changes within each group, and those changes are fairly meaningless. What matters is the differences in the groups and

Changes in BDI scores (between baseline and day 14) were significantly different among the three studied groups (F=5.48, p=0.007 overall) using ANOVA, and between the DWI group vs. Sham and T1 vs. Sham (p<0.05) using post hoc tests. Changes in HAMD24 scores (between baseline and day 14) were also compared among the 3 groups using ANOVA but the level of significance was slightly above the significance threshold (F=2.89, p=0.06).

Which is rather less convincing. There was a close-to-significant group difference in the HAMD24, and a significant but only just effect on the BDI. Remember that there were only 17 people in each group.

I'm inclined to think that this is one of the 5% of experiments which will produce a nominally significant result even assuming everything goes to plan and there are no confounds. My suspicion is that everyone in the trial got better (they were all on antidepressants, plus there's the placebo effect and the effect of time) - except a small number of people who didn't improve. And by chance they were all in the sham group.

The reason I'm skeptical is that I just can't see a plausible mechanism. The authors suggest that MRI scans might stimulate the brain in a similar way to TMS and that this could have antidepressant effects.

But there's a lot of problems with this: 1) the evidence is questionable whether TMS even works for depression 2) the magnetic stimulation of the brain generated during MRI is much weaker than in the case of TMS and 3) if MRI really stimulated the brain like TMS, then, like TMS, it would have a risk of triggering seizures in people with epilepsy. But it doesn't.

Vaziri-Bozorg SM, et al (2011). Antidepressant effects of magnetic resonance imaging-based stimulation on major depressive disorder: a double-blind randomized clinical trial. Brain imaging and behavior PMID: 22069111

Placebos And The Brain's Own Pot

According to a neat little new paper, the placebo effect relies on the brain's own marijuana-like chemicals, endocannabinoids.

Or rather, some kinds of placebo effects involve endocannabinoids. It turns out that "the placebo effect" is not one thing.

The authors, led by Fabrizio Benedetti, have previously shown that placebo "opioids" - i.e. when you expect to get a painkiller such as morphine, but actually it's just water - relieve pain via the brain's own opioid system (endorphins). Blocking endorphins with certain drugs blocks the power of placebo morphine.

But there are many painkillers that aren't opioids, leaving open the question of whether all placebo effects on pain are mediated by endorphins.

The new study claims that endocannabinoids are involved in non-opioid placebo analgesia. They used rimonabant, a weight loss drug that was pulled from the market shortly after it appeared, because it caused depression. Rimonabant worked by blocking CB1 receptors, which are the main target of the psychoactive chemicals in cannabis - and also key players in the endogenous cannabinoid system.

Here's the headline result:

The graph on the left shows the relationship between the pain relieving power of morphine, and the pain relief caused by placebo "morphine" given on a subsequent day. As you can see, there was a strong correlation. People who had a strong response to real morphine, later responded well to the fake morphine. But rimonabant had no effect at all.

Pain relief was measured using tolerance to the pain caused by a tightly fitting tourniquet.

However, rimonabant did have a strong effect on the placebo response to a different drug, ketorolac, which is related to the better-known ibuprofen (Nurofen). As you can see in the graph on the right, people given rimonabant had a much lower response to the placebo "ketorolac".

In other experiments, they showed that rimonabant alone had no effect on pain tolerance.

This is a nice result. It shows that the placebo effect is not a single thing, but that it depends upon the nature of the drug that you believe you've got. It also reminds us that the placebo effect is not some magical power of mind-over-matter, but is in fact, well, matter-over-matter.

Interestingly, ketorolac has no effect on endocannabinoids, or at least no direct effect. The mechanism of action, which is fairly well understood, has nothing to do with cannabinoids. Yet placebo "ketorolac" still seems to set endocannabinoids buzzing.

Benedetti F, Amanzio M, Rosato R, & Blanchard C (2011). Nonopioid placebo analgesia is mediated by CB1 cannabinoid receptors. Nature medicine, 17 (10), 1228-30 PMID: 21963514