3D fMRI Promises Deeper Neuroscience

A new approach to fMRI scanning offers a three-dimensional look at brain activation.

fMRI is already a 3D technique, of course, but in the case of the cerebral cortex - which is what the great majority of neuroscientists are most interested in - the 3D data are effectively just 2D images folded up in space.

The cortex can be thought of a big sheet crumpled up into the shape of a brain, and it's possible to use software to 'unfold' the cortex into a 2D map for the purposes of fMRI data visualization. It's more informative because it shows you which areas are closest to each other.

But the cortex isn't really a sheet. It's more like six sheets stacked up - the cortex is formed of six layers, each with distinct cell types, connections, and functions. The difference between Layer III and Layer V of a particular cortical area is, in some ways, as important as the difference between two adjacent areas, but fMRI can't distinguish them because they're too close together.

Until now. In a new paper, Minnesota neuroscientists Olman et al say that they've given fMRI a  third dimension - Layer-specific FMRI reflects different neuronal computations at different depths in human v1.

They used a powerful 7 Tesla MRI scanner and a T2-weighted 3D GRASE pulse sequence that provides extremely high spatial resolution (0.7 mm - whereas 3 mm is the fMRI standard). The trade-off was that they were only able to scan a small chunk of the brain, namely the primary visual cortex. However, this is a good place to start, because it has a very well-understood layering system.

Does it work?

Probably, although the data they present are a little messy. By showing volunteers various kinds of pictures, they tried to find evidence of layer-specific visual cortex activation. However, most of the stimuli they used activated all layers equally. In my view the best evidence for layer-specific results was this, from two people -

Showing that the upper layers of the cortex were more activated by colourful stimuli that activate "P cells" compared to rapidly changing stimuli that act on "M cells".

We'll need more data to be sure that this technique works, but if it does, it promises some awesome science in the future. Still, it's not all good news for us neuroscientists. We'll have to relearn all the facts about cortical layers that most of us studied in Neuroscience 101 and then promptly forgot about.

Someone remind me, is Layer I or VI the top one...?

Olman CA, Harel N, Feinberg DA, He S, Zhang P, Ugurbil K, and Yacoub E (2012). Layer-specific FMRI reflects different neuronal computations at different depths in human v1. PloS one, 7 (3) PMID: 22448223

Brain Scanning - Just the Tip of the Iceberg?

Neuroimaging studies may be giving us a misleading picture of the brain, according to two big papers just out.

By big, I don't just mean important. Both studies made use of a much larger set of data than is usual in neuroimaging studies. Thyreau et al scanned 1,326 people. For comparison, a lot of fMRI studies have more like n=13. Gonzalez-Castillo et al, on the other hand, only had 3 people - but each one was scanned while performing the same task 500 times over.

Both studies found that pretty much the whole brain "lit up" when people are doing simple tasks. In one case it was seeing videos of people's faces, in the other it was deciding whether stimuli on the screen were letters or numbers.

With all that data, the authors could detect effects too small to be noticed in most fMRI experiments, and it turned out that pretty much everywhere was activated. The signal was stronger in some areas than others, but it wasn't limited to particular "blobs".

So conventional fMRI experiments may just be showing us the tip of the iceberg of brain activity. In a small study, only the strongest activations pass the statistical threshold to show up as blobs, but that doesn't mean the rest of the brain is inactive. It just means it's less active. The idea that only small parts of the brain are 'involved' in any particular task may be a statistical artefact.

In fact, I wonder if the whole idea of treating statistically significant blobs as different from nearly-significant areas is itself a form of the error of interacting effects?

As if that wasn't enough, Gonzalez-Castillo further show that there are lots of activations in the brain - even to very simple stimuli - that might go undetected in conventional studies, because they don't follow the time-course predicted by the usual models.

Have a look -

This shows the average neural activation from various regions of the brain during a letter-number task. The two areas I've highlighted in red are the primary visual cortex, and they do follow the expected 'boxcar' pattern - the brain is active when the stimuli are on the screen, inactive when they're not. But you can see that all kinds of other brain areas are also responding to the stimuli - just in different ways.

For example, the left primary motor cortex was activated during the task. That area controls the right hand, and that makes sense, as people responded by pressing buttons with the right hand. But interestingly, the same area on the other side of the brain was deactivated at exactly the same time, even though people weren't doing anything with their left hand.

These papers illustrate the fact that conventional fMRI is a blunt instrument that often only tells us about the most straightforward events that happen in the brain. A bit like how we only hear the shouts and screams from through our neighbor's walls, not their normal conversations, which aren't loud enough to reach our ears.

That's the bad news, but every blob has a silver lining. fMRI is clearly more powerful than most neuroscientists have realized, and this holds out hope for cracking some of the trickiest questions. As Gonzalez-Castillo et al put it

This result helps narrow the gap between thousands of fMRI manuscripts showing limited activation in response to tasks and cognition theories that defend that cognition—understood as the process of “configuring the way in which sensory information becomes linked to adaptive responses and meaningful experiences”—can only result from the distributed collaboration of primary sensory, upstream and downstream unimodal, heteromodal, paralimbic, and limbic regions... [we were able to] switch from a regime where activity detection relates primary to sensory processing to a more sensitive regime, where activity detection includes also cognitive processes with subtler BOLD signatures.

Link: See also the interesting discussion here: Surely, God loves the .06 (blob) nearly as much as the .05.


Thyreau, B., Schwartz, Y., Thirion, B., Frouin, V., Loth, E., Vollstädt-Klein, S., Paus, T., Artiges, E., Conrod, P., Schumann, G., Whelan, R., and Poline, J. (2012). Very large fMRI study using the IMAGEN database: Sensitivity–specificity and population effect modeling in relation to the underlying anatomy NeuroImage DOI: 10.1016/j.neuroimage.2012.02.083

Gonzalez-Castillo, J., Saad, Z., Handwerker, D., Inati, S., Brenowitz, N., and Bandettini, P. (2012). Whole-brain, time-locked activation with simple tasks revealed using massive averaging and model-free analysis Proceedings of the National Academy of Sciences DOI: 10.1073/pnas.1121049109

The Blinking Brain - A Problem For fMRI?

Every time we blink, a wave of activity sweeps through our brain - and this could be a serious problem for some fMRI researchers.

French neuroscientists Hupé et al report on A BOLD signature of eyeblinks in the visual cortex. They found that spontaneous blinks are associated with a neural activation pattern over the occipital cortex areas responsible for processing vision.

In many ways this is not surprising - when you blink, everything goes dark, and then lights up again, all within a fraction of second, which means that blinks are a kind of very dramatic visual stimulus, equivalent to a big black object suddenly appearing and then vanishing again. However, it's long been believed that blink suppression mechanisms in the eye and brain somehow block out the responses that would otherwise happen during a blink.

Don't be so sure, say Hupé et al. In an elegant experiment, they showed volunteers a standard set of visual stimuli during fMRI scanning, while recording blinks using an eye tracking camera. Then they simply treated the blinks as events, and used standard analysis methods to find neural activation associated with them.

Blinks caused a significant BOLD response over a number of "visual" areas.

Compared to the "real" visual stimuli in the task, the blink signal was less extensive, but no less strong.

So what? The great majority of fMRI experiments don't use eyetracking to measure blinks, so this study raises the scary possibility that blinks could lie behind some of the "stimulus-related" activations that we all know and love. It would be a problem if subject blinks were correlated with the stimuli or tasks, which they might be, because blink rate may vary with our psychological state.

I don't think we should be too worried yet. The blink blobs were essentially confined to parts of the visual cortex. So any study that's not focussed on vision is probably in the clear (although that's just the average response: in some individual subjects, the activations were a lot wider.)

However, as the authors point out, there is a risk that alterations in blink rate, caused, perhaps, by emotional or cognitive stress, might be wrongly "found" to be causing visual cortex activation, which might call into question claims of "top-down" influences on early visual cortex... oh dear.

Hupé, J., Bordier, C., and Dojat, M. (2012). A BOLD signature of eyeblinks in the visual cortex NeuroImage DOI: 10.1016/j.neuroimage.2012.03.001

Ketamine - Magic Antidepressant, or Expensive Illusion?

Not one but two new papers have appeared from the Carlos Zarate group at NIMH reporting that a single injection of the drug ketamine has rapid, powerful antidepressant effects.

One placebo-controlled study found a benefit in depressed bipolar patients who were already on mood stabilizers. The other found benefits in treatment-resistant major depression, though ketamine wasn't compared to placebo that time. Here's the bipolar trial:

There have now been several studies finding dramatic antidepressant effects of ketamine, a compound that all journalists seem contractually bound to call either a or a "club drug" or a "horse-tranquilizer". Great news?

If you believe it. But hold your, er, horses... there's a problem. As I said almost 3 years ago about one of the earlier ketamine trials:

In theory, the trial was double blind - neither the patients nor the doctors knew whether they were getting ketamine or placebo. But you'll know when you've been injected with 0.5mg/kg ketamine. You get high. That's why people take it [recreationally]. The study can't really be called double blind.

To their credit, Zarate et al did acknowledge this, and suggested that in future ketamine could be compared to another drug which produces noticeable effects. But they really should have done that to begin with.

It's now 2012, and there have still not been any published studies comparing ketamine to an active comparator i.e. a different drug that produces noticable psychoactive effects, to avoid unblinding. This means it's 12 years since the initial pilot report on ketamine in depression, and 6 years since the first large trial appeared.

The authors of the 2006 paper themselves wrote that "limitations in preserving study blind may have biased patient reporting... One potential study design in future studies with ketamine might be to include an active comparator" and suggested amphetamine for the big role.

Good idea. But six years later, we're still waiting. Which is really a bit silly. There have been dozens of papers written about the possible antidepressant effects of ketamine, from human trials to mouse work. That's a lot of research dollars (and dead mice) on something that might just be an active placebo.

Looking at the registered ketamine research on clinicaltrials.gov, I found that four active-comparator ketamine trials are in the pipeline (1,2,3,4), plus one cancelled (5). Only one is for depression though. The others being for OCD, cocaine dependence and suicidal ideation.

In all of these trials a benzodiazepine is the active comparator. Is that a good idea? Well, it's certainly better than nothing, but I wonder.

An active comparator has to "make an impression" on the patient equal to that produced by the real drug.  The null hypothesis, remember, is that ketamine has no specific antidepressant effect. That means it produces improvement through a combination of a) the placebo effect (expectation) and b) non-specific psychoactive changes.

More on that second one: any psychoactive drug might relieve depression by "taking your mind off it" and a change in mental state, as provided by a drug, also provides a demonstration that "I won't always feel this way". By showing that states of consciousness are products of brain chemistry, almost any drug could therefore offer a "glimmer of hope" to the depressed. If all this sounds very subjective, it is, but that's the point. Psychiatry is.

Would a benzo make as big an impression as 0.5 mg/kg ketamine IV? It's impossible to predict, really; so we'd need to ask people about the subjective strength of the drug effect. Personally, I worry that a lot of people just get sleepy on benzos and don't really feel much, so I'd prefer they used something a bit more hard-hitting like amphetamine, but maybe that's just me.

There's a deeper problem though. Suppose our ketamine-benzo trial finds no difference between ketamine and benzo. A critic could say, ah, but maybe it was just a "failed trial", so it doesn't overturn the positive studies. The patients weren't properly diagnosed, or weren't depressed enough, or were too depressed, etc.

Nitpicking such differences between studies is a well-practiced art.

Critics could complain in other ways if the study did find a benefit of ketamine. As I see it, the only way to settle this once and for all is to do a three-way randomized controlled trial - inactive placebo vs. active comparator vs. ketamine.

That way, if it's a failed trial, we'd know: there'd be no difference between ketamine and the inactive placebo. If there was a difference, but the active comparator was just as good as ketamine, that means it was all about nonspecific effets. Finally, if ketamine was better than the other two conditions, we could be pretty confident it was really working.

Also important is the question of volunteer expertise; subjects shouldn't be able to tell what drug they're on, but people who'd taken ketamine and/or the comparator drug before might be able to do that, so you'd want naive volunteers.

In conclusion: It's possible that ketamine has no specific antidepressant effects. To find out we ideally need a three-way trial, with both active and inactive comparators, careful monitoring of subjective drug effects and patient knowledge and expectations. Until that happens, I will be skeptical of ketamine in depression.

This is not because I just think it's impossible. Ketamine profoundly affects the brain in ways that we don't understand. I've suffered depression and I know it can come and go in a matter of minutes. So I think it's entirely possible that it works - but it's also possible that it's a nonspecific effect.

Look. I really want to know the answer to this. Both as a neuroscientist, and as a depression sufferer, this is very important to me. That's why we urgently need a good trial.

Link: See also the discussion and the comments over at The Neurocritic and this Scientific American piece which is pretty good except that it doesn't cover the active placebo issue.


Zarate CA Jr, Brutsche NE, Ibrahim L, Franco-Chaves J, Diazgranados N, Cravchik A, Selter J, Marquardt CA, Liberty V, and Luckenbaugh DA (2012). Replication of Ketamine's Antidepressant Efficacy in Bipolar Depression: A Randomized Controlled Add-On Trial. Biological psychiatry PMID: 22297150

Ibrahim, L., et al. (2012). Course of Improvement in Depressive Symptoms to a Single Intravenous Infusion of Ketamine vs Add-on Riluzole: Results from a 4-Week, Double-Blind, Placebo-Controlled Study Neuropsychopharmacology DOI: 10.1038/npp.2011.338

Bringing the Real World into Brain Scanning

Canadian Neuroscientists Jacqueline Snow et al propose a new method of making brain scanning studies a bit more realistic.

Typically, in an fMRI or other neuroimaging study, any visual stimuli shown to the volunteer are just pictures on a screen. Sometimes videos will be used, but in almost all cases they're just 2D images. Is that adaquate? People have hoped so.

Snow et al's data suggest that it might not be.

They created a contraption for presenting subjects with real objects during a scan. See above. Now, to the uninitiated this might not seem like a big deal, but those with MRI experience will appreciate how impressive this is.

Everything from the angle of the volunteer's head to the LED lighting is an achivement, given the nature of MRI. The stimuli were controlled by one of the researchers, who had to sit next to the scanner, in total darkness, and operate the turntable with the help of some glow-in-the-dark stickers.

Having built this device, they then used it to compare the brain's responses to real objects vs photos of those same objects. The experiment was designed to test fMRI adaptation - the phenemenon whereby if you present the same stimuli repeatedly, the neural responses are reduced.

fMRI adaptation has been found to happen in many studies using 2D pictures, but Snow et al show that the effect was much smaller, maybe entirely absent, when people were repeatedly shown real objects: this graph shows the BOLD neural response in the lateral occipital complex. Seeing the same pictures over and over led to a weaker response, as expected; but seeing the same 3D objects didn't:

This is a good study and an important result, which suggests that the much-studied fMRI adaptation might not be a universal phenemonon. And the potential implications are big, as the authors write:

Finally, our preliminary fMRI results raise the provocative suggestion that the presence of real-world objects (i.e., as indicated initially via stereoscopic cues) invokes qualitatively different computations to those elicited by 2D images. Researchers in the field of behavioral psychophysics have expressed long-standing concern about the extent to which pictures of objects capture the properties of their real-world counterparts (i.e., their ecological validity), with reservations as to their appropriateness as stimuli with which to examine the nature of human object perception...

Snow, J., Pettypiece, C., McAdam, T., McLean, A., Stroman, P., Goodale, M., and Culham, J. (2011). Bringing the real world into the fMRI scanner: Repetition effects for pictures versus real objects Scientific Reports, 1 DOI: 10.1038/srep00130

Visualizing The Connected Brain

So it seems as though the "connectome" is the latest big thing in neuroscience. This is the brain's wiring diagram, in terms of the connections between neurons and on a larger scale, between brain regions.

We certainly won't understand the brain without getting to grips with the connections but equally, it's not the whole story. I previously emphasised that the brain is not made of soup; it's not made of spaghetti, either.

Connectomics does however unquestionably provide some of the prettiest images in neuroscience. And they just got prettier, with a new technique for visualizing connections, just revealed in Neuroimage: Circular representation of human cortical networks for subject and population-level connectomic visualization.

See above. It's a rather lovely vista (for which the authors Irimia et al share credit with the folks behind the Circos visualization tool they used).

All you need are some MRI scans, and a lot of image processing, and you can produce one of these "Connectograms". But what does it mean? Here's the authors' description:

The outermost ring shows the various brain regions arranged by lobe (fr — frontal; ins — insula; lim — limbic; tem — temporal; par — parietal; occ — occipital; nc — non-cortical; bs — brain stem; CeB — cerebellum) and further ordered anterior-to-posterior. The color map of each region is lobe-specific and maps to the color of each regional parcellation.

In other words, the outer ring is just a list of brain regions, each with an assigned colour. The inner rings tell us about those regions:

Proceeding inward towards the center of the circle, these measures are: total GM volume, total area of the surface associated with the GM–WM interface (at the base of the cortical ribbon), mean cortical thickness, mean curvature and connectivity per unit volume. For non-cortical regions, only average regional volume is shown.

So each of the five inner rings displays data about one aspect of brain anatomy, for each of the regions. The colors are a heat map of the numbers.

Finally, the lines between regions represent the degrees of connectivity between regions via white matter tracts, as measured with diffusion tensor imaging:

The links represent the computed degrees of connectivity between segmented brain regions. Links shaded in blue represent DTI tractography pathways in the lower third of the distribution of FA, green lines the middle third, and red lines the top third (see text for details).

You can also make a pooled connectogram of the average neuroanatomy across a group of people. Still, it remains to be seen whether these are as useful as they are beautiful.

Irimia A, Chambers MC, Torgerson CM, and Van Horn JD (2012). Circular representation of human cortical networks for subject and population-level connectomic visualization. NeuroImage PMID: 22305988

Psychiatry's True Blood? Pt 1.

Imagine that there was a blood test that could detect depression. Wouldn't that be useful?

It depends.

Ridge Diagnostics are a US company who offer such a test. They've just published some results of the technology in Molecular Psychiatry. In two samples of patients with major depressive disorder (MDD), they report differences in the"MDDScore", between the patients and healthy controls.

The MDDScore is an aggregate value, calculated from the levels of 9 metabolites in blood serum. They're all well-known molecules, including hormones, such as cortisol and prolactin. The novelty is in how they're put together to make the MDDScore. We're given equations - but the key variables are not provided, because they're proprietary:

Long-term Neuroskeptic readers will recall that this "secret ingredients" approach to publishing science was also adopted by another company offering a different depression test.

Anyway, the performance of the test was impressive. In both the pilot and the replication samples, the MDDScore was significantly higher in the depressed people than in the controls. In both cases, the test had a sensitivity of over 91% and a specificity of over 81%, which is pretty good. Ridge Diagnostics are already offering the MDDScore clinically. For $745 a pop.

However...

Although there were two depressed patient groups (n=36 and 34), there was only one set of controls (n=43); both patient samples were compared to it. This means the second, "replication", test was not fully independent of the first one. If the first finding was a fluke caused by the control group having weird results by chance, for instance, then the second study would just repeat the fluke.

The patients were significantly older, and with a higher BMI, than the controls. They did control for these variables, which is good, but this raises the question of whether these folks differed in other ways, that they didn't measure, and hence couldn't control for.

In both samples, the patients had a very significantly higher MDDScore than the controls (p less than 0.0001, both times). But in both cases, the difference in levels of EGF (epidermal growth factor) was almost as strong: p=0.0003 and p less than 0.0001, respectively. Other metabolites weren't far behind. Testing for EGF would almost certainly be cheaper than getting an MDDScore.

Finally, all these data demonstrate is that the test can distinguish between people with MDD and entirely healthy people. But how often are doctors going to need to do that? More likely, they'll want to distinguish depression from other things that are often confused with it, such as: bipolar disorder, anxiety disorders, chronic fatigue syndrome, bereavement, "stress", and all manner of physical illnesses e.g. thyroid problems. Daniel Carlat said last year that

If the test cannot distinguish different psychiatric problems, then the MDDScore is simply a non-specific "biomarker" for emotional difficulties of all stripes, and would be essentially useless.

How disorder-specific is the MDDScore? This paper doesn't tell us. And to date, this is the only published paper mentioning the MDDScore. The website mentions some conference presentations, but none have yet appeared in a peer reviewed journal.

Ridge Diagnostics have an interesting history. But that's another story - stay tuned for Part 2.

Papakostas, G., Shelton, R., Kinrys, G., Henry, M., Bakow, B., Lipkin, S., Pi, B., Thurmond, L., and Bilello, J. (2011). Assessment of a multi-assay, serum-based biological diagnostic test for major depressive disorder: a Pilot and Replication Study Molecular Psychiatry DOI: 10.1038/mp.2011.166

Voodoo Neuroscience Revisited

Two years ago, neuroscientists were shaken by the appearance of a draft paper showing that half of the published work in a particular field had fallen prey to a major statistical error.

Originally called "Voodoo Correlations in Social Neuroscience", it ended up with the less snappy name of Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. I prefer the old title.

The error in question is now known variously as the "circular analysis problem", "non-independence problem" or "double-dipping" although I still call it the "voodoo problem". In a nutshell it arises whenever you take a large set of data, search for data points which are statistically significantly different from some baseline (null hypothesis), and then go on to perform further statistics only on those significant data points.

The problem is that when you picked out the statistically significant observations, you selected the data points that were especially "good", so if you then do some more analyses only on those data, you are almost guaranteed to find something "good". To avoid this you need to make sure that your second analysis is truly independent of your first one.

Anyway, Vul and Pashler, the main authors of the original voodoo article, have just written a short piece in NeuroImage offering some reflections on the paper and the aftermath. They don't make any major new arguments but it's a good read. Particularly fun is their explanation of what inspired them to look into the voodoo problem:

In early 2005 a speaker in our department reported that BOLD activity in a small region of the brain can account for the great majority of the variance in speed with which subjects walk out of the experiment several hours later (this finding was never published as far as we know). The implications of this result struck us as puzzling, to say the least: Are walking speeds really so reliable that most of their variability can be predicted? Does a focal cortical region determine walking speeds? Are walking speeds largely predetermined hours in advance? These implications all struck us as far-fetched...

But they reveal that it was one paper in particular that set them off voodoo-hunting

Our interest in probing the matter was further whetted by an episode occurring a short while later: Grill-Spector et al. (2006) reported that individual voxels in face selective regions have a variety of stable stimulus preferences; in a critical commentary, Baker et al. (2007) found that the analysis used to ascertain this fact implicitly built these conclusions into the method, such that the same analysis applied to noise data (voxels from the nasal cavity) revealed a similar variety of stable preferences. It occurred to us that a similar circularity might underlie the puzzlingly high correlations.

To their credit, Grill-Spector et al quickly accepted Baker et al's criticism and admitted that some of their original conclusions had been wrong.

Vul, E., and Pashler, H. (2012). Voodoo and circularity errors NeuroImage DOI: 10.1016/j.neuroimage.2012.01.027

The Trojan Horses of Medicine

Dodgy science is being smuggled into medical journals thanks to a loophole in the regulations, say Italian psychiatrists Barbui and Cipriani in an important article.

They focus on agomelatine, a recently-approved antidepressant. But their point applies to all of medicine, not just psychiatry.

Here's the problem. Nowadays, major medical journals have rules governing systematic reviews and meta-analyses of clinical trial data. If you want to review the evidence about how well a certain drug works, or its safety, you've got to do it properly. You have to consider all of the data, not just focus on the results that suit you. And so on.

However, these rules don't apply to "narrative" review papers, which is a broad term meaning any kind of article meant to give a discussion of the pharmacology, history, chemistry etc. behind a particular drug. For a narrative review, there are no rules.

In particular, you can write about the clinical trial data in such articles with no restrictions. Unlike in a proper systematic review, you can cherry-pick trials and so on to your heart's content. Some narrative reviews have so much clinical data in them that they end up being, in effect, a bad systematic review. One that would never have been deemed acceptable as a systematic review.

Barbui and Cipriani argue that narrative reviews are often used in this way, namely to paint drugs in a positive light. In the case of agomelatine, they mention a number of recent narrative reviews which were supposedly about the drug's mechanism of action, but which actually contained extensive (but biased) reviews of the clinical trial data.

It's not hard to see how pharmaceutical companies might take advantage of this process.

However, the problem is surely not limited to agomelatine. It's a loophole that affects every branch of medicine:

Most medical journals require adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. Adherence to PRISMA is not required in review articles dealing with basic science issues as these articles are not focused on clinical trials.

In practice, however, the agomelatine case indicates that clinical data are regularly included and reviewed with no reference to the rigorous requirements of the PRISMA approach. These articles have this way became a modern Trojan horse for reintroducing the brave old world of narrative-based medicine into medical journals.

How do we stop this? It's simple, the authors say: just make all references to clinical data subject to PRISMA, or other accepted regulations, whatever the supposed 'primary focus' of the paper:

We argue that medical journals should urgently apply this higher standard of reporting, which is already available, easy to implement and inexpensive, to any form of clinical data presentation.

Of course, there are plenty of good narrative reviews that really do cover the pharmacology or other science in a useful way. The problem is not narrative reviews as such, but the way they're used.

Barbui, C., and Cipriani, A. (2012). Agomelatine and the brave old world of narrative-based medicine Evidence-Based Mental Health, 15 (1), 2-3 DOI: 10.1136/ebmh.2011.100485

Antidepressants: Bad Drugs... Or Bad Patients?

Why is it that modern trials of antidepressant drugs increasingly show no benefit of the drugs over placebo? This is the question asked by Cornell psychiatrists Brody et al in an American Journal of Psychiatry opinion piece.

They suggest that maybe it's the patients fault:

Participation that is induced by cash payments may lead subjects to exaggerate their symptoms [i.e. in order to get included into the trial]... Another contributing factor to high placebo response rates may be the extent to which the volunteers in antidepressant trials are really generalizable to patients in clinical practice.

Since the initial antidepressant trials in the 1960s, participants have gone from being patients who were recruited primarily from inpatient psychiatric populations to outpatient volunteers who are often recruited by advertisements. At times, these symptomatic volunteers have participated in other trials. When we contact potential participants to schedule screening, they often ask to be reminded which trial we are screening for or mistake our research trial for a different protocol in which they recently participated.

They then recount the tale of two "professional subjects" who claimed to be depressed and enrolled in two antidepressant trials simultaneously, without telling the researchers; it only came to light when someone involved in both studies spotted the duplicate names.

I've been the victim of such nonsense myself, as have many of colleagues - it's a perennial watercooler topic. A few years ago I was running a study recruiting people who'd recovered from psychiatric illness. The main source of volunteers was online adverts.

That study was a learning experience. What I learned is that House was right. We recruited about 20 people. No fewer than 3 turned out to have enrolled in other studies and lied about it. After I realized this I Googled the offender's names and two of them turned up in the court pages of the local newspaper pleading guilty to various petty crimes.

Another volunteer was left handed and, upon realizing that I was only recruiting right-handed people, discretely switched his pen to his right hand and then took 5 minutes trying to fill out a form with his off hand. He didn't make it in, but if I hadn't been paying attention he would have.

So yes, it is a problem. However, it would have to be taking place on a massive scale for it to be having a significant effect on antidepressant trial results and this really seems pretty unlikely.

In my view, the authors miss out on the real problem with recruiting depressed people through adverts:  depressed people don't tend to respond to adverts, because depressed people don't do anything. That's why they call it depression.

Getting recruited into a modern clinical trial is actually quite a challenge. There are many pieces of paper to fill in, calls to return, appointments to attend. Turn up late to the screening visit, or otherwise make life difficult for the study staff, and you'll be marked down as "unreliable" and they'll find someone who plays by the rules. Modern trials are very expensive. The last thing a study sponsor wants is a volunteer who will end up forgetting to take their pills on time.

Depression, unfortunately, makes you bad at doing things. You procrastinate, you forget, you put things off until too late, you have a change of heart and decide not to, you get cold feet, you can't be bothered... That goes for things as simple as cooking dinner in severe cases, let alone something as complicated as taking part in a trial.

So while you wouldn't go looking for aquaphobic people in a swimming pool, I'm not sure we should be looking for depressed people through adverts.

Brody B, Leon AC, and Kocsis JH (2011). Antidepressant clinical trials and subject recruitment: just who are symptomatic volunteers? The American journal of psychiatry, 168 (12), 1245-7 PMID: 22193668

How Realistic is fMRI?

How representative are fMRI experiments? Is "the brain" that we investigate with fMRI the same brain that we use outside the MRI scanner?

A new paper from Bernhard Hommel and colleagues of Leiden in the Netherlands offers some important caveats. They looked to see what effect playing some recorded MRI scanner sounds had on people's ability to perform some simple cognitive tasks, while sitting outside the scanner.

MRI is notoriously noisy. When you have an MRI scan you have to wear earplugs to protect against the sound but they only block out some of it. Opinions differ on whether the sound is pleasant or not. Personally I find the repetitive tick-tock rather soothing now, but then I've heard it many times over the years. First-timers can find it quite overwhelming.

Anyway, Hommel et al found that while scanner noise had no overall effects on reaction time or accuracy, it actually improved performance on three measures of "cognitive control".

For instance in a task in which participants had to respond to the colour of a circle by pressing the left or the right arrow key, they were slower to react when the circle appeared on the "wrong" side of the screen, i.e. on the left when the correct answer was the right arrow. This slowing of responses caused by a stimulus-response clash is called the Simon effect.

The results showed that the Simon effect was reduced by noise. The same thing happened in two other studies: noise meant better performance.

All of the noise effects were modest and the sample sizes were also quite small (14-18 per task, with everyone studied twice, noisy vs silent) but this paper joins a number of others raising questions about the representativeness of fMRI, with evidence that fMRI activates the brain and maybe even improves mood (although I doubt that last one).

The authors' interpretation is that the noise made people pay more attention to the tasks, to compensate for the distraction, and that this means that fMRI studies may be biased in their measurements of cognitive control:

Generalizing from fMRI findings to behavioral observations and vice versa seems to be more problematic than commonly thought, at least as far as control  processes are concerned. In a sense, then, investigating cognitive processes by means of  fMRI... is inevitably facing Heisenberg’s (1927) uncertainty principle, according to which the act of measurement can change what is being measured.

To my mind the biggest weakness of this is that it only looked at noise. While scanners are noisy, that's not the only distracting thing about them: during an fMRI study you also have to lie down, in a small confined tube, and your only way to see the "screen" on which experimental stimuli are shown is indirectly via a small mirror which often doesn't give a good view.

So ironically, I'm not sure how realistic this study is...


Hommel, B., Fischer, R., Colzato, L., van den Wildenberg, W. and Cellini, C. (2011). The effect of fMRI (noise) on cognitive control. Journal of Experimental Psychology: Human Perception and Performance DOI: 10.1037/a0026353

An Objective Measure of Consciousness...?

Could a puff of air in the eye offer a way to evaluate whether someone is conscious or not?

Yes it could, say Cambridge's Tristan Bekinschtein and colleagues in a new paper about Sea slugs, subliminal pictures, and vegetative state patients.

It's all about classical conditioning of the kind made famous by Pavlov. This is learning caused by the pairing of two stimuli, one of them somehow meaningful (usually unpleasant). So if I were to ring a little bell before, say, pepper spraying you, and I did that repeatedly, you would probably close your eyes whenever I rang that bell. Or just punch me, but you see the point.

Anyway, the key is that there are two kinds of classical conditioning. In the unhelpfully named "delay" conditioning, the warning stimulus overlaps with the painful one. Like if I started ringing my bell, then kept ringing it while I sprayed you with my other hand. In other words, there is no delay between the two stimuli... I said it was badly named.

By contrast in "trace", conditioning there is a delay - the warning stops shortly before the second stimulus. Bekinschtein et al argue that trace conditioning requires conciousness. While delay conditioning can occur without awareness of the link between the two stimuli, only conscious awareness can bridge the time gap in trace conditioning.

In trace experiments (in which rather than pepper spray, the unpleasant stimulus is just a puff of air in the eye), people who, when asked, can't explain the relationship ("sound means puff") don't learn to blink when they hear the sound. But with delay conditioning, this "unconscious" conditioning can occur. Likewise, under anaesthesia, trace conditioning is lost.

At first glance this looks like a piece of psychological trivia, but it could have literally life-or-death consequences. If trace conditioning is a measure of concious awareness then it could be used as a way of working out whether brain-injured people in a "coma" or "vegetative state" are aware or not.

This paper is in fact a follow-up to the author's own 2009 study showing that some people in a vegetative state do show trace conditioning - and the ones who did were more likely to subsequently wake up.

One snag is that the humble sea slug, Aplysia, can undergo trace conditioning, yet it is presumably not conscious, at least not in any recognizable sense.

But Bekinschtein et al say that trace conditioning is a product of convergent evolution. Alplysia can do it and we can do it, but we use different means to the same end. Their argument is that while in Alpysia trace conditioning is known to be dependent on just a handful of individual neurons in the creature's tiny "brain", in humans it requires an intact hippocampus (containing millions of cells). People with hippocampal damage, who suffer amnesia, also can't do trace conditioning.

That's a good point but does that mean such hippocampal patients aren't conscious? That would be weird because, apart from the amnesia, they seem perfectly normal. Presumably they're just not conscious of the relationship between things separated in time...

Also, primitive pathways for conditioning might still exist in humans, able to reactivate under special conditions. They do acknowledge this with a discussion of experiments showing that trace conditioning in the absence of conscious awareness of the relationship can occur but only when the warning stimuli are "scary", like pictures of snakes. They say that with generic, neutral stimuli there is no good evidence of unconscious trace conditioning, but this seems like a fairly fine distinction.


Ultimately, it's a very nice idea but only more studies on "unconscious" patients will tell us whether it's really able to measure consciousness in a useful way.

Bekinschtein TA, Peeters M, Shalom D, and Sigman M (2011). Sea slugs, subliminal pictures, and vegetative state patients: boundaries of consciousness in classical conditioning. Frontiers in psychology, 2 PMID: 22164148

Genes for Intelligence - Back to Square One

Here's a paper - soon to appear in Psychological Science - which says that Most Reported Genetic Associations with General Intelligence Are Probably False Positives

The authors tried to replicate published associations between particular genetic variants (SNPs) and IQ (specifically the g factor). They looked at three datasets, a total of about 10,000 people, and didn't confirm any of the 12 associations.

As Razib Khan says in his post on this, "My hunch is that these results will be unsatisfying to many people." I'd go further and say that no-one will be happy with these.

For those who believe that IQ is purely environmental and not genetic, any satisfaction they might feel will be short lived because these authors did replicate the recent finding that genetic variants explain about 50% of the variance in IQ. Looking at all SNPs together, there was a strong correlation between "genetic similarity" and similarity in IQ. That independently confirms what the much-criticized twin studies of IQ said - IQ is about 50% heritable.

But for people who do believe in the genetics of intelligence, this shows us that we have no idea what the genes are, and that everything published so far has been pretty much for naught.

There's another implication. We actually do know of many "IQ genes" in that we know genes that, when mutated, cause mental retardation (very low IQ).

Now many researchers have hoped that if a certain gene causes you to have an IQ of, say, 50 when it's completely deleted by a mutation, then more subtle variants in that gene would have minor effects on IQ. Maybe a variant that reduces expression of the gene by 10% would knock off 5 IQ points.

In other words, if big mutations cause big phenotypes, then small mutations in the same place ought to cause small phenotypes. It seems to make sense - but today's IQ literature shows that it's just not true.

That's not just a problem for IQ though. Take autism or ADHD, we know that there are rare, severe mutations that cause these conditions. Many people are hoping that common variation in the same genes might also be interesting - but if IQ is anything to go by, it won't be.

Perhaps this is not so surprising. Breaking your neck and becoming paraplegic is going to seriously impair your ability to play baseball. That doesn't mean that normal variation in baseballing skill has much to do with minor neck injuries.

Chabris, C. F. et al (2011). Most Reported Genetic Associations with General Intelligence Are Probably False Positives Psychological Science

Do Antidepressants Make Some People Worse?

Antidepressants may help depression in some people but make it worse for others, according to a new paper.

This is a tough one so bear with me.

Gueorguieva, Mallinckrodt and Krystal re-analysed the data from a number of trials of duloxetine (Cymbalta) vs placebo. Most of the trials also had another antidepressant (an SSRI) as well. And the SSRIs and duloxetine seemed to be indistinguishable so from now on I'll just call it antidepressants vs. placebo as the authors did.

People on placebo got, on average, moderately better over 8 weeks.

People on antidepressants fell into two classes. The largest class got, on average, a lot better. But about 25% did poorly, staying just as depressed as before. This "nonresponder" group did much worse than the placebo group - again on average. Here you can see the mean "trajectories" of depression symptoms (HAMD scores) in the three groups:

This raises the scary possibility that while antidepressants are helping some people, they're harming others. But hang on. It's complicated.

First off, maybe this is all a statistical illusion. When the authors say that the people on drug fell into two classes, what they mean is that when you try to model the data according to a certain mathematical model, assuming either 1, 2, 3 or 4 underlying classes, the 2 class solution was the best fit. While for placebo a 1 class solution was best.

We considered linear, quadratic, and cubic trends over time, with between 1 and 4 trajectory classes. We also considered piecewise models with a change point at 2 weeks, linear change before week 2, and quadratic change after week 2. The selection of the best model was based on the Schwartz-Bayesian information criterion and on the Lo-Mendell-Rubin (LMR) likelihood ratio test...

That's nice... but they don't present the raw data. They don't tell us whether, looking at the individual trajectories of people on antidepressants, you'd actually see two classes. What I want is a graph of how likely people are to get better by a certain amount. If Gueorguieva et al are right, I want it to look like this i.e. bimodal -

We're not shown this graph. I'll eat my hat if it does look like that, frankly, because if it did people would have noticed the bimodality in antidepressant trials ages ago.

True, statistical models can tell us things that aren't obvious by inspection, so even if this isn't what the data look like, they might still be right. It could be that the two "peaks" are so broad, and there's so much random noise, that they blur into one.

However, it's also true that you can fit an infinite number of models to any set of data and at some point you have to step back and say - am I making this more complicated than it needs to be?

It could be that a 2-class model is better than a 1-class model for the people on antidepressants, but only because they're both crap, and really, every patient has a different, unpredictable trajectory which is poorly captured by such models.

Let's assume however that this is true. What would it mean?

Firstly, the fact that one class of people on antidepressants does worse than people on placebo doesn't mean that antidepressants are harming them. The authors miss this point, when they say

there are 2 trajectories for patients treated with antidepressants and 1 trajectory for patients treated with placebo [so] some patients would seem to be more effectively treated with placebo than with a serotonergic antidepressant.

But that's fallacious. It treats a purely statistical entity as representing individual people. Suppose that what antidepressants do is to take people who, on placebo, would have improved a bit, and make them improve a bit more than they otherwise would have. You'd then end up with more people doing well, but also fewer people doing moderately because they'd have been "moved up" out of the middle ground.

That "nudging people off the fence" could lead to a bimodal distribution and two distinct classes. But in this case the people doing badly would have done badly either way. The drug didn't make them do badly, it just made doing-badly into a class. On the other hand it's consistent with antidepressants doing real harm. We can't tell.

We do know that other randomized controlled trials show very convincingly that in a small minority of people, mostly but not exclusively young people, antidepressants do worsen suicidal thoughts and behaviours. So it's plausible. But we just don't know yet.

What worries me is that this paper is the latest in a series of attempts  to use, well, creative statistical approaches to antidepressant trial data. This one is nowhere near as dodgy as the Cherrypicker's Manifesto I discussed last year, but it cites that paper and others by the same group. The first sentence of the Abstract of this paper makes the intention clear:

The high percentage of failed clinical trials in depression may be due to high placebo response rates and the failure of standard statistical approaches to capture heterogeneity in treatment response.

In other words, the reason clinical trials of new antidepressants often fail to show a benefit over placebo is not because the drugs are crap but because the statistics aren't subtle enough. And you can see where this is going: if only we could use statistical models to find the people who do benefit from antidepressants, and compare them to placebo, there'd be no problem...

Gueorguieva R, Mallinckrodt C, and Krystal JH (2011). Trajectories of depression severity in clinical trials of duloxetine: insights into antidepressant and placebo responses. Archives of General Psychiatry, 68 (12), 1227-37 PMID: 22147842