The Geography of Faces

How much can you tell about where someone comes from, just from their face?

The other day I was in London and came across a group of young people in Muslim attire who were waving (or in some cases wearing) a particular flag. I thought it was the Iranian flag, but, I thought, they didn't look Iranian. They looked more like Somalis, but it certainly wasn't the blue and white Somali flag. I decided that maybe they were some kind of pro-Iranian demonstrators, but I later worked out that it was the flag of the unrecognised state of Somaliland.

This got me thinking about how reliable these "they look they're from..." judgements are.

Clearly on a basic level, we can usually tell which continent someone's ancestors were from, in terms of the familiar "races" of Europeans, Africans, East Asians etc. But what about shorter distances?


Could you tell, just from looking at them (and setting aside dress, hairstyle, jewellery etc.) whether someone was from Spain as opposed to France? Korea or Japan? Russia or Germany?

I can only speak for England, but there's certainly a vague but widespread belief that every part of Europe has a  distinct 'look'. In the past, people were very fond of talking about that kind of thing; today, we're rather embarrassed by the idea but the belief lives on.

I don't know, but I'd be very surprised if there weren't analogous beliefs in other countries.

But how accurate are these folk beliefs, really?

Supposing you were the world expert on human faces - or suppose you were a supercomputer with face-recognition software and access to Facebook's entire dataset. How accurately could you place someone's origins on the map, on average? To within 1000 km? 100? With what degree of accuracy? In an ideal world, could the ultimate face-placer judge someone as French vs German 75% of the time? 90%? Or only slightly better than chance?

I suspect that if you researched this, you'd find that a supercomputer could do very well, in most parts of the world, but that the majority of actual people are less accurate than they think they are.

Brain Scanning - Just the Tip of the Iceberg?

Neuroimaging studies may be giving us a misleading picture of the brain, according to two big papers just out.

By big, I don't just mean important. Both studies made use of a much larger set of data than is usual in neuroimaging studies. Thyreau et al scanned 1,326 people. For comparison, a lot of fMRI studies have more like n=13. Gonzalez-Castillo et al, on the other hand, only had 3 people - but each one was scanned while performing the same task 500 times over.

Both studies found that pretty much the whole brain "lit up" when people are doing simple tasks. In one case it was seeing videos of people's faces, in the other it was deciding whether stimuli on the screen were letters or numbers.

With all that data, the authors could detect effects too small to be noticed in most fMRI experiments, and it turned out that pretty much everywhere was activated. The signal was stronger in some areas than others, but it wasn't limited to particular "blobs".

So conventional fMRI experiments may just be showing us the tip of the iceberg of brain activity. In a small study, only the strongest activations pass the statistical threshold to show up as blobs, but that doesn't mean the rest of the brain is inactive. It just means it's less active. The idea that only small parts of the brain are 'involved' in any particular task may be a statistical artefact.

In fact, I wonder if the whole idea of treating statistically significant blobs as different from nearly-significant areas is itself a form of the error of interacting effects?

As if that wasn't enough, Gonzalez-Castillo further show that there are lots of activations in the brain - even to very simple stimuli - that might go undetected in conventional studies, because they don't follow the time-course predicted by the usual models.

Have a look -

This shows the average neural activation from various regions of the brain during a letter-number task. The two areas I've highlighted in red are the primary visual cortex, and they do follow the expected 'boxcar' pattern - the brain is active when the stimuli are on the screen, inactive when they're not. But you can see that all kinds of other brain areas are also responding to the stimuli - just in different ways.

For example, the left primary motor cortex was activated during the task. That area controls the right hand, and that makes sense, as people responded by pressing buttons with the right hand. But interestingly, the same area on the other side of the brain was deactivated at exactly the same time, even though people weren't doing anything with their left hand.

These papers illustrate the fact that conventional fMRI is a blunt instrument that often only tells us about the most straightforward events that happen in the brain. A bit like how we only hear the shouts and screams from through our neighbor's walls, not their normal conversations, which aren't loud enough to reach our ears.

That's the bad news, but every blob has a silver lining. fMRI is clearly more powerful than most neuroscientists have realized, and this holds out hope for cracking some of the trickiest questions. As Gonzalez-Castillo et al put it

This result helps narrow the gap between thousands of fMRI manuscripts showing limited activation in response to tasks and cognition theories that defend that cognition—understood as the process of “configuring the way in which sensory information becomes linked to adaptive responses and meaningful experiences”—can only result from the distributed collaboration of primary sensory, upstream and downstream unimodal, heteromodal, paralimbic, and limbic regions... [we were able to] switch from a regime where activity detection relates primary to sensory processing to a more sensitive regime, where activity detection includes also cognitive processes with subtler BOLD signatures.

Link: See also the interesting discussion here: Surely, God loves the .06 (blob) nearly as much as the .05.


Thyreau, B., Schwartz, Y., Thirion, B., Frouin, V., Loth, E., Vollstädt-Klein, S., Paus, T., Artiges, E., Conrod, P., Schumann, G., Whelan, R., and Poline, J. (2012). Very large fMRI study using the IMAGEN database: Sensitivity–specificity and population effect modeling in relation to the underlying anatomy NeuroImage DOI: 10.1016/j.neuroimage.2012.02.083

Gonzalez-Castillo, J., Saad, Z., Handwerker, D., Inati, S., Brenowitz, N., and Bandettini, P. (2012). Whole-brain, time-locked activation with simple tasks revealed using massive averaging and model-free analysis Proceedings of the National Academy of Sciences DOI: 10.1073/pnas.1121049109

Voodoo Neuroscience Revisited

Two years ago, neuroscientists were shaken by the appearance of a draft paper showing that half of the published work in a particular field had fallen prey to a major statistical error.

Originally called "Voodoo Correlations in Social Neuroscience", it ended up with the less snappy name of Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. I prefer the old title.

The error in question is now known variously as the "circular analysis problem", "non-independence problem" or "double-dipping" although I still call it the "voodoo problem". In a nutshell it arises whenever you take a large set of data, search for data points which are statistically significantly different from some baseline (null hypothesis), and then go on to perform further statistics only on those significant data points.

The problem is that when you picked out the statistically significant observations, you selected the data points that were especially "good", so if you then do some more analyses only on those data, you are almost guaranteed to find something "good". To avoid this you need to make sure that your second analysis is truly independent of your first one.

Anyway, Vul and Pashler, the main authors of the original voodoo article, have just written a short piece in NeuroImage offering some reflections on the paper and the aftermath. They don't make any major new arguments but it's a good read. Particularly fun is their explanation of what inspired them to look into the voodoo problem:

In early 2005 a speaker in our department reported that BOLD activity in a small region of the brain can account for the great majority of the variance in speed with which subjects walk out of the experiment several hours later (this finding was never published as far as we know). The implications of this result struck us as puzzling, to say the least: Are walking speeds really so reliable that most of their variability can be predicted? Does a focal cortical region determine walking speeds? Are walking speeds largely predetermined hours in advance? These implications all struck us as far-fetched...

But they reveal that it was one paper in particular that set them off voodoo-hunting

Our interest in probing the matter was further whetted by an episode occurring a short while later: Grill-Spector et al. (2006) reported that individual voxels in face selective regions have a variety of stable stimulus preferences; in a critical commentary, Baker et al. (2007) found that the analysis used to ascertain this fact implicitly built these conclusions into the method, such that the same analysis applied to noise data (voxels from the nasal cavity) revealed a similar variety of stable preferences. It occurred to us that a similar circularity might underlie the puzzlingly high correlations.

To their credit, Grill-Spector et al quickly accepted Baker et al's criticism and admitted that some of their original conclusions had been wrong.

Vul, E., and Pashler, H. (2012). Voodoo and circularity errors NeuroImage DOI: 10.1016/j.neuroimage.2012.01.027

New Study: Openly Gay Applicants 40% Less Likely to Get Job Interviews

Openly gay applicants are 40 percent less likely to be granted an interview than their heterosexual counterparts, according to a study published Tuesday in the American Journal of Sociology. The study was the first of its kind to test the receptiveness of employers to gay male job applicants. It sent two fictitious resumes to more than 1,700 entry-level, white collar job openings in the U.S. The resumes were nearly identical, except each mentioned a different affiliation with a school organization. Read more here. 

Do Antidepressants Make Some People Worse?

Antidepressants may help depression in some people but make it worse for others, according to a new paper.

This is a tough one so bear with me.

Gueorguieva, Mallinckrodt and Krystal re-analysed the data from a number of trials of duloxetine (Cymbalta) vs placebo. Most of the trials also had another antidepressant (an SSRI) as well. And the SSRIs and duloxetine seemed to be indistinguishable so from now on I'll just call it antidepressants vs. placebo as the authors did.

People on placebo got, on average, moderately better over 8 weeks.

People on antidepressants fell into two classes. The largest class got, on average, a lot better. But about 25% did poorly, staying just as depressed as before. This "nonresponder" group did much worse than the placebo group - again on average. Here you can see the mean "trajectories" of depression symptoms (HAMD scores) in the three groups:

This raises the scary possibility that while antidepressants are helping some people, they're harming others. But hang on. It's complicated.

First off, maybe this is all a statistical illusion. When the authors say that the people on drug fell into two classes, what they mean is that when you try to model the data according to a certain mathematical model, assuming either 1, 2, 3 or 4 underlying classes, the 2 class solution was the best fit. While for placebo a 1 class solution was best.

We considered linear, quadratic, and cubic trends over time, with between 1 and 4 trajectory classes. We also considered piecewise models with a change point at 2 weeks, linear change before week 2, and quadratic change after week 2. The selection of the best model was based on the Schwartz-Bayesian information criterion and on the Lo-Mendell-Rubin (LMR) likelihood ratio test...

That's nice... but they don't present the raw data. They don't tell us whether, looking at the individual trajectories of people on antidepressants, you'd actually see two classes. What I want is a graph of how likely people are to get better by a certain amount. If Gueorguieva et al are right, I want it to look like this i.e. bimodal -

We're not shown this graph. I'll eat my hat if it does look like that, frankly, because if it did people would have noticed the bimodality in antidepressant trials ages ago.

True, statistical models can tell us things that aren't obvious by inspection, so even if this isn't what the data look like, they might still be right. It could be that the two "peaks" are so broad, and there's so much random noise, that they blur into one.

However, it's also true that you can fit an infinite number of models to any set of data and at some point you have to step back and say - am I making this more complicated than it needs to be?

It could be that a 2-class model is better than a 1-class model for the people on antidepressants, but only because they're both crap, and really, every patient has a different, unpredictable trajectory which is poorly captured by such models.

Let's assume however that this is true. What would it mean?

Firstly, the fact that one class of people on antidepressants does worse than people on placebo doesn't mean that antidepressants are harming them. The authors miss this point, when they say

there are 2 trajectories for patients treated with antidepressants and 1 trajectory for patients treated with placebo [so] some patients would seem to be more effectively treated with placebo than with a serotonergic antidepressant.

But that's fallacious. It treats a purely statistical entity as representing individual people. Suppose that what antidepressants do is to take people who, on placebo, would have improved a bit, and make them improve a bit more than they otherwise would have. You'd then end up with more people doing well, but also fewer people doing moderately because they'd have been "moved up" out of the middle ground.

That "nudging people off the fence" could lead to a bimodal distribution and two distinct classes. But in this case the people doing badly would have done badly either way. The drug didn't make them do badly, it just made doing-badly into a class. On the other hand it's consistent with antidepressants doing real harm. We can't tell.

We do know that other randomized controlled trials show very convincingly that in a small minority of people, mostly but not exclusively young people, antidepressants do worsen suicidal thoughts and behaviours. So it's plausible. But we just don't know yet.

What worries me is that this paper is the latest in a series of attempts  to use, well, creative statistical approaches to antidepressant trial data. This one is nowhere near as dodgy as the Cherrypicker's Manifesto I discussed last year, but it cites that paper and others by the same group. The first sentence of the Abstract of this paper makes the intention clear:

The high percentage of failed clinical trials in depression may be due to high placebo response rates and the failure of standard statistical approaches to capture heterogeneity in treatment response.

In other words, the reason clinical trials of new antidepressants often fail to show a benefit over placebo is not because the drugs are crap but because the statistics aren't subtle enough. And you can see where this is going: if only we could use statistical models to find the people who do benefit from antidepressants, and compare them to placebo, there'd be no problem...

Gueorguieva R, Mallinckrodt C, and Krystal JH (2011). Trajectories of depression severity in clinical trials of duloxetine: insights into antidepressant and placebo responses. Archives of General Psychiatry, 68 (12), 1227-37 PMID: 22147842

Beware Dead Fish Statistics

An editorial in the Journal of Physiology offers some important notes on statistics.

But even more importantly, it refers to a certain blog in the process:

The Student’s t-test merely quantifies the ‘Lack of support’ for no effect. It is left to the user of the test to decide how convincing this lack might be. A further difficulty is evident in the repeated samples we show in Figure 2: one of those samples was quite improbable because the P-value was 0.03, which suggests a substantial lack of support, but that’s chance for you! A parody of this effect of multiple sampling, taken to extremes, can be found at http://neuroskeptic.blogspot.com/2009/09/fmri-gets-slap-in-face-with-dead-fish.html

This makes it the second academic paper to refer to this blog as far. Although I feel rather bad about this one, since the citation ought to have been to the original dead salmon brain scanning study by Craig Bennett. I just wrote about it.

Actually, though, this editorial was published in five separate journals: The Journal of Physiology, Experimental Physiology, the British Journal of Pharmacology, Advances in Physiology Education, Microcirculation, and Clinical and Experimental Pharmacology and Physiology. Phew.

In fact, you could say that this makes not two but six citations for Neuroskeptic now. Yes. Let's go with that.

Anyway, after discussing the history of the ubiquitous Student's t-test - which was invented in a brewery - it reminds us that the p value you get from such a t-test doesn't tell you how likely it is that your results are "real".

Rather, it tells you how often you'd get the result you did, if there was no effect and it was just random chance. That's a big difference. A p value of 0.01 doesn't mean your results are 99% likely to be real. It means that there's a 1% chance that you'd get them, by chance. But if you did say 100 experiments, or more likely, 100 statistical tests on the same data, then you'd expect to get at least one result with a p value of 0.01 purely by chance.

In that case it would be silly to think that the finding was only 1% likely to be a fluke. Of course it could be true. But we'd have no particular reason to think so until we get some more data.

This is what the dead salmon study was all about. This multiple comparisons issue is very old, but very important. Arguably the biggest problem in science today is that we're doing too many comparisons and only reporting the significant ones.

Drummond GB, & Tom BD (2011). Statistics, probability, significance, likelihood: words mean what we define them to mean. British journal of pharmacology, 164 (6), 1573-6 PMID: 22022804

One in Four Revisited

In a recent Telegraph article, professional contrarian Brendan O'Neill argues against the idea that one in four people experience mental illness - and indeed against the idea that one in four people are bullied, abused or whatever else:

Can it really be true that a quarter of Brits are bullied or beaten up at home or are mentally ill, or is this simply a case of social campaigners exaggerating how bad life is in order that they can continue to make headlines, make an impact, and get funding? I reckon it's the latter. Next time you see the "one in four" figure, be very sceptical – it's probably Dickensian-style doom-mongering disguised as social research, where the aim is to convince us, against the evidence of our own eyes and ears, that loads of the people we encounter everyday are basket cases in need of rescue.

I say "argues against", but he doesn't actually provide any arguments. He just links to the claims and says they're silly.

As Neuroskeptic readers know, I am myself skeptical of the idea that one in four people are mentally ill, but I'm skeptical of it because I've looked at the evidence and it doesn't support that figure. Actually, if you take the available evidence at face value, it says that the true figure for the lifetime prevalence is much higher than one in four. I don't think those figures are very useful however because of various methodological issues.

So in my view we just don't know how many people are mentally ill, largely because we don't have any clear definition of what "mentally ill" means. But that doesn't mean we can just assume that it can't possibly be one in four just because "our own eyes and ears" tell us that most people are not "basket cases".

Much mental illness goes undiagnosed and unnoticed, and I'd imagine also that Brendan O'Neill and the kind of people who read him don't tend to "encounter everyday" people from groups such as the unemployed, the elderly and so forth, in whom the rates are higher.

But even beyond that, it's a silly argument because of selection bias. If you as a healthy person encounter someone everyday, chances are they're not severely ill - mentally or physically - because if they were, they'd be less likely to be around in places for you to encounter. Unless you're a doctor or whatever, you live your life in the world of healthy people.

It's like saying that you don't believe children or the elderly exist, because in your life as a working age adult, you never meet any of them.

Modern War-fMRI : Graphics Cards for Science

Videogames and neuroscience have a rocky relationship.

On the one hand you have Susan Greenfield and her games-hurt-the-brain theory. But she's not representative of neuroscientists as a whole: games have also helped neuroscience, for example, in this study of the neural correlates of "flow" experiences.

Now neuroscientists have another reason to be thankful for games, according to a new paper. It turns out that modern 3D graphics cards - which mostly exist in order to render videogame visuals - can be used to do fMRI data analysis.

According to Sweden's Eklund et al, a graphics card can perform intensive fMRI analysis hundreds of times faster than a regular processor of the equivalent speed, because graphics processors make use of parallel computing optimized for 3D images and that's ultimately what all brain scans are.

They developed a way to run non-parametric statistical analyses of brain imaging data. Proponents say that non-parametric stats have many advantages over conventional parametric ones - and they're certainly becoming increasingly popular. But they involve doing far more calculations. Thousands of times more, in some cases.

It turns out though that armed with 2.5 GHz CPU and three NVidia GTX 480s, and making use of NVidia's graphics programming language, they were able to cut the time to analyse one person's brain with 100,000 permutations, from 24 hrs to just 9 minutes. The whole setup cost $4000, so it's not cheap, but they say it's "a fraction of the price for a PC cluster with equivalent computational performance" i.e. one relying on lots of general purpose processors, rather than graphics cards. Even on GTX480 did the job very well.

Best of all, this gives neuroscientists an excuse to spend their grant money on awesome gaming rigs. Why do I want the latest GForce on my work computer? To do non-parametric data analysis, obviously. Sure, it would also allow me to run Modern Warfare 3 at the highest settings... but that's not why I want it.

Eklund A, Andersson M, Knutsson H (2011). Fast random permutation tests enable objective evaluation of methods for single-subject FMRI analysis. International journal of biomedical imaging, 2011 PMID: 22046176

New Stat on Support for Marriage Equality by Age

No surprises in the pattern here. Again, I'd be curious to see how the question was phrased. As I always caveat when sharing marriage equality data, in my opinion the question should be phrased as, "Do you believe same-sex couples should have the right to access civil marriage licenses?" This is because there is a difference between "favoring something" and believing people should have the right to it. As an example, some redneck American out there might not personally "favor" interracial marriage, but that doesn't necessarily mean he believes interracial couples out there shouldn't have the right to marry. Furthermore, the distinction between the two types of marriage - civil and religious - must always be made clear when it comes to polling on this issue. Language matters. 

Study Paints a Better Picture of Today's Real Modern Family

Edelman has just wrapped up an interesting study on changing US family dynamic and demographics. Among the most interesting findings include that just 4% of US families have stay-at-home moms and working dads with children under 18 years old. That's right, just 4%. Furthermore, the study points out how economic pressures and blended family models have redefined individual roles within the family: skill sets have replaced gender, and 62 percent of moms and 54 percent of dads feel that parenting roles will be redefined away from the traditional “mom and dad” roles of the past. This creates a new opportunity for marketers to think in terms of skill set versus gender, opening up the entire family as a target. The study, which was commissioned in August 2011, included in-depth interviews with 2,482 consumers among a cross-section of today’s modern family: single parents, working moms, gay partners, multicultural heads of households and grandparents.

The Facebook Brain

Facebook friend tally is associated with differences in brain structure

People with lots of Facebook friends have denser grey matter in three regions of the brain, a study suggests

When I heard about this, my heart sank. The Facebook area of the brain? It had all the hallmarks of a piece of media neuro-nonsense: a hook (Facebook!), a simplistic neo-phrenological story (bigger brains are better!)... so I was expecting to discover that the fuss was all about some tiny, statistically questionable study, which wasn't really about what the newspapers said it was, as is so often the case.

So I was very surprised to find that it's actually an extremely good paper.

Kanai et al from London took 125 young Facebookers (mostly students) and correlated their friend count with grey matter density across the brain. They found some correlations:

The numbers seem solid. It was a large study. They used whole-brain correction for multiple comparisons (a=0.05 FWE corrected), controlling for age, gender and overall brain grey matter.

Most importantly, they included a replication sample, something that very few neuroscience papers do. After having done the first 125 people, they got another 40, and looked in the areas where they'd previously found results. They found the same correlation in all three cases - in fact, it was even stronger.

They even made sure to only display the scatterplots from the replication sample, thus avoiding the dreaded voodoo correlations problem that so often plagues such graphs. Note that the correlations are actually with the square root of the number of friends.

As if this wasn't enough, they confirmed a previously reported correlation between amygdala size and social network size, in both of their samples. And to cap it all, they show that Facebook friends are correlated (albeit not hugely) with other measures of number of friends.

So, as unlikely as it sounds, this Facebook finding is stronger than a good 90% of similar papers.

What does it mean that the size of the amygdala, left MTG, right STS and right entorhinal cortex are correlated with your Friend count? Good question. The authors discuss the result in terms of the known functions of these areas, e.g. the entorhinal cortex is involved in learning to associate pairs of stimuli, such as matching names to faces, which might be related to keeping track of your friends... but frankly this is just a post-hoc story.

You could tell an equally convincing tale about almost any part of the brain, if you found a correlation there. And as the authors point out, they didn't find correlations with other "social" areas you might expect like the mirror neuron system.

But that doesn't change the fact that the results of the study seem rock solid. So what's going on? It could be that having lots of friends makes your brain bigger. Or it could be the reverse, that having a certain kind of brain wins you friends, or at least Facebook ones. Or it could be that there's some third factor underlying the correlation, although who knows what that is.

Kanai, R., Bahrami, B., Roylance, R., & Rees, G. (2011). Online social network size is reflected in human brain structure Proceedings of the Royal Society B: Biological Sciences DOI: 10.1098/rspb.2011.1959

Same-Sex Marriage vs. Cousin Marriage

Came across this photo on Facebook and did some informal research (Google) to learn that of the 50 states, 19 (including California, Alaska, New York, and Florida) permit restriction-free marriages between first cousins. This compares to just five states that recognize marriage equality. I'm not usually a big fan of this "if Brittany Spears can get married" strategy. It basically reinforces the notion that our equality only warrants consideration in the context of a comparison to another, even more inferior group. But nonetheless, this is pretty crazy. I think another fine example of just how wide the political and ideological spectrum runs in the USA.

The Recession and Death

The present economic crisis has led to more suicides in Europe - but fewer deaths in road traffic accidents.

So says a brief report in The Lancet. The authors show that suicide rates in people under the age of 65, which have been falling for several years in Europe, rose in 2008 and again in 2009, in line with unemployment figures. The overall effect was fairly small - 2009 was no worse than 2006. It still corresponds to a 5% annual increase in most countries. In Greece, Ireland, and Latvia the rise was about 15%.

That's sad but not perhaps very surprising.

What's interesting though is that road traffic fatalities fell sharply. In Lithuania, they dropped by nearly half, although they were very high to begin with, and in Spain and Ireland they fell by 25%.

This presumably reflects the fact that people are just driving less, and perhaps slower. We've got less money to spend on fuel, and fewer jobs and things to need to drive to.

The authors note that although fewer road deaths is generally a good thing, there's one downside - a shortage of donor organs for transplantation. Road accidents are a prime source of organs because they're one of the few times that young, healthy people die leaving most of the body intact.

Stuckler D, Basu S, Suhrcke M, Coutts A, & McKee M (2011). Effects of the 2008 recession on health: a first look at European data. Lancet, 378 (9786), 124-5 PMID: 21742166

Neuroscience Fails Stats 101?

According to a new paper, a full half of neuroscience papers that try to do a (very simple) statistical comparison are getting it wrong: Erroneous analyses of interactions in neuroscience: a problem of significance.

Here's the problem. Suppose you want to know whether a certain 'treatment' has an affect on a certain variable. The treatment could be a drug, an environmental change, a genetic variant, whatever. The target population could be animals, humans, brain cells, or anything else.

So you give the treatment to some targets and give a control treatment to others. You measure the outcome variable. You use a t-test of significance to see whether the effect is large enough that it wouldn't have happened by chance. You find that it was significant.

That's fine. Then you try a different treatment, and it doesn't cause a significant effect against the control. Does that mean the first treatment was more powerful than the second?

No. It just doesn't. The only way to find that out would be to compare the two treatments directly - and that would be very easy to do, because you have all the data to hand. If you just compare the two treatments to control you might end up with this scenario:

Both treatments are very similar but one (B) is slightly better so it's significantly different from control, while A isn't. But they're basically the same. It's probably just fluke that B did slightly better than A. If you compared A and B directly you'd find they were not significantly different.

An analogy: Passing a significance test is like winning a prize. You can only do it if you're much better than the average. But that doesn't mean you're much better than everyone who didn't win the prize, because some of them will have almost been good enough.

Usain Bolt is the fastest man in the world (when he's not false-starting himself out of races). Much faster than me. But he's not much faster than the second fastest man in the world.




Nieuwenhuis S, Forstmann BU, & Wagenmakers EJ (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nature neuroscience, 14 (9), 1105-7 PMID: 21878926

New Data on Brands that Support LGBT Nonprofits/Causes

97% of LGBT Internet users say they are likely to consider brands that support nonprofits/causes that are important to them as a gay or lesbian person. That's up from 85% in 2007.