Do Implicit Bias Tests Work? What They Measure, What They Miss, and a Different Question Worth Asking

If you have taken an implicit bias test, you probably remember the result better than the instructions. A label arrives at the end — slight, moderate, strong — and it lands like a verdict. It is worth knowing what that label is built on, because the answer is more interesting, and more useful, than either “it proves you are biased” or “it is junk science”.

The question underneath the question

Before asking whether a test is accurate, there is a duller question that has to come first: is it stable? If you take the same test twice a fortnight apart and get two different answers, then at least one of them is wrong, and the test cannot tell you which. Psychologists call this test–retest reliability, and it is the ceiling on everything else. A measure that does not agree with itself cannot agree with reality.

This matters more than it sounds. Reliability is not a technicality that only researchers care about — it is the difference between a bathroom scale and a magic 8-ball. Both give you a number.

How the implicit association test scores on its own terms

The Implicit Association Test (IAT) asks you to sort words and images into paired categories as fast as you can. When two categories share a key and the pairing feels natural, you are quicker. The gap between your speed on one pairing and the other becomes a score. It is an ingenious design, it has been studied for nearly thirty years, and it has generated genuinely important research.

Predictive validity — whether the score forecasts what someone actually does — has drifted downward as samples have grown. A 2013 meta-analysis in JPSP put the correlation with discriminatory behaviour at about 0.148, roughly 2% of the variance, and found the IAT performed no better than simple self-report questions⁴. A 2024 analysis pooling ten intergroup domains and 250 outcomes put it at 0.079³.

And the mechanism itself is now contested. A 2026 registered report in Nature Human Behaviour, covering 39 topics and more than 115,000 participants, found that how cautiously a person trades speed for accuracy explained more of the variance in scores than decision ease — the associative component the score is meant to capture⁵. That is not a fringe objection; it is the largest and best-controlled study of the instrument’s mechanics yet published.

The step that usually gets skipped

Suppose the measurement problem were solved tomorrow. There is still a link in the chain that rarely gets examined: does knowing your score change what you do?

The largest attempt to answer that is a network meta-analysis of 492 studies and more than 87,000 participants. Implicit measures can be nudged, weakly. Effects on behaviour were described as trivial. And decisively: changes in implicit measures did not mediate changes in behaviour⁶. A companion study tested nine interventions; all nine moved the score immediately, and none survived a delay of a few hours to a few days⁷.

There is one outcome that does hold up, and it is worth being precise about. A meta-analysis of 260 independent samples and more than 29,000 participants separated four kinds of result. Reactions, attitudes and behaviour all decayed over 0–24 months. Cognitive learning — knowing what a pattern is, what it is called, how to spot it — stayed stable and sometimes grew⁸. What sticks is the knowledge, not the sentiment.

A different question: what did you not notice?

A blind spot is not an attitude. It is the gap between what was in front of you and what you registered. That distinction turns out to matter enormously, because a gap has a ground truth and an attitude does not. If a transcript shows that one person proposed something at 09:12 and a second person was assigned it at 09:19, that is a fact you can point at. Whether anybody harboured a feeling about anyone is not.

That is the design behind our Blind Spots exercise. You read short workplace moments — a meeting transcript, a close-out report, two sets of appraisal notes — and say whether you think something is going wrong. What is planted in each one is structural and checkable: who said it first, what the contributions table lists, how many times a standing item was cut for time. Not a claim about anyone’s motives.

And here is the part most tests of this kind get wrong. About half the moments have nothing planted in them at all. They contain ordinary friction with an innocent cause stated right there in the text — a proposal declined because the budget was committed, a curt reply from someone who has been handling an escalation since 4am, feedback that is thin for everyone rather than for one person.

Without those, the exercise would be worthless. Consider two readers who both catch 9 of 10 planted moments. One of them flagged 1 ordinary moment; the other flagged 7. Their accuracy on the problems is identical, and they are not remotely the same reader. Only the ordinary moments reveal it — the approach is borrowed from signal detection theory, which separates how well you tell two things apart from how readily you call something a problem⁹.

The five patterns it looks for

What it reports, and what it refuses to

Two numbers. Sensitivity is how reliably you separated the planted moments from the ordinary ones. Threshold is how readily you call something a problem — and neither end of it is the good end. Flagging everything and flagging nothing are both costly, in different ways, so the exercise describes where you set the bar rather than grading it.

That refusal is deliberate, and it is not squeamishness. Priming someone to feel objective has been shown to increase discriminatory decisions in hiring tasks¹³. A screen that congratulates you on being fair-minded is the single most counterproductive thing a tool like this could produce — which is why there is no badge, no share card, and no “you passed”.

What this is worth to a team

Here is the claim we will not make: that taking an exercise makes anyone less biased. Section three of this post is the reason. What the evidence supports is narrower and, we think, more useful — people retain what they learn about a pattern, and changing the procedure changes the outcome regardless of what anyone believes.

So the exercise is an on-ramp and a shared vocabulary, not a fix. The fixes are procedural, and these are the ones with real evidence behind them:

• Structure your interviews. Structured interviews predict job performance roughly twice as well as unstructured ones — 0.42 against 0.19 in the current best meta-analytic estimates¹⁴ — and a field study of nearly 20,000 real applicants found demographic-similarity effects in highly structured interviews were trivially small²⁰.
• Prompt at the moment of decision. In a randomised trial across 3,644 managers and 367,805 applications, a seven-minute video watched immediately before shortlisting made a woman 3 percentage points more likely to reach the short list, and raised the hiring of non-national women by 41% in relative terms. Hiring of women overall did not move significantly — the effect is on who gets seen¹⁵.
• Compare candidates side by side rather than one at a time. In a controlled study, evaluating separately produced a 16-point gap favouring the stereotype-advantaged candidate; evaluating jointly eliminated it and substituted actual performance¹⁶.
• Attribute contributions in the notes as they are made, and make individual contribution legible before the credit conversation rather than during it¹¹.
• Log incidents. Judge the third against the log rather than the second against a feeling.
• Make it voluntary. Across 829 firms and three decades, mandatory diversity training was associated with falls in managerial representation for some groups, while voluntary training and accountability structures such as task forces showed the largest gains in the dataset¹⁷.

It is equally worth knowing what does not hold up. Anonymising applications is widely recommended and has repeatedly disappointed in the field: an Australian trial of more than 2,100 public servants found de-identification made female and Indigenous candidates less likely to be shortlisted, because reviewers had been applying a deliberate thumb on the scale that blinding removed¹⁸. A French randomised trial found anonymous résumés widened the gap for minority candidates in the firms that volunteered for the scheme¹⁹. Good intentions and good evidence are different things.

What we do not know yet

The obvious question to ask of any new instrument is the one we asked of the IAT at the top: does it hold steady between sittings? We do not know yet, and we are not going to pretend otherwise. The exercise ships in two parallel forms so that people can take one, come back a week later, take the other, and let us compare the two.

We have written down in advance what each result means. If the correlation comes back above 0.60, the personal reading stands. Between 0.40 and 0.60, we will only show a number to people who have completed both forms. Below 0.40, the personal number gets withdrawn and the exercise becomes a teaching tool with a group-level readout. Committing to that before seeing the data is the only point at which it can be decided honestly.