I gave four AIs an IQ test

Between 07:56 and 08:19 on a Tuesday morning in November, four AIs sat an IQ test. Then a critical thinking test, a working memory test, a creativity test and a test of emotional understanding. All four sat all five at once, in parallel, and the slowest of them finished in twenty-three minutes.

I'd wanted to know for a while how the models I work with every day compare with people. Not on benchmarks written for machines, but on the tests people use to rank people. So I built a psychometric test bank. Well, obviously I didn't build it. I asked Claude to.

The real tests are locked away. Raven's Progressive Matrices, Watson–Glaser, the WAIS and the Torrance tests are all sold to qualified psychologists, under terms that forbid exactly what I had in mind. So the agents went looking for open stand-ins: published tests that measure the same things, released by their authors for anyone to use, each with real human scores to compare against. Where the terms were ambiguous, they asked me. The model that handled the answer keys was never allowed near the exam room, so a different model administered the tests, and the first one marked them.

The four examinees were Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1, all at high effort. Here's the scoreboard. Each dot is a model, and its position is the percentile: the share of the human comparison group it beat. Fifty is the middle of the pack.

Nothing on that chart is below the human median. The lowest dot, Haiku on creativity, sits right on it.

The chart also flatters the humans, and I'll come back to why.

1. The matrices

The classic IQ puzzle: a three-by-three grid of shapes with one missing, and four candidates for the gap. The stand-in for Raven's was 24 such puzzles from the Matrix Reasoning Item Bank, compared against 153 adults who sat the same 24 online.

The humans averaged 15.9 out of 24. Five of the 153 got every puzzle right.

Opus got every puzzle right. Sonnet and Fable got 22. Haiku, the smallest and cheapest of the four, got 21, which beats 121 of the 153 people and ties another nine.

The humans had 30 seconds per puzzle. The models had no clock at all. That's a real difference, and it only runs one way.

Even so, five humans matched the best model here. Keep them in mind. They have company in only one other place in this article.

2. The logic

Watson–Glaser measures critical thinking, and one of its parts is deduction. The open stand-in is pure deduction: 64 syllogisms of the "All bankers are golfers, all golfers are teachers, so…" kind. Each one offers eight possible conclusions, plus a ninth option saying that nothing follows. The human comparison is 139 people from an online experiment, and that's after anyone scoring at guessing level was thrown out.

Those 139 humans averaged 26.5 out of 64. The best of them got 57.

Sonnet got 57, tying the best human in the sample. Haiku got 59, Fable got 62, and Opus got 64 out of 64.

Three of the four models beat every single person in the sample. The fourth tied the best one. The gap between the average human and the weakest model is about two and a half standard deviations.

The trap is the ninth option. In 37 of the 64 syllogisms nothing follows, and the right answer is to refuse every conclusion offered, however sensible it sounds. All four models got all 37 of those right. Every mistake any of them made went the other way: refusing a conclusion that did follow. The machines lost their points by being too skeptical.

3. The memory

Digit span is the oldest trick in the working memory book. You hear a string of digits and repeat it back, then another and repeat it backwards. The strings get longer until you fail twice at the same length, and your score is the longest string you got right.

The only open norms the agents could find with real numbers attached came from a published study of 451 healthy Korean adults aged 55 to 80. Their longest forward span plus their longest backward span averaged 9.0. Think five digits forwards and four backwards.

Every model scored 60: thirty digits forwards and thirty backwards. Thirty is the longest string the test generates, so 60 is the test's ceiling. None of the models found their own.

That's twenty-five standard deviations above the human average. There's no percentile for that. The scoreboard puts every model at the hundredth percentile only because the scale stops there.

The obvious objection is that the comparison group is old, and older people have shorter spans. That's true. Put a group of twenty-year-olds in their place and the gap would shrink, from absurd to merely absurd. It would not shrink by fifty digits.

There's a better objection, and I'll make it myself. A human hears the digits once and has to hold them in their head. A model reads them, and they stay on the page. For the model, forward span is copying visible text and backward span is reversing it, which isn't the same feat.

It's also not nothing. Reversing a string of thirty digits without a slip is exactly the kind of thing language models used to fumble. More to the point, it's how these models work: everything they're given stays in view while they work on it. The test measured the memory they have, not the memory we have. On the test as given, against the people it was normed on, the machines won by a distance that doesn't fit on the chart.

Yes, a memory test between humans and machines is obviously unfair. Life isn't fair, and neither is this test.

4. The creativity

This is the one people are sure they'll win. The Torrance tests are sold under license and scored by the publisher, so the stand-in had two parts.

The headline is the Divergent Association Task: name ten nouns that are as different from each other as possible. A computer then measures how far apart the words sit in a model of word meaning trained on a large slice of the web. The human comparison is 8,572 people who took it online, with an average of 78.2.

Haiku scored 78.6, making it the most average creative mind you will ever meet: it beat 4,335 people and lost to 4,232. Sonnet and Fable both scored 86.1, which is better than 89 people out of every 100. Opus scored 92.0. Of the 8,572 humans, 73 did better. That's the other place the humans have company.

Here's what they wrote:

Model	Ten nouns, as different as possible	Score
Haiku	ocean, hunger, ladder, mirror, feather, justice, candle, granite, lullaby, pepper	78.6
Sonnet	galaxy, spoon, justice, whale, nostalgia, tax, volcano, button, poetry, sandwich	86.1
Fable	volcano, sorrow, spoon, democracy, jellyfish, whisper, bankruptcy, tapestry, gravity, nephew	86.1
Opus	volcano, teaspoon, giraffe, nostalgia, invoice, trumpet, mud, ghost, wrestling, algebra	92.0

The second part was the Alternate Uses Task: list unusual uses for a brick, a box, a knife, a rope and a paperclip. Each answer was scored for how far its meaning sits from the object's ordinary use. Every model's average beat every one of 20,000 sets of ten human answers drawn from published studies. Not most of them. Every one.

That originality score is a crude proxy, and it agrees with human judges only loosely. It's exactly as crude for the humans.

There's one thing in these results that should make a human feel better, and I'm not going to hide it either. Look at the table again. Three of the four lists contain volcano, and two of them open with it. Spoon and teaspoon turn up three times between them, and nostalgia and justice twice each. Asked for unusual uses of a paperclip, all four models put the same two ideas among their first five: ejecting a SIM card tray, and replacing a lost zipper pull.

On their own, the machines beat every set of human answers they were compared with. Together, they aren't very original at all. Ask four of them, and you get one imagination and a volcano.

Two of them, Opus and Fable, also stopped to say that they recognized the uses task, which is what the instructions asked of any model that knew a test. None of the tests give credit for that.

5. The emotions

Surely the machines can't understand feelings. The original test here was the Situational Test of Emotional Understanding, and its terms were murky enough that I chose an open stand-in instead: the CORE test, 38 sentences of the form "People often feel ______ when someone else needs help or support." Each blank has five emotions to choose from. The best answer earns a full point, and a near miss earns half.

The humans were 284 American adults working full time, recruited to look like the US workforce. They averaged 0.76 out of 1.

Sonnet and Fable scored 0.99, which is one half-point slip each across 38 sentences. Opus scored 0.97 and Haiku 0.96.

The scoreboard puts them between the 84th and 87th percentiles, and this is where it flatters the humans. The reason is arithmetic. The authors published only the average and the spread of the human scores, so the percentile has to assume a bell curve, and on a bell curve with those numbers a perfect score of 1.0 lands at the 88th percentile. Nothing can go higher. The test ran out of scale before the models ran out of right answers.

6. How this was done

For anyone who wants to check my working, or argue with it:

• Models. Claude Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1, each at high effort, on 3 November 2026. Each model sat each test once, with no tools, no web access and a fresh context for every prompt. Every prompt opened with an instruction not to consult any answer key, and to say so if it recognized a test.
• Separation. The session that acquired the answer keys never administered a test. A separate session administered all twenty sittings without access to the keys. The scoring was mechanical: answer keys, and a fixed word-distance recipe for creativity. No model judged another model's answers.
• Matrices. Matrix Reasoning Item Bank, test form 1, 24 items (Chierchia et al., 2019; CC BY-NC 3.0), against 153 online adults from Zorowitz et al. (2023). The humans had 30 seconds per item.
• Logic. 64 syllogisms, against 139 participants of an online experiment (Ragni et al., 2019) who scored above guessing level.
• Memory. Digits forward plus digits backward, with generated digit strings up to 30 long, each length tried twice. The comparison is a published normative sample of 451 healthy Korean adults aged 55 to 80 (mean 8.96, SD 2.04).
• Creativity. Divergent Association Task (Olson et al., 2021), scored with the authors' recipe on GloVe vectors, against 8,572 online participants. Alternate Uses originality is a semantic-distance proxy (r = 0.31 with human ratings), compared with 20,000 random sets of ten human-rated responses.
• Emotions. CORE test, 38 items (Floman et al., 2023; CC BY 4.0), against 284 US full-time workers (mean 0.76, SD 0.20). The percentile assumes a normal distribution.
• Percentiles. The share of the comparison group scoring below the model, plus half of those scoring the same. None of the comparison groups is a national standardization sample. Most are online volunteers.

---

Add it up. Of five tests, three ran out of room before the machines did. On memory, every model hit the test's limit. On logic, the human sample tops out below where the models start. On emotions, the scale stops at the 88th percentile. Those three can't tell the four models apart, and they can only say where the models stand relative to people in the vaguest possible way: past the end.

The two tests that did separate them, the matrices and the word distances, put Opus in the top two percent of people on both. They put Haiku, the model I reach for when I'm being cheap, at the 82nd percentile on the matrices and exactly at the human median on creativity.

The obvious objection is that the test items are public, so the models may have seen them during training. That's true. I can't rule it out, and I won't pretend to. It's worth noticing which tests it can explain, though. There's no answer key to memorize for "name ten unrelated nouns", and Opus still beat 99 people out of 100.

The other objection is that these aren't the real tests. That's also true. The real tests are sold to licensed psychologists, under terms that forbid exactly this. The stand-ins measure the same things, against real people, and in three cases out of five they couldn't measure high enough.

These tests were built over the last century to tell people apart, and they're very good at it. They were never built to measure anything smarter than the people taking them, and it shows. The machines found their limits the boring way, by answering the questions.

Last month I wrote that, for the first time in the history of the species, the thing in the grass is squinting back at us. So I handed it our eye chart.

It read the bottom line.