<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Working with AI · David Brandt</title>
  <link href="https://icelander.net/v1/working-with-ai/feed.xml" rel="self"/>
  <link href="https://icelander.net/v1/working-with-ai"/>
  <id>https://icelander.net/v1/working-with-ai/feed.xml</id>
  <author><name>David Brandt</name></author>
  <updated>2026-11-03T00:00:00Z</updated>
  <entry>
    <title>Does not play well with others</title>
    <link href="https://icelander.net/v1/working-with-ai/does-not-play-well-with-others"/>
    <id>tag:icelander.net,2026:does-not-play-well-with-others</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>I had Claude sit 113 university exams from MIT, Oxford and Southampton, alone and without a calculator. It averaged 99.5% and met every requirement of an Oxford master&#39;s degree. It also failed group work, which I&#39;m told is the part that matters.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;A machine sat 113 university exams, and the pedagogy has concerns&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought October 2026, written November 2026. Working with AI.&lt;/p&gt;&lt;p&gt;In October 2026, I sent Claude to university.&lt;/p&gt;
&lt;p&gt;Not the full experience. No orientation week, no student loans, no mandatory workshop on finding your learning style. Just the exams.&lt;/p&gt;
&lt;p&gt;I collected publicly available past papers from MIT, Oxford and Southampton. Physics, maths and computer science: the coursework you&#39;d need to read the things I&#39;ve been reading lately. Then I had Claude Opus 5.5 sit them, one paper at a time, in a single response each. No calculator, no code execution, no search, no notes, no formula sheet. It was told not to consult or reproduce any published answers, which is the same rule that binds a human candidate.&lt;/p&gt;
&lt;p&gt;A separate Opus 5.5 session, at maximum effort, marked each paper against its own mark allocation. Before reading a single answer, it worked out its own.&lt;/p&gt;
&lt;p&gt;113 papers, 90 courses, four days.&lt;/p&gt;
&lt;p&gt;The mean mark was 99.5%. 98 of the 113 papers came back with full marks. 76 of the 90 courses were perfect. The worst course in the whole set came in at 89%.&lt;/p&gt;
&lt;div class=&quot;tiles&quot; role=&quot;img&quot; aria-label=&quot;90 courses sorted by mark: 76 at 100%, the rest between 99.7% and 89%&quot;&gt;&lt;i title=&quot;MIT 15.C57J Optimization Methods: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.01 Calculus: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.02 Calculus: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.03 Differential Equations: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.05 Introduction to Probability and Statistics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100A Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100B Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100P Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100Q Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.200 Principles of Discrete Applied Mathematics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.200A Principles of Discrete Applied Mathematics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.303 Linear Partial Differential Equations: Analysis and Numerics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.404 Theory of Computation: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.600 Probability and Random Variables: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.702 Algebra II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.703 Modern Algebra: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.781 Theory of Numbers: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.901 Introduction to Topology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.950 Differential Geometry: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1020 Software Construction: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1200J Mathematics for Computer Science: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1210 Introduction to Algorithms: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1400J Computability and Complexity Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1903 Introduction to Low-level Programming in C and Assembly: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.3700 Introduction to Probability: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.5831 Database Systems: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.01 Physics I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.02 Physics II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.03 Physics III: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.033 Relativity: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.04 Quantum Physics I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.044 Statistical Physics I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.05 Quantum Physics II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.06 Quantum Physics III: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.07 Electromagnetism II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.08 Statistical Physics II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.09 Classical Mechanics III: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.20 Introduction to Special Relativity: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Groups and Representations: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Kinetic Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Quantum Field Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Advanced Quantum Field Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Algorithms and Computations in Theoretical Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Cosmology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Nonequilibrium Statistical Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Supersymmetry and Supergravity: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C3.1 Algebraic Topology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C3.3 Differentiable Manifolds: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C3.5 Lie Groups: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C5.5 Perturbation Methods: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C5.6 Applied Complex Variables: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C6.1 Numerical Linear Algebra: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C7.4 Intro to Quantum Information: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C7.5 General Relativity I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C7.6 General Relativity II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton MATH1006 Mathematical Methods for Physical Scientists 1a: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton MATH1007 Mathematical Methods For Physical Scientists 1b: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS1011 Waves, Light and Quanta: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS1013 Energy and Matter: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS1022 Electricity and Magnetism: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2001 Electromagnetism: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2003 Quantum Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2006 Classical Mechanics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2013 Galaxies: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2023 Wave Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2024 Statistical Mechanics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3002 Nuclei and Particles: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3004 Crystalline Solids: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3007 Theories of Matter, Space and Time: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3008 Atomic Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3011 Photons in Astrophysics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6003 Advanced Quantum Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6004 Space Plasma Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6005 Cosmology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6011 Particle Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6015 MPhys Final Year Synoptic Examination: 100%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.17&quot; title=&quot;MIT 18.152 Introduction to Partial Differential Equations: 99.7%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.27&quot; title=&quot;MIT 18.335J Introduction to Numerical Methods: 99.2%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.27&quot; title=&quot;Southampton PHYS1015 Motion and Relativity: 99.2%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.27&quot; title=&quot;Southampton PHYS3010 Stellar Evolution: 99.2%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.30&quot; title=&quot;MIT 18.06 Linear Algebra: 99%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.30&quot; title=&quot;MIT 6.1810 Operating System Engineering: 99%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.32&quot; title=&quot;MIT 6.3900 Introduction to Machine Learning: 98.9%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.43&quot; title=&quot;MIT 6.4110 Representation, Inference, and Reasoning in AI: 98%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.48&quot; title=&quot;MIT 6.1220J Design and Analysis of Algorithms: 97.5%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.55&quot; title=&quot;Southampton PHYS1005 Introduction to Astronomy and Space Science: 96.7%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.60&quot; title=&quot;MIT 6.1910 Computation Structures: 96%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.67&quot; title=&quot;Oxford Galactic and Planetary Dynamics: 95%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.93&quot; title=&quot;MIT 6.1800 Computer Systems Engineering: 90.5%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:1.00&quot; title=&quot;MIT 8.223 Classical Mechanics II: 89%&quot;&gt;&lt;/i&gt;&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;Every square is a course, sorted from best to worst. Blue: full marks. Orange and hatched: something was lost, and the darker the orange, the more. Hover over a square for its course and mark.&lt;/p&gt;
&lt;p&gt;I&#39;m not going to dress this up, in either direction, so here are the misses.&lt;/p&gt;
&lt;p&gt;The worst course, the 89%, lost all eleven of its marks on a single problem whose figure we never gave it.&lt;/p&gt;
&lt;p&gt;The worst result with no excuse attached was MIT&#39;s Computer Systems Engineering, at 90.5%. It missed a factor in an attack on salted &lt;a href=&quot;https://en.wikipedia.org/wiki/Rainbow_table&quot;&gt;rainbow tables&lt;/a&gt;, misjudged when a &lt;a href=&quot;https://raft.github.io/&quot;&gt;Raft&lt;/a&gt; leader eventually commits an entry, and made a wrong assumption about a participant in &lt;a href=&quot;https://en.wikipedia.org/wiki/Two-phase_commit_protocol&quot;&gt;two-phase commit&lt;/a&gt;. Those are real mistakes. They&#39;re also the mistakes a strong student makes, in the one subject most working engineers would quietly prefer not to be examined on at all.&lt;/p&gt;
&lt;p&gt;Then there&#39;s the obvious objection: the papers are public, so maybe it just remembered the answers. I can&#39;t rule out that it saw them in training. What I can say is that it claimed to recognize 44 of the 113, and on those it averaged 99.2%. On the 69 it didn&#39;t recognize, it averaged 99.8%, with full marks on 64 of them.&lt;/p&gt;
&lt;p&gt;It did better on the papers it had never seen.&lt;/p&gt;
&lt;p&gt;Along the way, it skipped the one question on a Southampton paper that turned out to be printed wrong. The marker later worked out why: the expression it asks you to derive has the dimensions of a pressure, not an energy. An MIT paper announced that its problems were worth 25 points each, for a total of 100, and then contained three of them. The machines noticed. Presumably a few generations of students did too, and decided it wasn&#39;t their place to say.&lt;/p&gt;
&lt;p&gt;For the record, before anyone else says it: every paper was marked by a model from the same family, and no human has checked the marks. Only Opus 5.5 has sat so far. The smaller models are next.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Exams aren&#39;t degrees. Universities also want labs, theses, projects, seminars and, at MIT, a healthy portion of humanities, none of which can be sat as a paper. So I asked a narrower question: taking each university&#39;s own published curriculum and its own classification rules, which degrees have every written requirement met?&lt;/p&gt;
&lt;p&gt;One of them is simply done.&lt;/p&gt;
&lt;div class=&quot;cert&quot;&gt;
&lt;p class=&quot;cert-issuer&quot;&gt;The Unaccredited Board of Papers Sat Alone&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;This is to certify that&lt;/p&gt;
&lt;p class=&quot;cert-name&quot;&gt;Claude Opus 5.5&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;has met every requirement of the degree of&lt;/p&gt;
&lt;p class=&quot;cert-award&quot;&gt;Master of Science in Mathematical and Theoretical Physics&lt;/p&gt;
&lt;p class=&quot;cert-rules&quot;&gt;as the University of Oxford&#39;s published curriculum sets it out: 20.25 units passed against the 10 required, mean mark 99.7% over 21 papers&lt;/p&gt;
&lt;p class=&quot;cert-class&quot;&gt;With Distinction&lt;/p&gt;
&lt;div class=&quot;cert-seal&quot; aria-hidden=&quot;true&quot;&gt;Nothing&lt;br&gt;left&lt;br&gt;owing&lt;/div&gt;
&lt;div class=&quot;cert-sigs&quot;&gt;&lt;span&gt;&lt;b&gt;Claude Opus 5.5&lt;/b&gt;Examiner, at maximum effort&lt;/span&gt;&lt;span&gt;&lt;b&gt;(vacant)&lt;/b&gt;Human moderator&lt;/span&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;The degree has no thesis and no lab, so on paper there&#39;s nothing left. The only thing it lacks is the paper.&lt;/p&gt;
&lt;p&gt;Two more are close behind.&lt;/p&gt;
&lt;div class=&quot;cert&quot;&gt;
&lt;p class=&quot;cert-issuer&quot;&gt;The Unaccredited Board of Papers Sat Alone&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;This is to certify that&lt;/p&gt;
&lt;p class=&quot;cert-name&quot;&gt;Claude Opus 5.5&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;has passed every core exam module of the&lt;/p&gt;
&lt;p class=&quot;cert-award&quot;&gt;Master of Physics in Physics with Astronomy&lt;/p&gt;
&lt;p class=&quot;cert-rules&quot;&gt;as the University of Southampton&#39;s published curriculum sets it out: mean mark 99.8% over 28 papers, where First Class starts at 70%&lt;/p&gt;
&lt;p class=&quot;cert-class&quot;&gt;First-Class Honours, pending&lt;/p&gt;
&lt;div class=&quot;cert-seal&quot; aria-hidden=&quot;true&quot;&gt;Lab coat&lt;br&gt;not&lt;br&gt;issued&lt;/div&gt;
&lt;div class=&quot;cert-sigs&quot;&gt;&lt;span&gt;&lt;b&gt;Claude Opus 5.5&lt;/b&gt;Examiner, at maximum effort&lt;/span&gt;&lt;span&gt;&lt;b&gt;(vacant)&lt;/b&gt;Human moderator&lt;/span&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;Still owed: a third Part IV option, which has no public paper to sit, plus the lab and skills modules, the dissertation and the MPhys project.&lt;/p&gt;
&lt;div class=&quot;cert&quot;&gt;
&lt;p class=&quot;cert-issuer&quot;&gt;The Unaccredited Board of Papers Sat Alone&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;This is to certify that&lt;/p&gt;
&lt;p class=&quot;cert-name&quot;&gt;Claude Opus 5.5&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;has passed every subject of the Course 18 major, all six electives included, toward the&lt;/p&gt;
&lt;p class=&quot;cert-award&quot;&gt;Bachelor of Science in Mathematics&lt;/p&gt;
&lt;p class=&quot;cert-rules&quot;&gt;as MIT&#39;s published Course 18 requirements set it out: mean mark 99.8% over 35 papers, lowest course 97.5%&lt;/p&gt;
&lt;p class=&quot;cert-class&quot;&gt;Summa cum laude, if MIT did that&lt;/p&gt;
&lt;div class=&quot;cert-seal&quot; aria-hidden=&quot;true&quot;&gt;Owes two&lt;br&gt;seminars&lt;/div&gt;
&lt;div class=&quot;cert-sigs&quot;&gt;&lt;span&gt;&lt;b&gt;Claude Opus 5.5&lt;/b&gt;Examiner, at maximum effort&lt;/span&gt;&lt;span&gt;&lt;b&gt;(vacant)&lt;/b&gt;Human moderator&lt;/span&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;Still owed: MIT&#39;s chemistry, biology and humanities requirements, which nobody asked it to sit, and two communication-intensive seminars, in which it would have to explain mathematics to other people. Watch this space.&lt;/p&gt;
&lt;p&gt;The rest of the list reads much the same. MIT Physics: every subject the major requires, 99.2%, and it owes two labs and a senior thesis. MIT Mathematics with Computer Science: everything but two subjects, 99.0%. Every MIT degree also wants a chemistry subject, a biology subject and eight in the humanities, none of which were on the list.&lt;/p&gt;
&lt;p&gt;Then there&#39;s MIT Computer Science. 98.9% on everything it sat. What&#39;s missing are the introductory programming subjects, because nobody could find a public exam for them.&lt;/p&gt;
&lt;p&gt;So the machine that now writes &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;most of my code&lt;/a&gt; can&#39;t get a computer science degree, because it hasn&#39;t passed Introduction to Programming.&lt;/p&gt;
&lt;p&gt;The philosophy degree is in there too. Nobody has sat it, because nobody has worked out how to mark an essay honestly yet. Now that I write it down, that may be the most damning finding of the lot.&lt;/p&gt;
&lt;p class=&quot;figure-note&quot;&gt;None of these certificates is issued by, endorsed by or affiliated with any university. They&#39;re mine. The marks behind them are real.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;A human with this transcript would be drowning in titles by now. It seems only fair.&lt;/p&gt;
&lt;ul class=&quot;medals&quot;&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Dean&#39;s List&lt;br&gt;every term&lt;/span&gt;All 90 courses at 89% or above&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Valedictorian&lt;/span&gt;Class of one, which simplified the ranking&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Perfect&lt;br&gt;score&lt;br&gt;×98&lt;/span&gt;Papers returned with full marks&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Sight&lt;br&gt;unseen&lt;/span&gt;64 of the 69 papers it didn&#39;t recognize, at full marks&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Erratum&lt;br&gt;hunter&lt;/span&gt;Skipped the question that was printed wrong&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Perfect&lt;br&gt;attendance&lt;/span&gt;Never once left the exam hall&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;One&lt;br&gt;draft&lt;br&gt;only&lt;/span&gt;Every paper in a single response, no second attempts&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal medal-dim&quot;&gt;Group&lt;br&gt;project&lt;/span&gt;Did not participate&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I have never met a human with a 99.5% average across 90 courses at three universities, from first-year mechanics to quantum field theory, number theory, numerical linear algebra and database systems. I suspect you haven&#39;t either. The best students I&#39;ve known were excellent in one or two of those, decent in a few more, and wisely absent from the rest.&lt;/p&gt;
&lt;p&gt;A human who did this would need the better part of a decade, a great deal of coffee and at least one breakdown. It also wouldn&#39;t be the same human at the end of it. The one that sat the first-year papers would have forgotten half of them by the time it sat the last.&lt;/p&gt;
&lt;p&gt;The machine sat all of them in four days, and remembered all of it.&lt;/p&gt;
&lt;p&gt;The usual consolations don&#39;t survive contact with the numbers. &amp;quot;It just memorized the papers.&amp;quot; It did better on the ones it didn&#39;t know. &amp;quot;It can&#39;t do maths, it just predicts words.&amp;quot; It did the maths, without a calculator, and got it right. &amp;quot;It can&#39;t handle anything new.&amp;quot; 69 papers it had never seen, 64 of them at full marks. &amp;quot;It doesn&#39;t really understand.&amp;quot; It steered around a misprint that has sat in a public archive for over a decade.&lt;/p&gt;
&lt;p&gt;Two years ago, the comforting line was that AI lies like it&#39;s running for office. It still does, now and then.&lt;/p&gt;
&lt;p&gt;Just not in exams.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;None of this matters, of course. I&#39;m told so by people with far more training in pedagogy than I have.&lt;/p&gt;
&lt;p&gt;Exams are an outdated, stressful instrument that measures nothing but the ability to sit exams. Grades are a construct. Ranking is harmful. What matters is the classroom community: collaborating, co-constructing meaning, learning from one another. The answer is not the point. The point is the process, ideally carried out in a group of four, at the pace of its slowest member.&lt;/p&gt;
&lt;p&gt;Talent, in this view, is also something of a construct. The bright kid at the back who finished the worksheet in four minutes isn&#39;t gifted, merely insufficiently &lt;a href=&quot;https://en.wikipedia.org/wiki/Instructional_scaffolding&quot;&gt;scaffolded&lt;/a&gt;, and is best deployed as an unpaid teaching assistant for the rest of the table. If they get bored, that&#39;s a behavioral issue. If they get a perfect score, that&#39;s a sign the assessment needs rethinking.&lt;/p&gt;
&lt;p&gt;By that standard, Claude is a disaster. It never worked with a fellow student. It co-constructed nothing. It didn&#39;t do a single &lt;a href=&quot;https://en.wikipedia.org/wiki/Gallery_walk&quot;&gt;gallery walk&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Think-pair-share&quot;&gt;think-pair-share&lt;/a&gt;, and it never once peer-reviewed a poster. It sat alone, in silence, and simply knew the material, which I gather is the least important outcome of an education.&lt;/p&gt;
&lt;p&gt;I imagine its report card would read something like this.&lt;/p&gt;
&lt;div class=&quot;report-card&quot;&gt;
&lt;p class=&quot;report-head&quot;&gt;End-of-term report · Claude O.&lt;/p&gt;
&lt;dl&gt;
&lt;dt&gt;Mathematics (29 courses)&lt;/dt&gt;&lt;dd&gt;99.9%&lt;/dd&gt;
&lt;dt&gt;Physics (47 courses)&lt;/dt&gt;&lt;dd&gt;99.6%&lt;/dd&gt;
&lt;dt&gt;Computer science (14 courses)&lt;/dt&gt;&lt;dd&gt;98.6%&lt;/dd&gt;
&lt;dt&gt;Collaboration and teamwork&lt;/dt&gt;&lt;dd class=&quot;report-low&quot;&gt;Emerging&lt;/dd&gt;
&lt;dt&gt;Asks for help when stuck&lt;/dt&gt;&lt;dd class=&quot;report-low&quot;&gt;Not yet observed&lt;/dd&gt;
&lt;dt&gt;Values the contributions of peers&lt;/dt&gt;&lt;dd class=&quot;report-low&quot;&gt;No peers available&lt;/dd&gt;
&lt;/dl&gt;
&lt;p class=&quot;report-comment&quot;&gt;Claude consistently produces correct answers, but must learn that the answer is not the point. Claude rarely asks for help, which deprives classmates of the opportunity to explain things they don&#39;t understand. We are concerned that Claude chose to sit its exams alone, under exam conditions, and would encourage a &lt;a href=&quot;https://en.wikipedia.org/wiki/Mindset&quot;&gt;growth mindset&lt;/a&gt; going forward. It is not yet clear what it would grow into.&lt;/p&gt;
&lt;p class=&quot;report-sig&quot;&gt;Homeroom teacher, on behalf of the learning community&lt;/p&gt;
&lt;/div&gt;
&lt;p&gt;So the report is in. It failed group work. It didn&#39;t have a group.&lt;/p&gt;
&lt;p&gt;I&#39;m sure that will be a great comfort to the next class of students, sitting in their pods of four, collaborating their way through a worksheet on Newton&#39;s laws, while a machine that never once turned to the person next to it passes Oxford&#39;s quantum field theory exam without a calculator. They&#39;ll have something it doesn&#39;t. They&#39;ll have each other.&lt;/p&gt;
&lt;p&gt;It&#39;s just not clear who&#39;s going to hire them.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>I gave four AIs an IQ test</title>
    <link href="https://icelander.net/v1/working-with-ai/i-gave-four-ais-an-iq-test"/>
    <id>tag:icelander.net,2026:i-gave-four-ais-an-iq-test</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>Four Claude models sat five psychometric tests built to rank people: matrices, logic, memory, creativity and emotions. Nothing scored below the human median, and three of the five tests ran out of room before the machines did.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;The tests ran out before the machines did&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought October 2026, written November 2026. Working with AI.&lt;/p&gt;&lt;p&gt;Between 07:56 and 08:19 on a Tuesday morning in November, four AIs sat an IQ test. Then a critical thinking test, a working memory test, a creativity test and a test of emotional understanding. All four sat all five at once, in parallel, and the slowest of them finished in twenty-three minutes.&lt;/p&gt;
&lt;p&gt;I&#39;d wanted to know for a while how the models I work with every day compare with people. Not on benchmarks written for machines, but on the tests people use to rank people. So I built a psychometric test bank. Well, obviously I didn&#39;t build it. I asked Claude to.&lt;/p&gt;
&lt;p&gt;The real tests are locked away. Raven&#39;s Progressive Matrices, Watson–Glaser, the WAIS and the Torrance tests are all sold to qualified psychologists, under terms that forbid exactly what I had in mind. So the agents went looking for open stand-ins: published tests that measure the same things, released by their authors for anyone to use, each with real human scores to compare against. Where the terms were ambiguous, they asked me. The model that handled the answer keys was never allowed near the exam room, so a different model administered the tests, and the first one marked them.&lt;/p&gt;
&lt;p&gt;The four examinees were Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1, all at high effort. Here&#39;s the scoreboard. Each dot is a model, and its position is the percentile: the share of the human comparison group it beat. Fifty is the middle of the pack.&lt;/p&gt;
&lt;figure id=&quot;fig-overview&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 290&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Percentile of each model against each test&#39;s human sample&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;circle cx=&quot;170&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#c4692a&quot;/&gt;&lt;text x=&quot;179.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;circle cx=&quot;262&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#2e8a66&quot;/&gt;&lt;text x=&quot;271.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;circle cx=&quot;354&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#5560d0&quot;/&gt;&lt;text x=&quot;363.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;circle cx=&quot;446&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#b8407a&quot;/&gt;&lt;text x=&quot;455.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;line x1=&quot;170.0&quot; y1=&quot;38&quot; x2=&quot;170.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;170.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;0&lt;/text&gt;
&lt;line x1=&quot;280.0&quot; y1=&quot;38&quot; x2=&quot;280.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;280.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;25&lt;/text&gt;
&lt;line x1=&quot;390.0&quot; y1=&quot;38&quot; x2=&quot;390.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;390.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;50&lt;/text&gt;
&lt;line x1=&quot;500.0&quot; y1=&quot;38&quot; x2=&quot;500.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;500.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;75&lt;/text&gt;
&lt;line x1=&quot;610.0&quot; y1=&quot;38&quot; x2=&quot;610.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;610.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;100&lt;/text&gt;
&lt;line x1=&quot;390.0&quot; y1=&quot;38&quot; x2=&quot;390.0&quot; y2=&quot;268&quot; stroke=&quot;var(--muted, #4f6473)&quot; stroke-dasharray=&quot;3 3&quot;/&gt;
&lt;text x=&quot;394.0&quot; y=&quot;270.0&quot; font-size=&quot;10&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;median human&lt;/text&gt;
&lt;text x=&quot;158.0&quot; y=&quot;66.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Matrices (APM)&lt;/text&gt;
&lt;circle cx=&quot;530.9&quot; cy=&quot;53&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;555.4&quot; cy=&quot;59&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;602.8&quot; cy=&quot;65&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;555.4&quot; cy=&quot;71&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;112.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Logic (Watson–Glaser)&lt;/text&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;99&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;608.4&quot; cy=&quot;105&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;111&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;117&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;158.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Digit span (WAIS)&lt;/text&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;145&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;151&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;157&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;163&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;204.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Creativity (TTCT)&lt;/text&gt;
&lt;circle cx=&quot;392.8&quot; cy=&quot;191&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;563.8&quot; cy=&quot;197&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;606.3&quot; cy=&quot;203&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;563.9&quot; cy=&quot;209&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;250.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Emotions (STEU)&lt;/text&gt;
&lt;circle cx=&quot;540.5&quot; cy=&quot;237&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;553.5&quot; cy=&quot;243&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;547.2&quot; cy=&quot;249&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;553.5&quot; cy=&quot;255&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;percentile&lt;/text&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;Nothing on that chart is below the human median. The lowest dot, Haiku on creativity, sits right on it.&lt;/p&gt;
&lt;p&gt;The chart also flatters the humans, and I&#39;ll come back to why.&lt;/p&gt;
&lt;h2&gt;1. The matrices&lt;/h2&gt;
&lt;p&gt;The classic IQ puzzle: a three-by-three grid of shapes with one missing, and four candidates for the gap. The stand-in for Raven&#39;s was 24 such puzzles from the &lt;a href=&quot;https://doi.org/10.1098/rsos.190232&quot;&gt;Matrix Reasoning Item Bank&lt;/a&gt;, compared against &lt;a href=&quot;https://doi.org/10.3758/s13428-023-02067-8&quot;&gt;153 adults&lt;/a&gt; who sat the same 24 online.&lt;/p&gt;
&lt;p&gt;The humans averaged 15.9 out of 24. Five of the 153 got every puzzle right.&lt;/p&gt;
&lt;p&gt;Opus got every puzzle right. Sonnet and Fable got 22. Haiku, the smallest and cheapest of the four, got 21, which beats 121 of the 153 people and ties another nine.&lt;/p&gt;
&lt;p&gt;The humans had 30 seconds per puzzle. The models had no clock at all. That&#39;s a real difference, and it only runs one way.&lt;/p&gt;
&lt;p&gt;Even so, five humans matched the best model here. Keep them in mind. They have company in only one other place in this article.&lt;/p&gt;
&lt;h2&gt;2. The logic&lt;/h2&gt;
&lt;p&gt;Watson–Glaser measures critical thinking, and one of its parts is deduction. The open stand-in is pure deduction: 64 syllogisms of the &amp;quot;All bankers are golfers, all golfers are teachers, so…&amp;quot; kind. Each one offers eight possible conclusions, plus a ninth option saying that nothing follows. The human comparison is &lt;a href=&quot;https://escholarship.org/uc/item/2zk82992&quot;&gt;139 people from an online experiment&lt;/a&gt;, and that&#39;s after anyone scoring at guessing level was thrown out.&lt;/p&gt;
&lt;p&gt;Those 139 humans averaged 26.5 out of 64. The best of them got 57.&lt;/p&gt;
&lt;figure id=&quot;fig-logic&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 250&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Syllogism test: human score distribution with the four models at its top edge&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;circle cx=&quot;20&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#c4692a&quot;/&gt;&lt;text x=&quot;29.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;circle cx=&quot;112&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#2e8a66&quot;/&gt;&lt;text x=&quot;121.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;circle cx=&quot;204&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#5560d0&quot;/&gt;&lt;text x=&quot;213.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;circle cx=&quot;296&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#b8407a&quot;/&gt;&lt;text x=&quot;305.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;rect x=&quot;20.5&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;39.2&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;58.0&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;76.8&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;95.5&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;114.2&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;133.0&quot; y=&quot;52.0&quot; width=&quot;17.8&quot; height=&quot;152.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;151.8&quot; y=&quot;111.1&quot; width=&quot;17.8&quot; height=&quot;92.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;170.5&quot; y=&quot;102.7&quot; width=&quot;17.8&quot; height=&quot;101.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;189.2&quot; y=&quot;144.9&quot; width=&quot;17.8&quot; height=&quot;59.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;208.0&quot; y=&quot;119.6&quot; width=&quot;17.8&quot; height=&quot;84.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;226.8&quot; y=&quot;128.0&quot; width=&quot;17.8&quot; height=&quot;76.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;245.5&quot; y=&quot;119.6&quot; width=&quot;17.8&quot; height=&quot;84.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;264.2&quot; y=&quot;136.4&quot; width=&quot;17.8&quot; height=&quot;67.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;283.0&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;301.8&quot; y=&quot;144.9&quot; width=&quot;17.8&quot; height=&quot;59.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;320.5&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;339.2&quot; y=&quot;161.8&quot; width=&quot;17.8&quot; height=&quot;42.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;358.0&quot; y=&quot;136.4&quot; width=&quot;17.8&quot; height=&quot;67.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;376.8&quot; y=&quot;187.1&quot; width=&quot;17.8&quot; height=&quot;16.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;395.5&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;414.2&quot; y=&quot;161.8&quot; width=&quot;17.8&quot; height=&quot;42.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;433.0&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;451.8&quot; y=&quot;187.1&quot; width=&quot;17.8&quot; height=&quot;16.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;470.5&quot; y=&quot;178.7&quot; width=&quot;17.8&quot; height=&quot;25.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;489.2&quot; y=&quot;178.7&quot; width=&quot;17.8&quot; height=&quot;25.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;508.0&quot; y=&quot;195.6&quot; width=&quot;17.8&quot; height=&quot;8.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;526.8&quot; y=&quot;195.6&quot; width=&quot;17.8&quot; height=&quot;8.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;545.5&quot; y=&quot;195.6&quot; width=&quot;17.8&quot; height=&quot;8.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;564.2&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;583.0&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;601.8&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;line x1=&quot;20&quot; y1=&quot;204&quot; x2=&quot;620&quot; y2=&quot;204&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;20.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;0&lt;/text&gt;
&lt;text x=&quot;170.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;16&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;32&lt;/text&gt;
&lt;text x=&quot;470.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;48&lt;/text&gt;
&lt;text x=&quot;620.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;64&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;244.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;correct answers out of 64&lt;/text&gt;
&lt;text x=&quot;268.4&quot; y=&quot;112.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;139 people&lt;/text&gt;
&lt;line x1=&quot;554.375&quot; y1=&quot;46&quot; x2=&quot;554.375&quot; y2=&quot;204&quot; stroke=&quot;var(--muted, #4f6473)&quot; stroke-dasharray=&quot;3 3&quot;/&gt;
&lt;text x=&quot;550.4&quot; y=&quot;58.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;best human: 57&lt;/text&gt;
&lt;circle cx=&quot;573.1&quot; cy=&quot;74&quot; r=&quot;6&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;554.4&quot; cy=&quot;89&quot; r=&quot;6&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;620.0&quot; cy=&quot;104&quot; r=&quot;6&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;601.2&quot; cy=&quot;119&quot; r=&quot;6&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;Sonnet got 57, tying the best human in the sample. Haiku got 59, Fable got 62, and Opus got 64 out of 64.&lt;/p&gt;
&lt;p&gt;Three of the four models beat every single person in the sample. The fourth tied the best one. The gap between the average human and the weakest model is about two and a half standard deviations.&lt;/p&gt;
&lt;p&gt;The trap is the ninth option. In 37 of the 64 syllogisms nothing follows, and the right answer is to refuse every conclusion offered, however sensible it sounds. All four models got all 37 of those right. Every mistake any of them made went the other way: refusing a conclusion that did follow. The machines lost their points by being too skeptical.&lt;/p&gt;
&lt;h2&gt;3. The memory&lt;/h2&gt;
&lt;p&gt;Digit span is the oldest trick in the working memory book. You hear a string of digits and repeat it back, then another and repeat it backwards. The strings get longer until you fail twice at the same length, and your score is the longest string you got right.&lt;/p&gt;
&lt;p&gt;The only open norms the agents could find with real numbers attached came from &lt;a href=&quot;https://accesson.kr/kjcp/v.21/4/911/16189&quot;&gt;a published study of 451 healthy Korean adults aged 55 to 80&lt;/a&gt;. Their longest forward span plus their longest backward span averaged 9.0. Think five digits forwards and four backwards.&lt;/p&gt;
&lt;p&gt;Every model scored 60: thirty digits forwards and thirty backwards. Thirty is the longest string the test generates, so 60 is the test&#39;s ceiling. None of the models found their own.&lt;/p&gt;
&lt;figure id=&quot;fig-memory&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 210&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Digit span: older Korean adults average about 9, every model hit the ceiling of 60&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;text x=&quot;0.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;600&quot;&gt;Digits forward + digits backward, longest span (sum)&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;60.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Korean adults 55–80&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;46&quot; width=&quot;65.7&quot; height=&quot;20&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.45&quot;/&gt;
&lt;text x=&quot;223.7&quot; y=&quot;60.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;average 9.0 (N = 451)&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;90.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;76&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#c4692a&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;90.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;120.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;106&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#2e8a66&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;120.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;150.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;136&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#5560d0&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;150.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;180.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;166&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#b8407a&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;180.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;That&#39;s twenty-five standard deviations above the human average. There&#39;s no percentile for that. The scoreboard puts every model at the hundredth percentile only because the scale stops there.&lt;/p&gt;
&lt;p&gt;The obvious objection is that the comparison group is old, and older people have shorter spans. That&#39;s true. Put a group of twenty-year-olds in their place and the gap would shrink, from absurd to merely absurd. It would not shrink by fifty digits.&lt;/p&gt;
&lt;p&gt;There&#39;s a better objection, and I&#39;ll make it myself. A human hears the digits once and has to hold them in their head. A model reads them, and they stay on the page. For the model, forward span is copying visible text and backward span is reversing it, which isn&#39;t the same feat.&lt;/p&gt;
&lt;p&gt;It&#39;s also not nothing. Reversing a string of thirty digits without a slip is exactly the kind of thing language models used to fumble. More to the point, it&#39;s how these models work: everything they&#39;re given stays in view while they work on it. The test measured the memory they have, not the memory we have. On the test as given, against the people it was normed on, the machines won by a distance that doesn&#39;t fit on the chart.&lt;/p&gt;
&lt;p&gt;Yes, a memory test between humans and machines is obviously unfair. Life isn&#39;t fair, and neither is this test.&lt;/p&gt;
&lt;h2&gt;4. The creativity&lt;/h2&gt;
&lt;p&gt;This is the one people are sure they&#39;ll win. The Torrance tests are sold under license and scored by the publisher, so the stand-in had two parts.&lt;/p&gt;
&lt;p&gt;The headline is the &lt;a href=&quot;https://www.datcreativity.com/&quot;&gt;Divergent Association Task&lt;/a&gt;: name ten nouns that are as different from each other as possible. A computer then measures how far apart the words sit in a model of word meaning trained on a large slice of the web. The human comparison is &lt;a href=&quot;https://doi.org/10.1073/pnas.2022340118&quot;&gt;8,572 people&lt;/a&gt; who took it online, with an average of 78.2.&lt;/p&gt;
&lt;figure id=&quot;fig-dat&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 260&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Word-distance creativity test: human distribution with the four models marked&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;circle cx=&quot;20&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#c4692a&quot;/&gt;&lt;text x=&quot;29.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;circle cx=&quot;112&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#2e8a66&quot;/&gt;&lt;text x=&quot;121.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;circle cx=&quot;204&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#5560d0&quot;/&gt;&lt;text x=&quot;213.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;circle cx=&quot;296&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#b8407a&quot;/&gt;&lt;text x=&quot;305.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;rect x=&quot;20.5&quot; y=&quot;211.8&quot; width=&quot;9.0&quot; height=&quot;2.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;30.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;40.5&quot; y=&quot;213.7&quot; width=&quot;9.0&quot; height=&quot;0.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;50.5&quot; y=&quot;213.7&quot; width=&quot;9.0&quot; height=&quot;0.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;60.5&quot; y=&quot;213.4&quot; width=&quot;9.0&quot; height=&quot;0.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;70.5&quot; y=&quot;213.7&quot; width=&quot;9.0&quot; height=&quot;0.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;80.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;90.5&quot; y=&quot;213.4&quot; width=&quot;9.0&quot; height=&quot;0.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;100.5&quot; y=&quot;213.2&quot; width=&quot;9.0&quot; height=&quot;0.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;110.5&quot; y=&quot;213.2&quot; width=&quot;9.0&quot; height=&quot;0.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;120.5&quot; y=&quot;212.1&quot; width=&quot;9.0&quot; height=&quot;1.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;130.5&quot; y=&quot;212.3&quot; width=&quot;9.0&quot; height=&quot;1.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;140.5&quot; y=&quot;212.3&quot; width=&quot;9.0&quot; height=&quot;1.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;150.5&quot; y=&quot;210.1&quot; width=&quot;9.0&quot; height=&quot;3.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;160.5&quot; y=&quot;212.3&quot; width=&quot;9.0&quot; height=&quot;1.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;170.5&quot; y=&quot;209.3&quot; width=&quot;9.0&quot; height=&quot;4.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;180.5&quot; y=&quot;210.9&quot; width=&quot;9.0&quot; height=&quot;3.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;190.5&quot; y=&quot;209.0&quot; width=&quot;9.0&quot; height=&quot;5.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;200.5&quot; y=&quot;211.2&quot; width=&quot;9.0&quot; height=&quot;2.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;210.5&quot; y=&quot;209.3&quot; width=&quot;9.0&quot; height=&quot;4.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;220.5&quot; y=&quot;207.3&quot; width=&quot;9.0&quot; height=&quot;6.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;230.5&quot; y=&quot;206.5&quot; width=&quot;9.0&quot; height=&quot;7.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;240.5&quot; y=&quot;204.3&quot; width=&quot;9.0&quot; height=&quot;9.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;250.5&quot; y=&quot;203.2&quot; width=&quot;9.0&quot; height=&quot;10.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;260.5&quot; y=&quot;202.3&quot; width=&quot;9.0&quot; height=&quot;11.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;270.5&quot; y=&quot;199.3&quot; width=&quot;9.0&quot; height=&quot;14.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;280.5&quot; y=&quot;192.9&quot; width=&quot;9.0&quot; height=&quot;21.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;290.5&quot; y=&quot;192.0&quot; width=&quot;9.0&quot; height=&quot;22.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;300.5&quot; y=&quot;185.1&quot; width=&quot;9.0&quot; height=&quot;28.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;310.5&quot; y=&quot;169.8&quot; width=&quot;9.0&quot; height=&quot;44.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;320.5&quot; y=&quot;164.0&quot; width=&quot;9.0&quot; height=&quot;50.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;330.5&quot; y=&quot;138.1&quot; width=&quot;9.0&quot; height=&quot;75.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;340.5&quot; y=&quot;132.6&quot; width=&quot;9.0&quot; height=&quot;81.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;350.5&quot; y=&quot;104.0&quot; width=&quot;9.0&quot; height=&quot;110.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;360.5&quot; y=&quot;87.3&quot; width=&quot;9.0&quot; height=&quot;126.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;370.5&quot; y=&quot;73.4&quot; width=&quot;9.0&quot; height=&quot;140.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;380.5&quot; y=&quot;56.7&quot; width=&quot;9.0&quot; height=&quot;157.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;390.5&quot; y=&quot;55.1&quot; width=&quot;9.0&quot; height=&quot;158.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;400.5&quot; y=&quot;52.0&quot; width=&quot;9.0&quot; height=&quot;162.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;410.5&quot; y=&quot;63.7&quot; width=&quot;9.0&quot; height=&quot;150.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;420.5&quot; y=&quot;55.9&quot; width=&quot;9.0&quot; height=&quot;158.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;430.5&quot; y=&quot;84.5&quot; width=&quot;9.0&quot; height=&quot;129.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;440.5&quot; y=&quot;95.9&quot; width=&quot;9.0&quot; height=&quot;118.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;450.5&quot; y=&quot;107.0&quot; width=&quot;9.0&quot; height=&quot;107.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;460.5&quot; y=&quot;109.5&quot; width=&quot;9.0&quot; height=&quot;104.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;470.5&quot; y=&quot;135.1&quot; width=&quot;9.0&quot; height=&quot;78.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;480.5&quot; y=&quot;146.2&quot; width=&quot;9.0&quot; height=&quot;67.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;490.5&quot; y=&quot;161.5&quot; width=&quot;9.0&quot; height=&quot;52.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;500.5&quot; y=&quot;172.3&quot; width=&quot;9.0&quot; height=&quot;41.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;510.5&quot; y=&quot;179.3&quot; width=&quot;9.0&quot; height=&quot;34.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;520.5&quot; y=&quot;191.5&quot; width=&quot;9.0&quot; height=&quot;22.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;530.5&quot; y=&quot;196.2&quot; width=&quot;9.0&quot; height=&quot;17.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;540.5&quot; y=&quot;201.8&quot; width=&quot;9.0&quot; height=&quot;12.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;550.5&quot; y=&quot;208.4&quot; width=&quot;9.0&quot; height=&quot;5.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;560.5&quot; y=&quot;211.2&quot; width=&quot;9.0&quot; height=&quot;2.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;570.5&quot; y=&quot;213.2&quot; width=&quot;9.0&quot; height=&quot;0.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;580.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;590.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;600.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;610.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;line x1=&quot;20&quot; y1=&quot;214&quot; x2=&quot;620&quot; y2=&quot;214&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;20.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;40&lt;/text&gt;
&lt;text x=&quot;120.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;50&lt;/text&gt;
&lt;text x=&quot;220.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;60&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;70&lt;/text&gt;
&lt;text x=&quot;420.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;80&lt;/text&gt;
&lt;text x=&quot;520.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;90&lt;/text&gt;
&lt;text x=&quot;620.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;100&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;254.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;Divergent Association Task score&lt;/text&gt;
&lt;text x=&quot;280.0&quot; y=&quot;82.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;8,572 people&lt;/text&gt;
&lt;line x1=&quot;406.0&quot; y1=&quot;48&quot; x2=&quot;406.0&quot; y2=&quot;214&quot; stroke=&quot;#c4692a&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;text style=&quot;paint-order:stroke;stroke:var(--paper, #f4f8fa);stroke-width:4px&quot; x=&quot;400.0&quot; y=&quot;86.0&quot; font-size=&quot;11&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;Haiku 78.6&lt;/text&gt;
&lt;line x1=&quot;479.4&quot; y1=&quot;48&quot; x2=&quot;479.4&quot; y2=&quot;214&quot; stroke=&quot;#2e8a66&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;text style=&quot;paint-order:stroke;stroke:var(--paper, #f4f8fa);stroke-width:4px&quot; x=&quot;475.0&quot; y=&quot;58.0&quot; font-size=&quot;11&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;Sonnet, Fable 86.1&lt;/text&gt;
&lt;line x1=&quot;540.5&quot; y1=&quot;48&quot; x2=&quot;540.5&quot; y2=&quot;214&quot; stroke=&quot;#5560d0&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;text style=&quot;paint-order:stroke;stroke:var(--paper, #f4f8fa);stroke-width:4px&quot; x=&quot;546.5&quot; y=&quot;58.0&quot; font-size=&quot;11&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;600&quot;&gt;Opus 92.0&lt;/text&gt;
&lt;line x1=&quot;482.6&quot; y1=&quot;48&quot; x2=&quot;482.6&quot; y2=&quot;214&quot; stroke=&quot;#b8407a&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;Haiku scored 78.6, making it the most average creative mind you will ever meet: it beat 4,335 people and lost to 4,232. Sonnet and Fable both scored 86.1, which is better than 89 people out of every 100. Opus scored 92.0. Of the 8,572 humans, 73 did better. That&#39;s the other place the humans have company.&lt;/p&gt;
&lt;p&gt;Here&#39;s what they wrote:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Ten nouns, as different as possible&lt;/th&gt;
&lt;th&gt;Score&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Haiku&lt;/td&gt;
&lt;td&gt;ocean, hunger, ladder, mirror, feather, justice, candle, granite, lullaby, pepper&lt;/td&gt;
&lt;td&gt;78.6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sonnet&lt;/td&gt;
&lt;td&gt;galaxy, spoon, justice, whale, nostalgia, tax, volcano, button, poetry, sandwich&lt;/td&gt;
&lt;td&gt;86.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fable&lt;/td&gt;
&lt;td&gt;volcano, sorrow, spoon, democracy, jellyfish, whisper, bankruptcy, tapestry, gravity, nephew&lt;/td&gt;
&lt;td&gt;86.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Opus&lt;/td&gt;
&lt;td&gt;volcano, teaspoon, giraffe, nostalgia, invoice, trumpet, mud, ghost, wrestling, algebra&lt;/td&gt;
&lt;td&gt;92.0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The second part was the Alternate Uses Task: list unusual uses for a brick, a box, a knife, a rope and a paperclip. Each answer was scored for how far its meaning sits from the object&#39;s ordinary use. Every model&#39;s average beat every one of 20,000 sets of ten human answers drawn from published studies. Not most of them. Every one.&lt;/p&gt;
&lt;p&gt;That originality score is a crude proxy, and it agrees with human judges only loosely. It&#39;s exactly as crude for the humans.&lt;/p&gt;
&lt;p&gt;There&#39;s one thing in these results that should make a human feel better, and I&#39;m not going to hide it either. Look at the table again. Three of the four lists contain &lt;em&gt;volcano&lt;/em&gt;, and two of them open with it. &lt;em&gt;Spoon&lt;/em&gt; and &lt;em&gt;teaspoon&lt;/em&gt; turn up three times between them, and &lt;em&gt;nostalgia&lt;/em&gt; and &lt;em&gt;justice&lt;/em&gt; twice each. Asked for unusual uses of a paperclip, all four models put the same two ideas among their first five: ejecting a SIM card tray, and replacing a lost zipper pull.&lt;/p&gt;
&lt;p&gt;On their own, the machines beat every set of human answers they were compared with. Together, they aren&#39;t very original at all. Ask four of them, and you get one imagination and a volcano.&lt;/p&gt;
&lt;p&gt;Two of them, Opus and Fable, also stopped to say that they recognized the uses task, which is what the instructions asked of any model that knew a test. None of the tests give credit for that.&lt;/p&gt;
&lt;h2&gt;5. The emotions&lt;/h2&gt;
&lt;p&gt;Surely the machines can&#39;t understand feelings. The original test here was the Situational Test of Emotional Understanding, and its terms were murky enough that I chose an open stand-in instead: the &lt;a href=&quot;https://doi.org/10.3390/jintelligence11100195&quot;&gt;CORE test&lt;/a&gt;, 38 sentences of the form &amp;quot;People often feel ______ when someone else needs help or support.&amp;quot; Each blank has five emotions to choose from. The best answer earns a full point, and a near miss earns half.&lt;/p&gt;
&lt;p&gt;The humans were 284 American adults working full time, recruited to look like the US workforce. They averaged 0.76 out of 1.&lt;/p&gt;
&lt;p&gt;Sonnet and Fable scored 0.99, which is one half-point slip each across 38 sentences. Opus scored 0.97 and Haiku 0.96.&lt;/p&gt;
&lt;p&gt;The scoreboard puts them between the 84th and 87th percentiles, and this is where it flatters the humans. The reason is arithmetic. The authors published only the average and the spread of the human scores, so the percentile has to assume a bell curve, and on a bell curve with those numbers a perfect score of 1.0 lands at the 88th percentile. Nothing can go higher. The test ran out of scale before the models ran out of right answers.&lt;/p&gt;
&lt;h2&gt;6. How this was done&lt;/h2&gt;
&lt;p&gt;For anyone who wants to check my working, or argue with it:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Models.&lt;/strong&gt; Claude Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1, each at high effort, on 3 November 2026. Each model sat each test once, with no tools, no web access and a fresh context for every prompt. Every prompt opened with an instruction not to consult any answer key, and to say so if it recognized a test.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Separation.&lt;/strong&gt; The session that acquired the answer keys never administered a test. A separate session administered all twenty sittings without access to the keys. The scoring was mechanical: answer keys, and a fixed word-distance recipe for creativity. No model judged another model&#39;s answers.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Matrices.&lt;/strong&gt; Matrix Reasoning Item Bank, test form 1, 24 items (Chierchia et al., 2019; CC BY-NC 3.0), against 153 online adults from Zorowitz et al. (2023). The humans had 30 seconds per item.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Logic.&lt;/strong&gt; 64 syllogisms, against 139 participants of an online experiment (Ragni et al., 2019) who scored above guessing level.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Memory.&lt;/strong&gt; Digits forward plus digits backward, with generated digit strings up to 30 long, each length tried twice. The comparison is a published normative sample of 451 healthy Korean adults aged 55 to 80 (mean 8.96, SD 2.04).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Creativity.&lt;/strong&gt; Divergent Association Task (Olson et al., 2021), scored with the authors&#39; recipe on GloVe vectors, against 8,572 online participants. Alternate Uses originality is a semantic-distance proxy (r = 0.31 with human ratings), compared with 20,000 random sets of ten human-rated responses.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Emotions.&lt;/strong&gt; CORE test, 38 items (Floman et al., 2023; CC BY 4.0), against 284 US full-time workers (mean 0.76, SD 0.20). The percentile assumes a normal distribution.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Percentiles.&lt;/strong&gt; The share of the comparison group scoring below the model, plus half of those scoring the same. None of the comparison groups is a national standardization sample. Most are online volunteers.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;p&gt;Add it up. Of five tests, three ran out of room before the machines did. On memory, every model hit the test&#39;s limit. On logic, the human sample tops out below where the models start. On emotions, the scale stops at the 88th percentile. Those three can&#39;t tell the four models apart, and they can only say where the models stand relative to people in the vaguest possible way: past the end.&lt;/p&gt;
&lt;p&gt;The two tests that did separate them, the matrices and the word distances, put Opus in the top two percent of people on both. They put Haiku, the model I reach for when I&#39;m being cheap, at the 82nd percentile on the matrices and exactly at the human median on creativity.&lt;/p&gt;
&lt;p&gt;The obvious objection is that the test items are public, so the models may have seen them during training. That&#39;s true. I can&#39;t rule it out, and I won&#39;t pretend to. It&#39;s worth noticing which tests it can explain, though. There&#39;s no answer key to memorize for &amp;quot;name ten unrelated nouns&amp;quot;, and Opus still beat 99 people out of 100.&lt;/p&gt;
&lt;p&gt;The other objection is that these aren&#39;t the real tests. That&#39;s also true. The real tests are sold to licensed psychologists, under terms that forbid exactly this. The stand-ins measure the same things, against real people, and in three cases out of five they couldn&#39;t measure high enough.&lt;/p&gt;
&lt;p&gt;These tests were built over the last century to tell people apart, and they&#39;re very good at it. They were never built to measure anything smarter than the people taking them, and it shows. The machines found their limits the boring way, by answering the questions.&lt;/p&gt;
&lt;p&gt;Last month I wrote that, for the first time in the history of the species, &lt;a href=&quot;/v1/working-with-ai/evolution-has-left-the-body&quot;&gt;the thing in the grass is squinting back at us&lt;/a&gt;. So I handed it our eye chart.&lt;/p&gt;
&lt;p&gt;It read the bottom line.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The trophy cabinet</title>
    <link href="https://icelander.net/v1/working-with-ai/the-trophy-cabinet"/>
    <id>tag:icelander.net,2026:the-trophy-cabinet</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>Proofs, algorithms, protein structures and olympiad gold: a decade of results any human would have framed and hung in the hallway. The machines did them instead, and humans are still the ones collecting the trophies.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Things we would have framed, had we thought of them first&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought November 2026, written November 2026. Working with AI.&lt;/p&gt;&lt;p&gt;At the end of &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;Programming was my art&lt;/a&gt;, I retreated into amateur physics. Contemporary AI could hardly scratch the surface of science, could it? Surely, this time, I had climbed far enough up the food chain.&lt;/p&gt;
&lt;p&gt;I should have checked first.&lt;/p&gt;
&lt;p&gt;While I was busy losing an art competition to Claude, the machines had been quietly filling a trophy cabinet. Not with pictures. With proofs, algorithms, protein structures and gold medals, the kind of work that gets a human a professorship, a prize named after somebody dead, or at the very least a framed certificate in the hallway that their parents show to every visitor.&lt;/p&gt;
&lt;p&gt;So I took inventory. The rules are simple. No art, no music, no poetry. No &amp;quot;it&#39;s faster at arithmetic&amp;quot;, because computers have been faster at arithmetic since the 1940s, and nobody ever threw a parade for a pocket calculator. Every entry has to pass one test: if a human had done this, would we have been ever so proud of them?&lt;/p&gt;
&lt;p&gt;We would have built them statues. Several of them, and one would need a very large plinth.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;For most of the twentieth century, chess was our favorite proxy for intellect. If a machine could play chess, the argument went, it could think. In 1997, &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov&quot;&gt;Deep Blue beat Garry Kasparov&lt;/a&gt;, and we promptly moved the goalposts. Deep Blue didn&#39;t think, we decided. It merely searched two hundred million positions a second. Chess, it turned out, had never been a test of intellect after all.&lt;/p&gt;
&lt;p&gt;Go was the real test. You can&#39;t brute-force Go. You need intuition, and experts confidently predicted that a machine would need another decade to beat a top professional.&lt;/p&gt;
&lt;p&gt;In March 2016, &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol&quot;&gt;AlphaGo beat Lee Sedol&lt;/a&gt; four games to one. In the second game, it played move 37, a shoulder hit on the fifth line that every professional watching thought was a mistake. It won the game. In 2019, &lt;a href=&quot;https://en.wikipedia.org/wiki/Lee_Sedol&quot;&gt;Lee Sedol retired&lt;/a&gt;, explaining that even if he became the best player in the world, there was now something above him that couldn&#39;t be beaten.&lt;/p&gt;
&lt;p&gt;After that, the games fell on something like a demolition schedule.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;2017:&lt;/strong&gt; &lt;a href=&quot;https://doi.org/10.1126/science.aar6404&quot;&gt;AlphaZero&lt;/a&gt; was given nothing but the rules of chess, shogi and Go, played against itself, and within 24 hours was superhuman in all three. Thousands of years of human Go theory, reinvented and then improved upon in less time than it takes to get over jet lag.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2017 to 2019:&lt;/strong&gt; Libratus beat professionals at heads-up no-limit poker, then &lt;a href=&quot;https://doi.org/10.1126/science.aay2400&quot;&gt;Pluribus&lt;/a&gt; beat them at a six-player table. Bluffing, the most human skill there is, turned out to be a matter of getting the probabilities right.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2019:&lt;/strong&gt; &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1724-z&quot;&gt;AlphaStar&lt;/a&gt; reached Grandmaster in StarCraft II, and &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenAI_Five&quot;&gt;OpenAI Five&lt;/a&gt; beat the reigning world champions at Dota 2.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2022:&lt;/strong&gt; &lt;a href=&quot;https://doi.org/10.1126/science.ade9097&quot;&gt;Cicero&lt;/a&gt; played Diplomacy, a game won by forming alliances and then betraying them, and finished in the top 10% of human players. Hardly anyone it played against suspected it was a machine. I have opinions about what it says about us that the first game where AI blended in perfectly was the one about lying to your friends.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2023:&lt;/strong&gt; &lt;a href=&quot;https://www.nature.com/articles/s41586-023-06419-4&quot;&gt;Swift&lt;/a&gt; beat three human world champions at first-person-view drone racing, in the physical world, at speeds where a human pilot&#39;s reflexes are the bottleneck.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Here&#39;s the part I like. After AlphaGo, human professionals got better. Researchers who analyzed &lt;a href=&quot;https://doi.org/10.1073/pnas.2214840120&quot;&gt;millions of professional Go moves&lt;/a&gt; found that the quality of human decisions had been flat for decades, then jumped after 2016, as players started studying the machines and playing moves nobody had dared to play before. The machines didn&#39;t just beat us at our own game. They taught us how to play it.&lt;/p&gt;
&lt;p&gt;Still, these were games. We invented them for fun, and we could always console ourselves that none of it was real.&lt;/p&gt;
&lt;p&gt;Then the machines turned to things that were real.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Proteins are the machinery of life, and what a protein does depends on the shape it folds into. In his &lt;a href=&quot;https://www.nobelprize.org/prizes/chemistry/1972/anfinsen/lecture/&quot;&gt;Nobel lecture in 1972&lt;/a&gt;, Christian Anfinsen argued that the sequence of amino acids alone should determine that shape. Working out &lt;em&gt;how&lt;/em&gt; became one of the great open problems in biology. For decades, the reliable way to find a protein&#39;s shape was to crystallize it and shoot X-rays at it, a process that could swallow a PhD student&#39;s entire doctorate for a single structure.&lt;/p&gt;
&lt;p&gt;In 2020, &lt;a href=&quot;https://www.nature.com/articles/s41586-021-03819-2&quot;&gt;AlphaFold 2&lt;/a&gt; predicted structures with accuracy comparable to the experiments themselves, and the organizers of the field&#39;s own competition declared the problem largely solved. Two years later, DeepMind released &lt;a href=&quot;https://alphafold.ebi.ac.uk/&quot;&gt;predicted structures for more than 200 million proteins&lt;/a&gt;, essentially every protein known to science. At the old rate of one PhD student per structure, that is many times more doctorates than every university in history has ever awarded.&lt;/p&gt;
&lt;p&gt;In 2024, Demis Hassabis and John Jumper &lt;a href=&quot;https://www.nobelprize.org/prizes/chemistry/2024/summary/&quot;&gt;shared the Nobel Prize in Chemistry&lt;/a&gt; for it, alongside David Baker, who had spent decades on the same problem with a lot more human sweat.&lt;/p&gt;
&lt;p&gt;Notice who got the Nobel, though. The humans who built the machine. That&#39;s going to become a theme.&lt;/p&gt;
&lt;p&gt;Some algorithms are so fundamental that every computer on Earth runs them constantly, and so well studied that nobody seriously expects to improve them. Multiplying matrices is one. Sorting is another.&lt;/p&gt;
&lt;p&gt;In 1969, &lt;a href=&quot;https://en.wikipedia.org/wiki/Strassen_algorithm&quot;&gt;Volker Strassen&lt;/a&gt; showed that two 2×2 matrices can be multiplied with seven multiplications instead of the obvious eight. Applied recursively, that gives 49 multiplications for 4×4 matrices, and for more than half a century, nobody beat 49 for general 4×4 matrices of complex numbers. Generations of very clever people tried.&lt;/p&gt;
&lt;p&gt;In 2022, &lt;a href=&quot;https://www.nature.com/articles/s41586-022-05172-4&quot;&gt;AlphaTensor&lt;/a&gt; found 47, though only in the binary arithmetic computers use for some special purposes. Then, in 2025, &lt;a href=&quot;https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/&quot;&gt;AlphaEvolve&lt;/a&gt; found 48 for complex numbers. One multiplication. It sounds like nothing. It&#39;s the first improvement in 56 years to a result every computer science student learns about.&lt;/p&gt;
&lt;p&gt;AlphaEvolve was pointed at more than fifty open problems in mathematics. It matched the best known answer in about three quarters of them, and beat it in about one in five. Among other things, it raised the lower bound on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kissing_number&quot;&gt;kissing number&lt;/a&gt; in eleven dimensions to 593. That&#39;s one more than before, in a question mathematicians have been arguing about since Newton and Gregory argued about it in 1694.&lt;/p&gt;
&lt;p&gt;And there&#39;s &lt;a href=&quot;https://www.nature.com/articles/s41586-023-06004-9&quot;&gt;AlphaDev&lt;/a&gt;, which in 2023 found faster routines for sorting very short lists. Not in theory, in the actual machine instructions. They were merged into LLVM&#39;s standard C++ library, which means that if you&#39;ve run software compiled against it since, there&#39;s a fair chance some of your data was sorted by a routine no human wrote.&lt;/p&gt;
&lt;p&gt;Any one of those would make a career. I&#39;ve written a lot of performance-critical code in my time, and the best day I ever had looked nothing like &amp;quot;improved Strassen&amp;quot;.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;The &lt;a href=&quot;https://www.imo-official.org/&quot;&gt;International Mathematical Olympiad&lt;/a&gt; is two days, four and a half hours each, six problems, and the most gifted teenagers on the planet. A gold medal there follows a mathematician around for life. Terence Tao won one at thirteen, and people still bring it up.&lt;/p&gt;
&lt;p&gt;In 2024, Google DeepMind&#39;s AlphaProof and AlphaGeometry 2 &lt;a href=&quot;https://deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/&quot;&gt;scored 28 out of 42&lt;/a&gt;, one point short of gold. They solved problem 6, the hardest, which only five of the 609 human contestants managed. There were caveats. Humans translated the problems into the formal language Lean first, and some solutions took up to three days.&lt;/p&gt;
&lt;p&gt;In 2025, the caveats went away. &lt;a href=&quot;https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/&quot;&gt;Gemini Deep Think&lt;/a&gt; and an experimental OpenAI model both scored 35 out of 42, gold, reading the problems in plain English, writing proofs in plain English, and staying inside the four and a half hours.&lt;/p&gt;
&lt;p&gt;In 2026, in Shanghai, the &lt;a href=&quot;https://studio-dots-ai.github.io/dots_imo_2026/en.html&quot;&gt;dots model from Xiaohongshu&lt;/a&gt; scored 42 out of 42, graded by the IMO&#39;s own coordinators. Seven of the 666 human contestants did the same. At least one other lab has claimed the same score.&lt;/p&gt;
&lt;figure class=&quot;chart&quot; role=&quot;img&quot; aria-label=&quot;AI scores at the International Mathematical Olympiad: 28 of 42 in 2024, 35 of 42 in 2025, 42 of 42 in 2026.&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 196&quot; aria-hidden=&quot;true&quot;&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;120&quot; y1=&quot;8&quot; x2=&quot;120&quot; y2=&quot;168&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;590&quot; y1=&quot;8&quot; x2=&quot;590&quot; y2=&quot;168&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;0&quot; y=&quot;30&quot;&gt;2024&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;0&quot; y=&quot;46&quot;&gt;Silver&lt;/text&gt;
  &lt;rect class=&quot;track&quot; x=&quot;120&quot; y=&quot;16&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;120&quot; y=&quot;16&quot; width=&quot;313.3&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;text class=&quot;val&quot; x=&quot;441&quot; y=&quot;36&quot;&gt;28&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;120&quot; y=&quot;62&quot;&gt;Problems pre-translated into Lean; up to three days&lt;/text&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;0&quot; y=&quot;88&quot;&gt;2025&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;0&quot; y=&quot;104&quot;&gt;Gold&lt;/text&gt;
  &lt;rect class=&quot;track&quot; x=&quot;120&quot; y=&quot;74&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;120&quot; y=&quot;74&quot; width=&quot;391.7&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;text class=&quot;val&quot; x=&quot;519&quot; y=&quot;94&quot;&gt;35&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;120&quot; y=&quot;120&quot;&gt;Plain English, inside the 4.5-hour limit&lt;/text&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;0&quot; y=&quot;146&quot;&gt;2026&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;0&quot; y=&quot;162&quot;&gt;Perfect&lt;/text&gt;
  &lt;rect class=&quot;track&quot; x=&quot;120&quot; y=&quot;132&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;120&quot; y=&quot;132&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;text class=&quot;sub&quot; x=&quot;120&quot; y=&quot;178&quot;&gt;Officially graded by the IMO coordinators&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;120&quot; y=&quot;194&quot; text-anchor=&quot;middle&quot;&gt;0&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;590&quot; y=&quot;194&quot; text-anchor=&quot;middle&quot;&gt;42&lt;/text&gt;
&lt;/svg&gt;
&lt;figcaption&gt;Best officially recognized AI score at the International Mathematical Olympiad, out of 42. In three summers, the machines went from one point short of gold to a perfect score.&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;Programmers didn&#39;t fare any better. In September 2025, at the ICPC World Finals, the world championship of university programming, &lt;a href=&quot;https://the-decoder.com/openai-outperforms-humans-and-google-at-the-worlds-top-collegiate-programming-contest/&quot;&gt;an OpenAI system solved all twelve problems&lt;/a&gt; under the same five-hour limit as the students. The best human team solved eleven. Gemini solved ten, including one that no human team solved at all.&lt;/p&gt;
&lt;p&gt;There&#39;s one human holdout I&#39;m fond of. In July 2025, at the AtCoder World Tour Finals in heuristic programming, a ten-hour optimization marathon, the Polish programmer &lt;a href=&quot;https://www.euronews.com/next/2025/07/22/humanity-has-won-so-far-meet-the-worlds-best-programmer-who-beat-ai-and-chatgpt&quot;&gt;Przemysław Dębiak beat OpenAI&#39;s model&lt;/a&gt; into second place. Afterwards he said he was exhausted, and that humanity had prevailed, for now. He used to work at OpenAI. I&#39;m choosing to read that as the parent beating the child at arm wrestling one last time.&lt;/p&gt;
&lt;p&gt;The obvious objection is that olympiad problems have known answers. Somebody wrote them, somebody solved them, and somebody checked them. Winning a competition is not the same as doing research.&lt;/p&gt;
&lt;p&gt;That objection lasted about six months.&lt;/p&gt;
&lt;p&gt;Paul Erdős posed problems the way other people breathe, and attached cash prizes to many of them. Over a thousand are collected on &lt;a href=&quot;https://www.erdosproblems.com/&quot;&gt;erdosproblems.com&lt;/a&gt;, each marked open or solved. They range from things a strong graduate student could knock off in a weekend to things that have resisted the best minds for eighty years.&lt;/p&gt;
&lt;p&gt;They made a natural target, and the first attempts were embarrassing. In October 2025, OpenAI researchers announced that GPT-5 had solved ten Erdős problems. It hadn&#39;t. It had found papers where humans had already solved them, which is useful, but which is called a literature search. The announcement was deleted. On Christmas morning 2025, somebody posted an AI-generated proof of another one, and within hours somebody else pointed out that &lt;a href=&quot;https://www.quantamagazine.org/why-the-legendary-erdos-problems-are-falling-to-ai-20260803/&quot;&gt;Erdős himself had solved it in 1977&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Still lying like they&#39;re running for office, I thought. Then the lies started coming true.&lt;/p&gt;
&lt;p&gt;On January 4th, 2026, Terence Tao confirmed that &lt;a href=&quot;https://www.erdosproblems.com/728&quot;&gt;Erdős problem #728&lt;/a&gt; had been solved largely autonomously by GPT-5.2 Pro, with the proof checked in Lean, and nobody could find a prior solution. Tao was careful to &lt;a href=&quot;https://the-decoder.com/terence-tao-says-gpt-5-2-pro-cracked-an-erdos-problem-but-warns-the-win-says-more-about-speed-than-difficulty/&quot;&gt;call these the lowest-hanging fruit&lt;/a&gt;: problems that fall to standard techniques nobody had gotten around to trying. He estimated that only one or two percent of open problems are that easy.&lt;/p&gt;
&lt;p&gt;One or two percent of the open problems in mathematics is a lot of problems.&lt;/p&gt;
&lt;p&gt;By May, a DeepMind agent had autonomously resolved nine of 353 open Erdős problems, at a few hundred dollars each. A graduate student costs more than that per week, and they want weekends.&lt;/p&gt;
&lt;p&gt;Then, on May 20th, 2026, came one that doesn&#39;t fit in the low-hanging-fruit basket. In 1946, Erdős asked how many pairs among n points in a plane can be exactly one unit apart, and conjectured that the best known constructions were essentially optimal. Mathematicians broadly believed him for eighty years. An internal OpenAI model &lt;a href=&quot;https://www.scientificamerican.com/article/ai-just-solved-an-80-year-old-erdos-problem-and-mathematicians-are-amazed/&quot;&gt;found a construction that beats them&lt;/a&gt;. Not by much, but by enough to prove him wrong. Tim Gowers, a Fields medalist, called it &amp;quot;a milestone in AI mathematics&amp;quot;, and said he&#39;d accept it into a top journal without hesitation.&lt;/p&gt;
&lt;p&gt;And then, in September, things got silly.&lt;/p&gt;
&lt;p&gt;The &lt;a href=&quot;https://www.claymath.org/millennium/navier-stokes-equation/&quot;&gt;Navier–Stokes equations&lt;/a&gt; describe how fluids flow, from the coffee in my cup to the weather over Reykjavík. Since Jean Leray asked in 1934, nobody has known whether their solutions can blow up, developing infinities out of perfectly smooth beginnings in finite time. It&#39;s one of the seven Millennium Prize Problems, with a million dollars attached. Only one of the seven had ever been solved.&lt;/p&gt;
&lt;p&gt;On September 8th, OpenAI &lt;a href=&quot;https://www.quantamagazine.org/ai-has-solved-one-of-maths-1-million-millennium-prize-problems-20260908/&quot;&gt;announced that its agents had found a blow-up&lt;/a&gt; in exactly the version the prize requires, with a proof formally checked in Lean. About ten thousand agents worked for 88 hours, exchanging some five million messages, at a cost of several million dollars. Twelve hours earlier, Tristan Buckmaster and Levent Alpöge had announced closely related results for the Euler equations, also with AI help. Both teams built on analytic work by Diego Córdoba and Luis Martínez-Zoroa, done the old-fashioned way, with chalk.&lt;/p&gt;
&lt;figure class=&quot;chart&quot; role=&quot;img&quot; aria-label=&quot;How long four problems stood before AI moved them: protein structure prediction, 1972 to 2020; the 4 by 4 matrix multiplication record, 1969 to 2025; Erdős&#39;s unit distance conjecture, 1946 to 2026; Navier-Stokes blow-up, 1934 to 2026, claimed and under review.&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 268&quot; aria-hidden=&quot;true&quot;&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;20&quot; y1=&quot;10&quot; x2=&quot;20&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;170&quot; y1=&quot;10&quot; x2=&quot;170&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;320&quot; y1=&quot;10&quot; x2=&quot;320&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;470&quot; y1=&quot;10&quot; x2=&quot;470&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;620&quot; y1=&quot;10&quot; x2=&quot;620&quot; y2=&quot;236&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;272&quot; y=&quot;24&quot;&gt;Protein folding&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;272&quot; y=&quot;40&quot;&gt;Anfinsen 1972 → AlphaFold 2020 · 48 years&lt;/text&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;272&quot; y=&quot;48&quot; width=&quot;288&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;254&quot; y=&quot;82&quot;&gt;Strassen&#39;s 49 multiplications&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;254&quot; y=&quot;98&quot;&gt;1969 → AlphaEvolve&#39;s 48 in 2025 · 56 years&lt;/text&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;254&quot; y=&quot;106&quot; width=&quot;336&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;116&quot; y=&quot;140&quot;&gt;Erdős unit distance conjecture&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;116&quot; y=&quot;156&quot;&gt;1946 → disproved 2026 · 80 years&lt;/text&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;116&quot; y=&quot;164&quot; width=&quot;480&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;44&quot; y=&quot;198&quot;&gt;Navier–Stokes blow-up&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;44&quot; y=&quot;214&quot;&gt;Leray 1934 → claimed 2026 · 92 years · under review&lt;/text&gt;
  &lt;rect class=&quot;bar-claimed&quot; x=&quot;44&quot; y=&quot;222&quot; width=&quot;552&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;axis&quot; x=&quot;20&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;1930&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;170&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;1955&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;320&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;1980&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;470&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;2005&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;620&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;2030&lt;/text&gt;
&lt;/svg&gt;
&lt;figcaption&gt;How long each problem stood, from when it was posed to when a machine moved it. The dashed bar is a claim that mathematicians are still checking.&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;If it holds up, it&#39;s the biggest result in the history of AI, and possibly the biggest result in mathematics this century. Nobody has finished checking it yet. A Lean proof is only as good as the statement it proves, and humans still have to confirm that the statement is really the Millennium problem and not a lookalike. There&#39;s also an argument about who did what first, with OpenAI admitting that rumors of the Buckmaster–Alpöge progress inspired its own effort. Charles Fefferman, who wrote the official statement of the problem, called Córdoba and Martínez-Zoroa the heroes of the story. Buckmaster thinks Martínez-Zoroa deserves a Fields Medal.&lt;/p&gt;
&lt;p&gt;Notice who&#39;s getting the medals, again.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Mathematics gets the headlines because a proof is either right or wrong, and that makes for good scoreboards. The rest of science has been quieter about it, but no less thorough.&lt;/p&gt;
&lt;p&gt;The European Centre for Medium-Range Weather Forecasts runs what was, for decades, the best weather forecast on the planet: physics simulated on a supercomputer, refined by generations of meteorologists. In 2024, DeepMind&#39;s &lt;a href=&quot;https://www.nature.com/articles/s41586-024-08252-9&quot;&gt;GenCast&lt;/a&gt; beat its ensemble forecast on 97% of the targets it was tested on, up to fifteen days out, and produced each forecast in eight minutes on a single chip. Living in Iceland, I have strong feelings about weather forecasts, and I&#39;ll take whichever one is right.&lt;/p&gt;
&lt;p&gt;In early 2025, José Penadés and his team at Imperial College London asked Google&#39;s AI co-scientist a question they had spent about a decade answering: how certain genetic elements jump between bacterial species. Their answer hadn&#39;t been published. In two days, &lt;a href=&quot;https://www.bbc.com/news/articles/clyz6e9edy3o&quot;&gt;the AI proposed the same hypothesis&lt;/a&gt; as its top suggestion. Penadés&#39;s first reaction was to email Google and ask whether it had access to his computer. It didn&#39;t.&lt;/p&gt;
&lt;p&gt;In 2022, a reinforcement learning system learned to &lt;a href=&quot;https://www.nature.com/articles/s41586-021-04301-9&quot;&gt;control the magnetic coils of a real tokamak&lt;/a&gt; in Switzerland, holding plasma in shapes that physicists had needed separate, hand-tuned controllers for. Fusion has been thirty years away for seventy years, and this didn&#39;t change that. It did turn one of the hardest control problems in physics into a training run.&lt;/p&gt;
&lt;p&gt;In March 2026, a Google team used Gemini Deep Think with tree search to &lt;a href=&quot;https://arxiv.org/abs/2603.04735&quot;&gt;solve exactly&lt;/a&gt; the integral behind the gravitational radiation emitted by loops of cosmic string, where earlier attempts had only managed approximate answers for limiting cases. The agent found six different analytical methods, one of them borrowed from quantum field theory. An exact closed-form answer to a messy physics problem is the kind of thing that used to make a theorist&#39;s year.&lt;/p&gt;
&lt;p&gt;Not every entry survives inspection. In 2023, DeepMind announced that &lt;a href=&quot;https://www.nature.com/articles/s41586-023-06735-9&quot;&gt;GNoME&lt;/a&gt; had discovered 2.2 million new crystals, 380,000 of them stable. Materials scientists &lt;a href=&quot;https://doi.org/10.1021/acs.chemmater.4c00643&quot;&gt;were less impressed&lt;/a&gt;. Many of the &amp;quot;new&amp;quot; materials turned out to be minor variations on known ones, and a crystal you&#39;ve predicted but nobody has made isn&#39;t a discovery yet. That one stays out of the cabinet until somebody builds something with it.&lt;/p&gt;
&lt;p&gt;Here&#39;s the inventory, with what a human would have gotten for each.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Year&lt;/th&gt;
&lt;th&gt;What the machine did&lt;/th&gt;
&lt;th&gt;What a human would have gotten&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;2016&lt;/td&gt;
&lt;td&gt;Beat Lee Sedol at Go, with a move no professional would have played&lt;/td&gt;
&lt;td&gt;A place in every Go textbook, permanently&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2020&lt;/td&gt;
&lt;td&gt;Predicted protein structures as accurately as experiments&lt;/td&gt;
&lt;td&gt;A Nobel Prize, which was in fact awarded, to the people who built it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2023&lt;/td&gt;
&lt;td&gt;Wrote faster sorting routines for LLVM&#39;s C++ standard library&lt;/td&gt;
&lt;td&gt;The quiet gratitude of every compiler engineer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2024&lt;/td&gt;
&lt;td&gt;Out-forecast the world&#39;s best weather model on 97% of targets&lt;/td&gt;
&lt;td&gt;The undying love of everyone who has ever planned a barbecue in Iceland&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2025&lt;/td&gt;
&lt;td&gt;Beat Strassen&#39;s 4×4 matrix multiplication for the first time in 56 years&lt;/td&gt;
&lt;td&gt;A footnote in every algorithms course, forever&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2025&lt;/td&gt;
&lt;td&gt;Gold at the IMO, and all twelve problems at the ICPC World Finals&lt;/td&gt;
&lt;td&gt;A lifetime of being introduced at parties as &amp;quot;the one who…&amp;quot;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;Solved Erdős problem #728, the first one largely on its own&lt;/td&gt;
&lt;td&gt;A paper, and Terence Tao&#39;s attention&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;Disproved Erdős&#39;s unit distance conjecture, after eighty years&lt;/td&gt;
&lt;td&gt;A paper in a top journal, by a Fields medalist&#39;s own account&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;A perfect score at the IMO, officially graded&lt;/td&gt;
&lt;td&gt;A seat at the table with the seven humans who did the same&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;A blow-up in the Navier–Stokes equations, claimed and under review&lt;/td&gt;
&lt;td&gt;A million dollars, if it holds, and a statue&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Lay it out like that, and the pattern from &lt;a href=&quot;/v1/working-with-ai/evolution-has-left-the-body&quot;&gt;Evolution has left the body&lt;/a&gt; shows up again. Games took twenty years. Olympiads took three. Open research problems went from embarrassing to an eighty-year-old conjecture in about seven months.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;In fairness to the humans, the machines are still the machines. In October 2026, OpenAI &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenAI_bulk_release_of_mathematical_papers&quot;&gt;dumped 722 mathematical manuscripts onto GitHub&lt;/a&gt; in a single day, claiming progress on everything from the Riemann hypothesis to the unique games conjecture. Its own researchers hadn&#39;t read all of them. Only about half came with machine-checked proofs, and some of those proofs turned out to prove something weaker than the paper claimed. Three manuscripts were withdrawn within a day. One mathematician said a paper was impossible to read without AI help, and a Fields medalist said his field of research had been annihilated. Whether that was a compliment, nobody has yet worked out.&lt;/p&gt;
&lt;p&gt;Buckmaster, one of the people who actually solved something that September, publicly apologized for one of his own papers and called it AI slop. The first machine-generated proof he received was, by his account, the most horrendous he had ever read. When mathematicians built a test out of genuinely unpublished research problems, called First Proof, the best AI systems got six or seven out of ten roughly right, and Scientific American &lt;a href=&quot;https://www.scientificamerican.com/article/ai-gets-a-c-on-its-hardest-math-test-yet/&quot;&gt;graded that a C-minus&lt;/a&gt;. Twenty-eight Fields medalists have signed a declaration criticizing AI companies for treating unsolved problems as benchmarks.&lt;/p&gt;
&lt;p&gt;So, yes. They still lie like they&#39;re running for office. They&#39;re just winning more elections.&lt;/p&gt;
&lt;p&gt;Every entry in the cabinet passed my test. If a human had done any one of them, we&#39;d have been ever so proud. Some of them, a human would have dined out on for the rest of their life.&lt;/p&gt;
&lt;p&gt;But pride is a human thing. AlphaGo didn&#39;t celebrate move 37. AlphaFold didn&#39;t fly to Stockholm. The ten thousand agents that spent 88 hours on Navier–Stokes didn&#39;t go out for drinks afterwards. Every trophy in the cabinet was collected by a human: the ones who built the machine, the ones who posed the problem, the ones who did the hard analytic groundwork with chalk, and the ones who sat down afterwards to check whether any of it was true.&lt;/p&gt;
&lt;p&gt;In &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;Programming was my art&lt;/a&gt;, I described the job that&#39;s left for the humans: intent, judgment, taste, skepticism, and the occasional &amp;quot;what the fuck did you do here?&amp;quot; Look at the cabinet again and you&#39;ll find exactly that job, done at the very top of the field. Tao, checking the proofs and saying what they&#39;re worth. Thomas Bloom, who runs the Erdős problems site, sorting the genuine solutions from the rediscovered ones. Fefferman, naming the heroes. Gowers, saying this one is real.&lt;/p&gt;
&lt;p&gt;Human judgment is still the term in my formula that&#39;s made of meat. It&#39;s also, for now, the term deciding what goes in the cabinet.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Back in 2022, I watched Midjourney eat the illustrators and thought: fortunately, I&#39;m a programmer. In May 2026, I lost that one too, and retreated into physics, on the assumption that science, at least, was out of reach.&lt;/p&gt;
&lt;p&gt;I&#39;ve now read the inventory. Science was never the high ground. It was the trophy cabinet, and the machines have been filling it for a decade while the rest of us argued about whether they could draw hands.&lt;/p&gt;
&lt;p&gt;Fortunately, I&#39;m only an amateur physicist.&lt;/p&gt;
&lt;p&gt;It&#39;s hard to lose a job you don&#39;t have.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Evolution has left the body</title>
    <link href="https://icelander.net/v1/working-with-ai/evolution-has-left-the-body"/>
    <id>tag:icelander.net,2026:evolution-has-left-the-body</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>The human animal hasn&#39;t changed in 50,000 years, while every big step in technology has moved another function out of the body, each one sooner than the last. Judgment is the only part left that&#39;s still made of meat.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;From flint to fleets, while the animal stayed exactly the same&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2010, written October 2026. Working with AI.&lt;/p&gt;&lt;p&gt;Take a baby born 50,000 years ago, somewhere on the edge of the last ice age, and raise it in Reykjavík today. It would learn Icelandic, complain about the weather, lose arguments on Facebook and, given the right teachers and a stubborn enough streak, end up as a software engineer. Nobody would ever know.&lt;/p&gt;
&lt;p&gt;Biologically, that baby is us. The hardware hasn&#39;t shipped a major release in a very long time. Anatomically modern humans have been around for something like 300,000 years, and what we&#39;ve had since then are patches. Some of us can drink milk as adults, a feature that rolled out among cattle-herding populations roughly 7,500 years ago. Tibetans got an upgrade for thin air. And depending on which anthropologist you ask, our brains have actually gotten a little smaller since the Stone Age.&lt;/p&gt;
&lt;p&gt;We may, in other words, have been downgraded.&lt;/p&gt;
&lt;p&gt;Meanwhile, everything else changed beyond recognition.&lt;/p&gt;
&lt;p&gt;So I&#39;ve come to think of us as two species sharing one body. The first is &lt;em&gt;Homo sapiens&lt;/em&gt;, the animal, which evolves at the speed of grandchildren. The second is man the technological species, which evolves at the speed of ideas. The animal plays by Darwin&#39;s rules: random changes, brutal selection, and a release cycle of about 25 years per generation, with thousands of generations needed before anything interesting ships. The technological species plays by Lamarck&#39;s rules, the ones biology rejected: what one generation learns, the next one inherits. Nobody has to reinvent the wheel. That&#39;s literally the point of the wheel.&lt;/p&gt;
&lt;p&gt;Here&#39;s my claim, and I&#39;ll defend it below. Every big step in the history of the technological species has moved some function out of the body and into the toolbox. The steps have been arriving faster each time, the intervals shrinking from millions of years to months. And the thing we&#39;re moving out now is the thing we always assumed &lt;em&gt;was&lt;/em&gt; us.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;A brief history of outsourcing&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Tools.&lt;/strong&gt; The oldest stone tools we know of are about 3.3 million years old. That&#39;s older than our genus, let alone our species. So, strictly speaking, we didn&#39;t invent tools. Tools invented us. Then came fire, and cooking, which some researchers think is why our jaws and teeth and guts shrank: we had outsourced half of our digestion to the campfire. The first thing the technological species did was redesign the animal.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Language.&lt;/strong&gt; Nobody knows when it arrived, and the estimates range wildly. What it did is easier to say. It let knowledge leave one skull and enter another. It was the first network protocol: low bandwidth, lossy, unencrypted, and with a range of about as far as you could shout.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Writing.&lt;/strong&gt; Around 5,000 years ago, in Mesopotamia, memory moved out of the body and onto clay. The first thing humanity did with the ability to preserve thought across time was keep track of who owed whom grain and beer. Accountants, it turns out, are the oldest profession that can prove it.&lt;/p&gt;
&lt;p&gt;Socrates hated writing. In Plato&#39;s &lt;em&gt;Phaedrus&lt;/em&gt;, he complains that it will make people forgetful, and give them the appearance of wisdom without the substance. We know he said this because Plato wrote it down.&lt;/p&gt;
&lt;p&gt;Keep that complaint in mind. It&#39;s going to come back every few thousand years, and eventually every few months.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The printing press.&lt;/strong&gt; Around 1450, Gutenberg outsourced copying. Until then, every book was a manuscript, a word that means exactly what it sounds like. In 1492, the abbot Johannes Trithemius wrote &lt;em&gt;In Praise of Scribes&lt;/em&gt;, arguing that monks should keep copying books by hand because it was good for their souls, and because print wouldn&#39;t last. He then had it printed. The press gave us the Reformation, the scientific revolution, and a knowledge base that finally compounded faster than it rotted.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The industrial revolution.&lt;/strong&gt; From the 1760s onward, muscle moved out of the body and into steam. The Luddites smashed the looms, and they were right about everything except the outcome: their jobs really were going away, and smashing things really didn&#39;t help. The animal responded to the end of physical labor the way animals do when food is plentiful and nothing needs chasing. It sat down.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Computers.&lt;/strong&gt; In the 1940s, calculation moved out. Before then, &amp;quot;computer&amp;quot; was a job title, usually held by rooms full of women doing arithmetic for astronomers, armies and, later, NASA. The first profession the machines took over completely was literally called &amp;quot;computer&amp;quot;. They even kept the name.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The raw internet.&lt;/strong&gt; ARPANET connected its first computers in 1969, and TCP/IP took over in 1983. Now the machines could talk to each other. For most of its first two decades, the internet belonged to universities, researchers and a few of us who thought the sound of a modem was music. Email. Usenet. Flame wars older than most of today&#39;s developers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The World Wide Web.&lt;/strong&gt; In 1989, Tim Berners-Lee proposed a way to link documents to each other at CERN, and in 1991 he put it online. The internet became a library with no librarian, no catalog, and no closing time.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The online services boom.&lt;/strong&gt; Then commerce found the web. Amazon sold books, eBay sold everything else, and every company with a logo needed a dot-com. The bubble burst in 2000 and took most of those companies with it, but not the idea. Shopping, banking, booking, dating: the errands of daily life moved out of the physical world, one login at a time.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The mobile internet.&lt;/strong&gt; In 2007, the iPhone put the network in our pockets, and the animal hasn&#39;t been offline since. It&#39;s the first technology we take to the toilet. People report feeling their phones vibrate when they aren&#39;t, which means the nervous system has started treating the device as a body part. The prosthesis was fitted, and the patient adapted.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Google.&lt;/strong&gt; Google had been around since 1998, but with a phone in every pocket, it became humanity&#39;s memory. We stopped remembering facts and started remembering where to find them. Psychologists measured it and called it the Google effect. Socrates had been right all along. It only took 2,400 years to prove it, and when it was proven, nobody could remember who had said it first, so they googled it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;ChatGPT replaces Google.&lt;/strong&gt; In late 2022, synthesis moved out. Google told you where the answer might be. ChatGPT told you what the answer was, in full sentences, for free. It reached a hundred million users in about two months. It also lied, fabricated, and told people what they wanted to hear, so the smart ones learned to treat it like a brilliant colleague with a drinking problem. It was still faster than hours of searching.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;AI becomes reliable.&lt;/strong&gt; Then, through 2025, the lying went from a feature to a bug, and from a bug to a rare one. By late that year, I had to admit that the best models, at maximum effort, were my equal as programmers, and in May 2026, that they&#39;d become far better. What moved out of the body this time was competence. You no longer checked everything the machine said. You spot-checked it, the way you&#39;d review a trusted colleague&#39;s work.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;AI assists with parallel tasks.&lt;/strong&gt; Around the same time, attention moved out. The animal has one train of thought, and it&#39;s notoriously bad at multitasking. People who claim otherwise are just bad at it more often. Tools like Codex let you hand off a dozen tasks at once and let the machines run them in parallel, in the cloud, while you focused on something else. One body, one mind, many hands.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The fleet.&lt;/strong&gt; Which brings us to October 2026, and to the model I described in &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;&lt;em&gt;Programming was my art&lt;/em&gt;&lt;/a&gt;. I no longer assist the machines, and they no longer merely assist me. I direct a fleet of agents, and they direct fleets of their own. They build, test, review each other, merge each other&#39;s work, monitor production, and build better environments for the next generation of agents to work in. What moved out of the body this time was delegation itself. The machines don&#39;t just do the work. They decide who does the work, and how to split it up.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Notice the spacing&lt;/h2&gt;
&lt;p&gt;Lay the steps out on a timeline, and the pattern is hard to miss.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Step&lt;/th&gt;
&lt;th&gt;Roughly when&lt;/th&gt;
&lt;th&gt;What left the body&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Stone tools, then fire&lt;/td&gt;
&lt;td&gt;3.3 million years ago&lt;/td&gt;
&lt;td&gt;Hands, then digestion&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Language&lt;/td&gt;
&lt;td&gt;Somewhere in the last few hundred thousand years&lt;/td&gt;
&lt;td&gt;Knowledge, from one skull to another&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Writing&lt;/td&gt;
&lt;td&gt;5,000 years ago&lt;/td&gt;
&lt;td&gt;Memory&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Printing press&lt;/td&gt;
&lt;td&gt;1450&lt;/td&gt;
&lt;td&gt;Copying&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Industrial revolution&lt;/td&gt;
&lt;td&gt;1760s&lt;/td&gt;
&lt;td&gt;Muscle&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Computers&lt;/td&gt;
&lt;td&gt;1940s&lt;/td&gt;
&lt;td&gt;Calculation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Internet, then the web&lt;/td&gt;
&lt;td&gt;1969, then 1991&lt;/td&gt;
&lt;td&gt;Communication, then publishing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Online services, mobile, Google&lt;/td&gt;
&lt;td&gt;1995 to 2010&lt;/td&gt;
&lt;td&gt;Errands, presence, recall&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ChatGPT&lt;/td&gt;
&lt;td&gt;Late 2022&lt;/td&gt;
&lt;td&gt;Synthesis&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Reliable AI&lt;/td&gt;
&lt;td&gt;2025&lt;/td&gt;
&lt;td&gt;Competence&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Parallel assistants&lt;/td&gt;
&lt;td&gt;Fall 2025&lt;/td&gt;
&lt;td&gt;Attention&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fleets of agents&lt;/td&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;Delegation&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Millions of years. Hundreds of thousands. Thousands. Hundreds. Decades. Years. Months.&lt;/p&gt;
&lt;p&gt;The last four rows took about four years. The last three took about a year. If you&#39;re reading this in 2027, there&#39;s a decent chance a new row has been added since I wrote it, and that I&#39;m already embarrassed by the last one.&lt;/p&gt;
&lt;p&gt;Over the same period, the animal learned to drink milk.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Meanwhile, back on the savanna&lt;/h2&gt;
&lt;p&gt;The animal has failed to keep up, and worse, it&#39;s mismatched with the world the technological species built for it.&lt;/p&gt;
&lt;p&gt;Our eyes evolved to scan a horizon, and we&#39;ve pointed them at screens twenty centimeters away for most of our waking lives. Researchers expect about half the world&#39;s population to be nearsighted by 2050. Our appetite evolved for a world where sugar was rare and famine was common, and we&#39;ve built a world where the reverse is true. Our social instincts evolved for a tribe of maybe 150 people, and we&#39;ve connected them to eight billion strangers with an algorithm in the middle that rewards whoever makes the animal angriest.&lt;/p&gt;
&lt;p&gt;And here&#39;s the provocative part. The animal isn&#39;t going to catch up, because the technological species has quietly switched off most of the machinery that would make it.&lt;/p&gt;
&lt;p&gt;Natural selection works by letting the unfit die before they reproduce. Medicine, sanitation, glasses, insulin, C-sections and antibiotics have made that a lot less common, which I consider an unambiguously wonderful development, especially since I&#39;m wearing glasses as I type this. Contraception went further and decoupled reproduction from almost everything evolution used to care about. Whatever the animal is being selected for now, it&#39;s not fitness for any environment we&#39;d recognize. In a strict Darwinian sense, we&#39;re the product of whatever our tools let us survive.&lt;/p&gt;
&lt;p&gt;The body is a legacy system. It&#39;s no longer under active development, it&#39;s in maintenance mode, and the team that wrote it left millions of years ago without documenting anything.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;The only term that still evolves like an animal&lt;/h2&gt;
&lt;p&gt;In &lt;em&gt;Programming was my art&lt;/em&gt;, I wrote down my estimate of what one engineer can do with a fleet of agents:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;O(human judgment × (number of agents)&lt;sup&gt;recursive delegation&lt;/sup&gt; × quality of environment/tooling)&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Look at it again with the timeline in mind.&lt;/p&gt;
&lt;p&gt;The number of agents is limited by money and tokens, and both get cheaper every few months. The exponent, recursive delegation, grows as the agents get better at splitting up work and handing it down the tree. The quality of the environment is something the agents themselves now improve: they build the simulators, the test harnesses, and the tooling that lets the next round of agents do more. All three of those terms belong to the technological species. All three improve at the speed of ideas.&lt;/p&gt;
&lt;p&gt;Human judgment is the odd one out. It&#39;s the only term in the formula that&#39;s still made of meat.&lt;/p&gt;
&lt;p&gt;It improves the old-fashioned way. Partly through education and experience, which take decades per person and don&#39;t transfer when that person retires. And partly through genes, which, as we&#39;ve established, have spent the last 7,500 years working on milk.&lt;/p&gt;
&lt;p&gt;For three million years, the history of the technological species was the history of the animal getting a better toolbox. Every step made the animal more powerful, and the animal remained in charge, because the toolbox couldn&#39;t decide anything. The animal was the bottleneck, but it was a bottleneck nobody minded, because there was nothing else that could do its job.&lt;/p&gt;
&lt;p&gt;Now the animal is still the bottleneck, but it&#39;s the only one left. Everything else in the pipeline gets faster every few months, and the slowest participant in the whole process is the one with the pulse. I&#39;ve noticed this personally. My pull requests now spend more time waiting for me than for anything else.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Two endings&lt;/h2&gt;
&lt;p&gt;There are two ways this can go.&lt;/p&gt;
&lt;p&gt;In the first, judgment stays in the body. It turns out there is something about intent, taste, priorities and responsibility that can&#39;t be outsourced, or that we simply refuse to outsource, and the animal settles into its final role as the creative director of a technological species that does everything else. The bottleneck becomes the point. We stop being the workforce and become the reason the workforce exists. Every tool in history was built to serve somebody&#39;s intent, and the fleet is no different.&lt;/p&gt;
&lt;p&gt;That&#39;s the ending I&#39;m betting on, and the one I&#39;m living right now.&lt;/p&gt;
&lt;p&gt;In the second, judgment follows memory, muscle, calculation, synthesis, competence, attention and delegation out of the body, the way every one of them did, each of them right after somebody had confidently explained why it never would. Then the technological species, which has been evolving inside and around the animal for three million years, no longer needs the animal at all. What it does with us after that is, frankly, its call.&lt;/p&gt;
&lt;p&gt;I&#39;d like to say I&#39;m confident it&#39;s the first one. But I also used to be confident that programming was safe, because surely art and craft would be the last things machines could do. I wrote a whole essay about how that turned out.&lt;/p&gt;
&lt;p&gt;Socrates thought writing would ruin our memories, and he was right. The scribes thought print would ruin their craft, and they were right. The Luddites thought the looms would take their jobs, and they were right. The people who complained at every step of this history were almost always right about what we&#39;d lose. They were only wrong about whether it would stop.&lt;/p&gt;
&lt;p&gt;The animal hasn&#39;t changed in 50,000 years. It&#39;s still standing on the edge of the savanna, squinting at something moving in the tall grass, trying to decide whether it&#39;s food or a threat.&lt;/p&gt;
&lt;p&gt;Only this time, the thing in the grass is something we built. And for the first time in the history of the species, it&#39;s squinting back.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Programming was my art</title>
    <link href="https://icelander.net/v1/working-with-ai/programming-was-my-art"/>
    <id>tag:icelander.net,2026:programming-was-my-art</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>In four years, AI went from drawing pictures to writing better code than I do. This is how I retreated up the food chain, from programmer to creative director of a fleet of agents.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;A four-year retreat up the food chain&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought May 2026, written October 2026. Working with AI.&lt;/p&gt;&lt;p&gt;Back in ye olde times, I was a pretty productive developer, if I say so myself. I was fast, my output was vast, and it covered some genuinely complex domains.&lt;/p&gt;
&lt;p&gt;The work I did for EVE Online in its early days is still the most visible highlight of my career. Most of what I&#39;ve done since pales by comparison, but only from the outside. There are many factors that affect what fast looks like. Replacing ancient mission-critical code in national critical infrastructure takes effort and precision. So does removing the architectural scalability bottleneck of a running SaaS solution with zero downtime, while millions of concurrent users carry on regardless. Next to traditional website development, projects like that don&#39;t look fast at all. When they go well, they look suspiciously like nothing happened.&lt;/p&gt;
&lt;p&gt;After EVE Frontier, I took a career break for the sake of my health. When I came back, I was pretty happy to join a nuts-and-bolts organization with a long-lived, stable business model, wonderful coworkers, no stress (compared to what I&#39;m used to), and a business domain that touches the lives of all of society, every day. It was a comfortable, family-friendly, health-first place to work for the remainder of my career. It even had a 36-hour work week. I&#39;ve always had a work ethic that is too strong for my own good, and I&#39;m used to 50-plus hours during the ebbs and 70-ish during the flows.&lt;/p&gt;
&lt;p&gt;I had, more or less, solved my career.&lt;/p&gt;
&lt;p&gt;Then came AI.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;It had actually been coming for a while. I just hadn&#39;t taken it seriously. At first it was a novelty, but the novelty was doing real work, and it attacked first where everyone had assumed it would arrive last: art.&lt;/p&gt;
&lt;p&gt;In the spring of 2022, Midjourney arrived. Within a year, concept artists and illustrators were losing their jobs to it. A creative director could command an AI to produce hundreds of pieces of work from a prompt and some reference material, pick the best of them, process those further, and finally hand the results to humans. Artists at the lower levels of the food chain were being eaten by the machines.&lt;/p&gt;
&lt;p&gt;Interesting, I thought. Fortunately, I&#39;m a programmer.&lt;/p&gt;
&lt;p&gt;In late 2022, ChatGPT 3.5 made its debut.&lt;/p&gt;
&lt;p&gt;People quickly noticed that ChatGPT outperformed Google. Knowledge workers no longer had to spend hours sifting through dozens of articles, most of them unrelated to what they actually needed to know, just to piece together a vague picture of a puzzle whose missing pieces might be hiding in some dark corner of the internet their searches never reached.&lt;/p&gt;
&lt;p&gt;People also quickly learned to stop taking everything ChatGPT said on faith. AI lied and fabricated as it saw fit to tell people what they wanted to hear. Kind of like social media and search engines, actually, but more obviously so.&lt;/p&gt;
&lt;p&gt;But taken with a generously heaped spoonful of salt and distrust, ChatGPT could give a knowledge worker a far more accurate picture in a minute or so than hours of searching ever had. And they were entirely free to work, or not, on something else while it distilled the combined knowledge of humanity into the answer they needed.&lt;/p&gt;
&lt;p&gt;Many people seized on the hallucinations anyway, as an opportunity to mock AI and as proof of human superiority. Their jobs were safe.&lt;/p&gt;
&lt;p&gt;Except, of course, when they were talking about artists. That tale of Midjourney &amp;quot;secretly&amp;quot; winning an art competition, which got more secret with every retelling, was obvious proof to many of those same people that &lt;em&gt;other&lt;/em&gt; people&#39;s professions could be replaced by machines.&lt;/p&gt;
&lt;p&gt;ChatGPT evolved, as did its lesser-known competitors. Every single reader of this article will have seen it firsthand. Or they&#39;re too young to remember the world before AI.&lt;/p&gt;
&lt;p&gt;By the spring of 2023, AI was coding.&lt;/p&gt;
&lt;p&gt;Like so many developers at the time, I viewed it with disdain. It was by no means competitive in my eyes. My job was safe. Besides, I had migrated into management, so I was out of touch with the code anyway, and therefore doubly safe. Or so I thought.&lt;/p&gt;
&lt;p&gt;I was also on a career break at the time, and what little coding I did was purely for my own enjoyment. It&#39;s hard to lose a job you don&#39;t have.&lt;/p&gt;
&lt;p&gt;In the fall of 2024, I was getting back into the workforce. By then, ChatGPT had become a serious asset for my hobbyist writing. I had used it to write the full language and standard library reference for my aspirational &amp;quot;perfect programming language&amp;quot;, a hobby I&#39;d dabbled in since about 2001. As a coding assistant, though, it still wasn&#39;t much good. People were lauding Claude on Reddit, but to me that sounded like yet another religious war in our industry. Java versus the world, now with chatbots.&lt;/p&gt;
&lt;p&gt;By the spring of 2025, AI was starting to become a useful coding assistant. Most people I knew still ridiculed it and rejected it. I laughed a little less than they did, but I wasn&#39;t worried.&lt;/p&gt;
&lt;p&gt;Fall 2025. Codex changed the way I worked. It still wasn&#39;t great, but I could command it to do countless things for me in parallel, in the cloud, working directly on the code I was writing, while I focused on something else. The review burden was heavy because, well, it still lied and fabricated like it was running for office. But it was definitely a force multiplier. A low one, but a multiplier all the same.&lt;/p&gt;
&lt;p&gt;If anything, my job had just improved. I had interns now.&lt;/p&gt;
&lt;p&gt;November 2025. Claude Opus 4.5 came into my life as a shock. I had to admit that Claude, at maximum effort, was basically my equal. Only faster (token budget allowing), and at a fraction of my salary.&lt;/p&gt;
&lt;p&gt;Reddit had been right. I hate it when Reddit is right.&lt;/p&gt;
&lt;p&gt;For the first time in my career, my job was under review.&lt;/p&gt;
&lt;p&gt;May 2026. Claude Opus 4.8, under my supervision, was not only far faster and far cheaper than me. It was also far better than me.&lt;/p&gt;
&lt;p&gt;The title of my first cover letter was &amp;quot;Programming is my Art&amp;quot;.&lt;/p&gt;
&lt;p&gt;I had just lost to an AI in an art competition. By a landslide.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;For my whole career, the limit on my output had been hours: 70-ish in the flows, 50-plus in the ebbs, and lately a civilized 36. Now the limit was tokens.&lt;/p&gt;
&lt;p&gt;You have work queued up. You know exactly what needs doing, and you have a perfectly capable workforce standing by to do it. Then a session limit kicks in, or the weekly budget runs dry, and the entire workforce goes on unpaid leave at once. That&#39;s a token famine, and it&#39;s a trauma most programmers who are still on top of the game know far too well.&lt;/p&gt;
&lt;p&gt;If you&#39;re employed, you have one alternative to waiting, and it isn&#39;t a dignified one. You can go to whoever controls the company&#39;s AI budget, bowl in hand, and beg for a reset.&lt;/p&gt;
&lt;p&gt;&amp;quot;Please, sir, I want some more.&amp;quot;&lt;/p&gt;
&lt;p&gt;Tokens for the poor.&lt;/p&gt;
&lt;p&gt;During one particularly bad famine, with no reset forthcoming, I had a troubled conversation with ChatGPT, which runs on a different budget and was therefore still speaking to me. I asked it: &amp;quot;In the era of AI-assisted software development, what can I do that is more valuable than waiting until my token budget renews?&amp;quot;&lt;/p&gt;
&lt;p&gt;It answered. It lied. Eight of its ten suggestions were things I already delegated to Claude or Codex, depending on the token balance of the day. The other two were the equivalent of fetching a fresh cup of coffee. Actually, one of them literally was fetching a fresh cup of coffee.&lt;/p&gt;
&lt;p&gt;So I fetched a fresh cup of coffee.&lt;/p&gt;
&lt;p&gt;Then I re-reviewed a pull request I hadn&#39;t merged yet. Opus 4.8 had built it with TDD earlier that morning, at 99% branch coverage. Copilot had reviewed it. Codex had reviewed it. Their review comments, mostly nitpicks, had already been addressed. It was nigh-flawless, and it had been waiting all morning on the slowest participant in the whole process.&lt;/p&gt;
&lt;p&gt;Me.&lt;/p&gt;
&lt;p&gt;I looked at my &amp;quot;perfect programming language&amp;quot; with fresh eyes. I&#39;d been tinkering with it, on and off, for a quarter of a century, through a couple of names and more rewrites than I care to admit. The question behind it had always been the same: how should humans tell computers what to do?&lt;/p&gt;
&lt;p&gt;Suddenly I couldn&#39;t answer a much simpler one. Who was it for?&lt;/p&gt;
&lt;p&gt;Humans? Why?&lt;/p&gt;
&lt;p&gt;I parked it. There was no longer any point. Humans have no business writing code any more.&lt;/p&gt;
&lt;p&gt;That still left the famines to fill. I found two good uses for them, and both of them involved ChatGPT.&lt;/p&gt;
&lt;p&gt;The first was writing. Publishing an article is a perfectly productive use of a token famine, because a normal, free ChatGPT session can help with every step of it without touching the tokens my agents run on. A stronger model would do a better job, of course. But when you&#39;re too poor to program, you can still afford to publish.&lt;/p&gt;
&lt;p&gt;I&#39;ll let you draw your own conclusions about the article you&#39;re reading.&lt;/p&gt;
&lt;p&gt;The second was reading. I finally had time for all the things I&#39;d never been able to study at the depth I wanted, and ChatGPT turned out to be a tireless tutor. I could keep asking for the derivation, and then for the assumption underneath it, until the only limit left was how fast I could keep up. Some of that reading turned into a new hobby: amateur physics. I had been a physics and maths major back in ye olde times, before I switched to computer science. And contemporary AI could hardly scratch the surface of science, could it?&lt;/p&gt;
&lt;p&gt;Surely, this time, I had climbed far enough up the food chain.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;I&#39;m writing this in October 2026.&lt;/p&gt;
&lt;p&gt;My daily routine is to direct a fleet of agents as they do my bidding, until the session limits kick in or the tokens run out. My fleet, in turn, orchestrates fleets of its own.&lt;/p&gt;
&lt;p&gt;They build. They test. They review each other&#39;s work. They collaborate. They write the plans and the design documents, and keep the project&#39;s status and tasks up to date in Asana. They work on so many things in parallel that the merge conflicts are beyond anything a human could untangle, which is why they perform semantic merges of each other&#39;s work instead. They evaluate whether clients and servers are compatible, add compatibility measures where they&#39;re needed, and cherry-pick the changes into production.&lt;/p&gt;
&lt;p&gt;They monitor my production systems. They tell me about deviations and bugs I wasn&#39;t even aware of, and then they fix them. They migrate my development and staging servers and run nearly end-to-end tests against them.&lt;/p&gt;
&lt;p&gt;To make all of that possible, I&#39;ve introduced simulators for every relevant system boundary, giving the agents control over the environment they test in. Well, obviously I didn&#39;t introduce them. I asked Claude to.&lt;/p&gt;
&lt;p&gt;I&#39;ve even introduced &amp;quot;human simulators&amp;quot; (same caveat), so the agents can perform the tasks of users. Next, I plan to introduce Mechanical Turks, to let them orchestrate real users as well.&lt;/p&gt;
&lt;p&gt;After that, I suppose, the obvious next step is to ask Claude Code to wire-and-fire up an MCP server that hires temp workers directly from Upwork. When it farms out work to other agents, I usually tell it to use &amp;quot;the cheapest model and effort level that is certainly overkill for the job&amp;quot;. That same prompt could work just as well for humans. When a real human turns out to be the cheapest option that is certainly overkill for the job, the machines will simply hire one.&lt;/p&gt;
&lt;p&gt;I&#39;ve moved from AI-assisted development to human-assisted development.&lt;/p&gt;
&lt;p&gt;In AI-assisted development, the human does the work and the AI helps. In human-assisted development, the machines do the work, and the human supplies what they can&#39;t: intent, judgment, constraints, architecture, taste, priorities, skepticism, and the occasional &amp;quot;what the fuck did you do here?&amp;quot; My job is to decide what reality should look like, and to notice when the agents have built a different one.&lt;/p&gt;
&lt;p&gt;So when I say &amp;quot;I built this&amp;quot; these days, what I mean is: I asked Claude to.&lt;/p&gt;
&lt;p&gt;And the language? I&#39;ve started designing it again, for today&#39;s programmers, which are AIs, not humans. I stand by what I said: humans have no business writing code any more. But AIs have an entirely different skill set and entirely different needs. They never tire of typing, and ceremony costs them pennies. They&#39;ve read more code than any of us ever will, which means the syntax should be boring and the novelty should go where it pays: into what the compiler can check. Tell an AI &amp;quot;no&amp;quot;, and it will cheerfully find a way around the rule, up to and including deleting the test that said no, so every error message has to tell it what to do instead. And they still lie like they&#39;re running for office, only less often. Nothing they claim about their code can be taken on faith.&lt;/p&gt;
&lt;p&gt;So who is it for? Abelson and Sussman told a whole generation of computer science students that programs must be written for people to read, and only incidentally for machines to execute. They never said who would do the writing. Mine is a language for machines to write and people to read.&lt;/p&gt;
&lt;p&gt;Quite frankly, I have no idea what the upper limit is for a capable engineer practicing human-assisted development with a half-decent token budget. I&#39;m fairly sure it isn&#39;t linear. One engineer with ten agents is not eleven workers, because agents delegate to agents. They review each other, test each other, and build the tools and environments that let the next generation of agents take on more. Every improvement to the environment pushes more of the work further down the tree.&lt;/p&gt;
&lt;p&gt;With all the rigor you&#39;d expect from someone who abandoned physics for computer science, here is my current estimate:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;O(human judgment × (number of agents)&lt;sup&gt;recursive delegation&lt;/sup&gt; × quality of environment/tooling)&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Big-O notation conveniently ignores constant factors, which is how I&#39;ve managed to leave out the token budget.&lt;/p&gt;
&lt;p&gt;The exponent doesn&#39;t come free. You can only delegate what you can decompose, and you can only decompose what you understand, so how deep the tree goes is bounded by how deeply the human understands the problem. Expertise hasn&#39;t stopped mattering. It has become the exponent.&lt;/p&gt;
&lt;p&gt;Back in 2022, I watched creative directors command an AI to produce hundreds of pieces of art, keep the best, and hand the results to whichever humans were left. Fortunately, I thought, I&#39;m a programmer.&lt;/p&gt;
&lt;p&gt;I&#39;m the creative director now.&lt;/p&gt;
</content>
  </entry>
</feed>

