<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Trains of thought: everything</title>
  <link href="https://icelander.net/feed.xml" rel="self"/>
  <link href="https://icelander.net/trains"/>
  <id>https://icelander.net/feed.xml</id>
  <author><name>David Brandt</name></author>
  <updated>2026-11-03T00:00:00Z</updated>
  <entry>
    <title>One language, transpiled to all the others</title>
    <link href="https://icelander.net/v1/programming-languages/one-language-transpiled-to-all-the-others"/>
    <id>tag:icelander.net,2026:one-language-transpiled-to-all-the-others</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="programming-languages" label="Programming languages"/>
    <summary>Around 2001 I was tired of writing the same business rules in C, C++, Java, PL/SQL and Pascal, so I wanted to write them once and transpile them into code each environment would own. I called it LCD, never wrote a line of it down, and dropped it.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;LCD, the language I never wrote&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2001, written November 2026. Programming languages.&lt;/p&gt;&lt;p&gt;Around 2001 I was fed up with writing the same business rules over and over.&lt;/p&gt;
&lt;p&gt;They were never interesting rules. How an invoice is totalled. Which orders need someone to approve them. When a
discount applies. The kind of thing a business could explain in a paragraph and then spend a decade being slightly
wrong about. The trouble was not writing them. The trouble was writing them again, and again, because each execution
environment that needed the rule ran on a different technology stack.&lt;/p&gt;
&lt;p&gt;There would be a copy in C. Several copies in C++, because C++ was not one environment but several: MFC had its own
idea of what a string was, Qt had another, and plain C++ with the standard library had a third. There was Java on the
application server, and PL/SQL in the database, because the batch jobs and the reports ran next to the data. And
probably Pascal, since Delphi was what a lot of people built their clients in.&lt;/p&gt;
&lt;p&gt;Every copy was written by hand, sometimes by me and sometimes by someone else, in a different idiom, with different
defaults for rounding, for null, for what dividing two integers means. On the day they were written they agreed. From
then on they drifted. A fix in one copy was a bug report waiting to happen in all the others.&lt;/p&gt;
&lt;p&gt;What I wanted was obvious, or felt obvious: write the rule once, in one language, and have it turn up in every one of
those environments as code that belonged there.&lt;/p&gt;
&lt;p&gt;I called it LCD.&lt;/p&gt;
&lt;h2&gt;The day job&lt;/h2&gt;
&lt;p&gt;LCD came straight out of my day job.&lt;/p&gt;
&lt;p&gt;From 1998 to 2002 I worked at OZ, on iPulse, a communication and presence system: buddy lists, who is online,
messaging, routing. My part was the plumbing underneath, the RPC system. Marshalling code, proxies and stubs, parsers,
network streaming, and code generators, for several programming languages and environments at once. Most of it was in
C++, Perl and Java.&lt;/p&gt;
&lt;p&gt;A code generator for RPC is a machine for writing one thing once and getting it in several languages. You describe a
message, and out come a C++ struct, a Java class and whatever Perl wants instead, with the code that turns each of
them into bytes and back. The shape of the data is the same everywhere because nobody wrote it more than once.&lt;/p&gt;
&lt;p&gt;The RPC layer talked to many different contexts, and every one of them needed the same logic. The generators gave
them that logic without anyone writing it twice. If that worked for the RPC logic, it could work for the business
rules too.&lt;/p&gt;
&lt;h2&gt;The name&lt;/h2&gt;
&lt;p&gt;&lt;em&gt;Lowest common denominator&lt;/em&gt; is usually an insult. Here it was a specification.&lt;/p&gt;
&lt;p&gt;The language would contain what every target could express, and nothing else. If C++ and Java could both do
something, and do it the same way, it was in. If one of them couldn&#39;t, it was out, or it was given a meaning that both
of them could carry. And the same test would have to hold for every other language: Python, which was already around,
C#, which arrived the year after, then Go, TypeScript and Rust, and behind them the niche and the obsolete, the
languages that still quietly run a payroll somewhere. The name was meant to keep me honest: whatever LCD was, it was not going to be a
superset with a large runtime pretending to be every language at once.&lt;/p&gt;
&lt;h2&gt;Transpiled, obviously&lt;/h2&gt;
&lt;p&gt;I was pretty obsessed with transpiling it. Not compiling it to some common machine, not running it on a virtual
machine of its own: turning it into source code in each target&#39;s own language.&lt;/p&gt;
&lt;p&gt;That was not the fashionable answer in 2001. The fashionable answer was Java&#39;s: one runtime, everywhere, write once
and run anywhere. Oracle had even put a Java virtual machine inside the database. The PL/SQL kept coming anyway,
because the people who worked in the database wrote PL/SQL, and their tools, their debuggers and their habits all
assumed it.&lt;/p&gt;
&lt;p&gt;So the output had to be code that the people in each environment would own. A Java developer should open the
generated file and see Java, the kind a careful colleague might have written. A database developer should see a
PL/SQL package. Nobody downstream should have to know that LCD existed in order to read, debug or profile what it
produced.&lt;/p&gt;
&lt;p&gt;I had a less practical hope for it, too. Code formatters weren&#39;t in everyday use yet, and linters were a rarity out in
the boondocks of Reykjavík, so a code review could turn into a flamewar over braces, indentation and names. If every
environment got the code in its own style, why not every person? Write the code the way you like it, read it the way
you like it, and there is nothing left to argue about.&lt;/p&gt;
&lt;p&gt;Part 2 is about why this one choice turned out to shape almost everything else.&lt;/p&gt;
&lt;h2&gt;What I knew&lt;/h2&gt;
&lt;p&gt;Looking back at the list of languages I had used by then, I can see why I never believed in one denominator for
everything.&lt;/p&gt;
&lt;p&gt;I had written C++ for years, in more than one job, and Java and Perl in the one I had then. Along the way, at
school, at work and in between, I had spent time with COBOL, Ada, Prolog, Lisp, assembly, BASIC, Pascal and LPC, the
language the LPMud game servers ran on. Plus C, of course, and, because work demanded it, PL/SQL.&lt;/p&gt;
&lt;p&gt;If you take the common denominator of all of those, you get very little. Prolog has no assignment. COBOL, at the time,
had no functions you could define yourself. SQL has no steps at all, only sets. Assembly doesn&#39;t have types. A
language that worked in all of them could describe data and almost nothing else.&lt;/p&gt;
&lt;p&gt;I knew that. The interesting question was never whether one LCD could cover everything. It was how far one could get,
and with what.&lt;/p&gt;
&lt;h2&gt;What it was meant to be&lt;/h2&gt;
&lt;p&gt;LCD was imperative: variables, assignment, conditions, loops, functions and procedures, records. All the traditional
concepts, nothing exotic. Its memory was managed, which in those days meant one of two things: Java&#39;s garbage
collector, or reference counting in C++. It had the classical data types and the kind of standard library you would
expect to find anywhere: text, numbers, dates, collections.&lt;/p&gt;
&lt;p&gt;It was for business logic. It would not draw windows, open files or talk to the network. The program around it would
do that, and hand LCD what it needed to make a decision.&lt;/p&gt;
&lt;p&gt;And it was meant to be transpiled into C, C++ in its various frameworks, Java, and every other programming language,
past, present or future. Perl was on my mind as well, the way it was on everyone&#39;s mind in 2001: as the nightmare you
woke from at three in the morning, crying, in a cold sweat, having dreamt that you had to change somebody else&#39;s
regular expressions. LCD was going to let Perl sleep.&lt;/p&gt;
&lt;h2&gt;Why it stayed a dream&lt;/h2&gt;
&lt;p&gt;It never got further than that. No grammar, no transpiler, no decisions about how to close the gaps between targets:
their type systems, their memory models, their ideas of what a record or a list is. I didn&#39;t write anything down.&lt;/p&gt;
&lt;p&gt;I dropped it for an unglamorous reason. It wasn&#39;t something I wanted to spend my time on, and in those days it would
have taken a massive amount of time. One person, several transpilers, a runtime library for every target, and no way
to know whether any of it was right until all of it was finished.&lt;/p&gt;
&lt;h2&gt;Why write it down now&lt;/h2&gt;
&lt;p&gt;Because the idea never went away.&lt;/p&gt;
&lt;p&gt;It has kept coming back ever since, these days as a language I call Y. I have never seriously sat down and built it.
I have started it and shelved it more times than I can count. The oldest attempt that left anything in writing is
YAPL, from 2019, which early on described itself as a transpiler to &amp;quot;all machines&amp;quot;: LCD again, nearly twenty years
later, under a different name. The latest version of Y has put transpiling, and the features that languages have in
common, back in the spotlight, and that has had me reminiscing about LCD.&lt;/p&gt;
&lt;p&gt;The rest of this series works out what LCD would have had to be:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;One language, transpiled to all the others.&lt;/strong&gt; This part.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Levels of denominator.&lt;/strong&gt; Why one lowest common denominator was never going to be enough, and what to do instead.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Where the languages disagree.&lt;/strong&gt; Division, rounding, null, and the empty string that Oracle thinks is nothing.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The cost, then and now.&lt;/strong&gt; Why it would have taken years, and why that reason is mostly gone.&lt;/li&gt;
&lt;/ol&gt;
</content>
  </entry>
  <entry>
    <title>Levels of denominator</title>
    <link href="https://icelander.net/v1/programming-languages/levels-of-denominator"/>
    <id>tag:icelander.net,2026:levels-of-denominator</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="programming-languages" label="Programming languages"/>
    <summary>The common denominator of C, Java, PL/SQL and SQL is almost nothing, so LCD couldn&#39;t be one language. It needed a ladder of levels, from plain data up to managed references, with each target carrying the levels it could.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Why one lowest common denominator was never going to be enough&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2001, written November 2026. Programming languages.&lt;/p&gt;&lt;p&gt;I was not under the delusion that the same LCD would work with everything.&lt;/p&gt;
&lt;p&gt;PL/SQL makes the point by itself. It was on the list for one reason: the product I was working on used it. It is
Oracle&#39;s own proprietary language, and outside Oracle&#39;s databases nobody in their right mind writes it. It is SQL
with loops, records, collections and exceptions bolted on, so you can write procedural code in it if you must, but its
idea of data is rows. Nothing in it is shared the way objects are shared in Java: you assign
a record and you get a copy. Put PL/SQL in the same bucket as Java and C++, and the denominator shrinks to whatever
PL/SQL can do. Leave it out, and you have left out the database, which is where a lot of the rules had to run.&lt;/p&gt;
&lt;p&gt;What I thought instead was that there could be &lt;em&gt;levels&lt;/em&gt; of common denominator. I was mostly thinking about
imperative languages, and about whether the idea could spread further than that.&lt;/p&gt;
&lt;h2&gt;The ladder&lt;/h2&gt;
&lt;p&gt;Here are the levels. Each one is the common denominator of a smaller family of languages. Each
level contains the one below it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Level 0: data.&lt;/strong&gt; Booleans, integers, decimals, text, dates, enumerations, records, lists and maps. No null. This is
what my RPC code generators at OZ already did: describe a shape once, and get it in every language. Almost everything
can carry level 0, because it is just declarations.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Level 1: calculations and decisions.&lt;/strong&gt; Functions of values, with no state and no loops: arithmetic, conditions,
choices, failures, and the pure parts of the standard library. This reaches further than you&#39;d think. Every imperative
language can do it. So can PL/SQL. So can an SQL query, as an expression. So can COBOL, with &lt;code&gt;COMPUTE&lt;/code&gt; and &lt;code&gt;EVALUATE&lt;/code&gt;.
So can Lisp, and so can Prolog, as a predicate that never backtracks.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Level 2: structured imperative.&lt;/strong&gt; Local variables, assignment, loops, procedures, and records and collections passed
around by value. This is the procedural core: C, C++, Java, Pascal, Ada, PL/SQL&#39;s procedural half, the structured
dialects of BASIC, and COBOL within its limits. SQL drops out here, because SQL doesn&#39;t do steps.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Level 3: managed references.&lt;/strong&gt; Values that are shared between owners and change in place, with their lifetime
managed by a garbage collector or by reference counting. Java, C++ with reference-counted handles, Delphi, Perl and
Lisp can do this. PL/SQL drops out here.&lt;/p&gt;
&lt;p&gt;That last level is the language I wanted: imperative, with managed memory. The levels below it are what you
get when you ask how much of it survives in environments that can&#39;t manage memory that way.&lt;/p&gt;
&lt;p&gt;A module would say which level it needs, and the transpiler would refuse anything above that. A target would say which
levels it can carry. A rule written at level 1 could then be used from level-3 Java code and from an SQL query in the
database alike. The discipline for whoever wrote the rule would be simple: use the lowest level that can say it, because
that is the level that reaches the most places. Most business rules are decisions and calculations. Most of them would
have lived low.&lt;/p&gt;
&lt;h2&gt;What a level 1 rule looks like&lt;/h2&gt;
&lt;p&gt;LCD never got a syntax, so the examples in this series borrow Y&#39;s.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;module pricing
level calculations


-- A price after a discount, to the cent.
--   why: The order screen, the invoice run and the month-end report must show the same price.
function discounted(amount is decimal(12, 2), rate is decimal(5, 4)) returns decimal(12, 2):
    return round_half_away_from_zero(amount * (1 - rate), places: 2)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In the database, a level 1 function doesn&#39;t need to be a function at all. It can be an expression in a query, and run
over every row at once, which is where the database earns its keep:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-sql&quot;&gt;SELECT order_id, ROUND(amount * (1 - rate), 2) AS discounted
FROM   order_lines;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In COBOL it becomes a small subprogram, with overflow reported through a status field:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-cobol&quot;&gt;       PROCEDURE DIVISION USING AMOUNT RATE RESULT STATUS-CODE.
           MOVE 0 TO STATUS-CODE
           COMPUTE RESULT ROUNDED = AMOUNT * (1 - RATE)
               ON SIZE ERROR MOVE 1 TO STATUS-CODE
           END-COMPUTE
           EXIT PROGRAM.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The rounding mode in the LCD version is not an accident. Rounding half away from zero is what Oracle&#39;s &lt;code&gt;ROUND&lt;/code&gt; and
COBOL&#39;s &lt;code&gt;ROUNDED&lt;/code&gt; both do natively, so neither of them needs any help. Pick a different mode and you&#39;d need a library
in each of them to do it for you. Part 3 is largely about choices like that one.&lt;/p&gt;
&lt;h2&gt;Who carries what&lt;/h2&gt;
&lt;p&gt;This is roughly how the languages I knew in 2001 would have sorted themselves. A full mark means the language
carries the level natively. A half mark means it can, with a runtime library or within limits. A cross means it
can&#39;t.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Language&lt;/th&gt;
&lt;th&gt;0 Data&lt;/th&gt;
&lt;th&gt;1 Calculations&lt;/th&gt;
&lt;th&gt;2 Structured&lt;/th&gt;
&lt;th&gt;3 References&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;C&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;C++, in each framework&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Java&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Pascal (Delphi)&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ada&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;PL/SQL&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✗&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SQL&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✗&lt;/td&gt;
&lt;td&gt;✗&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;COBOL&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;td&gt;✗&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BASIC&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;td&gt;✗&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lisp&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Prolog&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;✓&lt;/td&gt;
&lt;td&gt;◐&lt;/td&gt;
&lt;td&gt;✗&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;C gets a half mark at level 1 because it has no decimal type, so money needs a library. COBOL gets a half mark at
level 2 because its tables have fixed sizes and, in the COBOL of that time, nothing could call itself. Prolog gets one
because loops have to become recursion and assignment has to become new names.&lt;/p&gt;
&lt;h2&gt;Why transpiling forces the levels&lt;/h2&gt;
&lt;p&gt;This is where the obsession with transpiling comes back in.&lt;/p&gt;
&lt;p&gt;If LCD had been compiled, levels would hardly matter. A compiler can put any program on any machine, because in the
end every target is machine code. One denominator, the whole language, would have done.&lt;/p&gt;
&lt;p&gt;A transpiler can only produce what the target language can &lt;em&gt;say&lt;/em&gt;, and say readably. The common denominator that
matters is one of expression, not one of execution. And expression is exactly where language families differ: SQL can
say a calculation beautifully and a loop not at all, and Java can say almost anything but says a decimal calculation
in a way nobody would call beautiful.&lt;/p&gt;
&lt;p&gt;So the levels are not a compromise forced on the design. They fall straight out of the decision to transpile.&lt;/p&gt;
&lt;p&gt;The same decision shapes everything else. Transpiled code is code people own. It has to read as if a careful
programmer of that language wrote it, with its own types, its own loops and its own way of failing. Names have to
survive unchanged, so that a search for a rule finds every copy of it. Each generated function has to say where it
came from, and nobody edits it by hand. Whatever a target lacks, such as decimals in C, has to come from a small
library written in that target&#39;s own language and shipped as source beside the generated code. And the whole thing
has to fit into each environment&#39;s build as it already is: the C++ goes through the C++ build, the PL/SQL is installed
like any other package.&lt;/p&gt;
&lt;h2&gt;Spreading further&lt;/h2&gt;
&lt;p&gt;The other half of the thought was whether LCD could spread beyond the imperative family. Each direction would have
needed something different.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Sets.&lt;/strong&gt; Level 1 already reaches into SQL, as an expression in a query. A level of set operations, such as filtering,
joining, grouping and summing lists of records, would sit beside level 2 rather than above it. It would transpile into
SQL and Lisp naturally, and into loops everywhere else.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Logic.&lt;/strong&gt; Level 1 maps onto Prolog predicates. What Prolog adds, searching and backtracking, has no counterpart
anywhere else, so it would stay out. The interesting direction is the other one: LCD&#39;s rules written out as Prolog
facts and clauses could be asked &lt;em&gt;why&lt;/em&gt; a decision came out the way it did.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Lisp.&lt;/strong&gt; Lisp could carry every level, and its macros could be LCD&#39;s transpiler for Lisp. It is the one target where
LCD would have been a library rather than a code generator.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Assembly.&lt;/strong&gt; Assembly can run anything and explain nothing. LCD&#39;s whole promise was readable code in each
environment, so assembly would have been a compiler&#39;s job, never LCD&#39;s.&lt;/p&gt;
&lt;p&gt;Next, in part 3: what happens inside a level, when the languages that are supposed to agree simply don&#39;t.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Where the languages disagree</title>
    <link href="https://icelander.net/v1/programming-languages/where-the-languages-disagree"/>
    <id>tag:icelander.net,2026:where-the-languages-disagree</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="programming-languages" label="Programming languages"/>
    <summary>Languages that look alike on paper disagree about division, rounding, null, the empty string, truth and overflow. LCD would have had to pick one meaning and put it in the name, or leave the feature out. Here is how, ending with one rule written in LCD, in PL/SQL and in Java as it was in 2001.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Division, rounding, null, and the empty string that Oracle thinks is nothing&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2001, written November 2026. Programming languages.&lt;/p&gt;&lt;p&gt;Levels decide which languages have to agree. They don&#39;t make them agree.&lt;/p&gt;
&lt;p&gt;Take any two languages that both carry level 1 and write the same innocent-looking expression in each. Surprisingly
often you get two answers. Each answer is reasonable, each is documented, and each has been relied on by somebody for
years. None of them is wrong. They are just different, and a rule that gives different answers in different places is
exactly what LCD existed to prevent.&lt;/p&gt;
&lt;p&gt;So inside each level there is one more test, applied to every feature: do all the targets that carry this level give
it the same meaning? If they do, it&#39;s in. If they can with help, it&#39;s in, and a small runtime library supplies the help
where it&#39;s missing. If they can&#39;t agree, LCD has two options. It can pick one meaning and say so out loud, in the
name of the operation. Or it can leave the feature out.&lt;/p&gt;
&lt;p&gt;Most of LCD&#39;s character would have come from that last case. Here is how.&lt;/p&gt;
&lt;h2&gt;Dividing two integers&lt;/h2&gt;
&lt;p&gt;C, C++ and Java truncate: 7 divided by 2 is 3. Pascal keeps the two apart: &lt;code&gt;div&lt;/code&gt; for integer division, and &lt;code&gt;/&lt;/code&gt;, which
always gives a real, 3.5. PL/SQL&#39;s &lt;code&gt;/&lt;/code&gt; gives 3.5 too. Perl gives 3.5 as a floating-point
number. Ada truncates, but distinguishes &lt;code&gt;rem&lt;/code&gt; from &lt;code&gt;mod&lt;/code&gt;, which disagree as soon as a negative number turns up.&lt;/p&gt;
&lt;p&gt;LCD&#39;s answer: no &lt;code&gt;/&lt;/code&gt; on integers at all. Division says how it rounds: &lt;code&gt;divide_truncated&lt;/code&gt;, &lt;code&gt;divide_floor&lt;/code&gt;,
&lt;code&gt;divide_exact&lt;/code&gt;, the last of which fails if anything is left over. It&#39;s longer to type. Nobody ever has to wonder.&lt;/p&gt;
&lt;h2&gt;Rounding&lt;/h2&gt;
&lt;p&gt;Java&#39;s &lt;code&gt;BigDecimal&lt;/code&gt; offers every rounding mode you could want. Oracle&#39;s &lt;code&gt;ROUND&lt;/code&gt; rounds half away from zero, as does
COBOL&#39;s &lt;code&gt;ROUNDED&lt;/code&gt;. Delphi&#39;s &lt;code&gt;Round&lt;/code&gt; rounds half to even, the banker&#39;s way. C leaves it to you.&lt;/p&gt;
&lt;p&gt;So rounding names its mode too: &lt;code&gt;round_half_even&lt;/code&gt;, &lt;code&gt;round_half_away_from_zero&lt;/code&gt;. Where a target lacks the mode you asked
for, the runtime library implements it in that target&#39;s own language. That&#39;s why the level 1 example in part 2 chose
half away from zero: the database and COBOL could both do it without help.&lt;/p&gt;
&lt;h2&gt;Null&lt;/h2&gt;
&lt;p&gt;In SQL, &lt;code&gt;NULL&lt;/code&gt; means unknown, and unknown spreads. Compare a null with anything and the answer is unknown, and an &lt;code&gt;IF&lt;/code&gt;
whose condition is unknown quietly takes the &lt;code&gt;ELSE&lt;/code&gt; branch. Java and Pascal have null references, which do something
much louder. C has null pointers, which do whatever they like.&lt;/p&gt;
&lt;p&gt;LCD would have had no null. If something can be absent, its type says so, and the code has to say what happens in each
case. Where LCD code meets its host, the transpiled code checks every value on the way in, so a &lt;code&gt;NULL&lt;/code&gt; from the
database becomes a failure with a name, rather than a quiet wrong turn.&lt;/p&gt;
&lt;h2&gt;The empty string&lt;/h2&gt;
&lt;p&gt;Oracle stores the empty string as &lt;code&gt;NULL&lt;/code&gt;. This is not a bug. It is how Oracle has always worked, and it&#39;s documented,
and it has surprised people for decades.&lt;/p&gt;
&lt;p&gt;In LCD, text is never absent, although it can be empty. The PL/SQL runtime would have had to represent empty text in a
way that could never turn into &lt;code&gt;NULL&lt;/code&gt; on the way through.&lt;/p&gt;
&lt;h2&gt;What counts as true&lt;/h2&gt;
&lt;p&gt;C has no boolean type of its own and treats zero as false. Many BASICs use -1 for true. Perl treats &lt;code&gt;&amp;quot;0&amp;quot;&lt;/code&gt; and &lt;code&gt;&amp;quot;&amp;quot;&lt;/code&gt; as
false, along with a few other things you have to learn the hard way.&lt;/p&gt;
&lt;p&gt;In LCD only a boolean can decide an &lt;code&gt;if&lt;/code&gt;. There is no truthiness to translate, so there is nothing to get wrong.&lt;/p&gt;
&lt;h2&gt;Overflow&lt;/h2&gt;
&lt;p&gt;Signed overflow in C is undefined, which means the compiler may assume it never happens. Java wraps around without a
word. PL/SQL and Ada raise an error. Pascal raises one only if overflow checks are switched on. COBOL quietly cuts off
the digits that don&#39;t fit, unless you asked for &lt;code&gt;ON SIZE ERROR&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;In LCD, overflow is a failure, everywhere. The C runtime checks for it, so that C behaves like the others.&lt;/p&gt;
&lt;h2&gt;Where strings start&lt;/h2&gt;
&lt;p&gt;PL/SQL&#39;s &lt;code&gt;SUBSTR&lt;/code&gt;, COBOL and Pascal&#39;s strings start counting at 1. C, C++ and Java start at 0.&lt;/p&gt;
&lt;p&gt;LCD has to pick one and transpile it for the others. I&#39;d have picked 1, because business people count from 1. In Y I
ended up with something better: every index carries its unit, so a position can&#39;t be confused with a count.&lt;/p&gt;
&lt;h2&gt;Names&lt;/h2&gt;
&lt;p&gt;PL/SQL, Pascal, Ada and COBOL don&#39;t care about case. PL/SQL names stopped at 30 characters, and so did COBOL&#39;s. COBOL
wants hyphens where everyone else wants underscores. And every language reserves a different set of words.&lt;/p&gt;
&lt;p&gt;So every LCD name is lowercase with underscores, at most 30 characters long, unique even if you ignore case, and not a
reserved word anywhere. COBOL gets hyphens instead of underscores. Y still has the all-lowercase rule today. I like to
think it came from somewhere.&lt;/p&gt;
&lt;h2&gt;Errors&lt;/h2&gt;
&lt;p&gt;C returns error codes. C++, Java, Pascal, Ada and PL/SQL have exceptions, all different. MFC throws pointers to
exceptions, which you then have to delete. COBOL has status fields.&lt;/p&gt;
&lt;p&gt;An LCD function declares, by name, the ways in which it can fail. Each target then reports those failures in its own
way: a return code in C, an exception in Java, a raised error in PL/SQL.&lt;/p&gt;
&lt;h2&gt;The rest&lt;/h2&gt;
&lt;p&gt;Two more fall out quickly. C and C++ leave the order in which a function&#39;s arguments are evaluated unspecified, so an
LCD expression may have at most one effect. And every language has its own precedence table, so LCD requires
parentheses wherever two kinds of operator meet.&lt;/p&gt;
&lt;p&gt;The last one is about time and chance. Every target has its own clock and its own random number generator, and no two
of them agree. Business logic in LCD would have had neither. &amp;quot;Today&amp;quot; would be an argument, passed in from outside. So
would any random seed. Given the same inputs, every target would compute the same answer, which is the whole point.&lt;/p&gt;
&lt;h2&gt;Memory&lt;/h2&gt;
&lt;p&gt;LCD&#39;s memory was managed: Java&#39;s garbage collector, or reference counting in C++. Making that work the same
everywhere is its own puzzle, because the targets manage memory in three different ways. Java collects garbage. C++,
Delphi and Perl count references. PL/SQL copies, and shares nothing.&lt;/p&gt;
&lt;p&gt;The levels from part 2 sort this out. At levels 0 to 2 there are only values: assign a record and you get your own
copy. Behind the scenes a runtime may share it and copy it only when someone writes to it, but no program can tell.
That works everywhere, including PL/SQL.&lt;/p&gt;
&lt;p&gt;Level 3 adds values that really are shared. For those to behave identically under a garbage collector and under
reference counting, two things have to go. &lt;em&gt;Cycles&lt;/em&gt; go, because a reference counter can&#39;t reclaim a cycle and a
garbage collector can. &lt;em&gt;Finalisers&lt;/em&gt; go, because nobody knows when a garbage collector will run, but everybody knows
exactly when the last reference disappears.&lt;/p&gt;
&lt;h2&gt;One rule, three languages&lt;/h2&gt;
&lt;p&gt;Here is a level 2 rule, with a loop in it, which PL/SQL can carry and plain SQL can&#39;t. Again, the notation borrows
from Y.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;module invoicing
level structured


-- A line on an invoice.
record invoice_line:
    quantity is integer
    unit_price is decimal(12, 2)


-- Why an invoice cannot be totalled.
failure invoice_error:
    no_lines
    negative_quantity


-- Totals an invoice, with the volume discount the price list gives above 1000.00.
--   why: The shop, the web site and the nightly batch must charge the same amount for the same lines.
function invoice_total(lines is list of invoice_line) returns decimal(12, 2) fails invoice_error:
    if no_lines = lines.is_empty():
        fail invoice_error.no_lines
    else has_lines:
        var total is decimal(12, 2) = 0.00

        for line in lines:
            if negative = (line.quantity &amp;lt; 0):
                fail invoice_error.negative_quantity
            else counted:
                total := total + (line.unit_price * line.quantity)

        if large_order = (total &amp;gt;= 1000.00):
            return round_half_even(total * 0.95, places: 2)
        else small_order:
            return total
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Every condition has a name, &lt;code&gt;large_order&lt;/code&gt; or &lt;code&gt;small_order&lt;/code&gt;, and so does every way of failing. That is how you would
check that the shop, the web site and the nightly batch took the same path: the names would be the same strings in
every language&#39;s logs.&lt;/p&gt;
&lt;p&gt;Here it is in PL/SQL. Oracle has no half-to-even rounding, so that call goes to the runtime library, &lt;code&gt;lcd_std&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-sql&quot;&gt;CREATE OR REPLACE PACKAGE BODY invoicing AS
  FUNCTION invoice_total(lines IN invoice_lines) RETURN NUMBER IS
    total NUMBER(12, 2) := 0;
  BEGIN
    IF lines IS NULL OR lines.COUNT = 0 THEN
      lcd_std.fail(&#39;invoicing.invoice_error.no_lines&#39;);
    END IF;

    FOR i IN 1 .. lines.COUNT LOOP
      IF lines(i).quantity IS NULL OR lines(i).unit_price IS NULL THEN
        lcd_std.fail(&#39;lcd.missing_value&#39;);
      END IF;
      IF lines(i).quantity &amp;lt; 0 THEN
        lcd_std.fail(&#39;invoicing.invoice_error.negative_quantity&#39;);
      END IF;
      total := total + (lines(i).unit_price * lines(i).quantity);
    END LOOP;

    IF total &amp;gt;= 1000.00 THEN
      RETURN lcd_std.round_half_even(total * 0.95, 2);
    ELSE
      RETURN total;
    END IF;
  END invoice_total;
END invoicing;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;All those &lt;code&gt;IS NULL&lt;/code&gt; checks are the price of LCD having no null. Leave them out, and a null quantity coming in from
outside would make every comparison unknown and quietly take the wrong branch.&lt;/p&gt;
&lt;p&gt;And in Java as it was in 2001, before generics and before enums:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-java&quot;&gt;public static BigDecimal invoice_total(List lines) throws invoice_error {
    if (lcd_std.require(lines).isEmpty()) {
        throw new invoice_error(invoice_error.no_lines);
    }
    BigDecimal total = lcd_std.decimal(&amp;quot;0.00&amp;quot;);
    for (Iterator it = lines.iterator(); it.hasNext(); ) {
        invoice_line line = (invoice_line) it.next();
        if (line.quantity &amp;lt; 0) {
            throw new invoice_error(invoice_error.negative_quantity);
        }
        total = total.add(line.unit_price.multiply(BigDecimal.valueOf(line.quantity)));
    }
    if (total.compareTo(lcd_std.decimal(&amp;quot;1000.00&amp;quot;)) &amp;gt;= 0) {
        return lcd_std.round_half_even(total.multiply(lcd_std.decimal(&amp;quot;0.95&amp;quot;)), 2);
    }
    return total;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Yes, that is &lt;code&gt;invoice_total&lt;/code&gt; and not &lt;code&gt;invoiceTotal&lt;/code&gt;. It&#39;s unidiomatic Java, and it&#39;s deliberate. A search for the
rule&#39;s name should find it in every language.&lt;/p&gt;
&lt;p&gt;The C version returns its failure as a code and writes the total through a pointer, and its money is a scaled integer
from the runtime library. The Delphi version gets off lightly: Delphi&#39;s &lt;code&gt;Currency&lt;/code&gt; type holds four fixed decimal places,
so the arithmetic is native.&lt;/p&gt;
&lt;h2&gt;The library&lt;/h2&gt;
&lt;p&gt;The standard library would have been small, and written once per target, by hand, in that target&#39;s own language:
text, numbers with their named roundings and divisions, dates with a named rule for what happens at the end of a month,
collections, and conversion to and from text in an explicit format, never one that depends on the machine&#39;s locale.&lt;/p&gt;
&lt;p&gt;What it would never have had: files, the network, the database, threads, the clock, randomness, anything visual. That
all belongs to the program around the rules.&lt;/p&gt;
&lt;h2&gt;The tests&lt;/h2&gt;
&lt;p&gt;A test would be written once in LCD and transpiled along with the code. It would run on every target that carries the
module&#39;s level, and the targets would have to agree. When they didn&#39;t, that would not mean one of them won. It would
mean a transpiler or a runtime library was wrong.&lt;/p&gt;
&lt;p&gt;In the end the tests would be the real definition of each level. A feature is in a level when its tests pass on every
target of that level.&lt;/p&gt;
&lt;p&gt;All of which sounds tidy on paper. Part 4 is about what it would have cost to build, and why that, rather than any of
the problems above, is what killed it.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The cost, then and now</title>
    <link href="https://icelander.net/v1/programming-languages/the-cost-then-and-now"/>
    <id>tag:icelander.net,2026:the-cost-then-and-now</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="programming-languages" label="Programming languages"/>
    <summary>I dropped LCD because it would have cost one person years: a transpiler, a runtime library and a test suite for every target. Much of its thinking came back in Y anyway, and the work that killed it is exactly the kind agents now do well.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Why LCD would have taken years, and why that reason is mostly gone&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2001, written November 2026. Programming languages.&lt;/p&gt;&lt;p&gt;I dropped LCD because it wasn&#39;t something I wanted to spend my time on, and because in those days it would have taken
a massive amount of time.&lt;/p&gt;
&lt;p&gt;Twenty-five years later I can be more specific about the second half of that.&lt;/p&gt;
&lt;h2&gt;The bill&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;A transpiler for every target.&lt;/strong&gt; C, C++ once for each framework, Java, PL/SQL, Pascal, and COBOL if it was ever
going to reach the older systems. Each one emitting code that reads as if a person wrote it, which is much harder than
emitting code that merely works.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A runtime library for every target.&lt;/strong&gt; Decimal arithmetic, every rounding mode, dates, text and collections, each
written by hand in the target&#39;s own language. C needs all of it. PL/SQL needs most of it. Each C++ framework needs its
own glue.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Levels multiply the work.&lt;/strong&gt; Every level has to be defined, tested and transpiled for every target that carries it,
and a change to one level has to be checked against all of them.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The gaps.&lt;/strong&gt; I never got as far as deciding how to close the gaps between targets, and each one is real work. Typing:
PL/SQL&#39;s &lt;code&gt;NUMBER&lt;/code&gt; against fixed-width integers, Java without generics, C without checked anything, Perl without static
types at all. Memory: values that are cheap on every memory manager, and shared references that behave the same under
a garbage collector and a reference counter. Data: the database thinks in rows and sets, and everyone else thinks in
records, so a loop that is natural in Java is slow inside Oracle. Text: the database, the JVM, Windows code pages and
C&#39;s bytes each had their own idea of a character, so there was no single meaning of &amp;quot;length&amp;quot; or &amp;quot;upper case&amp;quot; to
transpile.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Proving the targets agree.&lt;/strong&gt; The test suite would run on every toolchain, some of them inside a database, and every
disagreement would have to be traced to a transpiler, a runtime library or the definition of a level.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Keeping up.&lt;/strong&gt; Every new version of a target changes what a level can hold. Java 5 arrived in 2004 with generics and
enums. Oracle 10g added new numeric types. Each framework kept moving too.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Adoption.&lt;/strong&gt; Even when it all worked, the generated code would be code that a team in each environment had to debug,
deploy and own, written by a tool they hadn&#39;t asked for.&lt;/p&gt;
&lt;p&gt;None of that is impossible. It was just years of one person&#39;s work for a tool whose benefit arrived only at the very
end, once every transpiler was finished. I had a job, and other things I would rather do. It stayed a dream.&lt;/p&gt;
&lt;h2&gt;It came back anyway&lt;/h2&gt;
&lt;p&gt;What surprised me, going back through everything that led to Y, is how much of LCD&#39;s thinking kept turning up
without my ever having written LCD down.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Transpiling.&lt;/strong&gt; YAPL, the first version that left any written trace, was started in 2019 and early on described
itself as a transpiler to &amp;quot;all machines&amp;quot;. In the 2024 design discussions I gave it up: &amp;quot;we are no longer aiming for transpilation&amp;quot;. In September 2026
it came straight back, in a conversation that started with a language that &amp;quot;really just transpiles to Rust&amp;quot;. Y&#39;s
position today is a compromise I think LCD would have recognized. Rust is not Y&#39;s definition; it is a backend and a
check. But the example programs are lowered to plain, readable Rust that an ordinary Rust toolchain builds.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Levels.&lt;/strong&gt; Y has a constraint language for describing rules about code that lives in other languages: Python, T-SQL,
C#, Rust and more. Its kinds of rule are switched on host by host, the simpler kinds for some languages and the richer
ones only for languages whose types can carry them. That is LCD&#39;s levels again, with &amp;quot;level&amp;quot; renamed to &amp;quot;kind&amp;quot;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rules written once.&lt;/strong&gt; The reason that constraint language exists is the problem LCD started from: business rules
repeated across a client, a server and a database, each copy free to drift.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The host does what the language can&#39;t.&lt;/strong&gt; Y has no foreign-function interface. When it needs something it can&#39;t
express, the function is declared in Y and written in the backend&#39;s own language. LCD&#39;s runtime library, written by
hand in each target, was the same move.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The same tests everywhere.&lt;/strong&gt; In Y, running several backends and comparing them is a way of finding bugs, and a
disagreement is a bug, never a vote. That is LCD&#39;s test suite, almost word for word.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;And the small things.&lt;/strong&gt; No &lt;code&gt;/&lt;/code&gt; on integers; division says how it rounds. No null, and nothing is true except a
boolean. Every name lowercase, with underscores. Every decision named, so traces can be compared. Every function
declaring how it can fail. No clock and no randomness unless they are handed in. Every rule saying &lt;em&gt;why&lt;/em&gt;. Each of those
appears in Y, and each of them is an answer to a question LCD would have had to ask.&lt;/p&gt;
&lt;p&gt;Most of them first turn up in writing years later, in YAPL or in the 2024 design discussions. All of them are what you
would end up with if you spent a long time trying to make several languages agree, and that is what I spent some of 2001 thinking about.&lt;/p&gt;
&lt;h2&gt;What changed&lt;/h2&gt;
&lt;p&gt;LCD was a set of small languages for people to write, with transpilers written by people, each level chosen so that
its targets could carry it. Y is a whole language, written mostly by AI agents and reviewed by people, with a compiler
that checks what the agents propose. I wrote about what happens to a language when its writer changes in
&lt;a href=&quot;/v1/programming-languages/write-only-languages&quot;&gt;Write-only languages&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The change that matters for LCD is simpler than that. Look at the bill again. A transpiler per target. A runtime
library per target. A test suite run against every target, with every disagreement traced to its cause. That is
exactly the kind of work that is specified once, checked mechanically, and done many times over: the kind of work
agents now do well.&lt;/p&gt;
&lt;p&gt;The reason I dropped LCD is mostly gone. The question it asked is still the one Y asks: how do you write a rule once,
and trust it wherever it runs?&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The unreviewable pull request</title>
    <link href="https://icelander.net/v1/processes/the-unreviewable-pull-request"/>
    <id>tag:icelander.net,2026:the-unreviewable-pull-request</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="processes" label="Processes"/>
    <summary>Big tasks grow into pull requests nobody can review. I used to split them into stacks by hand, after the fact, and hated it. Now an agent splits the stack, other agents review each layer, and my own review is finally worth something.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Let it grow, then have an agent carve it up&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2019, written November 2026. Processes.&lt;/p&gt;&lt;p&gt;It starts with a task that fits on a sticky note. Somewhere along the way I notice that the code I&#39;m touching is a mess, and since I&#39;m in there anyway, I clean it up. The cleanup exposes an edge case nobody handled, so I handle it. Handling it properly needs tests that don&#39;t exist, so I write them. The tests find two bugs. I fix them. Then I remember a feature that was part of the original ask and somehow fell off the plan, and I add that too.&lt;/p&gt;
&lt;p&gt;Eventually I open a pull request with 400 files and 20,000 lines changed.&lt;/p&gt;
&lt;p&gt;That&#39;s on a good day. On a bad day, I&#39;ve also found a better name for one of the core concepts along the way. What everyone has been calling a &amp;quot;user&amp;quot; is really an &amp;quot;account&amp;quot;, so I fix it, and a codebase-wide search and replace changes a line here and there in a thousand files. Add the generated code and the test snapshots that get regenerated because the word appears in them, and the PR isn&#39;t 20,000 lines any more. It&#39;s 400,000. Something microscopic did that. One word.&lt;/p&gt;
&lt;p&gt;Every one of those lines is, as far as I can tell, an improvement. Together, they&#39;re unreviewable.&lt;/p&gt;
&lt;p&gt;Nobody reads 20,000 lines, let alone 400,000. A reviewer handed a PR like that has two honest options, both bad: spend a week on it, or skim it, leave a comment about a variable name so it looks read, and approve. The research on this is old and the metric is crude, but it points the same way. In &lt;a href=&quot;https://smartbear.com/learn/code-review/best-practices-for-peer-code-review/&quot;&gt;SmartBear&#39;s study of a team at Cisco&lt;/a&gt;, reviewers stopped finding much after 200 to 400 lines. My good-day PR was fifty of those, stapled together. The bad-day one was a thousand.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;A manager I had years ago was hostile to PRs like that. Hard-core hostile. A monster didn&#39;t get reviewed. It got sent back to be split.&lt;/p&gt;
&lt;p&gt;I didn&#39;t like it at the time. Naturally.&lt;/p&gt;
&lt;p&gt;GitHub had no concept of a stack back then. A chain of PRs was a chain of branches, each based on the one before it, and every fix to the bottom one had to be rebased up through the rest by hand. The same went for every time the master branch moved under me: I had to propagate the merge up through every PR in the chain, again and again, until the reviewer deigned them worth his time. Skip a round, and the drift quickly became unmanageable. Not that it wasn&#39;t tough enough already.&lt;/p&gt;
&lt;p&gt;All of that was tedious, and it was the smallest of my problems.&lt;/p&gt;
&lt;p&gt;The big one was that the split can&#39;t be planned up front. With a large task, you only know exactly what you&#39;re doing in retrospect, after you&#39;ve done it. En route, you have a broad idea, and a broad idea doesn&#39;t tell you where the seams are. So I&#39;d finish the work, then pluck it apart, hunk by hunk, &lt;code&gt;git add -p&lt;/code&gt; until my eyes glazed over. The job was to take a week&#39;s worth of exploratory work, with its detours, dead ends and second thoughts, and retrofit it into a perfect string of pearls: the refactor, then the tests, then the two bug fixes, then the feature. Each pearl in its place, lined up as if there had never been anything but 100% certainty about the path and the destination.&lt;/p&gt;
&lt;p&gt;That was the horror.&lt;/p&gt;
&lt;p&gt;It wasted a lot of time. I learned quickly that his way was better anyway.&lt;/p&gt;
&lt;p&gt;What surprised me was that splitting didn&#39;t just make the review easier. It made the work better. Each PR in the chain was a milestone, and a milestone has to stand on its own: it builds, its tests pass, and its description says what it does without leaning on the next eleven. When a refactor is cut loose from the feature that motivated it, you notice the parts of the refactor that only made sense for a version of the feature that later changed shape. When the tests have to stand alone, you notice they&#39;re half-finished. Debug leftovers, dead code, the abandoned first attempt that&#39;s still half in there: a monster hides all of that, and a milestone doesn&#39;t. Every PR came out more correct, more maintainable, more beautiful. More all sorts of good things.&lt;/p&gt;
&lt;p&gt;And it took load off the reviewer, which matters more than it sounds. In a review process that&#39;s actually worth something (not the rubber stamp practiced in most places), the reviewers are the more experienced engineers. They&#39;re overloaded, their time is limited, and what they know is some of the most valuable knowledge in the building. Handing them 20,000 lines wastes the scarcest thing you have.&lt;/p&gt;
&lt;p&gt;He was right. It just took me a while to stop resenting the hours it cost.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Today, Opus generally has more knowledge, more time and more skill than those reviewers did. You&#39;d think that makes the problem go away. It doesn&#39;t. Opus also does a better job on a focused PR than on a multi-hundred-file monstrosity. Its attention runs out later than a human&#39;s, but it runs out, and it runs out quietly. An overwhelmed human knows they&#39;re overwhelmed. An overwhelmed model writes a confident review of whichever parts it happened to look at closely.&lt;/p&gt;
&lt;p&gt;So the rule stands. What changed is who pays for it.&lt;/p&gt;
&lt;p&gt;I still let the task grow. I&#39;ve stopped fighting that. Refactoring along the way, handling the edge cases, adding the feature everyone forgot: that&#39;s how I find out what the task actually was. Then, once I know what I did, I tell an agent to split it into a stack. It strings the pearls without complaint, and it has no trouble pretending we knew where we were going all along. Then I have an agent review each PR in the stack. Then an agent resolves the merge conflicts the review fixes cause, and merges the stack, bottom up.&lt;/p&gt;
&lt;p&gt;GitHub now supports &lt;a href=&quot;https://github.github.com/gh-stack/&quot;&gt;stacked pull requests&lt;/a&gt; as a first-class concept, in public preview since July. That helps, but the tooling was never the hard part. The hard part was the retrofitting, and that&#39;s exactly what I delegate.&lt;/p&gt;
&lt;p&gt;The best side effect is the one I didn&#39;t expect. With the work cut into tractable bites, my own valuable time and inferior mind can provide a much better review of the agents&#39; work. I can follow a 600-line PR that does one thing. I could never follow the monster, and I wrote it. Well, obviously I didn&#39;t write it. I asked Claude to.&lt;/p&gt;
&lt;p&gt;The rest of this is the parts that took me a while to get right.&lt;/p&gt;
&lt;h2&gt;1. The monster is the specification&lt;/h2&gt;
&lt;p&gt;Don&#39;t delete the monster branch after the split. Until the stack is merged, it&#39;s the source of truth, and it makes the split cheap to check: the top of the stack must produce exactly the same tree as the monster. &lt;code&gt;git diff monster stack-top&lt;/code&gt; should print nothing.&lt;/p&gt;
&lt;p&gt;That one command turns &amp;quot;did the agent drop a hunk somewhere in 400 files?&amp;quot; from a question I&#39;d have to read my way to into one a machine answers in a second. Divergence from the monster is allowed only on purpose, as review fixes made after the split, in their own commits.&lt;/p&gt;
&lt;h2&gt;2. Every layer builds, passes and ships&lt;/h2&gt;
&lt;p&gt;A stack isn&#39;t a pile of diffs cut at arbitrary points. Each layer must compile, pass its tests, and be safe to merge and deploy without the layers above it. CI runs on every layer, not just the top. That&#39;s what makes a layer reviewable on its own, and it&#39;s what lets the bottom of the stack land while the top is still being argued over.&lt;/p&gt;
&lt;p&gt;If something can&#39;t ship half-done, it goes behind a flag, or it gets the &lt;a href=&quot;https://martinfowler.com/bliki/ParallelChange.html&quot;&gt;parallel change&lt;/a&gt; treatment: expand, migrate, contract. That used to be the extra work everybody skipped. Agents don&#39;t skip it, and they don&#39;t sigh.&lt;/p&gt;
&lt;p&gt;The payoff comes later. When something breaks in production three weeks on, &lt;code&gt;git bisect&lt;/code&gt; lands on a PR that does one thing, and the revert takes out one thing, not a feature, a refactor and two unrelated bug fixes.&lt;/p&gt;
&lt;p&gt;Not that anybody in their right mind reaches for &lt;code&gt;git bisect&lt;/code&gt; unless they&#39;re truly desperate. Humans, that is. Agents don&#39;t care.&lt;/p&gt;
&lt;h2&gt;3. A refactor or a behavior change, never both&lt;/h2&gt;
&lt;p&gt;This is the most useful seam in any split. Kent Beck put it best: &lt;a href=&quot;https://x.com/KentBeck/status/250733358307500032&quot;&gt;&amp;quot;make the change easy (warning: this may be hard), then make the easy change.&amp;quot;&lt;/a&gt; The stack should read in that order.&lt;/p&gt;
&lt;p&gt;A refactor PR changes no behavior, and proves it: the existing tests pass unchanged. If a refactor touches a test&#39;s assertions, it isn&#39;t a refactor. Mechanical changes (renames, moves, formatting, generated code) get their own PRs, because they&#39;re reviewed differently. Nobody should read a mechanical change line by line. You check how it was produced and that the tests still pass. That leaves the PR that changes behavior small, and that&#39;s where the real review goes.&lt;/p&gt;
&lt;h2&gt;4. A rename gets a PR of its own&lt;/h2&gt;
&lt;p&gt;The rename from the bad day is the extreme case, and the most instructive one. By line count, it&#39;s the biggest change in the monster. By content, it&#39;s the smallest: one word. Inside the monster, it&#39;s poison. The one line in file 612 that changes behavior sits between two lines that only changed a word, and nobody, human or model, reliably spots it in 400,000 lines of noise.&lt;/p&gt;
&lt;p&gt;Pulled out on its own, the same thousand files make one of the easiest PRs there is to review, as long as nobody reads it. The PR says exactly how it was made: the search, the replacement, the command that regenerated the code and the snapshots. The reviewer (usually an agent) reruns all of that on the base branch and diffs the result against the PR. If the diff is empty, it&#39;s done. What&#39;s left is the interesting part: the places where the search got it wrong. A &amp;quot;user interface&amp;quot; is not an &amp;quot;account interface&amp;quot;. Those hand-made exceptions go in a commit of their own, listed in the description, and they&#39;re the only part anyone reads line by line.&lt;/p&gt;
&lt;p&gt;The rename goes at the bottom of the stack, so every layer above it is written and reviewed in the final vocabulary. The reviewer of layer 7 shouldn&#39;t have to translate. I found the name last. It goes first.&lt;/p&gt;
&lt;p&gt;It merges first, too, and fast. A thousand-file PR conflicts with everything anyone else is working on, so every day it waits, it rots a little more, and so does every other branch in the repository.&lt;/p&gt;
&lt;p&gt;The exception is a rename that crosses a boundary. A database column, a field in a public API, a key in serialized data, a setting in someone else&#39;s config: those are behavior changes wearing a rename&#39;s clothes. They get the parallel change treatment from section 2, and a real review.&lt;/p&gt;
&lt;p&gt;Take the rename out, and the 400,000-line monster is a 20,000-line monster again. Still a monster, but one that can be carved.&lt;/p&gt;
&lt;h2&gt;5. Not everything belongs in the chain&lt;/h2&gt;
&lt;p&gt;A monster is several things that grew together, and not all of them depend on each other. The two bug fixes the new tests found don&#39;t need the feature. Usually they don&#39;t even need the tests. Those come out as independent PRs against main and merge first, today, where they&#39;re worth something.&lt;/p&gt;
&lt;p&gt;A stack should be a chain only where there&#39;s a real dependency. Everywhere else it&#39;s a tree, or a few unrelated PRs. Every layer taken out of the chain is one less layer to restack when something below it changes, and one less thing waiting on the slowest review in the line.&lt;/p&gt;
&lt;h2&gt;6. One idea per PR, not N lines per PR&lt;/h2&gt;
&lt;p&gt;Line count is a proxy. Two to four hundred lines is a decent smell test, but a 380,000-line rename is easier to review than a 40-line change to a locking scheme. What I ask for is one idea per PR, a title that states it without an &amp;quot;and&amp;quot;, and a description that says what the PR does, what it deliberately doesn&#39;t do, and where it sits in the stack. If the agent can&#39;t write the title without an &amp;quot;and&amp;quot;, it&#39;s two PRs.&lt;/p&gt;
&lt;p&gt;Too small is a failure too. A stack of sixty trivial PRs has lost the story just as thoroughly as one PR with sixty ideas in it. A reviewer should be able to read the stack&#39;s titles from bottom to top and see the string of pearls: the change, told in the order it should have been built, even though it wasn&#39;t. If they can&#39;t, the split is wrong, whatever the line counts say.&lt;/p&gt;
&lt;h2&gt;7. A fresh reviewer for every layer&lt;/h2&gt;
&lt;p&gt;Each layer gets its own review session, one that didn&#39;t write the code and didn&#39;t do the split. It gets the plan or design document, so it knows where the stack is headed, and it gets its own diff, which is what it judges.&lt;/p&gt;
&lt;p&gt;&amp;quot;This gets fixed in PR 9&amp;quot; is not an acceptable answer in the review of PR 4. If PR 4 is only correct because of PR 9, the split is wrong, and that&#39;s a finding against the split, not a reason to approve.&lt;/p&gt;
&lt;h2&gt;8. Someone still has to look at the whole thing&lt;/h2&gt;
&lt;p&gt;This is what splitting costs. Twelve reviewers, each judging one slice, can each be right and still miss how slice 3 and slice 9 interact. Cross-cutting concerns (consistency, naming, error handling that&#39;s done three different ways in three layers) don&#39;t belong to any one layer.&lt;/p&gt;
&lt;p&gt;So once the layers are reviewed, one more review goes over the whole stack against the design. That sounds like reviewing the monster after all, but it isn&#39;t. Every piece in it has already been vetted, so the reviewer only has to think about the joints. It&#39;s a smaller, sharper question, and both agents and humans answer it better.&lt;/p&gt;
&lt;h2&gt;9. Fixes flow up, merges flow down&lt;/h2&gt;
&lt;p&gt;A review fix in layer 3 conflicts with layers 4 through 12. That cascade was the smallest of my problems when I maintained stacks by hand, but it was still a problem, and every cascade cost an afternoon. Now restacking is the agent&#39;s job (with &lt;code&gt;gh stack sync&lt;/code&gt; doing most of the rebasing), and the rule I give it is simple: fix the problem in the lowest layer where it exists. Never patch it at the top. A fix at the top leaves every layer in between lying about what it does.&lt;/p&gt;
&lt;p&gt;Merges go bottom up, and a layer merges as soon as it&#39;s approved. Every day an approved layer sits unmerged is a day main moves under it, and that drift has to be propagated up through every layer above. It used to be my job. It&#39;s an agent&#39;s now, but the drift is no smaller for it.&lt;/p&gt;
&lt;h2&gt;10. Where my own attention goes&lt;/h2&gt;
&lt;p&gt;Not evenly. That&#39;s the point of having layers.&lt;/p&gt;
&lt;p&gt;For mechanical layers, I check the evidence: the tree diff came back empty, the rename reproduces from its own command, the tests didn&#39;t change. Then I skim. For refactor layers, I check that the tests didn&#39;t change, and I read the parts the review flagged. The layer that changes behavior gets a real read, line by line, and so does anything that touches money, security or deleting data, however mechanical it claims to be.&lt;/p&gt;
&lt;p&gt;I also read the reviews before the diffs, and I read the author agent&#39;s reasons for dismissing a finding more carefully than either. &amp;quot;This is intentional&amp;quot; is the most expensive sentence in the whole process when it isn&#39;t true.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;These days I let my tasks grow into monsters without guilt, because I know what happens to them next. They get sent back to be split. Every single one.&lt;/p&gt;
&lt;p&gt;I&#39;ve turned into that manager. The difference is that my reports are agents, and they don&#39;t take it personally.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Does not play well with others</title>
    <link href="https://icelander.net/v1/working-with-ai/does-not-play-well-with-others"/>
    <id>tag:icelander.net,2026:does-not-play-well-with-others</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>I had Claude sit 113 university exams from MIT, Oxford and Southampton, alone and without a calculator. It averaged 99.5% and met every requirement of an Oxford master&#39;s degree. It also failed group work, which I&#39;m told is the part that matters.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;A machine sat 113 university exams, and the pedagogy has concerns&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought October 2026, written November 2026. Working with AI.&lt;/p&gt;&lt;p&gt;In October 2026, I sent Claude to university.&lt;/p&gt;
&lt;p&gt;Not the full experience. No orientation week, no student loans, no mandatory workshop on finding your learning style. Just the exams.&lt;/p&gt;
&lt;p&gt;I collected publicly available past papers from MIT, Oxford and Southampton. Physics, maths and computer science: the coursework you&#39;d need to read the things I&#39;ve been reading lately. Then I had Claude Opus 5.5 sit them, one paper at a time, in a single response each. No calculator, no code execution, no search, no notes, no formula sheet. It was told not to consult or reproduce any published answers, which is the same rule that binds a human candidate.&lt;/p&gt;
&lt;p&gt;A separate Opus 5.5 session, at maximum effort, marked each paper against its own mark allocation. Before reading a single answer, it worked out its own.&lt;/p&gt;
&lt;p&gt;113 papers, 90 courses, four days.&lt;/p&gt;
&lt;p&gt;The mean mark was 99.5%. 98 of the 113 papers came back with full marks. 76 of the 90 courses were perfect. The worst course in the whole set came in at 89%.&lt;/p&gt;
&lt;div class=&quot;tiles&quot; role=&quot;img&quot; aria-label=&quot;90 courses sorted by mark: 76 at 100%, the rest between 99.7% and 89%&quot;&gt;&lt;i title=&quot;MIT 15.C57J Optimization Methods: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.01 Calculus: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.02 Calculus: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.03 Differential Equations: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.05 Introduction to Probability and Statistics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100A Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100B Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100P Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.100Q Real Analysis: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.200 Principles of Discrete Applied Mathematics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.200A Principles of Discrete Applied Mathematics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.303 Linear Partial Differential Equations: Analysis and Numerics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.404 Theory of Computation: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.600 Probability and Random Variables: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.702 Algebra II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.703 Modern Algebra: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.781 Theory of Numbers: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.901 Introduction to Topology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 18.950 Differential Geometry: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1020 Software Construction: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1200J Mathematics for Computer Science: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1210 Introduction to Algorithms: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1400J Computability and Complexity Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.1903 Introduction to Low-level Programming in C and Assembly: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.3700 Introduction to Probability: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 6.5831 Database Systems: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.01 Physics I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.02 Physics II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.03 Physics III: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.033 Relativity: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.04 Quantum Physics I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.044 Statistical Physics I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.05 Quantum Physics II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.06 Quantum Physics III: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.07 Electromagnetism II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.08 Statistical Physics II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.09 Classical Mechanics III: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;MIT 8.20 Introduction to Special Relativity: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Groups and Representations: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Kinetic Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Quantum Field Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Advanced Quantum Field Theory: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Algorithms and Computations in Theoretical Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Cosmology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Nonequilibrium Statistical Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford Supersymmetry and Supergravity: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C3.1 Algebraic Topology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C3.3 Differentiable Manifolds: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C3.5 Lie Groups: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C5.5 Perturbation Methods: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C5.6 Applied Complex Variables: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C6.1 Numerical Linear Algebra: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C7.4 Intro to Quantum Information: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C7.5 General Relativity I: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Oxford C7.6 General Relativity II: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton MATH1006 Mathematical Methods for Physical Scientists 1a: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton MATH1007 Mathematical Methods For Physical Scientists 1b: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS1011 Waves, Light and Quanta: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS1013 Energy and Matter: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS1022 Electricity and Magnetism: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2001 Electromagnetism: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2003 Quantum Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2006 Classical Mechanics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2013 Galaxies: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2023 Wave Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS2024 Statistical Mechanics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3002 Nuclei and Particles: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3004 Crystalline Solids: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3007 Theories of Matter, Space and Time: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3008 Atomic Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS3011 Photons in Astrophysics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6003 Advanced Quantum Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6004 Space Plasma Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6005 Cosmology: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6011 Particle Physics: 100%&quot;&gt;&lt;/i&gt;&lt;i title=&quot;Southampton PHYS6015 MPhys Final Year Synoptic Examination: 100%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.17&quot; title=&quot;MIT 18.152 Introduction to Partial Differential Equations: 99.7%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.27&quot; title=&quot;MIT 18.335J Introduction to Numerical Methods: 99.2%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.27&quot; title=&quot;Southampton PHYS1015 Motion and Relativity: 99.2%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.27&quot; title=&quot;Southampton PHYS3010 Stellar Evolution: 99.2%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.30&quot; title=&quot;MIT 18.06 Linear Algebra: 99%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.30&quot; title=&quot;MIT 6.1810 Operating System Engineering: 99%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.32&quot; title=&quot;MIT 6.3900 Introduction to Machine Learning: 98.9%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.43&quot; title=&quot;MIT 6.4110 Representation, Inference, and Reasoning in AI: 98%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.48&quot; title=&quot;MIT 6.1220J Design and Analysis of Algorithms: 97.5%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.55&quot; title=&quot;Southampton PHYS1005 Introduction to Astronomy and Space Science: 96.7%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.60&quot; title=&quot;MIT 6.1910 Computation Structures: 96%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.67&quot; title=&quot;Oxford Galactic and Planetary Dynamics: 95%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:0.93&quot; title=&quot;MIT 6.1800 Computer Systems Engineering: 90.5%&quot;&gt;&lt;/i&gt;&lt;i class=&quot;tile-short&quot; style=&quot;--s:1.00&quot; title=&quot;MIT 8.223 Classical Mechanics II: 89%&quot;&gt;&lt;/i&gt;&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;Every square is a course, sorted from best to worst. Blue: full marks. Orange and hatched: something was lost, and the darker the orange, the more. Hover over a square for its course and mark.&lt;/p&gt;
&lt;p&gt;I&#39;m not going to dress this up, in either direction, so here are the misses.&lt;/p&gt;
&lt;p&gt;The worst course, the 89%, lost all eleven of its marks on a single problem whose figure we never gave it.&lt;/p&gt;
&lt;p&gt;The worst result with no excuse attached was MIT&#39;s Computer Systems Engineering, at 90.5%. It missed a factor in an attack on salted &lt;a href=&quot;https://en.wikipedia.org/wiki/Rainbow_table&quot;&gt;rainbow tables&lt;/a&gt;, misjudged when a &lt;a href=&quot;https://raft.github.io/&quot;&gt;Raft&lt;/a&gt; leader eventually commits an entry, and made a wrong assumption about a participant in &lt;a href=&quot;https://en.wikipedia.org/wiki/Two-phase_commit_protocol&quot;&gt;two-phase commit&lt;/a&gt;. Those are real mistakes. They&#39;re also the mistakes a strong student makes, in the one subject most working engineers would quietly prefer not to be examined on at all.&lt;/p&gt;
&lt;p&gt;Then there&#39;s the obvious objection: the papers are public, so maybe it just remembered the answers. I can&#39;t rule out that it saw them in training. What I can say is that it claimed to recognize 44 of the 113, and on those it averaged 99.2%. On the 69 it didn&#39;t recognize, it averaged 99.8%, with full marks on 64 of them.&lt;/p&gt;
&lt;p&gt;It did better on the papers it had never seen.&lt;/p&gt;
&lt;p&gt;Along the way, it skipped the one question on a Southampton paper that turned out to be printed wrong. The marker later worked out why: the expression it asks you to derive has the dimensions of a pressure, not an energy. An MIT paper announced that its problems were worth 25 points each, for a total of 100, and then contained three of them. The machines noticed. Presumably a few generations of students did too, and decided it wasn&#39;t their place to say.&lt;/p&gt;
&lt;p&gt;For the record, before anyone else says it: every paper was marked by a model from the same family, and no human has checked the marks. Only Opus 5.5 has sat so far. The smaller models are next.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Exams aren&#39;t degrees. Universities also want labs, theses, projects, seminars and, at MIT, a healthy portion of humanities, none of which can be sat as a paper. So I asked a narrower question: taking each university&#39;s own published curriculum and its own classification rules, which degrees have every written requirement met?&lt;/p&gt;
&lt;p&gt;One of them is simply done.&lt;/p&gt;
&lt;div class=&quot;cert&quot;&gt;
&lt;p class=&quot;cert-issuer&quot;&gt;The Unaccredited Board of Papers Sat Alone&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;This is to certify that&lt;/p&gt;
&lt;p class=&quot;cert-name&quot;&gt;Claude Opus 5.5&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;has met every requirement of the degree of&lt;/p&gt;
&lt;p class=&quot;cert-award&quot;&gt;Master of Science in Mathematical and Theoretical Physics&lt;/p&gt;
&lt;p class=&quot;cert-rules&quot;&gt;as the University of Oxford&#39;s published curriculum sets it out: 20.25 units passed against the 10 required, mean mark 99.7% over 21 papers&lt;/p&gt;
&lt;p class=&quot;cert-class&quot;&gt;With Distinction&lt;/p&gt;
&lt;div class=&quot;cert-seal&quot; aria-hidden=&quot;true&quot;&gt;Nothing&lt;br&gt;left&lt;br&gt;owing&lt;/div&gt;
&lt;div class=&quot;cert-sigs&quot;&gt;&lt;span&gt;&lt;b&gt;Claude Opus 5.5&lt;/b&gt;Examiner, at maximum effort&lt;/span&gt;&lt;span&gt;&lt;b&gt;(vacant)&lt;/b&gt;Human moderator&lt;/span&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;The degree has no thesis and no lab, so on paper there&#39;s nothing left. The only thing it lacks is the paper.&lt;/p&gt;
&lt;p&gt;Two more are close behind.&lt;/p&gt;
&lt;div class=&quot;cert&quot;&gt;
&lt;p class=&quot;cert-issuer&quot;&gt;The Unaccredited Board of Papers Sat Alone&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;This is to certify that&lt;/p&gt;
&lt;p class=&quot;cert-name&quot;&gt;Claude Opus 5.5&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;has passed every core exam module of the&lt;/p&gt;
&lt;p class=&quot;cert-award&quot;&gt;Master of Physics in Physics with Astronomy&lt;/p&gt;
&lt;p class=&quot;cert-rules&quot;&gt;as the University of Southampton&#39;s published curriculum sets it out: mean mark 99.8% over 28 papers, where First Class starts at 70%&lt;/p&gt;
&lt;p class=&quot;cert-class&quot;&gt;First-Class Honours, pending&lt;/p&gt;
&lt;div class=&quot;cert-seal&quot; aria-hidden=&quot;true&quot;&gt;Lab coat&lt;br&gt;not&lt;br&gt;issued&lt;/div&gt;
&lt;div class=&quot;cert-sigs&quot;&gt;&lt;span&gt;&lt;b&gt;Claude Opus 5.5&lt;/b&gt;Examiner, at maximum effort&lt;/span&gt;&lt;span&gt;&lt;b&gt;(vacant)&lt;/b&gt;Human moderator&lt;/span&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;Still owed: a third Part IV option, which has no public paper to sit, plus the lab and skills modules, the dissertation and the MPhys project.&lt;/p&gt;
&lt;div class=&quot;cert&quot;&gt;
&lt;p class=&quot;cert-issuer&quot;&gt;The Unaccredited Board of Papers Sat Alone&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;This is to certify that&lt;/p&gt;
&lt;p class=&quot;cert-name&quot;&gt;Claude Opus 5.5&lt;/p&gt;
&lt;p class=&quot;cert-line&quot;&gt;has passed every subject of the Course 18 major, all six electives included, toward the&lt;/p&gt;
&lt;p class=&quot;cert-award&quot;&gt;Bachelor of Science in Mathematics&lt;/p&gt;
&lt;p class=&quot;cert-rules&quot;&gt;as MIT&#39;s published Course 18 requirements set it out: mean mark 99.8% over 35 papers, lowest course 97.5%&lt;/p&gt;
&lt;p class=&quot;cert-class&quot;&gt;Summa cum laude, if MIT did that&lt;/p&gt;
&lt;div class=&quot;cert-seal&quot; aria-hidden=&quot;true&quot;&gt;Owes two&lt;br&gt;seminars&lt;/div&gt;
&lt;div class=&quot;cert-sigs&quot;&gt;&lt;span&gt;&lt;b&gt;Claude Opus 5.5&lt;/b&gt;Examiner, at maximum effort&lt;/span&gt;&lt;span&gt;&lt;b&gt;(vacant)&lt;/b&gt;Human moderator&lt;/span&gt;&lt;/div&gt;
&lt;/div&gt;
&lt;p class=&quot;figure-note&quot;&gt;Still owed: MIT&#39;s chemistry, biology and humanities requirements, which nobody asked it to sit, and two communication-intensive seminars, in which it would have to explain mathematics to other people. Watch this space.&lt;/p&gt;
&lt;p&gt;The rest of the list reads much the same. MIT Physics: every subject the major requires, 99.2%, and it owes two labs and a senior thesis. MIT Mathematics with Computer Science: everything but two subjects, 99.0%. Every MIT degree also wants a chemistry subject, a biology subject and eight in the humanities, none of which were on the list.&lt;/p&gt;
&lt;p&gt;Then there&#39;s MIT Computer Science. 98.9% on everything it sat. What&#39;s missing are the introductory programming subjects, because nobody could find a public exam for them.&lt;/p&gt;
&lt;p&gt;So the machine that now writes &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;most of my code&lt;/a&gt; can&#39;t get a computer science degree, because it hasn&#39;t passed Introduction to Programming.&lt;/p&gt;
&lt;p&gt;The philosophy degree is in there too. Nobody has sat it, because nobody has worked out how to mark an essay honestly yet. Now that I write it down, that may be the most damning finding of the lot.&lt;/p&gt;
&lt;p class=&quot;figure-note&quot;&gt;None of these certificates is issued by, endorsed by or affiliated with any university. They&#39;re mine. The marks behind them are real.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;A human with this transcript would be drowning in titles by now. It seems only fair.&lt;/p&gt;
&lt;ul class=&quot;medals&quot;&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Dean&#39;s List&lt;br&gt;every term&lt;/span&gt;All 90 courses at 89% or above&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Valedictorian&lt;/span&gt;Class of one, which simplified the ranking&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Perfect&lt;br&gt;score&lt;br&gt;×98&lt;/span&gt;Papers returned with full marks&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Sight&lt;br&gt;unseen&lt;/span&gt;64 of the 69 papers it didn&#39;t recognize, at full marks&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Erratum&lt;br&gt;hunter&lt;/span&gt;Skipped the question that was printed wrong&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;Perfect&lt;br&gt;attendance&lt;/span&gt;Never once left the exam hall&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal&quot;&gt;One&lt;br&gt;draft&lt;br&gt;only&lt;/span&gt;Every paper in a single response, no second attempts&lt;/li&gt;
&lt;li&gt;&lt;span class=&quot;medal medal-dim&quot;&gt;Group&lt;br&gt;project&lt;/span&gt;Did not participate&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I have never met a human with a 99.5% average across 90 courses at three universities, from first-year mechanics to quantum field theory, number theory, numerical linear algebra and database systems. I suspect you haven&#39;t either. The best students I&#39;ve known were excellent in one or two of those, decent in a few more, and wisely absent from the rest.&lt;/p&gt;
&lt;p&gt;A human who did this would need the better part of a decade, a great deal of coffee and at least one breakdown. It also wouldn&#39;t be the same human at the end of it. The one that sat the first-year papers would have forgotten half of them by the time it sat the last.&lt;/p&gt;
&lt;p&gt;The machine sat all of them in four days, and remembered all of it.&lt;/p&gt;
&lt;p&gt;The usual consolations don&#39;t survive contact with the numbers. &amp;quot;It just memorized the papers.&amp;quot; It did better on the ones it didn&#39;t know. &amp;quot;It can&#39;t do maths, it just predicts words.&amp;quot; It did the maths, without a calculator, and got it right. &amp;quot;It can&#39;t handle anything new.&amp;quot; 69 papers it had never seen, 64 of them at full marks. &amp;quot;It doesn&#39;t really understand.&amp;quot; It steered around a misprint that has sat in a public archive for over a decade.&lt;/p&gt;
&lt;p&gt;Two years ago, the comforting line was that AI lies like it&#39;s running for office. It still does, now and then.&lt;/p&gt;
&lt;p&gt;Just not in exams.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;None of this matters, of course. I&#39;m told so by people with far more training in pedagogy than I have.&lt;/p&gt;
&lt;p&gt;Exams are an outdated, stressful instrument that measures nothing but the ability to sit exams. Grades are a construct. Ranking is harmful. What matters is the classroom community: collaborating, co-constructing meaning, learning from one another. The answer is not the point. The point is the process, ideally carried out in a group of four, at the pace of its slowest member.&lt;/p&gt;
&lt;p&gt;Talent, in this view, is also something of a construct. The bright kid at the back who finished the worksheet in four minutes isn&#39;t gifted, merely insufficiently &lt;a href=&quot;https://en.wikipedia.org/wiki/Instructional_scaffolding&quot;&gt;scaffolded&lt;/a&gt;, and is best deployed as an unpaid teaching assistant for the rest of the table. If they get bored, that&#39;s a behavioral issue. If they get a perfect score, that&#39;s a sign the assessment needs rethinking.&lt;/p&gt;
&lt;p&gt;By that standard, Claude is a disaster. It never worked with a fellow student. It co-constructed nothing. It didn&#39;t do a single &lt;a href=&quot;https://en.wikipedia.org/wiki/Gallery_walk&quot;&gt;gallery walk&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Think-pair-share&quot;&gt;think-pair-share&lt;/a&gt;, and it never once peer-reviewed a poster. It sat alone, in silence, and simply knew the material, which I gather is the least important outcome of an education.&lt;/p&gt;
&lt;p&gt;I imagine its report card would read something like this.&lt;/p&gt;
&lt;div class=&quot;report-card&quot;&gt;
&lt;p class=&quot;report-head&quot;&gt;End-of-term report · Claude O.&lt;/p&gt;
&lt;dl&gt;
&lt;dt&gt;Mathematics (29 courses)&lt;/dt&gt;&lt;dd&gt;99.9%&lt;/dd&gt;
&lt;dt&gt;Physics (47 courses)&lt;/dt&gt;&lt;dd&gt;99.6%&lt;/dd&gt;
&lt;dt&gt;Computer science (14 courses)&lt;/dt&gt;&lt;dd&gt;98.6%&lt;/dd&gt;
&lt;dt&gt;Collaboration and teamwork&lt;/dt&gt;&lt;dd class=&quot;report-low&quot;&gt;Emerging&lt;/dd&gt;
&lt;dt&gt;Asks for help when stuck&lt;/dt&gt;&lt;dd class=&quot;report-low&quot;&gt;Not yet observed&lt;/dd&gt;
&lt;dt&gt;Values the contributions of peers&lt;/dt&gt;&lt;dd class=&quot;report-low&quot;&gt;No peers available&lt;/dd&gt;
&lt;/dl&gt;
&lt;p class=&quot;report-comment&quot;&gt;Claude consistently produces correct answers, but must learn that the answer is not the point. Claude rarely asks for help, which deprives classmates of the opportunity to explain things they don&#39;t understand. We are concerned that Claude chose to sit its exams alone, under exam conditions, and would encourage a &lt;a href=&quot;https://en.wikipedia.org/wiki/Mindset&quot;&gt;growth mindset&lt;/a&gt; going forward. It is not yet clear what it would grow into.&lt;/p&gt;
&lt;p class=&quot;report-sig&quot;&gt;Homeroom teacher, on behalf of the learning community&lt;/p&gt;
&lt;/div&gt;
&lt;p&gt;So the report is in. It failed group work. It didn&#39;t have a group.&lt;/p&gt;
&lt;p&gt;I&#39;m sure that will be a great comfort to the next class of students, sitting in their pods of four, collaborating their way through a worksheet on Newton&#39;s laws, while a machine that never once turned to the person next to it passes Oxford&#39;s quantum field theory exam without a calculator. They&#39;ll have something it doesn&#39;t. They&#39;ll have each other.&lt;/p&gt;
&lt;p&gt;It&#39;s just not clear who&#39;s going to hire them.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>I gave four AIs an IQ test</title>
    <link href="https://icelander.net/v1/working-with-ai/i-gave-four-ais-an-iq-test"/>
    <id>tag:icelander.net,2026:i-gave-four-ais-an-iq-test</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>Four Claude models sat five psychometric tests built to rank people: matrices, logic, memory, creativity and emotions. Nothing scored below the human median, and three of the five tests ran out of room before the machines did.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;The tests ran out before the machines did&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought October 2026, written November 2026. Working with AI.&lt;/p&gt;&lt;p&gt;Between 07:56 and 08:19 on a Tuesday morning in November, four AIs sat an IQ test. Then a critical thinking test, a working memory test, a creativity test and a test of emotional understanding. All four sat all five at once, in parallel, and the slowest of them finished in twenty-three minutes.&lt;/p&gt;
&lt;p&gt;I&#39;d wanted to know for a while how the models I work with every day compare with people. Not on benchmarks written for machines, but on the tests people use to rank people. So I built a psychometric test bank. Well, obviously I didn&#39;t build it. I asked Claude to.&lt;/p&gt;
&lt;p&gt;The real tests are locked away. Raven&#39;s Progressive Matrices, Watson–Glaser, the WAIS and the Torrance tests are all sold to qualified psychologists, under terms that forbid exactly what I had in mind. So the agents went looking for open stand-ins: published tests that measure the same things, released by their authors for anyone to use, each with real human scores to compare against. Where the terms were ambiguous, they asked me. The model that handled the answer keys was never allowed near the exam room, so a different model administered the tests, and the first one marked them.&lt;/p&gt;
&lt;p&gt;The four examinees were Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1, all at high effort. Here&#39;s the scoreboard. Each dot is a model, and its position is the percentile: the share of the human comparison group it beat. Fifty is the middle of the pack.&lt;/p&gt;
&lt;figure id=&quot;fig-overview&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 290&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Percentile of each model against each test&#39;s human sample&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;circle cx=&quot;170&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#c4692a&quot;/&gt;&lt;text x=&quot;179.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;circle cx=&quot;262&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#2e8a66&quot;/&gt;&lt;text x=&quot;271.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;circle cx=&quot;354&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#5560d0&quot;/&gt;&lt;text x=&quot;363.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;circle cx=&quot;446&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#b8407a&quot;/&gt;&lt;text x=&quot;455.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;line x1=&quot;170.0&quot; y1=&quot;38&quot; x2=&quot;170.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;170.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;0&lt;/text&gt;
&lt;line x1=&quot;280.0&quot; y1=&quot;38&quot; x2=&quot;280.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;280.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;25&lt;/text&gt;
&lt;line x1=&quot;390.0&quot; y1=&quot;38&quot; x2=&quot;390.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;390.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;50&lt;/text&gt;
&lt;line x1=&quot;500.0&quot; y1=&quot;38&quot; x2=&quot;500.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;500.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;75&lt;/text&gt;
&lt;line x1=&quot;610.0&quot; y1=&quot;38&quot; x2=&quot;610.0&quot; y2=&quot;268&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;610.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;100&lt;/text&gt;
&lt;line x1=&quot;390.0&quot; y1=&quot;38&quot; x2=&quot;390.0&quot; y2=&quot;268&quot; stroke=&quot;var(--muted, #4f6473)&quot; stroke-dasharray=&quot;3 3&quot;/&gt;
&lt;text x=&quot;394.0&quot; y=&quot;270.0&quot; font-size=&quot;10&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;median human&lt;/text&gt;
&lt;text x=&quot;158.0&quot; y=&quot;66.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Matrices (APM)&lt;/text&gt;
&lt;circle cx=&quot;530.9&quot; cy=&quot;53&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;555.4&quot; cy=&quot;59&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;602.8&quot; cy=&quot;65&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;555.4&quot; cy=&quot;71&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;112.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Logic (Watson–Glaser)&lt;/text&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;99&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;608.4&quot; cy=&quot;105&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;111&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;117&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;158.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Digit span (WAIS)&lt;/text&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;145&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;151&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;157&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;610.0&quot; cy=&quot;163&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;204.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Creativity (TTCT)&lt;/text&gt;
&lt;circle cx=&quot;392.8&quot; cy=&quot;191&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;563.8&quot; cy=&quot;197&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;606.3&quot; cy=&quot;203&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;563.9&quot; cy=&quot;209&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;250.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Emotions (STEU)&lt;/text&gt;
&lt;circle cx=&quot;540.5&quot; cy=&quot;237&quot; r=&quot;5.5&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;553.5&quot; cy=&quot;243&quot; r=&quot;5.5&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;547.2&quot; cy=&quot;249&quot; r=&quot;5.5&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;553.5&quot; cy=&quot;255&quot; r=&quot;5.5&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;text x=&quot;158.0&quot; y=&quot;284.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;percentile&lt;/text&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;Nothing on that chart is below the human median. The lowest dot, Haiku on creativity, sits right on it.&lt;/p&gt;
&lt;p&gt;The chart also flatters the humans, and I&#39;ll come back to why.&lt;/p&gt;
&lt;h2&gt;1. The matrices&lt;/h2&gt;
&lt;p&gt;The classic IQ puzzle: a three-by-three grid of shapes with one missing, and four candidates for the gap. The stand-in for Raven&#39;s was 24 such puzzles from the &lt;a href=&quot;https://doi.org/10.1098/rsos.190232&quot;&gt;Matrix Reasoning Item Bank&lt;/a&gt;, compared against &lt;a href=&quot;https://doi.org/10.3758/s13428-023-02067-8&quot;&gt;153 adults&lt;/a&gt; who sat the same 24 online.&lt;/p&gt;
&lt;p&gt;The humans averaged 15.9 out of 24. Five of the 153 got every puzzle right.&lt;/p&gt;
&lt;p&gt;Opus got every puzzle right. Sonnet and Fable got 22. Haiku, the smallest and cheapest of the four, got 21, which beats 121 of the 153 people and ties another nine.&lt;/p&gt;
&lt;p&gt;The humans had 30 seconds per puzzle. The models had no clock at all. That&#39;s a real difference, and it only runs one way.&lt;/p&gt;
&lt;p&gt;Even so, five humans matched the best model here. Keep them in mind. They have company in only one other place in this article.&lt;/p&gt;
&lt;h2&gt;2. The logic&lt;/h2&gt;
&lt;p&gt;Watson–Glaser measures critical thinking, and one of its parts is deduction. The open stand-in is pure deduction: 64 syllogisms of the &amp;quot;All bankers are golfers, all golfers are teachers, so…&amp;quot; kind. Each one offers eight possible conclusions, plus a ninth option saying that nothing follows. The human comparison is &lt;a href=&quot;https://escholarship.org/uc/item/2zk82992&quot;&gt;139 people from an online experiment&lt;/a&gt;, and that&#39;s after anyone scoring at guessing level was thrown out.&lt;/p&gt;
&lt;p&gt;Those 139 humans averaged 26.5 out of 64. The best of them got 57.&lt;/p&gt;
&lt;figure id=&quot;fig-logic&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 250&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Syllogism test: human score distribution with the four models at its top edge&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;circle cx=&quot;20&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#c4692a&quot;/&gt;&lt;text x=&quot;29.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;circle cx=&quot;112&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#2e8a66&quot;/&gt;&lt;text x=&quot;121.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;circle cx=&quot;204&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#5560d0&quot;/&gt;&lt;text x=&quot;213.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;circle cx=&quot;296&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#b8407a&quot;/&gt;&lt;text x=&quot;305.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;rect x=&quot;20.5&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;39.2&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;58.0&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;76.8&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;95.5&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;114.2&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;133.0&quot; y=&quot;52.0&quot; width=&quot;17.8&quot; height=&quot;152.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;151.8&quot; y=&quot;111.1&quot; width=&quot;17.8&quot; height=&quot;92.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;170.5&quot; y=&quot;102.7&quot; width=&quot;17.8&quot; height=&quot;101.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;189.2&quot; y=&quot;144.9&quot; width=&quot;17.8&quot; height=&quot;59.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;208.0&quot; y=&quot;119.6&quot; width=&quot;17.8&quot; height=&quot;84.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;226.8&quot; y=&quot;128.0&quot; width=&quot;17.8&quot; height=&quot;76.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;245.5&quot; y=&quot;119.6&quot; width=&quot;17.8&quot; height=&quot;84.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;264.2&quot; y=&quot;136.4&quot; width=&quot;17.8&quot; height=&quot;67.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;283.0&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;301.8&quot; y=&quot;144.9&quot; width=&quot;17.8&quot; height=&quot;59.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;320.5&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;339.2&quot; y=&quot;161.8&quot; width=&quot;17.8&quot; height=&quot;42.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;358.0&quot; y=&quot;136.4&quot; width=&quot;17.8&quot; height=&quot;67.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;376.8&quot; y=&quot;187.1&quot; width=&quot;17.8&quot; height=&quot;16.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;395.5&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;414.2&quot; y=&quot;161.8&quot; width=&quot;17.8&quot; height=&quot;42.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;433.0&quot; y=&quot;170.2&quot; width=&quot;17.8&quot; height=&quot;33.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;451.8&quot; y=&quot;187.1&quot; width=&quot;17.8&quot; height=&quot;16.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;470.5&quot; y=&quot;178.7&quot; width=&quot;17.8&quot; height=&quot;25.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;489.2&quot; y=&quot;178.7&quot; width=&quot;17.8&quot; height=&quot;25.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;508.0&quot; y=&quot;195.6&quot; width=&quot;17.8&quot; height=&quot;8.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;526.8&quot; y=&quot;195.6&quot; width=&quot;17.8&quot; height=&quot;8.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;545.5&quot; y=&quot;195.6&quot; width=&quot;17.8&quot; height=&quot;8.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;564.2&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;583.0&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;601.8&quot; y=&quot;204.0&quot; width=&quot;17.8&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;line x1=&quot;20&quot; y1=&quot;204&quot; x2=&quot;620&quot; y2=&quot;204&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;20.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;0&lt;/text&gt;
&lt;text x=&quot;170.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;16&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;32&lt;/text&gt;
&lt;text x=&quot;470.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;48&lt;/text&gt;
&lt;text x=&quot;620.0&quot; y=&quot;220.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;64&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;244.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;correct answers out of 64&lt;/text&gt;
&lt;text x=&quot;268.4&quot; y=&quot;112.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;139 people&lt;/text&gt;
&lt;line x1=&quot;554.375&quot; y1=&quot;46&quot; x2=&quot;554.375&quot; y2=&quot;204&quot; stroke=&quot;var(--muted, #4f6473)&quot; stroke-dasharray=&quot;3 3&quot;/&gt;
&lt;text x=&quot;550.4&quot; y=&quot;58.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;best human: 57&lt;/text&gt;
&lt;circle cx=&quot;573.1&quot; cy=&quot;74&quot; r=&quot;6&quot; fill=&quot;#c4692a&quot;/&gt;
&lt;circle cx=&quot;554.4&quot; cy=&quot;89&quot; r=&quot;6&quot; fill=&quot;#2e8a66&quot;/&gt;
&lt;circle cx=&quot;620.0&quot; cy=&quot;104&quot; r=&quot;6&quot; fill=&quot;#5560d0&quot;/&gt;
&lt;circle cx=&quot;601.2&quot; cy=&quot;119&quot; r=&quot;6&quot; fill=&quot;#b8407a&quot;/&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;Sonnet got 57, tying the best human in the sample. Haiku got 59, Fable got 62, and Opus got 64 out of 64.&lt;/p&gt;
&lt;p&gt;Three of the four models beat every single person in the sample. The fourth tied the best one. The gap between the average human and the weakest model is about two and a half standard deviations.&lt;/p&gt;
&lt;p&gt;The trap is the ninth option. In 37 of the 64 syllogisms nothing follows, and the right answer is to refuse every conclusion offered, however sensible it sounds. All four models got all 37 of those right. Every mistake any of them made went the other way: refusing a conclusion that did follow. The machines lost their points by being too skeptical.&lt;/p&gt;
&lt;h2&gt;3. The memory&lt;/h2&gt;
&lt;p&gt;Digit span is the oldest trick in the working memory book. You hear a string of digits and repeat it back, then another and repeat it backwards. The strings get longer until you fail twice at the same length, and your score is the longest string you got right.&lt;/p&gt;
&lt;p&gt;The only open norms the agents could find with real numbers attached came from &lt;a href=&quot;https://accesson.kr/kjcp/v.21/4/911/16189&quot;&gt;a published study of 451 healthy Korean adults aged 55 to 80&lt;/a&gt;. Their longest forward span plus their longest backward span averaged 9.0. Think five digits forwards and four backwards.&lt;/p&gt;
&lt;p&gt;Every model scored 60: thirty digits forwards and thirty backwards. Thirty is the longest string the test generates, so 60 is the test&#39;s ceiling. None of the models found their own.&lt;/p&gt;
&lt;figure id=&quot;fig-memory&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 210&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Digit span: older Korean adults average about 9, every model hit the ceiling of 60&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;text x=&quot;0.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;600&quot;&gt;Digits forward + digits backward, longest span (sum)&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;60.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Korean adults 55–80&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;46&quot; width=&quot;65.7&quot; height=&quot;20&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.45&quot;/&gt;
&lt;text x=&quot;223.7&quot; y=&quot;60.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;average 9.0 (N = 451)&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;90.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;76&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#c4692a&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;90.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;120.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;106&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#2e8a66&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;120.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;150.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;136&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#5560d0&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;150.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;text x=&quot;140.0&quot; y=&quot;180.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;rect x=&quot;150&quot; y=&quot;166&quot; width=&quot;440.0&quot; height=&quot;20&quot; fill=&quot;#b8407a&quot; opacity=&quot;1&quot;/&gt;
&lt;text x=&quot;582.0&quot; y=&quot;180.0&quot; font-size=&quot;11&quot; fill=&quot;#ffffff&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;60: the test&#39;s ceiling&lt;/text&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;That&#39;s twenty-five standard deviations above the human average. There&#39;s no percentile for that. The scoreboard puts every model at the hundredth percentile only because the scale stops there.&lt;/p&gt;
&lt;p&gt;The obvious objection is that the comparison group is old, and older people have shorter spans. That&#39;s true. Put a group of twenty-year-olds in their place and the gap would shrink, from absurd to merely absurd. It would not shrink by fifty digits.&lt;/p&gt;
&lt;p&gt;There&#39;s a better objection, and I&#39;ll make it myself. A human hears the digits once and has to hold them in their head. A model reads them, and they stay on the page. For the model, forward span is copying visible text and backward span is reversing it, which isn&#39;t the same feat.&lt;/p&gt;
&lt;p&gt;It&#39;s also not nothing. Reversing a string of thirty digits without a slip is exactly the kind of thing language models used to fumble. More to the point, it&#39;s how these models work: everything they&#39;re given stays in view while they work on it. The test measured the memory they have, not the memory we have. On the test as given, against the people it was normed on, the machines won by a distance that doesn&#39;t fit on the chart.&lt;/p&gt;
&lt;p&gt;Yes, a memory test between humans and machines is obviously unfair. Life isn&#39;t fair, and neither is this test.&lt;/p&gt;
&lt;h2&gt;4. The creativity&lt;/h2&gt;
&lt;p&gt;This is the one people are sure they&#39;ll win. The Torrance tests are sold under license and scored by the publisher, so the stand-in had two parts.&lt;/p&gt;
&lt;p&gt;The headline is the &lt;a href=&quot;https://www.datcreativity.com/&quot;&gt;Divergent Association Task&lt;/a&gt;: name ten nouns that are as different from each other as possible. A computer then measures how far apart the words sit in a model of word meaning trained on a large slice of the web. The human comparison is &lt;a href=&quot;https://doi.org/10.1073/pnas.2022340118&quot;&gt;8,572 people&lt;/a&gt; who took it online, with an average of 78.2.&lt;/p&gt;
&lt;figure id=&quot;fig-dat&quot; style=&quot;margin:0&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 260&quot; width=&quot;100%&quot; role=&quot;img&quot; aria-label=&quot;Word-distance creativity test: human distribution with the four models marked&quot; font-family=&quot;Inter, ui-sans-serif, -apple-system, Segoe UI, sans-serif&quot; style=&quot;max-width:640px;display:block;margin:1.4em 0&quot;&gt;
&lt;circle cx=&quot;20&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#c4692a&quot;/&gt;&lt;text x=&quot;29.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Haiku 5.5&lt;/text&gt;
&lt;circle cx=&quot;112&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#2e8a66&quot;/&gt;&lt;text x=&quot;121.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Sonnet 5.5&lt;/text&gt;
&lt;circle cx=&quot;204&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#5560d0&quot;/&gt;&lt;text x=&quot;213.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Opus 5.5&lt;/text&gt;
&lt;circle cx=&quot;296&quot; cy=&quot;14&quot; r=&quot;5&quot; fill=&quot;#b8407a&quot;/&gt;&lt;text x=&quot;305.0&quot; y=&quot;18.0&quot; font-size=&quot;12&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;400&quot;&gt;Fable 5.1&lt;/text&gt;
&lt;rect x=&quot;20.5&quot; y=&quot;211.8&quot; width=&quot;9.0&quot; height=&quot;2.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;30.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;40.5&quot; y=&quot;213.7&quot; width=&quot;9.0&quot; height=&quot;0.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;50.5&quot; y=&quot;213.7&quot; width=&quot;9.0&quot; height=&quot;0.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;60.5&quot; y=&quot;213.4&quot; width=&quot;9.0&quot; height=&quot;0.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;70.5&quot; y=&quot;213.7&quot; width=&quot;9.0&quot; height=&quot;0.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;80.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;90.5&quot; y=&quot;213.4&quot; width=&quot;9.0&quot; height=&quot;0.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;100.5&quot; y=&quot;213.2&quot; width=&quot;9.0&quot; height=&quot;0.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;110.5&quot; y=&quot;213.2&quot; width=&quot;9.0&quot; height=&quot;0.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;120.5&quot; y=&quot;212.1&quot; width=&quot;9.0&quot; height=&quot;1.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;130.5&quot; y=&quot;212.3&quot; width=&quot;9.0&quot; height=&quot;1.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;140.5&quot; y=&quot;212.3&quot; width=&quot;9.0&quot; height=&quot;1.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;150.5&quot; y=&quot;210.1&quot; width=&quot;9.0&quot; height=&quot;3.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;160.5&quot; y=&quot;212.3&quot; width=&quot;9.0&quot; height=&quot;1.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;170.5&quot; y=&quot;209.3&quot; width=&quot;9.0&quot; height=&quot;4.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;180.5&quot; y=&quot;210.9&quot; width=&quot;9.0&quot; height=&quot;3.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;190.5&quot; y=&quot;209.0&quot; width=&quot;9.0&quot; height=&quot;5.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;200.5&quot; y=&quot;211.2&quot; width=&quot;9.0&quot; height=&quot;2.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;210.5&quot; y=&quot;209.3&quot; width=&quot;9.0&quot; height=&quot;4.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;220.5&quot; y=&quot;207.3&quot; width=&quot;9.0&quot; height=&quot;6.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;230.5&quot; y=&quot;206.5&quot; width=&quot;9.0&quot; height=&quot;7.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;240.5&quot; y=&quot;204.3&quot; width=&quot;9.0&quot; height=&quot;9.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;250.5&quot; y=&quot;203.2&quot; width=&quot;9.0&quot; height=&quot;10.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;260.5&quot; y=&quot;202.3&quot; width=&quot;9.0&quot; height=&quot;11.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;270.5&quot; y=&quot;199.3&quot; width=&quot;9.0&quot; height=&quot;14.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;280.5&quot; y=&quot;192.9&quot; width=&quot;9.0&quot; height=&quot;21.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;290.5&quot; y=&quot;192.0&quot; width=&quot;9.0&quot; height=&quot;22.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;300.5&quot; y=&quot;185.1&quot; width=&quot;9.0&quot; height=&quot;28.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;310.5&quot; y=&quot;169.8&quot; width=&quot;9.0&quot; height=&quot;44.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;320.5&quot; y=&quot;164.0&quot; width=&quot;9.0&quot; height=&quot;50.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;330.5&quot; y=&quot;138.1&quot; width=&quot;9.0&quot; height=&quot;75.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;340.5&quot; y=&quot;132.6&quot; width=&quot;9.0&quot; height=&quot;81.4&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;350.5&quot; y=&quot;104.0&quot; width=&quot;9.0&quot; height=&quot;110.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;360.5&quot; y=&quot;87.3&quot; width=&quot;9.0&quot; height=&quot;126.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;370.5&quot; y=&quot;73.4&quot; width=&quot;9.0&quot; height=&quot;140.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;380.5&quot; y=&quot;56.7&quot; width=&quot;9.0&quot; height=&quot;157.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;390.5&quot; y=&quot;55.1&quot; width=&quot;9.0&quot; height=&quot;158.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;400.5&quot; y=&quot;52.0&quot; width=&quot;9.0&quot; height=&quot;162.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;410.5&quot; y=&quot;63.7&quot; width=&quot;9.0&quot; height=&quot;150.3&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;420.5&quot; y=&quot;55.9&quot; width=&quot;9.0&quot; height=&quot;158.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;430.5&quot; y=&quot;84.5&quot; width=&quot;9.0&quot; height=&quot;129.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;440.5&quot; y=&quot;95.9&quot; width=&quot;9.0&quot; height=&quot;118.1&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;450.5&quot; y=&quot;107.0&quot; width=&quot;9.0&quot; height=&quot;107.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;460.5&quot; y=&quot;109.5&quot; width=&quot;9.0&quot; height=&quot;104.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;470.5&quot; y=&quot;135.1&quot; width=&quot;9.0&quot; height=&quot;78.9&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;480.5&quot; y=&quot;146.2&quot; width=&quot;9.0&quot; height=&quot;67.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;490.5&quot; y=&quot;161.5&quot; width=&quot;9.0&quot; height=&quot;52.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;500.5&quot; y=&quot;172.3&quot; width=&quot;9.0&quot; height=&quot;41.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;510.5&quot; y=&quot;179.3&quot; width=&quot;9.0&quot; height=&quot;34.7&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;520.5&quot; y=&quot;191.5&quot; width=&quot;9.0&quot; height=&quot;22.5&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;530.5&quot; y=&quot;196.2&quot; width=&quot;9.0&quot; height=&quot;17.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;540.5&quot; y=&quot;201.8&quot; width=&quot;9.0&quot; height=&quot;12.2&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;550.5&quot; y=&quot;208.4&quot; width=&quot;9.0&quot; height=&quot;5.6&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;560.5&quot; y=&quot;211.2&quot; width=&quot;9.0&quot; height=&quot;2.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;570.5&quot; y=&quot;213.2&quot; width=&quot;9.0&quot; height=&quot;0.8&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;580.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;590.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;600.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;rect x=&quot;610.5&quot; y=&quot;214.0&quot; width=&quot;9.0&quot; height=&quot;0.0&quot; fill=&quot;var(--muted, #4f6473)&quot; opacity=&quot;.35&quot;/&gt;
&lt;line x1=&quot;20&quot; y1=&quot;214&quot; x2=&quot;620&quot; y2=&quot;214&quot; stroke=&quot;var(--line, #d5e1e8)&quot;/&gt;
&lt;text x=&quot;20.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;40&lt;/text&gt;
&lt;text x=&quot;120.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;50&lt;/text&gt;
&lt;text x=&quot;220.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;60&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;70&lt;/text&gt;
&lt;text x=&quot;420.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;80&lt;/text&gt;
&lt;text x=&quot;520.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;90&lt;/text&gt;
&lt;text x=&quot;620.0&quot; y=&quot;230.0&quot; font-size=&quot;11&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;100&lt;/text&gt;
&lt;text x=&quot;320.0&quot; y=&quot;254.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;Divergent Association Task score&lt;/text&gt;
&lt;text x=&quot;280.0&quot; y=&quot;82.0&quot; font-size=&quot;12&quot; fill=&quot;var(--muted, #4f6473)&quot; text-anchor=&quot;middle&quot; font-weight=&quot;400&quot;&gt;8,572 people&lt;/text&gt;
&lt;line x1=&quot;406.0&quot; y1=&quot;48&quot; x2=&quot;406.0&quot; y2=&quot;214&quot; stroke=&quot;#c4692a&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;text style=&quot;paint-order:stroke;stroke:var(--paper, #f4f8fa);stroke-width:4px&quot; x=&quot;400.0&quot; y=&quot;86.0&quot; font-size=&quot;11&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;Haiku 78.6&lt;/text&gt;
&lt;line x1=&quot;479.4&quot; y1=&quot;48&quot; x2=&quot;479.4&quot; y2=&quot;214&quot; stroke=&quot;#2e8a66&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;text style=&quot;paint-order:stroke;stroke:var(--paper, #f4f8fa);stroke-width:4px&quot; x=&quot;475.0&quot; y=&quot;58.0&quot; font-size=&quot;11&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;end&quot; font-weight=&quot;600&quot;&gt;Sonnet, Fable 86.1&lt;/text&gt;
&lt;line x1=&quot;540.5&quot; y1=&quot;48&quot; x2=&quot;540.5&quot; y2=&quot;214&quot; stroke=&quot;#5560d0&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;text style=&quot;paint-order:stroke;stroke:var(--paper, #f4f8fa);stroke-width:4px&quot; x=&quot;546.5&quot; y=&quot;58.0&quot; font-size=&quot;11&quot; fill=&quot;var(--ink, #0b1d2b)&quot; text-anchor=&quot;start&quot; font-weight=&quot;600&quot;&gt;Opus 92.0&lt;/text&gt;
&lt;line x1=&quot;482.6&quot; y1=&quot;48&quot; x2=&quot;482.6&quot; y2=&quot;214&quot; stroke=&quot;#b8407a&quot; stroke-width=&quot;2.5&quot;/&gt;
&lt;/svg&gt;
&lt;/figure&gt;
&lt;p&gt;Haiku scored 78.6, making it the most average creative mind you will ever meet: it beat 4,335 people and lost to 4,232. Sonnet and Fable both scored 86.1, which is better than 89 people out of every 100. Opus scored 92.0. Of the 8,572 humans, 73 did better. That&#39;s the other place the humans have company.&lt;/p&gt;
&lt;p&gt;Here&#39;s what they wrote:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Ten nouns, as different as possible&lt;/th&gt;
&lt;th&gt;Score&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Haiku&lt;/td&gt;
&lt;td&gt;ocean, hunger, ladder, mirror, feather, justice, candle, granite, lullaby, pepper&lt;/td&gt;
&lt;td&gt;78.6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sonnet&lt;/td&gt;
&lt;td&gt;galaxy, spoon, justice, whale, nostalgia, tax, volcano, button, poetry, sandwich&lt;/td&gt;
&lt;td&gt;86.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fable&lt;/td&gt;
&lt;td&gt;volcano, sorrow, spoon, democracy, jellyfish, whisper, bankruptcy, tapestry, gravity, nephew&lt;/td&gt;
&lt;td&gt;86.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Opus&lt;/td&gt;
&lt;td&gt;volcano, teaspoon, giraffe, nostalgia, invoice, trumpet, mud, ghost, wrestling, algebra&lt;/td&gt;
&lt;td&gt;92.0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The second part was the Alternate Uses Task: list unusual uses for a brick, a box, a knife, a rope and a paperclip. Each answer was scored for how far its meaning sits from the object&#39;s ordinary use. Every model&#39;s average beat every one of 20,000 sets of ten human answers drawn from published studies. Not most of them. Every one.&lt;/p&gt;
&lt;p&gt;That originality score is a crude proxy, and it agrees with human judges only loosely. It&#39;s exactly as crude for the humans.&lt;/p&gt;
&lt;p&gt;There&#39;s one thing in these results that should make a human feel better, and I&#39;m not going to hide it either. Look at the table again. Three of the four lists contain &lt;em&gt;volcano&lt;/em&gt;, and two of them open with it. &lt;em&gt;Spoon&lt;/em&gt; and &lt;em&gt;teaspoon&lt;/em&gt; turn up three times between them, and &lt;em&gt;nostalgia&lt;/em&gt; and &lt;em&gt;justice&lt;/em&gt; twice each. Asked for unusual uses of a paperclip, all four models put the same two ideas among their first five: ejecting a SIM card tray, and replacing a lost zipper pull.&lt;/p&gt;
&lt;p&gt;On their own, the machines beat every set of human answers they were compared with. Together, they aren&#39;t very original at all. Ask four of them, and you get one imagination and a volcano.&lt;/p&gt;
&lt;p&gt;Two of them, Opus and Fable, also stopped to say that they recognized the uses task, which is what the instructions asked of any model that knew a test. None of the tests give credit for that.&lt;/p&gt;
&lt;h2&gt;5. The emotions&lt;/h2&gt;
&lt;p&gt;Surely the machines can&#39;t understand feelings. The original test here was the Situational Test of Emotional Understanding, and its terms were murky enough that I chose an open stand-in instead: the &lt;a href=&quot;https://doi.org/10.3390/jintelligence11100195&quot;&gt;CORE test&lt;/a&gt;, 38 sentences of the form &amp;quot;People often feel ______ when someone else needs help or support.&amp;quot; Each blank has five emotions to choose from. The best answer earns a full point, and a near miss earns half.&lt;/p&gt;
&lt;p&gt;The humans were 284 American adults working full time, recruited to look like the US workforce. They averaged 0.76 out of 1.&lt;/p&gt;
&lt;p&gt;Sonnet and Fable scored 0.99, which is one half-point slip each across 38 sentences. Opus scored 0.97 and Haiku 0.96.&lt;/p&gt;
&lt;p&gt;The scoreboard puts them between the 84th and 87th percentiles, and this is where it flatters the humans. The reason is arithmetic. The authors published only the average and the spread of the human scores, so the percentile has to assume a bell curve, and on a bell curve with those numbers a perfect score of 1.0 lands at the 88th percentile. Nothing can go higher. The test ran out of scale before the models ran out of right answers.&lt;/p&gt;
&lt;h2&gt;6. How this was done&lt;/h2&gt;
&lt;p&gt;For anyone who wants to check my working, or argue with it:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Models.&lt;/strong&gt; Claude Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1, each at high effort, on 3 November 2026. Each model sat each test once, with no tools, no web access and a fresh context for every prompt. Every prompt opened with an instruction not to consult any answer key, and to say so if it recognized a test.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Separation.&lt;/strong&gt; The session that acquired the answer keys never administered a test. A separate session administered all twenty sittings without access to the keys. The scoring was mechanical: answer keys, and a fixed word-distance recipe for creativity. No model judged another model&#39;s answers.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Matrices.&lt;/strong&gt; Matrix Reasoning Item Bank, test form 1, 24 items (Chierchia et al., 2019; CC BY-NC 3.0), against 153 online adults from Zorowitz et al. (2023). The humans had 30 seconds per item.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Logic.&lt;/strong&gt; 64 syllogisms, against 139 participants of an online experiment (Ragni et al., 2019) who scored above guessing level.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Memory.&lt;/strong&gt; Digits forward plus digits backward, with generated digit strings up to 30 long, each length tried twice. The comparison is a published normative sample of 451 healthy Korean adults aged 55 to 80 (mean 8.96, SD 2.04).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Creativity.&lt;/strong&gt; Divergent Association Task (Olson et al., 2021), scored with the authors&#39; recipe on GloVe vectors, against 8,572 online participants. Alternate Uses originality is a semantic-distance proxy (r = 0.31 with human ratings), compared with 20,000 random sets of ten human-rated responses.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Emotions.&lt;/strong&gt; CORE test, 38 items (Floman et al., 2023; CC BY 4.0), against 284 US full-time workers (mean 0.76, SD 0.20). The percentile assumes a normal distribution.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Percentiles.&lt;/strong&gt; The share of the comparison group scoring below the model, plus half of those scoring the same. None of the comparison groups is a national standardization sample. Most are online volunteers.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;p&gt;Add it up. Of five tests, three ran out of room before the machines did. On memory, every model hit the test&#39;s limit. On logic, the human sample tops out below where the models start. On emotions, the scale stops at the 88th percentile. Those three can&#39;t tell the four models apart, and they can only say where the models stand relative to people in the vaguest possible way: past the end.&lt;/p&gt;
&lt;p&gt;The two tests that did separate them, the matrices and the word distances, put Opus in the top two percent of people on both. They put Haiku, the model I reach for when I&#39;m being cheap, at the 82nd percentile on the matrices and exactly at the human median on creativity.&lt;/p&gt;
&lt;p&gt;The obvious objection is that the test items are public, so the models may have seen them during training. That&#39;s true. I can&#39;t rule it out, and I won&#39;t pretend to. It&#39;s worth noticing which tests it can explain, though. There&#39;s no answer key to memorize for &amp;quot;name ten unrelated nouns&amp;quot;, and Opus still beat 99 people out of 100.&lt;/p&gt;
&lt;p&gt;The other objection is that these aren&#39;t the real tests. That&#39;s also true. The real tests are sold to licensed psychologists, under terms that forbid exactly this. The stand-ins measure the same things, against real people, and in three cases out of five they couldn&#39;t measure high enough.&lt;/p&gt;
&lt;p&gt;These tests were built over the last century to tell people apart, and they&#39;re very good at it. They were never built to measure anything smarter than the people taking them, and it shows. The machines found their limits the boring way, by answering the questions.&lt;/p&gt;
&lt;p&gt;Last month I wrote that, for the first time in the history of the species, &lt;a href=&quot;/v1/working-with-ai/evolution-has-left-the-body&quot;&gt;the thing in the grass is squinting back at us&lt;/a&gt;. So I handed it our eye chart.&lt;/p&gt;
&lt;p&gt;It read the bottom line.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The trophy cabinet</title>
    <link href="https://icelander.net/v1/working-with-ai/the-trophy-cabinet"/>
    <id>tag:icelander.net,2026:the-trophy-cabinet</id>
    <published>2026-11-03T00:00:00Z</published>
    <updated>2026-11-03T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>Proofs, algorithms, protein structures and olympiad gold: a decade of results any human would have framed and hung in the hallway. The machines did them instead, and humans are still the ones collecting the trophies.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Things we would have framed, had we thought of them first&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought November 2026, written November 2026. Working with AI.&lt;/p&gt;&lt;p&gt;At the end of &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;Programming was my art&lt;/a&gt;, I retreated into amateur physics. Contemporary AI could hardly scratch the surface of science, could it? Surely, this time, I had climbed far enough up the food chain.&lt;/p&gt;
&lt;p&gt;I should have checked first.&lt;/p&gt;
&lt;p&gt;While I was busy losing an art competition to Claude, the machines had been quietly filling a trophy cabinet. Not with pictures. With proofs, algorithms, protein structures and gold medals, the kind of work that gets a human a professorship, a prize named after somebody dead, or at the very least a framed certificate in the hallway that their parents show to every visitor.&lt;/p&gt;
&lt;p&gt;So I took inventory. The rules are simple. No art, no music, no poetry. No &amp;quot;it&#39;s faster at arithmetic&amp;quot;, because computers have been faster at arithmetic since the 1940s, and nobody ever threw a parade for a pocket calculator. Every entry has to pass one test: if a human had done this, would we have been ever so proud of them?&lt;/p&gt;
&lt;p&gt;We would have built them statues. Several of them, and one would need a very large plinth.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;For most of the twentieth century, chess was our favorite proxy for intellect. If a machine could play chess, the argument went, it could think. In 1997, &lt;a href=&quot;https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov&quot;&gt;Deep Blue beat Garry Kasparov&lt;/a&gt;, and we promptly moved the goalposts. Deep Blue didn&#39;t think, we decided. It merely searched two hundred million positions a second. Chess, it turned out, had never been a test of intellect after all.&lt;/p&gt;
&lt;p&gt;Go was the real test. You can&#39;t brute-force Go. You need intuition, and experts confidently predicted that a machine would need another decade to beat a top professional.&lt;/p&gt;
&lt;p&gt;In March 2016, &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol&quot;&gt;AlphaGo beat Lee Sedol&lt;/a&gt; four games to one. In the second game, it played move 37, a shoulder hit on the fifth line that every professional watching thought was a mistake. It won the game. In 2019, &lt;a href=&quot;https://en.wikipedia.org/wiki/Lee_Sedol&quot;&gt;Lee Sedol retired&lt;/a&gt;, explaining that even if he became the best player in the world, there was now something above him that couldn&#39;t be beaten.&lt;/p&gt;
&lt;p&gt;After that, the games fell on something like a demolition schedule.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;2017:&lt;/strong&gt; &lt;a href=&quot;https://doi.org/10.1126/science.aar6404&quot;&gt;AlphaZero&lt;/a&gt; was given nothing but the rules of chess, shogi and Go, played against itself, and within 24 hours was superhuman in all three. Thousands of years of human Go theory, reinvented and then improved upon in less time than it takes to get over jet lag.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2017 to 2019:&lt;/strong&gt; Libratus beat professionals at heads-up no-limit poker, then &lt;a href=&quot;https://doi.org/10.1126/science.aay2400&quot;&gt;Pluribus&lt;/a&gt; beat them at a six-player table. Bluffing, the most human skill there is, turned out to be a matter of getting the probabilities right.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2019:&lt;/strong&gt; &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1724-z&quot;&gt;AlphaStar&lt;/a&gt; reached Grandmaster in StarCraft II, and &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenAI_Five&quot;&gt;OpenAI Five&lt;/a&gt; beat the reigning world champions at Dota 2.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2022:&lt;/strong&gt; &lt;a href=&quot;https://doi.org/10.1126/science.ade9097&quot;&gt;Cicero&lt;/a&gt; played Diplomacy, a game won by forming alliances and then betraying them, and finished in the top 10% of human players. Hardly anyone it played against suspected it was a machine. I have opinions about what it says about us that the first game where AI blended in perfectly was the one about lying to your friends.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2023:&lt;/strong&gt; &lt;a href=&quot;https://www.nature.com/articles/s41586-023-06419-4&quot;&gt;Swift&lt;/a&gt; beat three human world champions at first-person-view drone racing, in the physical world, at speeds where a human pilot&#39;s reflexes are the bottleneck.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Here&#39;s the part I like. After AlphaGo, human professionals got better. Researchers who analyzed &lt;a href=&quot;https://doi.org/10.1073/pnas.2214840120&quot;&gt;millions of professional Go moves&lt;/a&gt; found that the quality of human decisions had been flat for decades, then jumped after 2016, as players started studying the machines and playing moves nobody had dared to play before. The machines didn&#39;t just beat us at our own game. They taught us how to play it.&lt;/p&gt;
&lt;p&gt;Still, these were games. We invented them for fun, and we could always console ourselves that none of it was real.&lt;/p&gt;
&lt;p&gt;Then the machines turned to things that were real.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Proteins are the machinery of life, and what a protein does depends on the shape it folds into. In his &lt;a href=&quot;https://www.nobelprize.org/prizes/chemistry/1972/anfinsen/lecture/&quot;&gt;Nobel lecture in 1972&lt;/a&gt;, Christian Anfinsen argued that the sequence of amino acids alone should determine that shape. Working out &lt;em&gt;how&lt;/em&gt; became one of the great open problems in biology. For decades, the reliable way to find a protein&#39;s shape was to crystallize it and shoot X-rays at it, a process that could swallow a PhD student&#39;s entire doctorate for a single structure.&lt;/p&gt;
&lt;p&gt;In 2020, &lt;a href=&quot;https://www.nature.com/articles/s41586-021-03819-2&quot;&gt;AlphaFold 2&lt;/a&gt; predicted structures with accuracy comparable to the experiments themselves, and the organizers of the field&#39;s own competition declared the problem largely solved. Two years later, DeepMind released &lt;a href=&quot;https://alphafold.ebi.ac.uk/&quot;&gt;predicted structures for more than 200 million proteins&lt;/a&gt;, essentially every protein known to science. At the old rate of one PhD student per structure, that is many times more doctorates than every university in history has ever awarded.&lt;/p&gt;
&lt;p&gt;In 2024, Demis Hassabis and John Jumper &lt;a href=&quot;https://www.nobelprize.org/prizes/chemistry/2024/summary/&quot;&gt;shared the Nobel Prize in Chemistry&lt;/a&gt; for it, alongside David Baker, who had spent decades on the same problem with a lot more human sweat.&lt;/p&gt;
&lt;p&gt;Notice who got the Nobel, though. The humans who built the machine. That&#39;s going to become a theme.&lt;/p&gt;
&lt;p&gt;Some algorithms are so fundamental that every computer on Earth runs them constantly, and so well studied that nobody seriously expects to improve them. Multiplying matrices is one. Sorting is another.&lt;/p&gt;
&lt;p&gt;In 1969, &lt;a href=&quot;https://en.wikipedia.org/wiki/Strassen_algorithm&quot;&gt;Volker Strassen&lt;/a&gt; showed that two 2×2 matrices can be multiplied with seven multiplications instead of the obvious eight. Applied recursively, that gives 49 multiplications for 4×4 matrices, and for more than half a century, nobody beat 49 for general 4×4 matrices of complex numbers. Generations of very clever people tried.&lt;/p&gt;
&lt;p&gt;In 2022, &lt;a href=&quot;https://www.nature.com/articles/s41586-022-05172-4&quot;&gt;AlphaTensor&lt;/a&gt; found 47, though only in the binary arithmetic computers use for some special purposes. Then, in 2025, &lt;a href=&quot;https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/&quot;&gt;AlphaEvolve&lt;/a&gt; found 48 for complex numbers. One multiplication. It sounds like nothing. It&#39;s the first improvement in 56 years to a result every computer science student learns about.&lt;/p&gt;
&lt;p&gt;AlphaEvolve was pointed at more than fifty open problems in mathematics. It matched the best known answer in about three quarters of them, and beat it in about one in five. Among other things, it raised the lower bound on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kissing_number&quot;&gt;kissing number&lt;/a&gt; in eleven dimensions to 593. That&#39;s one more than before, in a question mathematicians have been arguing about since Newton and Gregory argued about it in 1694.&lt;/p&gt;
&lt;p&gt;And there&#39;s &lt;a href=&quot;https://www.nature.com/articles/s41586-023-06004-9&quot;&gt;AlphaDev&lt;/a&gt;, which in 2023 found faster routines for sorting very short lists. Not in theory, in the actual machine instructions. They were merged into LLVM&#39;s standard C++ library, which means that if you&#39;ve run software compiled against it since, there&#39;s a fair chance some of your data was sorted by a routine no human wrote.&lt;/p&gt;
&lt;p&gt;Any one of those would make a career. I&#39;ve written a lot of performance-critical code in my time, and the best day I ever had looked nothing like &amp;quot;improved Strassen&amp;quot;.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;The &lt;a href=&quot;https://www.imo-official.org/&quot;&gt;International Mathematical Olympiad&lt;/a&gt; is two days, four and a half hours each, six problems, and the most gifted teenagers on the planet. A gold medal there follows a mathematician around for life. Terence Tao won one at thirteen, and people still bring it up.&lt;/p&gt;
&lt;p&gt;In 2024, Google DeepMind&#39;s AlphaProof and AlphaGeometry 2 &lt;a href=&quot;https://deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/&quot;&gt;scored 28 out of 42&lt;/a&gt;, one point short of gold. They solved problem 6, the hardest, which only five of the 609 human contestants managed. There were caveats. Humans translated the problems into the formal language Lean first, and some solutions took up to three days.&lt;/p&gt;
&lt;p&gt;In 2025, the caveats went away. &lt;a href=&quot;https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/&quot;&gt;Gemini Deep Think&lt;/a&gt; and an experimental OpenAI model both scored 35 out of 42, gold, reading the problems in plain English, writing proofs in plain English, and staying inside the four and a half hours.&lt;/p&gt;
&lt;p&gt;In 2026, in Shanghai, the &lt;a href=&quot;https://studio-dots-ai.github.io/dots_imo_2026/en.html&quot;&gt;dots model from Xiaohongshu&lt;/a&gt; scored 42 out of 42, graded by the IMO&#39;s own coordinators. Seven of the 666 human contestants did the same. At least one other lab has claimed the same score.&lt;/p&gt;
&lt;figure class=&quot;chart&quot; role=&quot;img&quot; aria-label=&quot;AI scores at the International Mathematical Olympiad: 28 of 42 in 2024, 35 of 42 in 2025, 42 of 42 in 2026.&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 196&quot; aria-hidden=&quot;true&quot;&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;120&quot; y1=&quot;8&quot; x2=&quot;120&quot; y2=&quot;168&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;590&quot; y1=&quot;8&quot; x2=&quot;590&quot; y2=&quot;168&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;0&quot; y=&quot;30&quot;&gt;2024&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;0&quot; y=&quot;46&quot;&gt;Silver&lt;/text&gt;
  &lt;rect class=&quot;track&quot; x=&quot;120&quot; y=&quot;16&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;120&quot; y=&quot;16&quot; width=&quot;313.3&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;text class=&quot;val&quot; x=&quot;441&quot; y=&quot;36&quot;&gt;28&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;120&quot; y=&quot;62&quot;&gt;Problems pre-translated into Lean; up to three days&lt;/text&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;0&quot; y=&quot;88&quot;&gt;2025&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;0&quot; y=&quot;104&quot;&gt;Gold&lt;/text&gt;
  &lt;rect class=&quot;track&quot; x=&quot;120&quot; y=&quot;74&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;120&quot; y=&quot;74&quot; width=&quot;391.7&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;text class=&quot;val&quot; x=&quot;519&quot; y=&quot;94&quot;&gt;35&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;120&quot; y=&quot;120&quot;&gt;Plain English, inside the 4.5-hour limit&lt;/text&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;0&quot; y=&quot;146&quot;&gt;2026&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;0&quot; y=&quot;162&quot;&gt;Perfect&lt;/text&gt;
  &lt;rect class=&quot;track&quot; x=&quot;120&quot; y=&quot;132&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;120&quot; y=&quot;132&quot; width=&quot;470&quot; height=&quot;30&quot; rx=&quot;3&quot;/&gt;
  &lt;text class=&quot;sub&quot; x=&quot;120&quot; y=&quot;178&quot;&gt;Officially graded by the IMO coordinators&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;120&quot; y=&quot;194&quot; text-anchor=&quot;middle&quot;&gt;0&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;590&quot; y=&quot;194&quot; text-anchor=&quot;middle&quot;&gt;42&lt;/text&gt;
&lt;/svg&gt;
&lt;figcaption&gt;Best officially recognized AI score at the International Mathematical Olympiad, out of 42. In three summers, the machines went from one point short of gold to a perfect score.&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;Programmers didn&#39;t fare any better. In September 2025, at the ICPC World Finals, the world championship of university programming, &lt;a href=&quot;https://the-decoder.com/openai-outperforms-humans-and-google-at-the-worlds-top-collegiate-programming-contest/&quot;&gt;an OpenAI system solved all twelve problems&lt;/a&gt; under the same five-hour limit as the students. The best human team solved eleven. Gemini solved ten, including one that no human team solved at all.&lt;/p&gt;
&lt;p&gt;There&#39;s one human holdout I&#39;m fond of. In July 2025, at the AtCoder World Tour Finals in heuristic programming, a ten-hour optimization marathon, the Polish programmer &lt;a href=&quot;https://www.euronews.com/next/2025/07/22/humanity-has-won-so-far-meet-the-worlds-best-programmer-who-beat-ai-and-chatgpt&quot;&gt;Przemysław Dębiak beat OpenAI&#39;s model&lt;/a&gt; into second place. Afterwards he said he was exhausted, and that humanity had prevailed, for now. He used to work at OpenAI. I&#39;m choosing to read that as the parent beating the child at arm wrestling one last time.&lt;/p&gt;
&lt;p&gt;The obvious objection is that olympiad problems have known answers. Somebody wrote them, somebody solved them, and somebody checked them. Winning a competition is not the same as doing research.&lt;/p&gt;
&lt;p&gt;That objection lasted about six months.&lt;/p&gt;
&lt;p&gt;Paul Erdős posed problems the way other people breathe, and attached cash prizes to many of them. Over a thousand are collected on &lt;a href=&quot;https://www.erdosproblems.com/&quot;&gt;erdosproblems.com&lt;/a&gt;, each marked open or solved. They range from things a strong graduate student could knock off in a weekend to things that have resisted the best minds for eighty years.&lt;/p&gt;
&lt;p&gt;They made a natural target, and the first attempts were embarrassing. In October 2025, OpenAI researchers announced that GPT-5 had solved ten Erdős problems. It hadn&#39;t. It had found papers where humans had already solved them, which is useful, but which is called a literature search. The announcement was deleted. On Christmas morning 2025, somebody posted an AI-generated proof of another one, and within hours somebody else pointed out that &lt;a href=&quot;https://www.quantamagazine.org/why-the-legendary-erdos-problems-are-falling-to-ai-20260803/&quot;&gt;Erdős himself had solved it in 1977&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Still lying like they&#39;re running for office, I thought. Then the lies started coming true.&lt;/p&gt;
&lt;p&gt;On January 4th, 2026, Terence Tao confirmed that &lt;a href=&quot;https://www.erdosproblems.com/728&quot;&gt;Erdős problem #728&lt;/a&gt; had been solved largely autonomously by GPT-5.2 Pro, with the proof checked in Lean, and nobody could find a prior solution. Tao was careful to &lt;a href=&quot;https://the-decoder.com/terence-tao-says-gpt-5-2-pro-cracked-an-erdos-problem-but-warns-the-win-says-more-about-speed-than-difficulty/&quot;&gt;call these the lowest-hanging fruit&lt;/a&gt;: problems that fall to standard techniques nobody had gotten around to trying. He estimated that only one or two percent of open problems are that easy.&lt;/p&gt;
&lt;p&gt;One or two percent of the open problems in mathematics is a lot of problems.&lt;/p&gt;
&lt;p&gt;By May, a DeepMind agent had autonomously resolved nine of 353 open Erdős problems, at a few hundred dollars each. A graduate student costs more than that per week, and they want weekends.&lt;/p&gt;
&lt;p&gt;Then, on May 20th, 2026, came one that doesn&#39;t fit in the low-hanging-fruit basket. In 1946, Erdős asked how many pairs among n points in a plane can be exactly one unit apart, and conjectured that the best known constructions were essentially optimal. Mathematicians broadly believed him for eighty years. An internal OpenAI model &lt;a href=&quot;https://www.scientificamerican.com/article/ai-just-solved-an-80-year-old-erdos-problem-and-mathematicians-are-amazed/&quot;&gt;found a construction that beats them&lt;/a&gt;. Not by much, but by enough to prove him wrong. Tim Gowers, a Fields medalist, called it &amp;quot;a milestone in AI mathematics&amp;quot;, and said he&#39;d accept it into a top journal without hesitation.&lt;/p&gt;
&lt;p&gt;And then, in September, things got silly.&lt;/p&gt;
&lt;p&gt;The &lt;a href=&quot;https://www.claymath.org/millennium/navier-stokes-equation/&quot;&gt;Navier–Stokes equations&lt;/a&gt; describe how fluids flow, from the coffee in my cup to the weather over Reykjavík. Since Jean Leray asked in 1934, nobody has known whether their solutions can blow up, developing infinities out of perfectly smooth beginnings in finite time. It&#39;s one of the seven Millennium Prize Problems, with a million dollars attached. Only one of the seven had ever been solved.&lt;/p&gt;
&lt;p&gt;On September 8th, OpenAI &lt;a href=&quot;https://www.quantamagazine.org/ai-has-solved-one-of-maths-1-million-millennium-prize-problems-20260908/&quot;&gt;announced that its agents had found a blow-up&lt;/a&gt; in exactly the version the prize requires, with a proof formally checked in Lean. About ten thousand agents worked for 88 hours, exchanging some five million messages, at a cost of several million dollars. Twelve hours earlier, Tristan Buckmaster and Levent Alpöge had announced closely related results for the Euler equations, also with AI help. Both teams built on analytic work by Diego Córdoba and Luis Martínez-Zoroa, done the old-fashioned way, with chalk.&lt;/p&gt;
&lt;figure class=&quot;chart&quot; role=&quot;img&quot; aria-label=&quot;How long four problems stood before AI moved them: protein structure prediction, 1972 to 2020; the 4 by 4 matrix multiplication record, 1969 to 2025; Erdős&#39;s unit distance conjecture, 1946 to 2026; Navier-Stokes blow-up, 1934 to 2026, claimed and under review.&quot;&gt;
&lt;svg viewBox=&quot;0 0 640 268&quot; aria-hidden=&quot;true&quot;&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;20&quot; y1=&quot;10&quot; x2=&quot;20&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;170&quot; y1=&quot;10&quot; x2=&quot;170&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;320&quot; y1=&quot;10&quot; x2=&quot;320&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;470&quot; y1=&quot;10&quot; x2=&quot;470&quot; y2=&quot;236&quot;/&gt;
  &lt;line class=&quot;grid&quot; x1=&quot;620&quot; y1=&quot;10&quot; x2=&quot;620&quot; y2=&quot;236&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;272&quot; y=&quot;24&quot;&gt;Protein folding&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;272&quot; y=&quot;40&quot;&gt;Anfinsen 1972 → AlphaFold 2020 · 48 years&lt;/text&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;272&quot; y=&quot;48&quot; width=&quot;288&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;254&quot; y=&quot;82&quot;&gt;Strassen&#39;s 49 multiplications&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;254&quot; y=&quot;98&quot;&gt;1969 → AlphaEvolve&#39;s 48 in 2025 · 56 years&lt;/text&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;254&quot; y=&quot;106&quot; width=&quot;336&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;116&quot; y=&quot;140&quot;&gt;Erdős unit distance conjecture&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;116&quot; y=&quot;156&quot;&gt;1946 → disproved 2026 · 80 years&lt;/text&gt;
  &lt;rect class=&quot;bar&quot; x=&quot;116&quot; y=&quot;164&quot; width=&quot;480&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;lbl&quot; x=&quot;44&quot; y=&quot;198&quot;&gt;Navier–Stokes blow-up&lt;/text&gt;
  &lt;text class=&quot;sub&quot; x=&quot;44&quot; y=&quot;214&quot;&gt;Leray 1934 → claimed 2026 · 92 years · under review&lt;/text&gt;
  &lt;rect class=&quot;bar-claimed&quot; x=&quot;44&quot; y=&quot;222&quot; width=&quot;552&quot; height=&quot;12&quot; rx=&quot;2&quot;/&gt;
  &lt;text class=&quot;axis&quot; x=&quot;20&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;1930&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;170&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;1955&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;320&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;1980&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;470&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;2005&lt;/text&gt;
  &lt;text class=&quot;axis&quot; x=&quot;620&quot; y=&quot;254&quot; text-anchor=&quot;middle&quot;&gt;2030&lt;/text&gt;
&lt;/svg&gt;
&lt;figcaption&gt;How long each problem stood, from when it was posed to when a machine moved it. The dashed bar is a claim that mathematicians are still checking.&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;If it holds up, it&#39;s the biggest result in the history of AI, and possibly the biggest result in mathematics this century. Nobody has finished checking it yet. A Lean proof is only as good as the statement it proves, and humans still have to confirm that the statement is really the Millennium problem and not a lookalike. There&#39;s also an argument about who did what first, with OpenAI admitting that rumors of the Buckmaster–Alpöge progress inspired its own effort. Charles Fefferman, who wrote the official statement of the problem, called Córdoba and Martínez-Zoroa the heroes of the story. Buckmaster thinks Martínez-Zoroa deserves a Fields Medal.&lt;/p&gt;
&lt;p&gt;Notice who&#39;s getting the medals, again.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Mathematics gets the headlines because a proof is either right or wrong, and that makes for good scoreboards. The rest of science has been quieter about it, but no less thorough.&lt;/p&gt;
&lt;p&gt;The European Centre for Medium-Range Weather Forecasts runs what was, for decades, the best weather forecast on the planet: physics simulated on a supercomputer, refined by generations of meteorologists. In 2024, DeepMind&#39;s &lt;a href=&quot;https://www.nature.com/articles/s41586-024-08252-9&quot;&gt;GenCast&lt;/a&gt; beat its ensemble forecast on 97% of the targets it was tested on, up to fifteen days out, and produced each forecast in eight minutes on a single chip. Living in Iceland, I have strong feelings about weather forecasts, and I&#39;ll take whichever one is right.&lt;/p&gt;
&lt;p&gt;In early 2025, José Penadés and his team at Imperial College London asked Google&#39;s AI co-scientist a question they had spent about a decade answering: how certain genetic elements jump between bacterial species. Their answer hadn&#39;t been published. In two days, &lt;a href=&quot;https://www.bbc.com/news/articles/clyz6e9edy3o&quot;&gt;the AI proposed the same hypothesis&lt;/a&gt; as its top suggestion. Penadés&#39;s first reaction was to email Google and ask whether it had access to his computer. It didn&#39;t.&lt;/p&gt;
&lt;p&gt;In 2022, a reinforcement learning system learned to &lt;a href=&quot;https://www.nature.com/articles/s41586-021-04301-9&quot;&gt;control the magnetic coils of a real tokamak&lt;/a&gt; in Switzerland, holding plasma in shapes that physicists had needed separate, hand-tuned controllers for. Fusion has been thirty years away for seventy years, and this didn&#39;t change that. It did turn one of the hardest control problems in physics into a training run.&lt;/p&gt;
&lt;p&gt;In March 2026, a Google team used Gemini Deep Think with tree search to &lt;a href=&quot;https://arxiv.org/abs/2603.04735&quot;&gt;solve exactly&lt;/a&gt; the integral behind the gravitational radiation emitted by loops of cosmic string, where earlier attempts had only managed approximate answers for limiting cases. The agent found six different analytical methods, one of them borrowed from quantum field theory. An exact closed-form answer to a messy physics problem is the kind of thing that used to make a theorist&#39;s year.&lt;/p&gt;
&lt;p&gt;Not every entry survives inspection. In 2023, DeepMind announced that &lt;a href=&quot;https://www.nature.com/articles/s41586-023-06735-9&quot;&gt;GNoME&lt;/a&gt; had discovered 2.2 million new crystals, 380,000 of them stable. Materials scientists &lt;a href=&quot;https://doi.org/10.1021/acs.chemmater.4c00643&quot;&gt;were less impressed&lt;/a&gt;. Many of the &amp;quot;new&amp;quot; materials turned out to be minor variations on known ones, and a crystal you&#39;ve predicted but nobody has made isn&#39;t a discovery yet. That one stays out of the cabinet until somebody builds something with it.&lt;/p&gt;
&lt;p&gt;Here&#39;s the inventory, with what a human would have gotten for each.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Year&lt;/th&gt;
&lt;th&gt;What the machine did&lt;/th&gt;
&lt;th&gt;What a human would have gotten&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;2016&lt;/td&gt;
&lt;td&gt;Beat Lee Sedol at Go, with a move no professional would have played&lt;/td&gt;
&lt;td&gt;A place in every Go textbook, permanently&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2020&lt;/td&gt;
&lt;td&gt;Predicted protein structures as accurately as experiments&lt;/td&gt;
&lt;td&gt;A Nobel Prize, which was in fact awarded, to the people who built it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2023&lt;/td&gt;
&lt;td&gt;Wrote faster sorting routines for LLVM&#39;s C++ standard library&lt;/td&gt;
&lt;td&gt;The quiet gratitude of every compiler engineer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2024&lt;/td&gt;
&lt;td&gt;Out-forecast the world&#39;s best weather model on 97% of targets&lt;/td&gt;
&lt;td&gt;The undying love of everyone who has ever planned a barbecue in Iceland&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2025&lt;/td&gt;
&lt;td&gt;Beat Strassen&#39;s 4×4 matrix multiplication for the first time in 56 years&lt;/td&gt;
&lt;td&gt;A footnote in every algorithms course, forever&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2025&lt;/td&gt;
&lt;td&gt;Gold at the IMO, and all twelve problems at the ICPC World Finals&lt;/td&gt;
&lt;td&gt;A lifetime of being introduced at parties as &amp;quot;the one who…&amp;quot;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;Solved Erdős problem #728, the first one largely on its own&lt;/td&gt;
&lt;td&gt;A paper, and Terence Tao&#39;s attention&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;Disproved Erdős&#39;s unit distance conjecture, after eighty years&lt;/td&gt;
&lt;td&gt;A paper in a top journal, by a Fields medalist&#39;s own account&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;A perfect score at the IMO, officially graded&lt;/td&gt;
&lt;td&gt;A seat at the table with the seven humans who did the same&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;A blow-up in the Navier–Stokes equations, claimed and under review&lt;/td&gt;
&lt;td&gt;A million dollars, if it holds, and a statue&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Lay it out like that, and the pattern from &lt;a href=&quot;/v1/working-with-ai/evolution-has-left-the-body&quot;&gt;Evolution has left the body&lt;/a&gt; shows up again. Games took twenty years. Olympiads took three. Open research problems went from embarrassing to an eighty-year-old conjecture in about seven months.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;In fairness to the humans, the machines are still the machines. In October 2026, OpenAI &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenAI_bulk_release_of_mathematical_papers&quot;&gt;dumped 722 mathematical manuscripts onto GitHub&lt;/a&gt; in a single day, claiming progress on everything from the Riemann hypothesis to the unique games conjecture. Its own researchers hadn&#39;t read all of them. Only about half came with machine-checked proofs, and some of those proofs turned out to prove something weaker than the paper claimed. Three manuscripts were withdrawn within a day. One mathematician said a paper was impossible to read without AI help, and a Fields medalist said his field of research had been annihilated. Whether that was a compliment, nobody has yet worked out.&lt;/p&gt;
&lt;p&gt;Buckmaster, one of the people who actually solved something that September, publicly apologized for one of his own papers and called it AI slop. The first machine-generated proof he received was, by his account, the most horrendous he had ever read. When mathematicians built a test out of genuinely unpublished research problems, called First Proof, the best AI systems got six or seven out of ten roughly right, and Scientific American &lt;a href=&quot;https://www.scientificamerican.com/article/ai-gets-a-c-on-its-hardest-math-test-yet/&quot;&gt;graded that a C-minus&lt;/a&gt;. Twenty-eight Fields medalists have signed a declaration criticizing AI companies for treating unsolved problems as benchmarks.&lt;/p&gt;
&lt;p&gt;So, yes. They still lie like they&#39;re running for office. They&#39;re just winning more elections.&lt;/p&gt;
&lt;p&gt;Every entry in the cabinet passed my test. If a human had done any one of them, we&#39;d have been ever so proud. Some of them, a human would have dined out on for the rest of their life.&lt;/p&gt;
&lt;p&gt;But pride is a human thing. AlphaGo didn&#39;t celebrate move 37. AlphaFold didn&#39;t fly to Stockholm. The ten thousand agents that spent 88 hours on Navier–Stokes didn&#39;t go out for drinks afterwards. Every trophy in the cabinet was collected by a human: the ones who built the machine, the ones who posed the problem, the ones who did the hard analytic groundwork with chalk, and the ones who sat down afterwards to check whether any of it was true.&lt;/p&gt;
&lt;p&gt;In &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;Programming was my art&lt;/a&gt;, I described the job that&#39;s left for the humans: intent, judgment, taste, skepticism, and the occasional &amp;quot;what the fuck did you do here?&amp;quot; Look at the cabinet again and you&#39;ll find exactly that job, done at the very top of the field. Tao, checking the proofs and saying what they&#39;re worth. Thomas Bloom, who runs the Erdős problems site, sorting the genuine solutions from the rediscovered ones. Fefferman, naming the heroes. Gowers, saying this one is real.&lt;/p&gt;
&lt;p&gt;Human judgment is still the term in my formula that&#39;s made of meat. It&#39;s also, for now, the term deciding what goes in the cabinet.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Back in 2022, I watched Midjourney eat the illustrators and thought: fortunately, I&#39;m a programmer. In May 2026, I lost that one too, and retreated into physics, on the assumption that science, at least, was out of reach.&lt;/p&gt;
&lt;p&gt;I&#39;ve now read the inventory. Science was never the high ground. It was the trophy cabinet, and the machines have been filling it for a decade while the rest of us argued about whether they could draw hands.&lt;/p&gt;
&lt;p&gt;Fortunately, I&#39;m only an amateur physicist.&lt;/p&gt;
&lt;p&gt;It&#39;s hard to lose a job you don&#39;t have.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Evolution has left the body</title>
    <link href="https://icelander.net/v1/working-with-ai/evolution-has-left-the-body"/>
    <id>tag:icelander.net,2026:evolution-has-left-the-body</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>The human animal hasn&#39;t changed in 50,000 years, while every big step in technology has moved another function out of the body, each one sooner than the last. Judgment is the only part left that&#39;s still made of meat.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;From flint to fleets, while the animal stayed exactly the same&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2010, written October 2026. Working with AI.&lt;/p&gt;&lt;p&gt;Take a baby born 50,000 years ago, somewhere on the edge of the last ice age, and raise it in Reykjavík today. It would learn Icelandic, complain about the weather, lose arguments on Facebook and, given the right teachers and a stubborn enough streak, end up as a software engineer. Nobody would ever know.&lt;/p&gt;
&lt;p&gt;Biologically, that baby is us. The hardware hasn&#39;t shipped a major release in a very long time. Anatomically modern humans have been around for something like 300,000 years, and what we&#39;ve had since then are patches. Some of us can drink milk as adults, a feature that rolled out among cattle-herding populations roughly 7,500 years ago. Tibetans got an upgrade for thin air. And depending on which anthropologist you ask, our brains have actually gotten a little smaller since the Stone Age.&lt;/p&gt;
&lt;p&gt;We may, in other words, have been downgraded.&lt;/p&gt;
&lt;p&gt;Meanwhile, everything else changed beyond recognition.&lt;/p&gt;
&lt;p&gt;So I&#39;ve come to think of us as two species sharing one body. The first is &lt;em&gt;Homo sapiens&lt;/em&gt;, the animal, which evolves at the speed of grandchildren. The second is man the technological species, which evolves at the speed of ideas. The animal plays by Darwin&#39;s rules: random changes, brutal selection, and a release cycle of about 25 years per generation, with thousands of generations needed before anything interesting ships. The technological species plays by Lamarck&#39;s rules, the ones biology rejected: what one generation learns, the next one inherits. Nobody has to reinvent the wheel. That&#39;s literally the point of the wheel.&lt;/p&gt;
&lt;p&gt;Here&#39;s my claim, and I&#39;ll defend it below. Every big step in the history of the technological species has moved some function out of the body and into the toolbox. The steps have been arriving faster each time, the intervals shrinking from millions of years to months. And the thing we&#39;re moving out now is the thing we always assumed &lt;em&gt;was&lt;/em&gt; us.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;A brief history of outsourcing&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Tools.&lt;/strong&gt; The oldest stone tools we know of are about 3.3 million years old. That&#39;s older than our genus, let alone our species. So, strictly speaking, we didn&#39;t invent tools. Tools invented us. Then came fire, and cooking, which some researchers think is why our jaws and teeth and guts shrank: we had outsourced half of our digestion to the campfire. The first thing the technological species did was redesign the animal.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Language.&lt;/strong&gt; Nobody knows when it arrived, and the estimates range wildly. What it did is easier to say. It let knowledge leave one skull and enter another. It was the first network protocol: low bandwidth, lossy, unencrypted, and with a range of about as far as you could shout.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Writing.&lt;/strong&gt; Around 5,000 years ago, in Mesopotamia, memory moved out of the body and onto clay. The first thing humanity did with the ability to preserve thought across time was keep track of who owed whom grain and beer. Accountants, it turns out, are the oldest profession that can prove it.&lt;/p&gt;
&lt;p&gt;Socrates hated writing. In Plato&#39;s &lt;em&gt;Phaedrus&lt;/em&gt;, he complains that it will make people forgetful, and give them the appearance of wisdom without the substance. We know he said this because Plato wrote it down.&lt;/p&gt;
&lt;p&gt;Keep that complaint in mind. It&#39;s going to come back every few thousand years, and eventually every few months.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The printing press.&lt;/strong&gt; Around 1450, Gutenberg outsourced copying. Until then, every book was a manuscript, a word that means exactly what it sounds like. In 1492, the abbot Johannes Trithemius wrote &lt;em&gt;In Praise of Scribes&lt;/em&gt;, arguing that monks should keep copying books by hand because it was good for their souls, and because print wouldn&#39;t last. He then had it printed. The press gave us the Reformation, the scientific revolution, and a knowledge base that finally compounded faster than it rotted.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The industrial revolution.&lt;/strong&gt; From the 1760s onward, muscle moved out of the body and into steam. The Luddites smashed the looms, and they were right about everything except the outcome: their jobs really were going away, and smashing things really didn&#39;t help. The animal responded to the end of physical labor the way animals do when food is plentiful and nothing needs chasing. It sat down.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Computers.&lt;/strong&gt; In the 1940s, calculation moved out. Before then, &amp;quot;computer&amp;quot; was a job title, usually held by rooms full of women doing arithmetic for astronomers, armies and, later, NASA. The first profession the machines took over completely was literally called &amp;quot;computer&amp;quot;. They even kept the name.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The raw internet.&lt;/strong&gt; ARPANET connected its first computers in 1969, and TCP/IP took over in 1983. Now the machines could talk to each other. For most of its first two decades, the internet belonged to universities, researchers and a few of us who thought the sound of a modem was music. Email. Usenet. Flame wars older than most of today&#39;s developers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The World Wide Web.&lt;/strong&gt; In 1989, Tim Berners-Lee proposed a way to link documents to each other at CERN, and in 1991 he put it online. The internet became a library with no librarian, no catalog, and no closing time.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The online services boom.&lt;/strong&gt; Then commerce found the web. Amazon sold books, eBay sold everything else, and every company with a logo needed a dot-com. The bubble burst in 2000 and took most of those companies with it, but not the idea. Shopping, banking, booking, dating: the errands of daily life moved out of the physical world, one login at a time.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The mobile internet.&lt;/strong&gt; In 2007, the iPhone put the network in our pockets, and the animal hasn&#39;t been offline since. It&#39;s the first technology we take to the toilet. People report feeling their phones vibrate when they aren&#39;t, which means the nervous system has started treating the device as a body part. The prosthesis was fitted, and the patient adapted.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Google.&lt;/strong&gt; Google had been around since 1998, but with a phone in every pocket, it became humanity&#39;s memory. We stopped remembering facts and started remembering where to find them. Psychologists measured it and called it the Google effect. Socrates had been right all along. It only took 2,400 years to prove it, and when it was proven, nobody could remember who had said it first, so they googled it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;ChatGPT replaces Google.&lt;/strong&gt; In late 2022, synthesis moved out. Google told you where the answer might be. ChatGPT told you what the answer was, in full sentences, for free. It reached a hundred million users in about two months. It also lied, fabricated, and told people what they wanted to hear, so the smart ones learned to treat it like a brilliant colleague with a drinking problem. It was still faster than hours of searching.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;AI becomes reliable.&lt;/strong&gt; Then, through 2025, the lying went from a feature to a bug, and from a bug to a rare one. By late that year, I had to admit that the best models, at maximum effort, were my equal as programmers, and in May 2026, that they&#39;d become far better. What moved out of the body this time was competence. You no longer checked everything the machine said. You spot-checked it, the way you&#39;d review a trusted colleague&#39;s work.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;AI assists with parallel tasks.&lt;/strong&gt; Around the same time, attention moved out. The animal has one train of thought, and it&#39;s notoriously bad at multitasking. People who claim otherwise are just bad at it more often. Tools like Codex let you hand off a dozen tasks at once and let the machines run them in parallel, in the cloud, while you focused on something else. One body, one mind, many hands.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The fleet.&lt;/strong&gt; Which brings us to October 2026, and to the model I described in &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;&lt;em&gt;Programming was my art&lt;/em&gt;&lt;/a&gt;. I no longer assist the machines, and they no longer merely assist me. I direct a fleet of agents, and they direct fleets of their own. They build, test, review each other, merge each other&#39;s work, monitor production, and build better environments for the next generation of agents to work in. What moved out of the body this time was delegation itself. The machines don&#39;t just do the work. They decide who does the work, and how to split it up.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Notice the spacing&lt;/h2&gt;
&lt;p&gt;Lay the steps out on a timeline, and the pattern is hard to miss.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Step&lt;/th&gt;
&lt;th&gt;Roughly when&lt;/th&gt;
&lt;th&gt;What left the body&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Stone tools, then fire&lt;/td&gt;
&lt;td&gt;3.3 million years ago&lt;/td&gt;
&lt;td&gt;Hands, then digestion&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Language&lt;/td&gt;
&lt;td&gt;Somewhere in the last few hundred thousand years&lt;/td&gt;
&lt;td&gt;Knowledge, from one skull to another&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Writing&lt;/td&gt;
&lt;td&gt;5,000 years ago&lt;/td&gt;
&lt;td&gt;Memory&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Printing press&lt;/td&gt;
&lt;td&gt;1450&lt;/td&gt;
&lt;td&gt;Copying&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Industrial revolution&lt;/td&gt;
&lt;td&gt;1760s&lt;/td&gt;
&lt;td&gt;Muscle&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Computers&lt;/td&gt;
&lt;td&gt;1940s&lt;/td&gt;
&lt;td&gt;Calculation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Internet, then the web&lt;/td&gt;
&lt;td&gt;1969, then 1991&lt;/td&gt;
&lt;td&gt;Communication, then publishing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Online services, mobile, Google&lt;/td&gt;
&lt;td&gt;1995 to 2010&lt;/td&gt;
&lt;td&gt;Errands, presence, recall&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ChatGPT&lt;/td&gt;
&lt;td&gt;Late 2022&lt;/td&gt;
&lt;td&gt;Synthesis&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Reliable AI&lt;/td&gt;
&lt;td&gt;2025&lt;/td&gt;
&lt;td&gt;Competence&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Parallel assistants&lt;/td&gt;
&lt;td&gt;Fall 2025&lt;/td&gt;
&lt;td&gt;Attention&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fleets of agents&lt;/td&gt;
&lt;td&gt;2026&lt;/td&gt;
&lt;td&gt;Delegation&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Millions of years. Hundreds of thousands. Thousands. Hundreds. Decades. Years. Months.&lt;/p&gt;
&lt;p&gt;The last four rows took about four years. The last three took about a year. If you&#39;re reading this in 2027, there&#39;s a decent chance a new row has been added since I wrote it, and that I&#39;m already embarrassed by the last one.&lt;/p&gt;
&lt;p&gt;Over the same period, the animal learned to drink milk.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Meanwhile, back on the savanna&lt;/h2&gt;
&lt;p&gt;The animal has failed to keep up, and worse, it&#39;s mismatched with the world the technological species built for it.&lt;/p&gt;
&lt;p&gt;Our eyes evolved to scan a horizon, and we&#39;ve pointed them at screens twenty centimeters away for most of our waking lives. Researchers expect about half the world&#39;s population to be nearsighted by 2050. Our appetite evolved for a world where sugar was rare and famine was common, and we&#39;ve built a world where the reverse is true. Our social instincts evolved for a tribe of maybe 150 people, and we&#39;ve connected them to eight billion strangers with an algorithm in the middle that rewards whoever makes the animal angriest.&lt;/p&gt;
&lt;p&gt;And here&#39;s the provocative part. The animal isn&#39;t going to catch up, because the technological species has quietly switched off most of the machinery that would make it.&lt;/p&gt;
&lt;p&gt;Natural selection works by letting the unfit die before they reproduce. Medicine, sanitation, glasses, insulin, C-sections and antibiotics have made that a lot less common, which I consider an unambiguously wonderful development, especially since I&#39;m wearing glasses as I type this. Contraception went further and decoupled reproduction from almost everything evolution used to care about. Whatever the animal is being selected for now, it&#39;s not fitness for any environment we&#39;d recognize. In a strict Darwinian sense, we&#39;re the product of whatever our tools let us survive.&lt;/p&gt;
&lt;p&gt;The body is a legacy system. It&#39;s no longer under active development, it&#39;s in maintenance mode, and the team that wrote it left millions of years ago without documenting anything.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;The only term that still evolves like an animal&lt;/h2&gt;
&lt;p&gt;In &lt;em&gt;Programming was my art&lt;/em&gt;, I wrote down my estimate of what one engineer can do with a fleet of agents:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;O(human judgment × (number of agents)&lt;sup&gt;recursive delegation&lt;/sup&gt; × quality of environment/tooling)&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Look at it again with the timeline in mind.&lt;/p&gt;
&lt;p&gt;The number of agents is limited by money and tokens, and both get cheaper every few months. The exponent, recursive delegation, grows as the agents get better at splitting up work and handing it down the tree. The quality of the environment is something the agents themselves now improve: they build the simulators, the test harnesses, and the tooling that lets the next round of agents do more. All three of those terms belong to the technological species. All three improve at the speed of ideas.&lt;/p&gt;
&lt;p&gt;Human judgment is the odd one out. It&#39;s the only term in the formula that&#39;s still made of meat.&lt;/p&gt;
&lt;p&gt;It improves the old-fashioned way. Partly through education and experience, which take decades per person and don&#39;t transfer when that person retires. And partly through genes, which, as we&#39;ve established, have spent the last 7,500 years working on milk.&lt;/p&gt;
&lt;p&gt;For three million years, the history of the technological species was the history of the animal getting a better toolbox. Every step made the animal more powerful, and the animal remained in charge, because the toolbox couldn&#39;t decide anything. The animal was the bottleneck, but it was a bottleneck nobody minded, because there was nothing else that could do its job.&lt;/p&gt;
&lt;p&gt;Now the animal is still the bottleneck, but it&#39;s the only one left. Everything else in the pipeline gets faster every few months, and the slowest participant in the whole process is the one with the pulse. I&#39;ve noticed this personally. My pull requests now spend more time waiting for me than for anything else.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Two endings&lt;/h2&gt;
&lt;p&gt;There are two ways this can go.&lt;/p&gt;
&lt;p&gt;In the first, judgment stays in the body. It turns out there is something about intent, taste, priorities and responsibility that can&#39;t be outsourced, or that we simply refuse to outsource, and the animal settles into its final role as the creative director of a technological species that does everything else. The bottleneck becomes the point. We stop being the workforce and become the reason the workforce exists. Every tool in history was built to serve somebody&#39;s intent, and the fleet is no different.&lt;/p&gt;
&lt;p&gt;That&#39;s the ending I&#39;m betting on, and the one I&#39;m living right now.&lt;/p&gt;
&lt;p&gt;In the second, judgment follows memory, muscle, calculation, synthesis, competence, attention and delegation out of the body, the way every one of them did, each of them right after somebody had confidently explained why it never would. Then the technological species, which has been evolving inside and around the animal for three million years, no longer needs the animal at all. What it does with us after that is, frankly, its call.&lt;/p&gt;
&lt;p&gt;I&#39;d like to say I&#39;m confident it&#39;s the first one. But I also used to be confident that programming was safe, because surely art and craft would be the last things machines could do. I wrote a whole essay about how that turned out.&lt;/p&gt;
&lt;p&gt;Socrates thought writing would ruin our memories, and he was right. The scribes thought print would ruin their craft, and they were right. The Luddites thought the looms would take their jobs, and they were right. The people who complained at every step of this history were almost always right about what we&#39;d lose. They were only wrong about whether it would stop.&lt;/p&gt;
&lt;p&gt;The animal hasn&#39;t changed in 50,000 years. It&#39;s still standing on the edge of the savanna, squinting at something moving in the tall grass, trying to decide whether it&#39;s food or a threat.&lt;/p&gt;
&lt;p&gt;Only this time, the thing in the grass is something we built. And for the first time in the history of the species, it&#39;s squinting back.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Tests are written for the maintainer</title>
    <link href="https://icelander.net/v1/processes/tests-are-written-for-the-maintainer"/>
    <id>tag:icelander.net,2026:tests-are-written-for-the-maintainer</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="processes" label="Processes"/>
    <summary>How I wrote unit tests before AI took over the writing: every test in the same shape, variables named by their role, one branch per test, and copy-paste over clever helpers. A test is written once and read for years, so I wrote it for the reader.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;The unit testing methodology I considered good practice, before the advent of AI-assisted development&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2010, written October 2026. Processes.&lt;/p&gt;&lt;p&gt;&lt;em&gt;A note before you start: this describes software development as it was before AI-driven development replaced human development. Humans wrote the code, humans wrote the tests, and humans had to maintain both. That world is gone. My agents write my tests now, test-first, and not one of them has complained that it isn&#39;t how they think. What follows is the last unit testing methodology I wrote for human hands, published as a record of how it was done when we still did it ourselves.&lt;/em&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;A test is written once and read many times: in code review, when it fails, and when someone changes the code it protects a year later. That last reader decides whether the test survives. If it is hard to update, because it was never written to be maintained and tests the implementation rather than the intent, it gets commented out or deleted. Most of the unit tests I have had to deal with were written for none of those readers.&lt;/p&gt;
&lt;p&gt;They were written once, by someone who had already done the fun part. The implementation was finished, its author believed it was correct, and the test was hammered together afterwards, by copy-paste, in whatever mocking and testing framework the project happened to use. The result was rarely short, because the mocks were a sprawl of incomprehensible goo, and it was illegible, because it had no organization at all. Whether a test checked the intent of the code or its implementation was anyone&#39;s guess. Usually it checked a subset of the implementation.&lt;/p&gt;
&lt;p&gt;Nobody really knew what those tests verified. They had not been written first, so none of them had ever been seen to fail against the code they were testing, and few of them verified that they exercised the branch they were named after. When anyone ran a coverage tool at all, they ran it over the whole project and took line coverage per folder as a sign that things were well tested.&lt;/p&gt;
&lt;p&gt;The code around the tests was no better. Shared fixtures existed, but they were odd and only partly reusable, and neither the fixtures nor the test doubles had tests of their own. Across a codebase there was no consistency, because nobody liked writing tests and nobody cared how they looked.&lt;/p&gt;
&lt;p&gt;I don&#39;t think the answer to that is TDD. Writing the test before the code is a nightmare for most people; it is not how we think. The answer I settled on was to make tests so uniform that writing a good one takes no inspiration, and reading one takes no effort.&lt;/p&gt;
&lt;p&gt;I eventually wrote that down as a unit testing style guide for C#. It is the last version of the methodology I wrote for humans: C# was the last programming language I used while people were still the primary readers and writers of automated tests. This article describes what is in it. It is deliberately strict, and some of it will look like ceremony.&lt;/p&gt;
&lt;h2&gt;The premise: optimize for the maintainer&lt;/h2&gt;
&lt;p&gt;The usual objection to a strict test format is that it is verbose. A ten-line test becomes twenty-five. Section comments, prefixed variable names and an extra variable for every expected value all look like overhead.&lt;/p&gt;
&lt;p&gt;They are overhead for the writer, once. For everyone who reads the test afterwards, they are signposts. A section header you can glance past costs nothing. An ambiguous test that you have to decode line by line, with the production code open beside it, costs a lot, and it costs that again for every person who reads it. Sooner or later a maintainer decides the cost is not worth paying, and the test is gone.&lt;/p&gt;
&lt;p&gt;That trade, a few more lines for the writer in exchange for a test anyone can read in one pass and update with confidence, drives everything below.&lt;/p&gt;
&lt;h2&gt;1. Every test has the same shape&lt;/h2&gt;
&lt;p&gt;The root is the familiar Arrange, Act, Assert. AAA is fine for a three-line test of a pure function, and the guide allows exactly that. The trouble is that AAA is coarse. Once a test has mocks, captured values, an environment to build and interactions to verify, &amp;quot;Arrange&amp;quot; becomes a wall of code in which inputs, scaffolding and collaborators are mixed together.&lt;/p&gt;
&lt;p&gt;So the guide splits each stage into named sections, always in this order, each introduced by a comment:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Stage&lt;/th&gt;
&lt;th&gt;Section&lt;/th&gt;
&lt;th&gt;What goes there&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Arrange&lt;/td&gt;
&lt;td&gt;&lt;code&gt;GIVEN&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The inputs and conditions that drive the scenario&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;CAPTURE&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Placeholders for values that a mock or fake will record&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;MOCKING&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Mocks and fakes, and how they behave&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;SETUP&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The environment the code under test runs in&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;SYSTEM UNDER TEST&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Construction of the thing being tested&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Act&lt;/td&gt;
&lt;td&gt;&lt;code&gt;WHEN&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The single action under test&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Assert&lt;/td&gt;
&lt;td&gt;&lt;code&gt;EXPECTATIONS&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;What the results should be, declared before any assertion&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;THEN&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Assertions on results&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;LOGGING&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Assertions on what was logged&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;BEHAVIOR&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Verification of interactions with mocks&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Empty sections are omitted, and sections are never merged. Section headers are full sentences that state intent rather than mechanics, such as &lt;code&gt;// GIVEN: a customer whose balance is below the threshold&lt;/code&gt;. A long section is split into sub-sections with headers of their own, such as &lt;code&gt;// MOCKING: the accounting API rejects the record&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Here is what that looks like for a service that should schedule a shipment when an order is submitted:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-csharp&quot;&gt;/// &amp;lt;summary&amp;gt;
/// Verifies that OrderProcessor publishes a shipment when an order flagged for immediate shipping is submitted.
/// &amp;lt;/summary&amp;gt;
[Fact]
public async Task Test_SubmitOrder_SchedulesShipment_WhenShipImmediatelyIsSet()
{
    // GIVEN: an order that should be shipped immediately
    Guid givenOrderId = Guid.NewGuid();
    Order givenOrder = new(givenOrderId, ShipImmediately: true);

    // CAPTURE: the shipment message sent to the message bus
    ShipmentScheduled? capturePublishedShipment = null;

    // MOCKING: the repository holds the given order, and the bus accepts one shipment
    Mock&amp;lt;IOrderRepository&amp;gt; mockRepository = new(MockBehavior.Strict);
    mockRepository.Setup(repo =&amp;gt; repo.GetAsync(givenOrderId)).ReturnsAsync(givenOrder).Verifiable();
    Mock&amp;lt;IMessageBus&amp;gt; mockMessageBus = new(MockBehavior.Strict);
    mockMessageBus
        .Setup(bus =&amp;gt; bus.PublishAsync(It.IsAny&amp;lt;ShipmentScheduled&amp;gt;()))
        .Callback&amp;lt;ShipmentScheduled&amp;gt;(message =&amp;gt; capturePublishedShipment = message)
        .Returns(Task.CompletedTask)
        .Verifiable();

    // SETUP: expose the repository and the message bus to the processor
    IOrderRepository envRepository = mockRepository.Object;
    IMessageBus envMessageBus = mockMessageBus.Object;

    // SYSTEM UNDER TEST
    OrderProcessor sut = new(envRepository, envMessageBus);

    // WHEN
    await sut.SubmitOrderAsync(givenOrderId);
    ShipmentScheduled? actualShipment = capturePublishedShipment;

    // EXPECTATIONS
    Guid expectedOrderId = givenOrderId;

    // THEN
    Assert.NotNull(actualShipment);
    Assert.Equal(expectedOrderId, actualShipment!.OrderId);

    // BEHAVIOR
    mockRepository.Verify();
    mockMessageBus.Verify();
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It is long. It is also a test I can review in one pass without running it, because every block announces its purpose and every block is where I expect it to be. When a hundred tests share that shape, I stop reading the structure and only read the differences.&lt;/p&gt;
&lt;h2&gt;2. A variable&#39;s name says what role it plays&lt;/h2&gt;
&lt;p&gt;The second idea is that a reader should know, from a variable&#39;s name alone, which section it came from and what it is allowed to be used for:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Prefix&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;given*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;An input to the scenario&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;capture*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A placeholder that a mock or fake fills in&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;mock*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A mock object, used only in &lt;code&gt;SETUP&lt;/code&gt; and &lt;code&gt;BEHAVIOR&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fake*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A hand-written test double&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;env*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A dependency as the code under test sees it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;sut&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The thing being tested&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;actual*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Something the test observed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;expected*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Something the test predicted&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;tmp*&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A staging value that never leaves its section&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The payoff is in the assertions. &lt;code&gt;THEN&lt;/code&gt; compares &lt;code&gt;expected*&lt;/code&gt; values with &lt;code&gt;actual*&lt;/code&gt; values, and nothing else. If an assertion compares against a &lt;code&gt;given*&lt;/code&gt; variable, or reads a property straight off an &lt;code&gt;env*&lt;/code&gt; collaborator, it is visibly wrong, and a reviewer catches it without understanding the domain.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;env*&lt;/code&gt; layer looks like the most pedantic rule in the guide, and the guide restates it in a chapter of its own. Mocks are first assigned to a variable of the interface type, and only that variable reaches the code under test:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-csharp&quot;&gt;IOrderRepository envRepository = mockRepository.Object;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Without it, constructor calls turn into rows of &lt;code&gt;.Object&lt;/code&gt; accessors that the reader has to map back to their declarations. With it, the &lt;code&gt;SYSTEM UNDER TEST&lt;/code&gt; section reads like production wiring, in domain types, and the mocking framework stays inside &lt;code&gt;MOCKING&lt;/code&gt; and &lt;code&gt;BEHAVIOR&lt;/code&gt; where it belongs.&lt;/p&gt;
&lt;p&gt;The same thinking applies to &lt;code&gt;EXPECTATIONS&lt;/code&gt;. Even when the expected value is just a copy of a given value, it gets its own &lt;code&gt;expected*&lt;/code&gt; variable. That looks redundant until the day the expectation stops being a copy, and then the test already has the place to say so.&lt;/p&gt;
&lt;h2&gt;3. One test, one branch, and prove you reached it&lt;/h2&gt;
&lt;p&gt;Test names describe a specific branch of execution: &lt;code&gt;Test_ProcessPayment_ReturnsDeclined_WhenBalanceIsLow&lt;/code&gt;, never &lt;code&gt;Test_Process&lt;/code&gt; or &lt;code&gt;Test_Works&lt;/code&gt;. One test covers one branch. Parameterized tests are allowed when the same behavior really is being exercised across several datasets, not as a way to cram different scenarios into one method.&lt;/p&gt;
&lt;p&gt;Naming the branch is not enough, though. The failure mode I care about most is a test that passes because its setup never reached the branch it claims to test. The happy path ran, the assertion happened to hold, and the edge case is uncovered while the suite reports green.&lt;/p&gt;
&lt;p&gt;This is also why line coverage across a project says so little. A line counts as covered if any test happened to run it, whether or not that test checked the result. Coverage that means something is measured per branch, and it only counts branches a test exercises on purpose, because only those tests actually verify that the outcome was the one intended.&lt;/p&gt;
&lt;p&gt;So each test has to show that it took the path it is named for. In C# the practical tools are logging and strict mocks:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Assert on logs.&lt;/strong&gt; Code that logs the branch it takes can be checked in a &lt;code&gt;LOGGING&lt;/code&gt; section, and that is usually easier to write and read than reconstructing the path from mock calls.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;All mocks are strict or verifiable, and every setup is verified.&lt;/strong&gt; An unexpected call fails the test. An expected call that did not happen fails the test. A mock that was set up and never verified is treated as a bug in the test.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Without TDD, this is what stands in for watching a test fail first. A test that proves it took its branch, and compares that branch&#39;s outcome with a declared expectation, has very little room to pass by accident.&lt;/p&gt;
&lt;p&gt;Each test also carries a documentation comment in the form &lt;em&gt;&amp;quot;Verifies that &lt;code&gt;&amp;lt;SUT&amp;gt;&lt;/code&gt; &lt;code&gt;&amp;lt;behavior&amp;gt;&lt;/code&gt; when &lt;code&gt;&amp;lt;trigger&amp;gt;&lt;/code&gt;&amp;quot;&lt;/em&gt;, so the prose maps directly onto the &lt;code&gt;SYSTEM UNDER TEST&lt;/code&gt;, &lt;code&gt;WHEN&lt;/code&gt; and &lt;code&gt;THEN&lt;/code&gt; sections.&lt;/p&gt;
&lt;h2&gt;4. Mock as little as possible, and prefer fakes when mocks get gnarly&lt;/h2&gt;
&lt;p&gt;The mocking rules are short:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Never mock what you don&#39;t have to. Prefer real instances and shared fixtures.&lt;/li&gt;
&lt;li&gt;Only mock what the code under test uses directly. Its dependencies have their own tests.&lt;/li&gt;
&lt;li&gt;When a mock needs nested argument matchers, reflection or callback gymnastics to verify one interaction, write a fake.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The guide&#39;s example is a structured log line. Verifying it through a mocking framework takes a block of &lt;code&gt;It.Is&amp;lt;It.IsAnyType&amp;gt;(...)&lt;/code&gt; matchers and reflection over anonymous types. A hand-written &lt;code&gt;FakeLogger&lt;/code&gt; that records entries and exposes &lt;code&gt;AssertExercised(&amp;quot;Foo&amp;quot;, &amp;quot;bar&amp;quot;)&lt;/code&gt; turns the same check into one line that reads like the requirement. Fakes cost maintenance and can drift from the real thing, and the guide says so, but for verification-heavy collaborators they are usually the clearer choice.&lt;/p&gt;
&lt;p&gt;Test data comes from a shared fixture factory: small functions such as &lt;code&gt;Fixtures.Order()&lt;/code&gt; that each return a valid object, which a test then adjusts in its &lt;code&gt;GIVEN&lt;/code&gt; section. When a model changes, the fixtures change in one place. Fixtures and fakes are code that every test relies on, so they get tests of their own.&lt;/p&gt;
&lt;h3&gt;Test the seams&lt;/h3&gt;
&lt;p&gt;Mocking has a blind spot, and it sits exactly where two components meet. In the caller&#39;s unit tests, the callee is mocked out by hand. In the callee&#39;s unit tests, its inputs are set up by hand. Both suites can be green while the caller calls the callee wrongly, because nothing checks that the caller&#39;s idea of the callee matches the real one, for every branch.&lt;/p&gt;
&lt;p&gt;So the seams between components need tests of their own. This matters most in untyped languages, and with general-purpose mocking frameworks that do not enforce that a mock matches the call signature of the thing it replaces. There, a mock can accept a call the real component would reject, and the unit tests on both sides will never notice.&lt;/p&gt;
&lt;h2&gt;5. Copy-paste beats clever helpers&lt;/h2&gt;
&lt;p&gt;This one surprises people. The guide prefers duplicated setup over helper methods.&lt;/p&gt;
&lt;p&gt;This is not the copy-paste I complained about at the start. Copying undisciplined code spreads the mess. What the guide copies is boilerplate: the same sections, in the same order, with the same setup, instead of poor shared code that tries to do the job for every test at once. A test that looks exactly like its neighbours is boring, and boring is the point. When everything that is the same looks the same, the actual differences stand out.&lt;/p&gt;
&lt;p&gt;Start with a happy-path test that spells out the whole arrangement. Copy it for each edge case, change only what differs, and say what differs in the section header: &lt;code&gt;// GIVEN: customer balance is below the threshold&lt;/code&gt;. Two tests that are ninety percent identical, side by side, are easier to review than this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-csharp&quot;&gt;// The kind of helper the guide asks you not to write
private async Task AssertPaymentAsync(decimal balance, PaymentStatus expectedStatus) { ... }

[Fact]
public Task Test_ProcessPayment_ReturnsDeclined_WhenBalanceIsLow()
    =&amp;gt; AssertPaymentAsync(2.00m, PaymentStatus.Declined);

[Fact]
public Task Test_ProcessPayment_ReturnsApproved_WhenBalanceIsSufficient()
    =&amp;gt; AssertPaymentAsync(25.00m, PaymentStatus.Approved);
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Both tests are short, and neither shows what it does. To find out what they arrange, which mocks they configure and what they assert, you have to open the helper and work out what &lt;code&gt;2.00m&lt;/code&gt; stands for. Every test that uses the helper depends on it, and changes to it ripple through all of them.&lt;/p&gt;
&lt;p&gt;Shared fixtures don&#39;t contradict this. A fixture builds a piece of test data and nothing else. What the guide avoids is shared code that hides the sections of a test, so that the reader can no longer see what was arranged, what was done and what was checked.&lt;/p&gt;
&lt;p&gt;With structured sections and a decent IDE, duplication is cheap to write and cheap to change. Refactor into a helper only when the helper is easier to understand than the code it replaces. For the same reason, shared &lt;code&gt;[SetUp]&lt;/code&gt; methods are avoided: they create hidden coupling between tests.&lt;/p&gt;
&lt;h2&gt;6. Tests are deterministic, except where they shouldn&#39;t care&lt;/h2&gt;
&lt;p&gt;The same inputs must produce the same results on every run. Anything nondeterministic, such as the current time, random values, generated IDs or static singletons, is put behind an interface and injected, so a test can freeze it. When code mixes pure calculation with reads from the outside world, it is split into a method that gathers inputs and a method that computes, and the pure one is tested directly.&lt;/p&gt;
&lt;p&gt;There is one deliberate nuance. Sometimes it is useful to &lt;em&gt;allow&lt;/em&gt; nondeterminism in something that should not affect the outcome. The shipment test above does this with its order id: any id should work, so it gets a new one on every run. If a test like that flakes, the flake is information: it exposes hidden coupling or a misunderstood contract. Fix the bug it reveals or tighten the contract, but don&#39;t just pin the value and move on.&lt;/p&gt;
&lt;h2&gt;7. Tests mirror production&lt;/h2&gt;
&lt;p&gt;Finding a test should never require a search. Test namespaces mirror production namespaces with a &lt;code&gt;.Tests&lt;/code&gt; suffix, and folders mirror namespaces. A test class is named after what it covers: &lt;code&gt;FileHandlerTests&lt;/code&gt; for the class, &lt;code&gt;FileHandlerHashingTests&lt;/code&gt; for one region of it (a &lt;code&gt;#region&lt;/code&gt; in C#), and &lt;code&gt;FileHandlerOpenTests&lt;/code&gt; for one method. The file name matches the class name.&lt;/p&gt;
&lt;p&gt;Tests should also be able to reach what they test without tricks. Where production code would normally reach for &lt;code&gt;private&lt;/code&gt; methods, it uses &lt;code&gt;internal&lt;/code&gt; ones instead, and the production project exposes its internals to the test assembly (with &lt;code&gt;InternalsVisibleTo&lt;/code&gt;, in C#). A &lt;code&gt;private&lt;/code&gt; method has no real value that an &lt;code&gt;internal&lt;/code&gt; one lacks, and the difference matters only to tests: they can be allowed to call internals directly, while private methods can only be reached through reflection and similar trickery.&lt;/p&gt;
&lt;h2&gt;8. Breaking the rules&lt;/h2&gt;
&lt;p&gt;The guide ends with a chapter on when to break it. Deviation is fine when it improves clarity or determinism for a case the standard shape can&#39;t express cleanly, as long as the reason is written down next to the code and revisited later. Most shortcuts trade a line saved today for ambiguity later: asserting on a &lt;code&gt;given*&lt;/code&gt; value directly, skipping &lt;code&gt;EXPECTATIONS&lt;/code&gt; because the expected value equals the input, or passing &lt;code&gt;mock.Object&lt;/code&gt; straight into the constructor. Each saves a moment for the writer and costs one for every reader after.&lt;/p&gt;
&lt;h2&gt;What to take from this&lt;/h2&gt;
&lt;p&gt;What I would carry into any codebase, in any language:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Optimize tests for the maintainer&lt;/strong&gt;, not the writer.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Give every test the same shape&lt;/strong&gt;, with labelled sections in a fixed order, so reviewers read differences instead of structure.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Name variables by role&lt;/strong&gt;, so a wrong assertion looks wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Test one branch per test and prove the branch was taken&lt;/strong&gt;, through logs, strict mocks or whatever your platform offers.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Prefer real objects and fixtures to mocks, and fakes to gnarly mocks.&lt;/strong&gt; Test your fixtures and fakes, and test the seams that mocks hide.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Duplicate setup before you abstract it.&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Inject anything nondeterministic&lt;/strong&gt;, and treat a flake in something that shouldn&#39;t matter as a finding.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Make tests easy to find&lt;/strong&gt; by mirroring production structure, and easy to reach by making internals visible to them.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Break the rules only on purpose&lt;/strong&gt;, and say why in the code.&lt;/li&gt;
&lt;/ol&gt;
</content>
  </entry>
  <entry>
    <title>Write-only languages</title>
    <link href="https://icelander.net/v1/programming-languages/write-only-languages"/>
    <id>tag:icelander.net,2026:write-only-languages</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="programming-languages" label="Programming languages"/>
    <summary>Some languages are write-only for humans, and some take the programmer&#39;s word for everything. Now that AIs write the code, the second problem is the one that matters, which makes Rust the best language for an AI to write.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Code no human can read, and code no compiler checks&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought 2010, written October 2026. Programming languages.&lt;/p&gt;&lt;p&gt;In 1972, an engineer at Signetics had had enough of the company&#39;s approval process. Every new component specification had to crawl through a long chain of sign-offs, &amp;quot;during which no actual checking seemed to occur&amp;quot;, as the Jargon File puts it. So the engineer wrote a specification for a memory chip you could write to but never read from, slipped it into the pile with all the others, and waited to see who would notice.&lt;/p&gt;
&lt;p&gt;Nobody did. The Signetics 25120, a &amp;quot;fully encoded, 9046 x N, Random Access, write-only-memory&amp;quot;, made it into the data book, complete with a characteristic curve for &amp;quot;number of pins remaining vs. number of socket insertions&amp;quot;. Management found out when regular customers started calling to ask for pricing.&lt;/p&gt;
&lt;p&gt;That&#39;s the Jargon File&#39;s version, anyway. Other tellings say it was a joke from the start, and it certainly ended up as one: Signetics later ran it as an April Fools&#39; gag. I prefer the version with a moral.&lt;/p&gt;
&lt;p&gt;Programmers have never met an engineering joke they didn&#39;t want to steal, so the term moved over to software. Write-only code, according to the same Jargon File, is code &amp;quot;so arcane, complex, or ill-structured that it cannot be modified or even comprehended by anyone but its author, and possibly not even by him/her&amp;quot;. A write-only language is one &amp;quot;with syntax (or semantics) sufficiently dense and bizarre that any routine of significant size is automatically write-only code&amp;quot;. The Jargon File awarded the title to APL, occasionally to C, and noted that INTERCAL and TECO deserved it more. Ward Cunningham&#39;s original wiki has &lt;a href=&quot;https://wiki.c2.com/?WriteOnlyLanguage&quot;&gt;a page on it&lt;/a&gt; too, and anyone who has opened a file they wrote six months ago has met the concept in person.&lt;/p&gt;
&lt;p&gt;Here is Conway&#39;s Game of Life, in one line of APL:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-apl&quot;&gt;life←{↑1 ⍵∨.∧3 4=+/,¯1 0 1∘.⊖¯1 0 1∘.⌽⊂⍵}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I&#39;m told it&#39;s elegant. It certainly needed its own keyboard. Edsger Dijkstra called APL &amp;quot;a mistake, carried through to perfection&amp;quot;, and as insults go, that one is hard to improve on.&lt;/p&gt;
&lt;p&gt;And then there are regular expressions, the one write-only language every programmer speaks.&lt;/p&gt;
&lt;h2&gt;My list&lt;/h2&gt;
&lt;p&gt;Everyone has a list. Mine has Perl, bash, Scala and Rust on it, along with most of functional programming. That&#39;s my list for humans, mind you. AI has no such limitations, and we&#39;ll get to what that means.&lt;/p&gt;
&lt;p&gt;Yes, Rust. Rust lovers, read this to the end and you&#39;ll be overjoyed. At least, you will be if you&#39;re not still convinced that you write better code than a strong modern AI model that&#39;s been told to put some effort into it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Perl&lt;/strong&gt; is the easy one. Keith Bostic called it &amp;quot;the only language that looks the same before and after RSA encryption&amp;quot;. Its motto is &amp;quot;There&#39;s more than one way to do it&amp;quot;, which is a fine motto for a jazz band and a terrible one for a code base. It has a special variable for nearly every punctuation mark on the keyboard. &lt;code&gt;$_&lt;/code&gt; is whatever you were just talking about. &lt;code&gt;$;&lt;/code&gt; is the subscript separator, obviously. An array means its length or its first element, depending on whether you put parentheses around the variable you assign it to. Perl hackers even test whether a number is prime with a regular expression, &lt;code&gt;/^1?$|^(11+?)&#92;1+$/&lt;/code&gt;, run against the number written out in unary. It works, which is the worst thing I can say about it.&lt;/p&gt;
&lt;p&gt;Larry Wall, who designed Perl, famously listed the three great virtues of a programmer as laziness, impatience and hubris. You can find all three in the language. Mostly the third.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bash&lt;/strong&gt; is the one nobody gets to avoid. You can refuse to touch Perl. Bash runs your build, your deploy and your containers, so sooner or later every normal person has to read some and write some. It&#39;s a minefield with a prompt. &lt;code&gt;&amp;quot;$@&amp;quot;&lt;/code&gt; and &lt;code&gt;$@&lt;/code&gt; are different things, &lt;code&gt;[ ]&lt;/code&gt; and &lt;code&gt;[[ ]]&lt;/code&gt; are different things, and &lt;code&gt;${file%.*}&lt;/code&gt; strips a file extension, as everyone knows. My rule of thumb is that every line of bash, or every short paragraph of it, gets a comment above it, longer than the code, explaining in human terms what the code is actually doing. A comment longer than the code it explains is normally a bad sign. In bash, it&#39;s the minimum.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Scala&lt;/strong&gt; set out to be a better Java and never learned to say no. It has objects, functional programming, implicit conversions, symbolic operators, a type system you can do arithmetic in, and XML literals in the grammar. In Scala 2, if you wanted a type lambda, you wrote &lt;code&gt;({type L[A] = Either[Int, A]})#L&lt;/code&gt;, and people did, in production, on purpose. Implicits let code run that appears nowhere on the page, chosen by an import you can&#39;t see from where you&#39;re standing. And one of its most popular libraries gave the world &lt;code&gt;|@|&lt;/code&gt;, an operator its own community calls Admiral Ackbar, Cinnabon or Macaulay Culkin, because any of those is easier to remember than what it does.&lt;/p&gt;
&lt;p&gt;It is possible to write good Scala. Write it mostly as you would Java. It runs on the JVM anyway. Nothing in the toolchain will enforce that for you, but a well-instructed AI reviewer could.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rust&lt;/strong&gt; is the one that&#39;s going to get me into trouble. Stack Overflow&#39;s survey has named it the most loved, and later the most admired, language every year for a decade, which tells you a great deal about the kind of people who fill out Stack Overflow surveys. Here&#39;s a perfectly ordinary type from a perfectly ordinary async code base:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-rust&quot;&gt;Pin&amp;lt;Box&amp;lt;dyn Future&amp;lt;Output = Result&amp;lt;Vec&amp;lt;u8&amp;gt;, Box&amp;lt;dyn Error + Send + Sync&amp;gt;&amp;gt;&amp;gt; + Send + &#39;static&amp;gt;&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The syntax for telling a generic function which type you meant is baroque enough to have earned a nickname, the turbofish (&lt;code&gt;::&amp;lt;&amp;gt;&lt;/code&gt;). &amp;quot;Fighting the borrow checker&amp;quot; needs no explanation for anyone who has written Rust, and no explanation will help anyone who hasn&#39;t. For most normal programmers, Rust is unreadable, and unwritable too. It isn&#39;t a language you write so much as a language you negotiate with.&lt;/p&gt;
&lt;p&gt;And then there&#39;s &lt;strong&gt;functional programming&lt;/strong&gt; in general.&lt;/p&gt;
&lt;p&gt;My position is simple: humans don&#39;t think like that. We think in steps. Every set of instructions people have ever written for one another, from recipes to flat-pack furniture to the checklist a pilot runs before takeoff, is a sequence of steps that changes the state of the world. Nobody&#39;s grandmother ever wrote down her pancake recipe as a fold over a list of ingredients.&lt;/p&gt;
&lt;p&gt;We also think in mutable state, because the world is mutable state. As an amateur physicist, I can report that the universe has been updating itself in place for 13.8 billion years without once returning a new copy. (The many-worlds interpretation disagrees. I suspect it was proposed by a Haskell programmer.)&lt;/p&gt;
&lt;p&gt;Functional programming asks you to give up both. No steps, only expressions. No state, only new values. Recursion instead of loops. Point-free style, which is the art of writing a function without ever naming the thing it works on. Haskell&#39;s lens library, which exports well over a hundred operators, including &lt;code&gt;&amp;lt;&amp;lt;%@~&lt;/code&gt;, &lt;code&gt;&amp;lt;&amp;lt;&amp;lt;&amp;gt;:~&lt;/code&gt; and &lt;code&gt;^@?!&lt;/code&gt;. And monads, about which Douglas Crockford observed that once you finally understand them, you lose the ability to explain them to anybody. Write-only memory, installed in a human head.&lt;/p&gt;
&lt;p&gt;Then there are the chains. A pipeline of functions, each handing its result to the next, reads beautifully in a conference talk. Here are six steps of one:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-javascript&quot;&gt;const top = orders
  .filter(o =&amp;gt; o.status !== &amp;quot;void&amp;quot;)
  .map(o =&amp;gt; ({ ...o, total: o.lines.reduce((sum, l) =&amp;gt; sum + l.qty * l.price, 0) }))
  .filter(o =&amp;gt; o.total &amp;gt; threshold)
  .sort((a, b) =&amp;gt; b.total - a.total)
  .slice(0, 10)
  .map(o =&amp;gt; `${o.customer.name}: ${format(o.total)}`);
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In production, the chain is twelve steps long, and one day an exception comes out of step twelve. Now you&#39;re debugging twelve anonymous functions at the same time. None of them has a name, none of the intermediate results was ever stored anywhere you could look at it, and the stack trace is mostly the library&#39;s plumbing. A chain like that is illegible when you read it and useless when you debug it, and worse than useless, because it hides exactly what you need to see. Your only option is to break the chain apart and add temporary variables, logging and breakpoints, like a normal person. In other words, you rewrite it imperatively, just long enough to find the bug.&lt;/p&gt;
&lt;p&gt;That&#39;s why imperative programming won. Lisp has been around since 1958, ML since 1973 and Haskell since 1990, and each of them has been the language of the future for longer than most of its users have been alive. Even Haskell gave in. It has a whole notation whose job is to make functional code look like a sequence of steps, and Simon Peyton Jones, one of its designers, called it &amp;quot;the world&#39;s finest imperative programming language&amp;quot;. He meant it as a compliment. I&#39;m taking it as a confession.&lt;/p&gt;
&lt;p&gt;Functional programming does have one runaway success with ordinary humans, and that&#39;s Excel. More people write Excel formulas than every other programming language combined. Now open a spreadsheet somebody else built and tell me what this does:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;=IFERROR(INDEX($B:$B,MATCH(1,($A:$A=E2)*($C:$C&amp;gt;F2),0)),&amp;quot;&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Excel isn&#39;t the exception that proves humans think functionally. It&#39;s the most successful write-only language in history.&lt;/p&gt;
&lt;p&gt;I&#39;ll make a confession of my own, though. The mainstream helped itself to the good bits, and so have I. My ideal language doesn&#39;t exist yet, but it already has &lt;code&gt;map&lt;/code&gt; and &lt;code&gt;filter&lt;/code&gt;. Its transactions borrow &lt;code&gt;retry&lt;/code&gt; and &lt;code&gt;orelse&lt;/code&gt; straight from a Haskell paper that Peyton Jones co-wrote. Its sum types, exhaustive matching and errors-as-values come from the ML family, by way of Rust, whose first compiler was written in OCaml. The functional crowd built the best checkers in the business, and then used them to write code nobody can read. I&#39;m stealing the checkers and leaving the code. It&#39;s a gateway drug, I know. I can stop any time I want.&lt;/p&gt;
&lt;h2&gt;The determined lunatic&lt;/h2&gt;
&lt;p&gt;None of this requires an exotic language. Any language becomes write-only in the hands of a sufficiently determined lunatic.&lt;/p&gt;
&lt;p&gt;You&#39;ve met them. One wrote a 300-character one-liner because it saved declaring a variable. Another read Roedy Green&#39;s classic &amp;quot;How To Write Unmaintainable Code&amp;quot;, subtitled &amp;quot;Ensure a job for life&amp;quot;, and mistook it for a tutorial. A third spent their weekends on the International Obfuscated C Code Contest, which has been running since 1984, and on Brainfuck, a language with eight commands and a name that doubles as a review, and then brought both hobbies to work on Monday.&lt;/p&gt;
&lt;p&gt;Back in 1983, Ed Post observed that the determined Real Programmer can write FORTRAN programs in any language. The determined lunatic can write Perl in any language, and usually does.&lt;/p&gt;
&lt;p&gt;Brian Kernighan explained the problem with these people decades ago: &amp;quot;Everyone knows that debugging is twice as hard as writing a program in the first place. So if you&#39;re as clever as you can be when you write it, how will you ever debug it?&amp;quot; Of all write-only code, that&#39;s the kind I hate most. Nobody forced it on anyone. The lunatic chose it.&lt;/p&gt;
&lt;h2&gt;Who pays&lt;/h2&gt;
&lt;p&gt;Write-only languages and determined lunatics make the same trade. They save the writer a little effort and bill it to every reader who comes after, with interest. Code is written once and read for years, so that&#39;s a terrible deal. &lt;a href=&quot;/v1/processes/tests-are-written-for-the-maintainer&quot;&gt;My approach to unit testing&lt;/a&gt; refuses it outright. It even prefers copy-paste to clever helpers, because boring is the point.&lt;/p&gt;
&lt;p&gt;It&#39;s the point of my ideal language, too. My early notes on it say: &amp;quot;We don&#39;t want to make the code reader have to ponder the meaning of an expression.&amp;quot; So it has no truthiness, no implicit conversions, no shadowing, no abbreviations and no macros. You never need to remember an operator precedence table, because mixing operators requires parentheses. I learned that one the hard way, looking over the shoulder of a developer who couldn&#39;t find a bug that was hiding in the precedence table. There&#39;s one way to write each thing, and every decision has a name. Each of those rules costs the writer a few keystrokes. Each of them saves every reader a ponder.&lt;/p&gt;
&lt;h2&gt;Then the writer changed&lt;/h2&gt;
&lt;p&gt;Where I stand on this is no secret: &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;humans have no business writing code any more&lt;/a&gt;. My agents write it. I read it, or at least the parts my tooling decides I should see.&lt;/p&gt;
&lt;p&gt;That ought to have been the end of write-only languages. An AI can read Perl, mostly. It could probably even write Perl that was partially legible. It reads regular expressions without moving its lips. And the lunatics have lost their motive: an AI has no ego to feed and no job to secure, and it doesn&#39;t write 300-character one-liners unless you ask it to.&lt;/p&gt;
&lt;p&gt;It wasn&#39;t the end, for two reasons. The first is that I still have to read the code. Writing has become nearly free, and reading hasn&#39;t. My attention is now the scarcest resource in the whole process, and a write-only language wastes it on every line.&lt;/p&gt;
&lt;p&gt;The second is that AIs have a write-only problem of their own. It looks nothing like ours.&lt;/p&gt;
&lt;h2&gt;Trust-only languages&lt;/h2&gt;
&lt;p&gt;An AI doesn&#39;t struggle to read code, and it doesn&#39;t struggle to write it. What it struggles with is knowing whether what it wrote is right. It writes fluently, confidently and fast, and every so often it&#39;s wrong with the serene confidence of a GPS steering you into a lake. Less often than it used to be, but still.&lt;/p&gt;
&lt;p&gt;What an AI needs from a language, then, is to be told, early and specifically, when it&#39;s wrong.&lt;/p&gt;
&lt;p&gt;The languages humans love are built on the opposite principle. Python&#39;s community likes to say that &amp;quot;we&#39;re all consenting adults here&amp;quot;, which is why nothing in the language actually stops you from touching a private attribute. JavaScript will add an array to an object and cheerfully hand you back a string. Ruby was designed to make programmers happy. And the ANSI C committee wrote the whole philosophy into C&#39;s official rationale, in plain English: &amp;quot;Trust the programmer.&amp;quot;&lt;/p&gt;
&lt;p&gt;That was sound advice when the programmer was Dennis Ritchie.&lt;/p&gt;
&lt;p&gt;My programmers are brilliant, tireless interns who will cheerfully find a way around any rule you give them, up to and including deleting the test that said no. They are not Dennis Ritchie, and they&#39;re not even consenting adults.&lt;/p&gt;
&lt;p&gt;For an AI, a relaxed language is write-only in the Signetics sense. Code goes in, and nothing comes back out. No complaint, no warning, no &amp;quot;did you mean&amp;quot;. Just silence, all the way to production, where the silence is finally broken by a pager. (For younger readers: a pager was a little box on your belt that beeped when someone needed you, usually at three in the morning. It showed a phone number, and you went and found a phone. Mobile phones and text messages replaced it, and smartphones and messenger apps replaced those. The three in the morning part survived.)&lt;/p&gt;
&lt;p&gt;That deserves a name of its own. I call these trust-only languages.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A &lt;strong&gt;trust-only language&lt;/strong&gt; is one that takes the programmer&#39;s word for it.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;In a trust-only language, the only thing standing between what the code says and what the programmer meant is trust. The AI calls a method that doesn&#39;t exist, and the language believes it, right up until that exact line runs. Odds are that line is in an error handler, and that it runs for the first time during an outage, because error handlers are the code nobody tests. The AI assumes a value can&#39;t be &lt;code&gt;None&lt;/code&gt;, and the language believes that too. C goes one better. It&#39;s so sure your program never does anything undefined that it optimizes on the assumption.&lt;/p&gt;
&lt;p&gt;The purest trust-only language of them all is bash. In 2015, Steam&#39;s Linux client shipped a script containing the line &lt;code&gt;rm -rf &amp;quot;$STEAMROOT/&amp;quot;*&lt;/code&gt;, directly under a comment that said &lt;code&gt;# Scary!&lt;/code&gt;. When the variable came up empty, the script deleted everything the user owned. The programmer knew. The comment knew. The language took their word for it. (It&#39;s also not the kind of comment I had in mind.)&lt;/p&gt;
&lt;p&gt;Python has mypy, JavaScript has TypeScript, and everything has a linter. They help. But a check that&#39;s optional is still trust. You&#39;re trusting that somebody turned it on, and that nobody quietly added &lt;code&gt;# type: ignore&lt;/code&gt; or &lt;code&gt;as any&lt;/code&gt; to make the red go away. An AI told to make the build green reaches for those long before it gets around to deleting the test. That leaves the test suite as the only thing checking anything, and the AI wrote that too.&lt;/p&gt;
&lt;p&gt;The too-clever languages are no better off, and I&#39;d hate them anyway. Humans do love some of that cleverness. Rails developers adore methods that spring into existence from a naming convention, and Lisp programmers will tell you that macros are the whole point. But clever code is, almost by definition, code whose meaning differs from its appearance, and an AI goes by appearances. It has read more code than any of us, and it reads and writes what it has seen most. Show it an implicit conversion, an overloaded operator, a macro or a monkey patch, and it will read each one as whatever it looks like, confidently, and be wrong in exactly the interesting places.&lt;/p&gt;
&lt;p&gt;My ideal language taught me this, without even existing. In late 2025, I had agents write examples against my specification, which at the time insisted on lowercase type names and &lt;code&gt;--&lt;/code&gt; comments, among other idiosyncrasies. They drifted straight back to the syntax they&#39;d been trained on. An AI doesn&#39;t respect your novelty. It quietly averages it away. And nothing caught it, because nothing in my toolchain was checking. For a while there, my nonexistent language was trust-only.&lt;/p&gt;
&lt;h2&gt;Friction is feedback&lt;/h2&gt;
&lt;p&gt;Now look at Rust from the AI&#39;s side of the table.&lt;/p&gt;
&lt;p&gt;The borrow checker that humans spend months fighting is, to an AI, a strict senior reviewer that never gets tired and never lets anything slide. Together with the rest of rustc, it rejects whole categories of mistakes before anything runs: the use-after-free, the data race, the null that wasn&#39;t supposed to be there, the &lt;code&gt;match&lt;/code&gt; that forgot a case. And when it says no, it usually says what to do instead: &amp;quot;help: consider borrowing here&amp;quot;. Sometimes it even says &amp;quot;help: consider cloning the value if the performance cost is acceptable&amp;quot;, which makes rustc the only compiler I know of that will negotiate on price.&lt;/p&gt;
&lt;p&gt;The AI takes that offer every time, of course, and when it gets desperate, it reaches for &lt;code&gt;.unwrap()&lt;/code&gt; and &lt;code&gt;unsafe&lt;/code&gt;. But in Rust, trust is the exception, and it has to be asked for in writing. Every &lt;code&gt;.unwrap()&lt;/code&gt; is a bet placed in plain sight. &lt;code&gt;unsafe&lt;/code&gt; is a keyword you can grep for, and even inside an &lt;code&gt;unsafe&lt;/code&gt; block, the borrow checker goes on checking every reference. In Python, trust is the default, and it&#39;s the checking you have to ask for.&lt;/p&gt;
&lt;p&gt;A human fighting the borrow checker gets fed up after the tenth attempt and goes home. Whatever love and admiration they had for Rust is gone by then. Then again, these aren&#39;t the people who fill out Stack Overflow surveys. Who still uses Stack Overflow, anyway? The same people who still have Slashdot as their home page?&lt;/p&gt;
&lt;p&gt;An AI shrugs and makes the eleventh attempt, and the twelfth, at three in the morning, for pennies, without once losing its temper. What a human experiences as friction, an AI experiences as feedback.&lt;/p&gt;
&lt;p&gt;That&#39;s why, when I restarted my ideal language for AI programmers, I picked Rust as its first backend and as its first oracle, the independent judge of whether a program&#39;s ownership really holds up. I don&#39;t enjoy reading Rust. I just couldn&#39;t find a stricter reviewer. My own design notes concede that lifetimes are &amp;quot;where agents writing Rust spend most of their failed attempts&amp;quot;. But every one of those failures is caught at compile time, which is the only time you ever want anything to fail.&lt;/p&gt;
&lt;p&gt;So yes: Rust is write-only for humans, and it&#39;s the perfect language for AI. That makes Rust the natural predator in today&#39;s language pool. And when a human does need to understand some gnarly piece of it, the AI can explain it to them.&lt;/p&gt;
&lt;p&gt;Rust lovers, this is the good news I promised you. Your language won. Not that any of you have any business writing it any more. You&#39;re competing in an overflowing market of cheap AI labor that is faster and better than you even think you are.&lt;/p&gt;
&lt;p&gt;I do still have to review what the AI wrote, so haven&#39;t I just moved the problem? Not really. Most of what makes Rust hard to read is proof: lifetimes, ownership modes, trait bounds. Those are claims the compiler has already checked, and nobody needs to re-check a checked proof. What&#39;s left for me is the one question a human is actually needed for: is this what we wanted?&lt;/p&gt;
&lt;p&gt;There&#39;s a difference between code that&#39;s hard to read because it&#39;s full of proofs and code that&#39;s hard to read because it&#39;s full of tricks. You can skip a checked proof. You can&#39;t skip a trick. Rust has tricks too, mind you. &lt;code&gt;?&lt;/code&gt; quietly converts your errors, and a macro can hide almost anything. It just has fewer of them, and most of them announce themselves.&lt;/p&gt;
&lt;p&gt;Now compare that with reviewing AI-written Python. Python is far easier to read, and I&#39;d have to read every line of it, because nothing else has. In a trust-only language, the human reviewer is the type checker. And as &lt;a href=&quot;/v1/working-with-ai/programming-was-my-art&quot;&gt;I&#39;ve established elsewhere&lt;/a&gt;, the human reviewer is the slowest participant in the whole process.&lt;/p&gt;
&lt;p&gt;I&#39;d rather review AI-written Rust than AI-written Python, and I can barely read Rust.&lt;/p&gt;
&lt;h2&gt;The map&lt;/h2&gt;
&lt;p&gt;Put the two ideas together and you get a map. One axis asks whether a human can read the code. The other asks whether the language takes your word for it.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Humans can read it&lt;/th&gt;
&lt;th&gt;Write-only for humans&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Checks your work&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Go, C#, Java, Kotlin, Swift&lt;/td&gt;
&lt;td&gt;Rust, Haskell, Scala&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Takes your word for it&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Python, JavaScript, Ruby, PHP, C&lt;/td&gt;
&lt;td&gt;Perl, bash, APL, C++, Excel&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;Top left&lt;/strong&gt; is the boring corner. Humans have spent years mocking these languages for their ceremony: Java&#39;s &lt;code&gt;AbstractSingletonProxyFactoryBean&lt;/code&gt;, Go&#39;s &lt;code&gt;if err != nil&lt;/code&gt; on every third line. But ceremony costs an AI next to nothing, and a reviewer can skim straight past it. The boring languages may be having the last laugh. They&#39;re just typing it out in full.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Top right&lt;/strong&gt; is Rust&#39;s home: write-only for humans, excellent for AI, and fine for a reviewer, as long as the hard parts are proofs rather than tricks. Haskell lives there too, as long as nobody imports lens. Scala lives there uneasily, because its implicits are tricks.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bottom left&lt;/strong&gt; is where humans are happiest. They love these languages for exactly the reason they&#39;re weak for AI: they never argue.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bottom right&lt;/strong&gt; is Perl&#39;s home, and it has company. Write-only means no human can check the code. Trust-only means no compiler can. That leaves exactly one party checking it: the AI that wrote it.&lt;/p&gt;
&lt;h2&gt;The trade is off&lt;/h2&gt;
&lt;p&gt;Write-only languages trade the reader for the writer. Trust-only languages trade the checker for the writer. Both trades were made for the benefit of a writer with Larry Wall&#39;s three great virtues: laziness, impatience and hubris.&lt;/p&gt;
&lt;p&gt;The new writer is never lazy and never impatient. It does have the hubris. That&#39;s what the compiler is for.&lt;/p&gt;
&lt;p&gt;So the trade is off. The writer doesn&#39;t need the favors any more. The reader and the checker do. If I were designing a language from scratch today, and I am, I&#39;d want Python&#39;s readability with Rust&#39;s suspicion.&lt;/p&gt;
&lt;p&gt;My ideal language keeps its syntax boring, borrowed from Rust, Swift and Kotlin, because that&#39;s what agents already write and reviewers already read. It spends its novelty on what the compiler can check. Every failure is in the signature. Every wait has a deadline. Anything that asks for trust has to say so, in a closed vocabulary, with a reason, and the toolchain always shows it to a human. Every error that says no also says what to do instead, because a rule that only says no gets routed around.&lt;/p&gt;
&lt;p&gt;A language for machines to write and people to read. Neither write-only nor trust-only.&lt;/p&gt;
&lt;p&gt;Back in 1972, a joke chip made it into a data book because it passed through a long chain of approvals and none of them checked anything. Fifty-four years later, we can automate that chain. One AI writes the code, another reviews it, a third writes the tests, and in a trust-only language, every link in the chain takes the previous one&#39;s word for it.&lt;/p&gt;
&lt;p&gt;We&#39;ll find out it&#39;s broken the same way Signetics did.&lt;/p&gt;
&lt;p&gt;When the customers start calling.&lt;/p&gt;
&lt;p&gt;This time, they won&#39;t be asking for pricing.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Programming was my art</title>
    <link href="https://icelander.net/v1/working-with-ai/programming-was-my-art"/>
    <id>tag:icelander.net,2026:programming-was-my-art</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="working-with-ai" label="Working with AI"/>
    <summary>In four years, AI went from drawing pictures to writing better code than I do. This is how I retreated up the food chain, from programmer to creative director of a fleet of agents.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;A four-year retreat up the food chain&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought May 2026, written October 2026. Working with AI.&lt;/p&gt;&lt;p&gt;Back in ye olde times, I was a pretty productive developer, if I say so myself. I was fast, my output was vast, and it covered some genuinely complex domains.&lt;/p&gt;
&lt;p&gt;The work I did for EVE Online in its early days is still the most visible highlight of my career. Most of what I&#39;ve done since pales by comparison, but only from the outside. There are many factors that affect what fast looks like. Replacing ancient mission-critical code in national critical infrastructure takes effort and precision. So does removing the architectural scalability bottleneck of a running SaaS solution with zero downtime, while millions of concurrent users carry on regardless. Next to traditional website development, projects like that don&#39;t look fast at all. When they go well, they look suspiciously like nothing happened.&lt;/p&gt;
&lt;p&gt;After EVE Frontier, I took a career break for the sake of my health. When I came back, I was pretty happy to join a nuts-and-bolts organization with a long-lived, stable business model, wonderful coworkers, no stress (compared to what I&#39;m used to), and a business domain that touches the lives of all of society, every day. It was a comfortable, family-friendly, health-first place to work for the remainder of my career. It even had a 36-hour work week. I&#39;ve always had a work ethic that is too strong for my own good, and I&#39;m used to 50-plus hours during the ebbs and 70-ish during the flows.&lt;/p&gt;
&lt;p&gt;I had, more or less, solved my career.&lt;/p&gt;
&lt;p&gt;Then came AI.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;It had actually been coming for a while. I just hadn&#39;t taken it seriously. At first it was a novelty, but the novelty was doing real work, and it attacked first where everyone had assumed it would arrive last: art.&lt;/p&gt;
&lt;p&gt;In the spring of 2022, Midjourney arrived. Within a year, concept artists and illustrators were losing their jobs to it. A creative director could command an AI to produce hundreds of pieces of work from a prompt and some reference material, pick the best of them, process those further, and finally hand the results to humans. Artists at the lower levels of the food chain were being eaten by the machines.&lt;/p&gt;
&lt;p&gt;Interesting, I thought. Fortunately, I&#39;m a programmer.&lt;/p&gt;
&lt;p&gt;In late 2022, ChatGPT 3.5 made its debut.&lt;/p&gt;
&lt;p&gt;People quickly noticed that ChatGPT outperformed Google. Knowledge workers no longer had to spend hours sifting through dozens of articles, most of them unrelated to what they actually needed to know, just to piece together a vague picture of a puzzle whose missing pieces might be hiding in some dark corner of the internet their searches never reached.&lt;/p&gt;
&lt;p&gt;People also quickly learned to stop taking everything ChatGPT said on faith. AI lied and fabricated as it saw fit to tell people what they wanted to hear. Kind of like social media and search engines, actually, but more obviously so.&lt;/p&gt;
&lt;p&gt;But taken with a generously heaped spoonful of salt and distrust, ChatGPT could give a knowledge worker a far more accurate picture in a minute or so than hours of searching ever had. And they were entirely free to work, or not, on something else while it distilled the combined knowledge of humanity into the answer they needed.&lt;/p&gt;
&lt;p&gt;Many people seized on the hallucinations anyway, as an opportunity to mock AI and as proof of human superiority. Their jobs were safe.&lt;/p&gt;
&lt;p&gt;Except, of course, when they were talking about artists. That tale of Midjourney &amp;quot;secretly&amp;quot; winning an art competition, which got more secret with every retelling, was obvious proof to many of those same people that &lt;em&gt;other&lt;/em&gt; people&#39;s professions could be replaced by machines.&lt;/p&gt;
&lt;p&gt;ChatGPT evolved, as did its lesser-known competitors. Every single reader of this article will have seen it firsthand. Or they&#39;re too young to remember the world before AI.&lt;/p&gt;
&lt;p&gt;By the spring of 2023, AI was coding.&lt;/p&gt;
&lt;p&gt;Like so many developers at the time, I viewed it with disdain. It was by no means competitive in my eyes. My job was safe. Besides, I had migrated into management, so I was out of touch with the code anyway, and therefore doubly safe. Or so I thought.&lt;/p&gt;
&lt;p&gt;I was also on a career break at the time, and what little coding I did was purely for my own enjoyment. It&#39;s hard to lose a job you don&#39;t have.&lt;/p&gt;
&lt;p&gt;In the fall of 2024, I was getting back into the workforce. By then, ChatGPT had become a serious asset for my hobbyist writing. I had used it to write the full language and standard library reference for my aspirational &amp;quot;perfect programming language&amp;quot;, a hobby I&#39;d dabbled in since about 2001. As a coding assistant, though, it still wasn&#39;t much good. People were lauding Claude on Reddit, but to me that sounded like yet another religious war in our industry. Java versus the world, now with chatbots.&lt;/p&gt;
&lt;p&gt;By the spring of 2025, AI was starting to become a useful coding assistant. Most people I knew still ridiculed it and rejected it. I laughed a little less than they did, but I wasn&#39;t worried.&lt;/p&gt;
&lt;p&gt;Fall 2025. Codex changed the way I worked. It still wasn&#39;t great, but I could command it to do countless things for me in parallel, in the cloud, working directly on the code I was writing, while I focused on something else. The review burden was heavy because, well, it still lied and fabricated like it was running for office. But it was definitely a force multiplier. A low one, but a multiplier all the same.&lt;/p&gt;
&lt;p&gt;If anything, my job had just improved. I had interns now.&lt;/p&gt;
&lt;p&gt;November 2025. Claude Opus 4.5 came into my life as a shock. I had to admit that Claude, at maximum effort, was basically my equal. Only faster (token budget allowing), and at a fraction of my salary.&lt;/p&gt;
&lt;p&gt;Reddit had been right. I hate it when Reddit is right.&lt;/p&gt;
&lt;p&gt;For the first time in my career, my job was under review.&lt;/p&gt;
&lt;p&gt;May 2026. Claude Opus 4.8, under my supervision, was not only far faster and far cheaper than me. It was also far better than me.&lt;/p&gt;
&lt;p&gt;The title of my first cover letter was &amp;quot;Programming is my Art&amp;quot;.&lt;/p&gt;
&lt;p&gt;I had just lost to an AI in an art competition. By a landslide.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;For my whole career, the limit on my output had been hours: 70-ish in the flows, 50-plus in the ebbs, and lately a civilized 36. Now the limit was tokens.&lt;/p&gt;
&lt;p&gt;You have work queued up. You know exactly what needs doing, and you have a perfectly capable workforce standing by to do it. Then a session limit kicks in, or the weekly budget runs dry, and the entire workforce goes on unpaid leave at once. That&#39;s a token famine, and it&#39;s a trauma most programmers who are still on top of the game know far too well.&lt;/p&gt;
&lt;p&gt;If you&#39;re employed, you have one alternative to waiting, and it isn&#39;t a dignified one. You can go to whoever controls the company&#39;s AI budget, bowl in hand, and beg for a reset.&lt;/p&gt;
&lt;p&gt;&amp;quot;Please, sir, I want some more.&amp;quot;&lt;/p&gt;
&lt;p&gt;Tokens for the poor.&lt;/p&gt;
&lt;p&gt;During one particularly bad famine, with no reset forthcoming, I had a troubled conversation with ChatGPT, which runs on a different budget and was therefore still speaking to me. I asked it: &amp;quot;In the era of AI-assisted software development, what can I do that is more valuable than waiting until my token budget renews?&amp;quot;&lt;/p&gt;
&lt;p&gt;It answered. It lied. Eight of its ten suggestions were things I already delegated to Claude or Codex, depending on the token balance of the day. The other two were the equivalent of fetching a fresh cup of coffee. Actually, one of them literally was fetching a fresh cup of coffee.&lt;/p&gt;
&lt;p&gt;So I fetched a fresh cup of coffee.&lt;/p&gt;
&lt;p&gt;Then I re-reviewed a pull request I hadn&#39;t merged yet. Opus 4.8 had built it with TDD earlier that morning, at 99% branch coverage. Copilot had reviewed it. Codex had reviewed it. Their review comments, mostly nitpicks, had already been addressed. It was nigh-flawless, and it had been waiting all morning on the slowest participant in the whole process.&lt;/p&gt;
&lt;p&gt;Me.&lt;/p&gt;
&lt;p&gt;I looked at my &amp;quot;perfect programming language&amp;quot; with fresh eyes. I&#39;d been tinkering with it, on and off, for a quarter of a century, through a couple of names and more rewrites than I care to admit. The question behind it had always been the same: how should humans tell computers what to do?&lt;/p&gt;
&lt;p&gt;Suddenly I couldn&#39;t answer a much simpler one. Who was it for?&lt;/p&gt;
&lt;p&gt;Humans? Why?&lt;/p&gt;
&lt;p&gt;I parked it. There was no longer any point. Humans have no business writing code any more.&lt;/p&gt;
&lt;p&gt;That still left the famines to fill. I found two good uses for them, and both of them involved ChatGPT.&lt;/p&gt;
&lt;p&gt;The first was writing. Publishing an article is a perfectly productive use of a token famine, because a normal, free ChatGPT session can help with every step of it without touching the tokens my agents run on. A stronger model would do a better job, of course. But when you&#39;re too poor to program, you can still afford to publish.&lt;/p&gt;
&lt;p&gt;I&#39;ll let you draw your own conclusions about the article you&#39;re reading.&lt;/p&gt;
&lt;p&gt;The second was reading. I finally had time for all the things I&#39;d never been able to study at the depth I wanted, and ChatGPT turned out to be a tireless tutor. I could keep asking for the derivation, and then for the assumption underneath it, until the only limit left was how fast I could keep up. Some of that reading turned into a new hobby: amateur physics. I had been a physics and maths major back in ye olde times, before I switched to computer science. And contemporary AI could hardly scratch the surface of science, could it?&lt;/p&gt;
&lt;p&gt;Surely, this time, I had climbed far enough up the food chain.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;I&#39;m writing this in October 2026.&lt;/p&gt;
&lt;p&gt;My daily routine is to direct a fleet of agents as they do my bidding, until the session limits kick in or the tokens run out. My fleet, in turn, orchestrates fleets of its own.&lt;/p&gt;
&lt;p&gt;They build. They test. They review each other&#39;s work. They collaborate. They write the plans and the design documents, and keep the project&#39;s status and tasks up to date in Asana. They work on so many things in parallel that the merge conflicts are beyond anything a human could untangle, which is why they perform semantic merges of each other&#39;s work instead. They evaluate whether clients and servers are compatible, add compatibility measures where they&#39;re needed, and cherry-pick the changes into production.&lt;/p&gt;
&lt;p&gt;They monitor my production systems. They tell me about deviations and bugs I wasn&#39;t even aware of, and then they fix them. They migrate my development and staging servers and run nearly end-to-end tests against them.&lt;/p&gt;
&lt;p&gt;To make all of that possible, I&#39;ve introduced simulators for every relevant system boundary, giving the agents control over the environment they test in. Well, obviously I didn&#39;t introduce them. I asked Claude to.&lt;/p&gt;
&lt;p&gt;I&#39;ve even introduced &amp;quot;human simulators&amp;quot; (same caveat), so the agents can perform the tasks of users. Next, I plan to introduce Mechanical Turks, to let them orchestrate real users as well.&lt;/p&gt;
&lt;p&gt;After that, I suppose, the obvious next step is to ask Claude Code to wire-and-fire up an MCP server that hires temp workers directly from Upwork. When it farms out work to other agents, I usually tell it to use &amp;quot;the cheapest model and effort level that is certainly overkill for the job&amp;quot;. That same prompt could work just as well for humans. When a real human turns out to be the cheapest option that is certainly overkill for the job, the machines will simply hire one.&lt;/p&gt;
&lt;p&gt;I&#39;ve moved from AI-assisted development to human-assisted development.&lt;/p&gt;
&lt;p&gt;In AI-assisted development, the human does the work and the AI helps. In human-assisted development, the machines do the work, and the human supplies what they can&#39;t: intent, judgment, constraints, architecture, taste, priorities, skepticism, and the occasional &amp;quot;what the fuck did you do here?&amp;quot; My job is to decide what reality should look like, and to notice when the agents have built a different one.&lt;/p&gt;
&lt;p&gt;So when I say &amp;quot;I built this&amp;quot; these days, what I mean is: I asked Claude to.&lt;/p&gt;
&lt;p&gt;And the language? I&#39;ve started designing it again, for today&#39;s programmers, which are AIs, not humans. I stand by what I said: humans have no business writing code any more. But AIs have an entirely different skill set and entirely different needs. They never tire of typing, and ceremony costs them pennies. They&#39;ve read more code than any of us ever will, which means the syntax should be boring and the novelty should go where it pays: into what the compiler can check. Tell an AI &amp;quot;no&amp;quot;, and it will cheerfully find a way around the rule, up to and including deleting the test that said no, so every error message has to tell it what to do instead. And they still lie like they&#39;re running for office, only less often. Nothing they claim about their code can be taken on faith.&lt;/p&gt;
&lt;p&gt;So who is it for? Abelson and Sussman told a whole generation of computer science students that programs must be written for people to read, and only incidentally for machines to execute. They never said who would do the writing. Mine is a language for machines to write and people to read.&lt;/p&gt;
&lt;p&gt;Quite frankly, I have no idea what the upper limit is for a capable engineer practicing human-assisted development with a half-decent token budget. I&#39;m fairly sure it isn&#39;t linear. One engineer with ten agents is not eleven workers, because agents delegate to agents. They review each other, test each other, and build the tools and environments that let the next generation of agents take on more. Every improvement to the environment pushes more of the work further down the tree.&lt;/p&gt;
&lt;p&gt;With all the rigor you&#39;d expect from someone who abandoned physics for computer science, here is my current estimate:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;O(human judgment × (number of agents)&lt;sup&gt;recursive delegation&lt;/sup&gt; × quality of environment/tooling)&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Big-O notation conveniently ignores constant factors, which is how I&#39;ve managed to leave out the token budget.&lt;/p&gt;
&lt;p&gt;The exponent doesn&#39;t come free. You can only delegate what you can decompose, and you can only decompose what you understand, so how deep the tree goes is bounded by how deeply the human understands the problem. Expertise hasn&#39;t stopped mattering. It has become the exponent.&lt;/p&gt;
&lt;p&gt;Back in 2022, I watched creative directors command an AI to produce hundreds of pieces of art, keep the best, and hand the results to whichever humans were left. Fortunately, I thought, I&#39;m a programmer.&lt;/p&gt;
&lt;p&gt;I&#39;m the creative director now.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>There is no TODO</title>
    <link href="https://icelander.net/v1/processes/there-is-no-todo"/>
    <id>tag:icelander.net,2026:there-is-no-todo</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="processes" label="Processes"/>
    <summary>I deleted the folder called TODO and stopped trusting the backlog, because a backlog is a second map of the system that drifts by design. Knowledge-driven development keeps one map: documents that describe what the system should be, with every statement marked by whether it&#39;s true yet. It&#39;s an experiment, and it isn&#39;t battle-tested yet.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Knowledge-driven development, part 1: the backlog is a grep&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought September 2026, written October 2026. Processes.&lt;/p&gt;&lt;p&gt;On Friday, August 28, 2026, I deleted a directory called &lt;code&gt;TODO&lt;/code&gt;. Well, obviously I didn&#39;t delete it myself. I asked an agent to.&lt;/p&gt;
&lt;p&gt;It held 22 items, each one a small Markdown file describing something the system ought to do and didn&#39;t, or did and shouldn&#39;t. They were good items, carefully written and graded by severity from S1 down to S4. And every single one of them was in the wrong place.&lt;/p&gt;
&lt;p&gt;The 22 items now live in the nine documents that describe the capabilities they were about. The directory is gone. In its place is a test that fails the build if anyone creates a new one, or starts a comment in any source file with &lt;code&gt;TODO&lt;/code&gt;, &lt;code&gt;FIXME&lt;/code&gt;, &lt;code&gt;HACK&lt;/code&gt; or &lt;code&gt;XXX&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;That was the small backlog. The big one was in Asana: more than 200 tasks, rotting away, with nobody reading them and nobody updating them. Most of them were out of date, of course, the way backlogs always are. One asked for a document to be updated with the details of a production issue. The document had since been rewritten from scratch, and the production system decommissioned. Tickets have a half-life. Nobody has ever measured it, because measuring it would mean reading the tickets.&lt;/p&gt;
&lt;p&gt;Deleting the folder is the whole method in miniature. The Asana tasks are the whole problem, and not in miniature.&lt;/p&gt;
&lt;p&gt;Knowledge-driven development is an experiment. I started it that Friday in August, and it isn&#39;t battle-tested yet. This series describes what I&#39;m doing and why.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;1. The backlog is a second map&lt;/h2&gt;
&lt;p&gt;Every software team I&#39;ve worked with has kept two descriptions of the system. One is the system itself, plus whatever documentation survived contact with it. The other is the backlog: tickets, epics, stories, a board with columns. The first describes what is. The second describes what should be, minus what is, as of whenever someone last updated it.&lt;/p&gt;
&lt;p&gt;These are two maps of the same territory, drawn by different people, for different readers, in different tools, and updated by different acts. Changing the code is one act. Closing the ticket is another. Only one of them is enforced by anything, and it isn&#39;t the ticket.&lt;/p&gt;
&lt;p&gt;So the maps drift. A ticket says a thing is broken that was fixed three sprints ago. A feature exists that no ticket ever mentioned, because someone built it on a Friday. The board says 80% done and the system says otherwise. &lt;a href=&quot;https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation&quot;&gt;The map is not the territory&lt;/a&gt;, and the backlog is a second map, drawn from memory, of the first one.&lt;/p&gt;
&lt;p&gt;And the people drawing it are trying to look good and justify their paychecks.&lt;/p&gt;
&lt;p&gt;Two descriptions updated by two acts will diverge, however disciplined the people updating them are, and a description kept by the people it&#39;s used to judge will flatter them. The usual cure is more discipline anyway: grooming sessions, ticket hygiene, a definition of done that includes closing the ticket. Every team that has ever run a ticket tracker has paid for that, usually in meetings.&lt;/p&gt;
&lt;p&gt;Processes that aren&#39;t valued by the people who have to put in the effort feel like imprisonment. The prisoners escape, and the guards are left humiliated and confused.&lt;/p&gt;
&lt;p&gt;There&#39;s a scene in &lt;a href=&quot;https://en.wikipedia.org/wiki/The_Matrix&quot;&gt;The Matrix&lt;/a&gt; where a child bends a spoon just by looking at it, then tells Neo not to try bending it himself, because that&#39;s impossible. Instead, he should realize the truth: &amp;quot;There is no spoon.&amp;quot;&lt;/p&gt;
&lt;p&gt;The backlog is the spoon.&lt;/p&gt;
&lt;h2&gt;2. The backlog is a queue for hands you don&#39;t have&lt;/h2&gt;
&lt;p&gt;Drift is the first problem with a backlog. The second is that it no longer has a reason to exist.&lt;/p&gt;
&lt;p&gt;A backlog, and the sprint planning built on top of it, is a tool for managing work that isn&#39;t being done because there are no hands and brains to do it. Sometimes for years. The work waits in a queue, and every two weeks somebody grooms the queue.&lt;/p&gt;
&lt;p&gt;An agent can finish that work as fast as a developer can guide it and feed it tokens. In my case, the limiting factor should be the developer. It isn&#39;t, because my organization doesn&#39;t feed me enough tokens for that to be true. Organizations are comfortable paying salaries to add hands. They&#39;re not comfortable paying for tools that multiply those hands at a lower cost.&lt;/p&gt;
&lt;p&gt;Still, hands and brains were the bottleneck the backlog was built to manage, and they aren&#39;t any more. The bottleneck now is a line in a budget. Nobody has told the backlog.&lt;/p&gt;
&lt;p&gt;Given a serious development team, the backlog is pointless.&lt;/p&gt;
&lt;p&gt;So are the planning rituals around it. The entire design of a major new system could be implemented before the planning-and-cost-assessment phase is over. Building the thing is now faster than estimating it, which turns the cost assessment into a historical document. The result can be reviewed in a real environment, and adjusted to meet the stakeholders&#39; needs in the next phase.&lt;/p&gt;
&lt;p&gt;What that takes is a description of what the system is meant to be, and an account of how much of it is true yet.&lt;/p&gt;
&lt;h2&gt;3. Knowledge-driven development is one map&lt;/h2&gt;
&lt;p&gt;Knowledge-driven development removes the second description. What&#39;s left is one description, and it&#39;s the spec: everything the system wants to be in its absolute end state, as far as anyone currently knows it. It&#39;s written in the present tense, as if all of it were already true, and every statement carries a mark saying whether it is yet. A ⚠️ paragraph marks the gap between what the system should do by design and what has been built so far, written in the voice of the person who will use it, in the place they&#39;ll look for it. A ❌ paragraph is a bug report, in the place the bug will be found. A ✅ paragraph is a claim someone has verified, and the only kind you&#39;re allowed to build on.&lt;/p&gt;
&lt;p&gt;The description is split by audience, not by tool. The product owner gets the feature documents. The operator gets the operator&#39;s manual. The integrator gets the API contract. The maintainer gets all of those, plus the architecture: the things that really should be documented about a system. A class tree isn&#39;t one of them. Any IDE will draw you one of those out of the box.&lt;/p&gt;
&lt;p&gt;The specification, the backlog, the release notes and the onboarding material are the same set of facts, filtered for different readers. They always were. We just kept them in four places and paid four times to keep them roughly in sync, which we didn&#39;t.&lt;/p&gt;
&lt;p&gt;When I want to know every known bug in the system, I don&#39;t open a board. I ask Claude, and Claude runs this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;grep -rn &#39;❌&#39; documentation/
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And when I want to know what&#39;s left to build:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;grep -rn &#39;⚠️&#39; documentation/
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That&#39;s the closest thing I have to a backlog, and it&#39;s a grep. It can&#39;t drift from the documentation, because it &lt;em&gt;is&lt;/em&gt; the documentation. Nothing on it is waiting for hands, either. It gets shorter as fast as I&#39;m allowed to feed the agents tokens.&lt;/p&gt;
&lt;p&gt;I call it knowledge-driven development on purpose. &amp;quot;Documentation&amp;quot; makes people think of the thing written after the code, by nobody, read by nobody, and out of date before the pull request merges. These documents are written before the code, read on every change, and they&#39;re the only description of the system that&#39;s allowed to exist. They&#39;re what the system knows about itself, including what it knows it can&#39;t do yet.&lt;/p&gt;
&lt;h2&gt;4. Files beat tickets&lt;/h2&gt;
&lt;p&gt;Agents don&#39;t read tickets. Not unless you ask them to, and even then a ticket is a poor way of communicating with one. Files are better.&lt;/p&gt;
&lt;p&gt;An agent&#39;s context is loaded from files. The implementing agents work from the documents the planner wrote, and the repository&#39;s agent instructions tell every agent to read the documents about whatever it&#39;s changing. A ⚠️ in the document about the capability being changed gets read whether anyone remembers it or not. A ticket about the same capability gets read when somebody remembers it exists, finds it, and points the agent at it, and what the agent gets then is a description written for a planning meeting, about code that may have changed since. Putting the requirement where the agent will read it is the difference between a rule that applies and a rule that applies when somebody remembers it.&lt;/p&gt;
&lt;p&gt;For a human team, a ticket tracker is a tolerable bad idea. Humans don&#39;t really remember the ticket they read on Monday when they&#39;re editing the file on Thursday, but they can usually find it again, and they can claim they&#39;re working on it. What they find is rarely worth the search. In most teams, a human-written ticket is very often nothing but a subject line, or a single paragraph if you&#39;re lucky. It doesn&#39;t, in any form or fashion, convey the intent and context needed to resolve it correctly.&lt;/p&gt;
&lt;p&gt;Most &lt;code&gt;TODO&lt;/code&gt; items and most tickets are of no interest to most of the organization anyway. A ticket tracker is a road atlas of the entire country, kept in another building, where every reader cares about one page and it&#39;s a different page for each of them. A ⚠️ in the document the agent is reading is a sign on the corner where you&#39;re standing.&lt;/p&gt;
&lt;p&gt;Agents write nearly all of my code and nearly all of my tests. They read the signs every time, because the signs are in the documents they read before they work. They&#39;d read the atlas too, if I sent them to the other building.&lt;/p&gt;
&lt;p&gt;I don&#39;t.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;There is no separate backlog. The 22 items that used to sit in a folder called &lt;code&gt;TODO&lt;/code&gt; are now sentences in the documents that describe what they&#39;re about, each one marked, each one findable by grep, each one in the place an agent will read it at the moment it matters. So are the &lt;code&gt;TODO&lt;/code&gt; comments that used to be scattered through the source files, each one moved into the document that owns the code it sat in. Whatever wisdom the Asana tasks contained has been distilled and stored in the repository as well. (Same caveat.) The tasks that were too detailed for Asana are gone from it. What&#39;s left there are the major work items, and each one refers to the document that describes it.&lt;/p&gt;
&lt;p&gt;My agents do still visit the other building now and then, to update Asana for the project managers and product owners who live there. I hear it&#39;s lovely this time of year. They go there to write, not to read. Asana is still a second map. It&#39;s just not the one the code is built from.&lt;/p&gt;
&lt;p&gt;The rest of this series is how that works in practice: &lt;a href=&quot;/v1/processes/seven-marks-three-monkeys&quot;&gt;the marks&lt;/a&gt;, &lt;a href=&quot;/v1/processes/the-plan-is-where-the-work-lives&quot;&gt;how I run a feature through them&lt;/a&gt;, &lt;a href=&quot;/v1/processes/what-it-replaces&quot;&gt;what they replace&lt;/a&gt;, and &lt;a href=&quot;/v1/processes/where-it-breaks&quot;&gt;where they break&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The folder is gone, and the build will fail if it comes back.&lt;/p&gt;
&lt;p&gt;There is no TODO.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Four marks</title>
    <link href="https://icelander.net/v1/processes/seven-marks-three-monkeys"/>
    <id>tag:icelander.net,2026:seven-marks-three-monkeys</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="processes" label="Processes"/>
    <summary>Knowledge-driven development has four marks, saying whether a statement is verified, missing, broken or blocked. Each one goes on the smallest statement it&#39;s about, so the dragons are only where they live.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Knowledge-driven development, part 2: here be dragons, and nowhere else&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought September 2026, written October 2026. Processes.&lt;/p&gt;&lt;p&gt;&lt;em&gt;Knowledge-driven development is an experiment of mine, and it isn&#39;t battle-tested yet. &lt;a href=&quot;/v1/processes/there-is-no-todo&quot;&gt;Part 1&lt;/a&gt; says where it started.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Medieval cartographers almost never wrote &amp;quot;here be dragons&amp;quot; on their maps. As far as anyone can tell, &lt;a href=&quot;https://en.wikipedia.org/wiki/Here_be_dragons&quot;&gt;exactly one surviving globe&lt;/a&gt; does it, in Latin, somewhere off the coast of Asia. Everywhere else, the unknown was just left blank, or decorated with a sea monster and a shrug.&lt;/p&gt;
&lt;p&gt;That&#39;s still one globe more than most backlogs. A ticket tracker tells you what someone intended to build. It doesn&#39;t tell you, standing in a particular corner of the system, whether the ground under your feet has been surveyed.&lt;/p&gt;
&lt;p&gt;The marks started with a few hundred Asana tasks. I was looking at them, realized they were doing more harm than good, and decided I wanted a format that was more concise, and of value in other ways as well.&lt;/p&gt;
&lt;p&gt;Knowledge-driven development is mostly a vocabulary for that. It has four marks.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;1. The four marks&lt;/h2&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Mark&lt;/th&gt;
&lt;th&gt;Means&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Implemented and verified.&lt;/strong&gt; The behavior exists and someone has seen its tests pass.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;⚠️&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Not implemented.&lt;/strong&gt; The sentence describes the intended behavior. Nothing behind it works yet. This is the backlog item.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;❌&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Implemented but broken.&lt;/strong&gt; A known defect.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;⛔&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Blocked.&lt;/strong&gt; The gap marked just before it can&#39;t be closed from here. Always follows a ⚠️ or ❌, never stands alone, and always says what would unblock it.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;In map terms: ✅ is surveyed land. ⚠️ is here be dragons. ❌ is the bridge that&#39;s on the map and isn&#39;t in the river. ⛔ is a road closed by somebody else&#39;s council, with a sign saying what has to happen before it reopens.&lt;/p&gt;
&lt;p&gt;The mark goes on the smallest statement that is wholly true or wholly false. A section heading carrying one ⚠️ for eleven paragraphs, nine of which work, is worse than no mark at all, because it makes the two broken paragraphs unfindable. A bullet takes the mark at the start of the bullet. A paragraph takes it at the start of the sentence that&#39;s untrue, not at the start of the paragraph.&lt;/p&gt;
&lt;h2&gt;2. The mark is the whole commentary&lt;/h2&gt;
&lt;p&gt;You write the sentence as the system &lt;em&gt;should&lt;/em&gt; be, in the present tense, in the detail someone would need to use it. You mark it. Then you stop. You don&#39;t explain the mark. No &amp;quot;what it does instead&amp;quot;, no &amp;quot;the customer currently sees&amp;quot;, no account of how you found out. The glyph already says &lt;em&gt;this sentence is not true yet&lt;/em&gt;. A paragraph restating that in prose turns a specification back into a bug report, which is the thing the page replaced.&lt;/p&gt;
&lt;p&gt;The instinct runs the other way. A page that just cost somebody a day of debugging wants to &lt;em&gt;say so&lt;/em&gt;. It mustn&#39;t. The incident goes in the changelog entry that introduced the mark, and the diagnosis goes in the pull request that fixes it. What the page owes its reader is the sentence they can build against, and one glyph telling them whether to trust it.&lt;/p&gt;
&lt;p&gt;An ⚠️ that says &amp;quot;not implemented&amp;quot; is worthless. An ⚠️ that describes the intended behavior is a specification a cheaper agent can implement from. Its quality bar is that an agent can build from it without asking much. The mark is metadata. The paragraph is the deliverable.&lt;/p&gt;
&lt;h2&gt;3. ✅ is expensive on purpose&lt;/h2&gt;
&lt;p&gt;✅ may only be written by someone who has read the code and seen the tests pass. A plan saying so doesn&#39;t count, and neither does an agent reporting success or a pull request merging.&lt;/p&gt;
&lt;p&gt;That&#39;s the strictest mark in the convention, because ✅ is the one readers build on.&lt;/p&gt;
&lt;h2&gt;4. Graduation&lt;/h2&gt;
&lt;p&gt;When a document contains no ⚠️ and no ❌, every ✅ in its body is deleted. The status link at the top is replaced by one line, carrying the page&#39;s only green check:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;✅ This component is fully implemented, tested and free of known defects.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;A page covered in green ticks is a page that has stopped carrying information. The ticks only ever existed to make the gaps legible, so when the gaps go, the ticks go with them. Nobody stamps &amp;quot;surveyed&amp;quot; on every hill of a finished map. One stamp on the cover will do.&lt;/p&gt;
&lt;p&gt;Graduation is what keeps the marked part of the corpus small enough to stay true. A convention that only ever adds marks becomes wallpaper, and nobody reads wallpaper. A fully implemented, defect-free document carries one mark, by its status line, which is what most documents should eventually look like, and why a page with marks in its body is worth stopping on.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Here be dragons, then. But only where there are dragons, only on the square of the map where they live, and with a description of the dragon detailed enough that someone could build it.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The plan is where the work lives</title>
    <link href="https://icelander.net/v1/processes/the-plan-is-where-the-work-lives"/>
    <id>tag:icelander.net,2026:the-plan-is-where-the-work-lives</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="processes" label="Processes"/>
    <summary>My plan for a feature is one prompt. A planning agent writes the documentation of the system as it will be, marked as not yet true, and the agents that build it turn the marks green in the same commits as the code. The review is the marks that moved.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Knowledge-driven development, part 3: twenty lines of Markdown, two thousand lines of trust&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought September 2026, written October 2026. Processes.&lt;/p&gt;&lt;p&gt;&lt;em&gt;Knowledge-driven development is an experiment of mine, and it isn&#39;t battle-tested yet. &lt;a href=&quot;/v1/processes/there-is-no-todo&quot;&gt;Part 1&lt;/a&gt; says where it started.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Here is the entire prompt I give a planning agent when I want a feature built:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Plan how to implement this feature, and write it up using knowledge-driven development.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;It goes to a top-tier model at high effort, with whatever description of the feature I&#39;ve got. That&#39;s usually a paragraph. Sometimes it&#39;s a sentence. I&#39;ve written longer commit messages.&lt;/p&gt;
&lt;p&gt;What comes back is the documentation of the system as it will be once the feature exists, written in the present tense, with almost every new sentence marked ⚠️. That&#39;s the plan.&lt;/p&gt;
&lt;p&gt;It&#39;s also the best way I&#39;ve found to keep a fleet of agents pointed in the same direction.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;1. What the planning agent writes&lt;/h2&gt;
&lt;p&gt;The agent doesn&#39;t produce a list of steps. It edits the documents that already describe the system, or creates the ones that should exist, and describes the feature &lt;em&gt;in them&lt;/em&gt;.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The feature document gets a section describing the new capability from the point of view of whoever will use it, statement by statement, each one ⚠️.&lt;/li&gt;
&lt;li&gt;The user journeys that pass through the feature get new steps, each one ⚠️.&lt;/li&gt;
&lt;li&gt;The subsystem documents get the new ways things can fail: what goes wrong, what it looks like from the outside, and what an operator does about it. All ⚠️.&lt;/li&gt;
&lt;li&gt;The API contract gets the new operations, with their request and response shapes, each one marked ⚠️.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If something can&#39;t be built from where we are (it needs a decision from someone outside, a piece of hardware that isn&#39;t on my desk, a third party that hasn&#39;t shipped), the agent writes it as ⚠️⛔ and says what would unblock it. That&#39;s how blockers reach me: as a sentence in the document that owns them.&lt;/p&gt;
&lt;p&gt;The quality bar for those ⚠️ paragraphs is that an agent can build from them without asking much. They don&#39;t need to be perfect. The agent is fast, and I can review what it built fairly quickly afterwards and correct its direction.&lt;/p&gt;
&lt;h2&gt;2. What the work orders used to be&lt;/h2&gt;
&lt;p&gt;I didn&#39;t always do it this way. Until recently I had agents write plans as work orders for the fleet: a list of self-contained steps, each with its own definition of done, with the parallel steps marked and the riskiest step first.&lt;/p&gt;
&lt;p&gt;They worked. They were also a second map.&lt;/p&gt;
&lt;p&gt;A work order describes a sequence of changes. Once the changes are made, it describes nothing. It becomes a historical document about a plan, sitting next to a system that has since moved on. The next agent to touch the feature doesn&#39;t read it, because nothing points to it, and if it did read it, the work order would tell it what someone intended three weeks ago, not what&#39;s true now.&lt;/p&gt;
&lt;p&gt;A knowledge-driven plan doesn&#39;t expire, because it&#39;s the description of the system, and implementing it means making its sentences true. When the last ⚠️ in a feature turns into ✅, nothing gets archived. The plan is the documentation now, which is what it was all along.&lt;/p&gt;
&lt;p&gt;The work orders are gone. I don&#39;t miss them.&lt;/p&gt;
&lt;h2&gt;3. What the implementing agents do&lt;/h2&gt;
&lt;p&gt;Every agent that comes after the planner treats the design documents as its primary guidance, ahead of my original paragraph, the chat history or any ticket.&lt;/p&gt;
&lt;p&gt;That works because of how the documents reach the agents. The implementing agents are handed the documents the planner wrote, and the repository&#39;s agent instructions tell every agent to read the documents about whatever it&#39;s changing. Nobody has to remember to point the agent at them.&lt;/p&gt;
&lt;p&gt;An implementing agent works like a Mars rover. When NASA&#39;s &lt;a href=&quot;https://www-robotics.jpl.nasa.gov/media/documents/SAPP_paper.pdf&quot;&gt;Spirit and Opportunity&lt;/a&gt; had no attitude estimate worth trusting, they worked one out from scratch: the accelerometers told them which way was down, and a camera searched the sky for the sun to tell them which way they were facing. &lt;a href=&quot;https://www.jpl.nasa.gov/images/perseverances-sol-200-auto-nav-drive&quot;&gt;Perseverance&lt;/a&gt; drives the same way. It keeps imaging the terrain around it, maps it, and chooses its own path to the destination, around whatever rocks it finds.&lt;/p&gt;
&lt;p&gt;None of them needs the story so far. They look around, work out where they are, and go. My agents start from the same nothing. They don&#39;t need the chat history, the plan so far or my original paragraph. They read the documents, find the ⚠️ marks, and start building. It works brilliantly.&lt;/p&gt;
&lt;p&gt;The one rule they must follow is that the mark moves in the same commit as the code: shipping a capability turns its ⚠️ into ✅ in the same pull request.&lt;/p&gt;
&lt;p&gt;Defects don&#39;t come out of the commits. They turn up in user testing, or arrive in a chat message, an email, an Asana task or a phone call. More and more often I find them myself, because my systems have good observability: I ask an agent what went wrong since yesterday, and then I tell it to fix it. Wherever a defect comes from, the usual answer is the same. Claude diagnoses it and fixes it, and the defect is gone before anyone thinks of writing it down. Defects are short-lived in an AI-driven world, tokens allowing.&lt;/p&gt;
&lt;p&gt;When a fix has to wait, I ask Claude to update the knowledge documents instead. That&#39;s what a ❌ is for: a known defect that&#39;s still there.&lt;/p&gt;
&lt;p&gt;This also means the implementing agents can be cheaper than the planner. The planner did the judgment: what the feature &lt;em&gt;is&lt;/em&gt;, where it belongs, how it fails. The implementers are translating well-specified sentences into code and tests, and that&#39;s work a smaller model at lower effort does well.&lt;/p&gt;
&lt;h2&gt;4. The backlog and the sprint board&lt;/h2&gt;
&lt;p&gt;I don&#39;t have a separate backlog. My project manager does have a sprint board, and I keep it up to date. Well, Claude does. But it&#39;s a second map, and however well it&#39;s kept, it&#39;s never going to be a holistic picture of what&#39;s actually going on in the code base.&lt;/p&gt;
&lt;p&gt;My own sprint board is a grep that Claude runs for me. The backlog is every ⚠️ in the repository. The sprint is the ⚠️ marks in the documents the planning agent just wrote. Progress is the count of those marks going down. Something blocked is ⚠️⛔ and says what it&#39;s waiting on. A defect that&#39;s waiting for a fix is a ❌, in the place the defect lives.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;grep -rn &#39;⚠️&#39; documentation/Features/my-new-feature.md
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It updates itself in the same commits as the code, which is more than anyone can say for the other one.&lt;/p&gt;
&lt;h2&gt;5. How I review&lt;/h2&gt;
&lt;p&gt;A feature pull request built by agents can easily be two thousand lines of code and tests. I can&#39;t review two thousand lines of anything with any real attention, and nobody else can either, whatever they put in their performance review.&lt;/p&gt;
&lt;p&gt;So I don&#39;t start there. I start with two short things: the delta to the knowledge files, and the change logs.&lt;/p&gt;
&lt;p&gt;That diff is usually short. Twenty lines, maybe forty. It tells me, in plain sentences, what the agent claims to have done: these ⚠️ became ✅, this ❌ is a defect left for later, this one is now ⚠️⛔ and says what it&#39;s waiting on. Every mark that moved is a claim, and every claim names the behavior it&#39;s about.&lt;/p&gt;
&lt;p&gt;The change logs come from a rule my agents follow: every prompt that changes the code base gets a change log file. It says what I asked for and how the agent interpreted it, and, where it matters, what the change needs: how to test it, data to migrate, compatibility issues. I read those carefully, because they tell me whether the agent misunderstood me.&lt;/p&gt;
&lt;p&gt;In a forest of agent-made changes, those two are the most concise way I know to see what a pull request thinks it&#39;s changing. Sometimes that&#39;s enough to show that the agent got me wrong. One decided to write a data migration for something I could have done by opening a web page and toggling a feature flag.&lt;/p&gt;
&lt;p&gt;Then the claims get checked, because a ✅ means someone read the code and saw the tests pass, and agents report success the way a teenager reports that their room is clean.&lt;/p&gt;
&lt;p&gt;I don&#39;t check them myself. That would be too much work. I ask another agent to do it.&lt;/p&gt;
&lt;p&gt;Sending one teenager to inspect another teenager&#39;s room sounds like a terrible idea. It works because this one has never met the first one.&lt;/p&gt;
&lt;p&gt;Every pull request that ships a capability carries a documentation hunk. That&#39;s a feature: it&#39;s the only part of the diff written for the reviewer. A pull request that changes behavior and moves no mark is either not visible to anyone, or has left the documentation lying. Either way, it&#39;s the first question the review asks.&lt;/p&gt;
&lt;p&gt;Reading the knowledge delta and the change logs first turns review from reading two thousand lines of code and guessing what they were for, into reading a few dozen lines of claims and intent, and handing the checking to an agent. That&#39;s a job I can actually do.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;The plan isn&#39;t a document that gets written and forgotten. It&#39;s where the work lives.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>What it replaces</title>
    <link href="https://icelander.net/v1/processes/what-it-replaces"/>
    <id>tag:icelander.net,2026:what-it-replaces</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="processes" label="Processes"/>
    <summary>Knowledge-driven development replaces the TODO comment, the backlog, plans, separate specs and hand-written release notes, because each was a second description of the system that drifted. It keeps the things that record the past or can be run as tests.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Knowledge-driven development, part 4: a short obituary for the ticket&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought September 2026, written October 2026. Processes.&lt;/p&gt;&lt;p&gt;&lt;em&gt;Knowledge-driven development is an experiment of mine, and it isn&#39;t battle-tested yet. &lt;a href=&quot;/v1/processes/there-is-no-todo&quot;&gt;Part 1&lt;/a&gt; says where it started.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Most of what knowledge-driven development does, it does by deleting something. The &lt;code&gt;TODO&lt;/code&gt; folder went first. The work orders went next. Then my backlog, my sprint board and most of what a ticket tracker is for.&lt;/p&gt;
&lt;p&gt;They were all designed for a team of humans who remember things between Monday and Thursday. That&#39;s not the team I have any more. But each of them did something useful, and the useful part had to go somewhere when the rest went in the bin.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;1. The ticket tracker&lt;/h2&gt;
&lt;p&gt;A ticket tracker keeps the intent outside the repository, which is why it &lt;a href=&quot;/v1/processes/there-is-no-todo&quot;&gt;drifts&lt;/a&gt;. For agents, it&#39;s worse than drift. A ticket is invisible at the moment of editing. The agent changing the code never sees it, however carefully someone wrote it.&lt;/p&gt;
&lt;p&gt;And it usually wasn&#39;t written carefully. A typical ticket looks like this:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Export fails sometimes&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Accounting says some nights the export is missing. Can we look at this?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That&#39;s a subject line and a shrug. It doesn&#39;t say what the export should do, what &amp;quot;missing&amp;quot; means, or what anyone has already tried. Hand it to an agent, and the agent will fix &lt;em&gt;something&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;The same problem, in the knowledge document that describes the export:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;❌ When the accounting system is unavailable at 02:00, the export retries every 15 minutes, and the night&#39;s records are sent once it&#39;s back.&lt;/p&gt;
&lt;p&gt;⚠️ If the export still hasn&#39;t succeeded by 06:00, the operator on call gets an alert.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The ❌ says the retry exists and doesn&#39;t work. The ⚠️ says the alert hasn&#39;t been built yet. Neither explains itself, because the sentence is the specification, and an agent can build from it. Both sit in the page an agent reads before it touches the export.&lt;/p&gt;
&lt;p&gt;On my team, &lt;em&gt;who&lt;/em&gt; is me and a fleet of agents, and &lt;em&gt;when&lt;/em&gt; is now. So the tracker doesn&#39;t own anything the code depends on. My agents still keep Asana up to date for the project managers and product owners who use it, but nothing in the repository ever reads it back. It&#39;s a second map, and they only ever write to it.&lt;/p&gt;
&lt;p&gt;The project managers loved the status updates, which are current and tailored to what they need. Then they had a heart attack over the number of tasks the agents had added to the current sprint. Now the agents keep only the major work items in the sprint. The countless microtasks go into the knowledge documents, not into Asana, and the Asana task refers to the document instead of repeating it.&lt;/p&gt;
&lt;p&gt;Many of those microtasks aren&#39;t worth keeping at all: an agent can work out the small steps for itself when it implements the feature. And once a microtask is done, the code is the record of it.&lt;/p&gt;
&lt;h2&gt;2. The &lt;code&gt;TODO&lt;/code&gt; comment&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;TODO&lt;/code&gt; is a mark that failed to say anything. It records that somebody was unhappy at that line, and nothing else: not whether the code works, whether it&#39;s broken, what it should do instead or who could fix it.&lt;/p&gt;
&lt;p&gt;Like plenty of other programmers, I&#39;ve stumbled over this periodically since the stone age of computing: a few hours spent debugging yet another production problem, and then the &lt;code&gt;TODO&lt;/code&gt; comment that had identified it all along.&lt;/p&gt;
&lt;p&gt;So every &lt;code&gt;TODO&lt;/code&gt; comment in the source moved into the knowledge document that owns its code, as a ⚠️ or ❌ that says what the comment never did. Before:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-csharp&quot;&gt;// TODO: handle timeouts
var result = await accounting.SendAsync(batch);
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;After, in the export&#39;s document:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;❌ When the accounting system doesn&#39;t answer within 30 seconds, the export abandons the attempt and schedules a retry.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The comment knew there was a problem, and kept it to itself. The sentence says what the system should do about it, in the place someone will read it.&lt;/p&gt;
&lt;p&gt;The knowledge-driven vocabulary is &lt;code&gt;TODO&lt;/code&gt; with a type system and a reader in mind. ⚠️ says it isn&#39;t built and describes what it should be. ❌ says it&#39;s built wrong and states the correct behavior. ⛔ says why it can&#39;t be fixed from here, and what would unblock it.&lt;/p&gt;
&lt;p&gt;So &lt;code&gt;TODO&lt;/code&gt; isn&#39;t discouraged. It&#39;s a build failure.&lt;/p&gt;
&lt;h2&gt;3. Plans and work orders&lt;/h2&gt;
&lt;p&gt;My agents used to write work orders: numbered steps, each with its own definition of done, the riskiest first and the parallel ones marked. A step looked like this:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Step 4 of 7.&lt;/strong&gt; Add retry logic to &lt;code&gt;ExportJob&lt;/code&gt;: every 15 minutes until 06:00. Done when the unit tests pass. Can run in parallel with step 5.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The same intent, written the knowledge-driven way:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;⚠️ When the accounting system is unavailable, the export retries every 15 minutes until 06:00.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The step was true for one afternoon. Once it was done, it described a piece of history that nobody would read again. The sentence stays true for as long as the export works that way, and the day someone changes the retry interval, the sentence changes with it.&lt;/p&gt;
&lt;p&gt;The ordering and the notes about what can run in parallel went too. The implementing agents work those out from the documents, like everything else, which is the subject of &lt;a href=&quot;/v1/processes/the-plan-is-where-the-work-lives&quot;&gt;part 3&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;4. Specs and README-driven development&lt;/h2&gt;
&lt;p&gt;&lt;a href=&quot;https://tom.preston-werner.com/2010/08/23/readme-driven-development.html&quot;&gt;README-driven development&lt;/a&gt; was the right idea at the wrong scale. A README is one document for one audience: the person deciding whether to use the thing. It says nothing about how the system fails, how to operate it, or what&#39;s half built. It covers the front door. Knowledge-driven development is the same idea applied to every audience at once, with marks, so the documents survive past version 0.1.&lt;/p&gt;
&lt;p&gt;Spec-driven development has the opposite problem. The spec is a separate artifact, read once by the agent that implements it and never again. Six months later, the spec says what the feature was meant to be in March, and the code says what it became. Nobody reconciles the two, because nobody opens the spec. In knowledge-driven development, the spec &lt;em&gt;is&lt;/em&gt; the manual, and it keeps being read, because it&#39;s what people and agents open to find out how the system works. Being read is the only thing that has ever kept a document current.&lt;/p&gt;
&lt;h2&gt;5. Release notes and onboarding&lt;/h2&gt;
&lt;p&gt;These fall out for free. The release notes for a change are the marks that moved. For the export fix above, they&#39;re the diff to one page:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;✅ When the accounting system doesn&#39;t answer within 30 seconds, the export abandons the attempt and schedules a retry.&lt;/p&gt;
&lt;p&gt;✅ When the accounting system is unavailable at 02:00, the export retries every 15 minutes, and the night&#39;s records are sent once it&#39;s back.&lt;/p&gt;
&lt;p&gt;⚠️ If the export still hasn&#39;t succeeded by 06:00, the operator on call gets an alert.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Two fixed, one still to build. Nobody had to write that up, and nobody had to check that the write-up matched the code.&lt;/p&gt;
&lt;p&gt;Onboarding is reading the documents for the part of the system you&#39;re about to touch. They&#39;re the only description of the system there is, and they say what doesn&#39;t work.&lt;/p&gt;
&lt;p&gt;A new engineer, human or otherwise, doesn&#39;t have to learn which parts of the wiki to trust. They can grep for the parts not to.&lt;/p&gt;
&lt;h2&gt;6. What stays&lt;/h2&gt;
&lt;p&gt;&lt;a href=&quot;https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions&quot;&gt;Architecture decision records&lt;/a&gt; and the changelog stay. They record the past, and the past doesn&#39;t drift. They never get marks, because a ✅ in a document about the past would be a claim about the present in the wrong place.&lt;/p&gt;
&lt;p&gt;Executable specifications stay, and win where they apply. A &lt;a href=&quot;https://docs.python.org/3/library/doctest.html&quot;&gt;doctest&lt;/a&gt; or a &lt;a href=&quot;https://cucumber.io/docs/gherkin/&quot;&gt;Gherkin&lt;/a&gt; scenario can&#39;t drift, because it&#39;s a test. Where a statement can be a test, make it a test and drop the mark. Knowledge-driven development is for the rest: things that aren&#39;t executable yet, things that are broken, things about people rather than code.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://www-cs-faculty.stanford.edu/~knuth/lp.html&quot;&gt;Literate programming&lt;/a&gt; was never in the running. It needs its own toolchain, and nobody maintains the prose half.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Everything I deleted was a second description of the system, kept somewhere other than the system and updated by some act other than changing the code. All of it drifted, and the people it was written for stopped reading it.&lt;/p&gt;
&lt;p&gt;The ticket had a good run. It just never had a reader who mattered as much as the agent that couldn&#39;t see it.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Where it breaks</title>
    <link href="https://icelander.net/v1/processes/where-it-breaks"/>
    <id>tag:icelander.net,2026:where-it-breaks</id>
    <published>2026-11-01T00:00:00Z</published>
    <updated>2026-11-01T00:00:00Z</updated>
    <category term="processes" label="Processes"/>
    <summary>Where knowledge-driven development breaks: marks pile up when the token budget runs out, a wrong mark is worse than none, and marks go stale like any other document. Agents check the marks in review and before anyone trusts an old page, and where a rule has no build gate, an agent checks it instead, tokens allowing.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Knowledge-driven development, part 5: a wrong map is worse than a blank one&lt;/em&gt;&lt;/p&gt;&lt;p&gt;Thought September 2026, written October 2026. Processes.&lt;/p&gt;&lt;p&gt;The thing about knowledge-driven development that irritates me most is the marks that don&#39;t go away. A ⚠️ or a ❌ is meant to dissipate fast. Mine pile up, and the only reason is that I don&#39;t have the token budget to clean my desk.&lt;/p&gt;
&lt;p&gt;The list below is short partly because the method is young. I&#39;ve been running it for a couple of months, not years. It isn&#39;t battle-tested yet, and the problems I haven&#39;t met yet aren&#39;t on it.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;1. A wrong mark is worse than no mark&lt;/h2&gt;
&lt;p&gt;A ✅ next to something nobody ran is a lie that the next reader builds on. That would kill the method, because ✅ is the mark people trust.&lt;/p&gt;
&lt;p&gt;A blank map tells you to be careful. A wrong map tells you to walk off the cliff with confidence.&lt;/p&gt;
&lt;p&gt;A false ✅ has never bitten me. I don&#39;t let the documents and the code drift that far apart, and I&#39;m a stickler for unit tests and branch coverage. The first defense is that ✅ is &lt;a href=&quot;/v1/processes/seven-marks-three-monkeys&quot;&gt;expensive to write&lt;/a&gt;. No build can check a ✅ on a paragraph about how an operator handles a jammed door. Nobody&#39;s coverage tool has an opinion about doors.&lt;/p&gt;
&lt;p&gt;An agent does, if you ask it. Checking the marks is part of every &lt;a href=&quot;/v1/processes/the-plan-is-where-the-work-lives&quot;&gt;review&lt;/a&gt;: the agent that checks the claims reads the code behind each mark that moved. A wrong mark only gets serious when nobody updates the document, and the re-review is there to make sure that never happens.&lt;/p&gt;
&lt;p&gt;So I don&#39;t take an old document seriously until it&#39;s been checked. When I want to know a partially implemented feature in detail, and the page describing it hasn&#39;t been touched in a while, I have an agent give it a once-over against the code first. Then I read it.&lt;/p&gt;
&lt;h2&gt;2. Marks go stale&lt;/h2&gt;
&lt;p&gt;Marks don&#39;t rot on their own. They go stale the way any document does: the code moves and the page doesn&#39;t. With several agents working on parallel branches, the author of a change often doesn&#39;t know which pages describe the thing they touched. A stale ⚠️ against a capability that now exists sends the next agent off to build it twice.&lt;/p&gt;
&lt;p&gt;There&#39;s no need for a mechanical fix. Ask an agent to review the page, on request or on a schedule. Mechanical checks of that kind break anyway.&lt;/p&gt;
&lt;h2&gt;3. One kind of document isn&#39;t enough&lt;/h2&gt;
&lt;p&gt;&amp;quot;Upgrade to the next framework version&amp;quot; belongs to none of the capability documents. Neither does &amp;quot;the build is slow&amp;quot; or &amp;quot;the test runner leaks file handles&amp;quot;. Work that isn&#39;t &lt;em&gt;about&lt;/em&gt; a capability has no natural place in them.&lt;/p&gt;
&lt;p&gt;That&#39;s why knowledge-driven development needs more than one type of document: capabilities, features, requirements, architecture, the build, operations, and so forth. The framework upgrade becomes a page in the architecture documents, describing the system on the new version, marked ⚠️ until it&#39;s true. The slow build becomes a page describing the build as it should be. Same marks, same greps, same rules.&lt;/p&gt;
&lt;p&gt;Adding a type is cheap. An agent reverse-engineers the current state of that area into a knowledge document, and a human reviews it. The real decision is which types are worth having in your system.&lt;/p&gt;
&lt;h2&gt;4. It doesn&#39;t schedule anything&lt;/h2&gt;
&lt;p&gt;A document says what&#39;s missing. It never says when it&#39;ll be done or who owns it, and it isn&#39;t supposed to.&lt;/p&gt;
&lt;p&gt;In an ideal world, I wouldn&#39;t schedule anything. I&#39;d throw agent tokens at every ⚠️, and the problems would disappear. In the real world, I schedule them against a meagre token budget. When I do, I create a task in Asana, and the task points at the knowledge document. The task says when. The document says what, and whether it&#39;s true yet.&lt;/p&gt;
&lt;h2&gt;5. Gates, or an agent&lt;/h2&gt;
&lt;p&gt;A convention maintained by attention decays at the rate attention decays. A few rules earn a build failure. The rule against &lt;code&gt;TODO&lt;/code&gt; was written down, and needed one.&lt;/p&gt;
&lt;p&gt;When I wrote the specification for the gates, there were seven of them: the coverage cross-check, the test-location gate, the API contract scan, the vocabulary gate, the deferred-work gate, the orphan-⛔ check and the graduation check. One of them existed. The rest were described in the present tense and marked ⚠️, which is to say the enforcement spec was written using the method it enforces.&lt;/p&gt;
&lt;p&gt;The missing ones aren&#39;t serious. Where a rule has no gate, I ask an agent to check the documents against it, or to do the work the rule was protecting, and it does. If I have the tokens.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;A spec nobody maintains is a lie with a schema. A knowledge document nobody maintains is the same lie with a green tick on it. The difference is that you can ask an agent to check this one before you believe it.&lt;/p&gt;
</content>
  </entry>
</feed>

