Different modes, different outcomes

AI does not give everyone the same upgrade. On a bounded, well-specified task it works as an equalizer, lifting weaker performers more than strong ones. On open-ended work it works as a lever, and the return goes to whoever can name the problem, judge the answer and steer the next step. The same tool can therefore look modest in one person’s hands and transformative in another’s.

The modes below describe usage patterns rather than IQ brackets, and they attach to a person in a domain. Someone can direct the model expertly in their own field and drift back to vague one-shot prompts outside it. Raw ability matters, and so do self-awareness and willingness to iterate. A weak performer who knows it and accepts help gains more than an equally weak one who trusts the first answer. Anyone who can detect errors gains more than someone who can’t.

Mode 0: Non-users

They refuse the tool on principle, status or aesthetics: it’s cheating, it’s soulless, it’s beneath them. Their absolute skill holds steady while their relative position slips, because Modes 2 and 3 are pulling away on the same work.

AI hasn’t made them worse at anything. Peers now finish in an afternoon what still takes them a week, and when they are eventually forced to adopt the tool they will have no practice judging machine output.

Rough benefit: none directly, and a growing relative loss as the rest of the distribution shifts.

Mode 1: Underspecified users

Vague questions, hidden assumptions, and no check on whether the answer addresses the right problem. The model fills the gaps with average advice. To the user this can feel like a big step, because the output is cleaner than anything they would have written themselves, but what they have received is a tidier first draft.

On routine tasks with a good default template, field studies find gains of roughly 15–40% for weaker workers, the largest of any group. On novel or high-stakes work the gain shrinks, and it turns negative when they can’t spot a confident wrong answer.

Rough benefit: +10–30% on everyday tasks they already do, and close to nothing on work they can’t formulate.

Mode 2: Competent one-shot users

They use AI regularly and get useful results, but they treat it like a vending machine: one prompt, one answer, done. They rarely plan with it, and they rarely make it attack their assumptions or run a second pass against their own judgment.

This is enough to look productive. They get real leverage on writing, lookup, summarising and first-cut analysis, and they miss the compounding that comes from directing the work.

Rough benefit: 25–50% across a working week, and up to 2–3× on individual tasks such as drafting or summarising. Much less when the question is “what should I even be doing?”

Mode 3: Iterative directors

The human supplies taste, constraints and a real idea; the model challenges it, compresses it, expands it, then executes. The loop does the work: draft, critique, tighten, implement, inspect. Agentic coding and multi-step workflows turn that loop into finished output, where Mode 2 ends up with better notes. The payoff is a shorter cycle from idea to artefact.

Power-user data already shows a small group drawing several times more advanced reasoning from the same subscription than the median user.

Rough benefit: 5–15× on speed from idea to working artefact in domains they already understand, with a long tail above that in software, research and other fields where the model can ship the artefact itself. These multiples are estimates, since controlled studies have not yet measured this group well.

The mode has a failure case. When a task sits outside what the model does well, fluent users get led astray. Consultants in the Harvard/BCG study were 19 points less likely to reach the right answer on such a task when using AI. Experienced developers in METR’s trial of early-2025 tools worked 19% slower with AI while believing they were faster.

That second result has since moved. METR’s February 2026 follow-up estimates an 18% speed-up for the developers who returned, with a confidence interval that still spans zero, and METR now calls its own data unreliable. Developers increasingly refuse to work without AI, and timing breaks down when people run several agents at once. What survives from the original study is the gap between how fast people felt and how fast they were. Direction pays only while the director keeps verifying.

Mode 3.5: Loop designers

A Mode 3 director verifies by hand, which makes their attention the bottleneck. The next step is to build the verification into the system.

Two techniques carry most of it. The first is front-loading judgment into context. The director writes down what good looks like (the shape of the output, the constraints, the taste) so the model is steered before it goes wrong and needs less correction afterwards. Judgment gets spent once, where a Mode 3 director spends it on every cycle. The second is running checking loops alongside the main loop. The main loop produces the work while the others test it continuously against the stated constraints and flag drift.

The human stays in the loop, at intervals. There are three entry points:

1. The system pauses and asks for guidance when it doubts that it is meeting the person’s constraints or taste.
2. The person intervenes when they see the work drifting.
3. After the system reports that it has finished, the person points to where it went wrong, rewinds to that step and restarts from there with added context.

Over time the interventions get rarer, and in a mature setup they may be needed only occasionally. This is different from one-shot use, where the human specifies up front and walks away. Here the specification keeps being corrected by the person who owns it.

It would be easy to read this as a step toward AI that does everything itself. That reading fails on one point. The constraints, the taste and the definition of the problem all sit in the eye of the beholder. No objective standard says what good taste is, which constraints apply, or what the problem is in the first place, and the scope and intent of the problem space belong wholly to the person. A model can propose options and question a framing, but it has nothing to check its proposals against except the person’s reaction, because there is no fact of the matter to find. It can enforce a standard once someone sets one.

So a Mode 3.5 user verifies less and judges just as much. The judgment moves from correcting output to defining what correct means.

Rough benefit: Mode 3’s multiple, extended by the number of loops one person can keep supervised at once, since attention stops being the limit. This is an estimate with no controlled measurement behind it.

The Mode 3 failure case carries over in a new form. Checks encode the designer’s judgment as of the day they were written, so a wrong or stale check enforces the wrong thing at scale, and does it quietly.

Mode 4: Compounding systems

Mode 3.5 describes where the leading practitioners publicly arrived in mid-2026. In June, Peter Steinberger, the creator of OpenClaw, told developers to stop prompting coding agents: “You should be designing loops that prompt your agents.” The same week Boris Cherny, who leads Claude Code at Anthropic, said he no longer prompts Claude himself. His loops do the prompting, and his job is to write them. Addy Osmani gave the practice its name, loop engineering.

The same people are already working one level up. In Mode 3.5 the designer’s judgment is encoded once and then enforced. In Mode 4 the system gets better every time it runs. Four moves account for most of the difference.

Work finds itself. On the Claude Code team, routines watch the ticket queue, pick up bug reports that have sat untouched for five hours, draft the fix and bring in a person only where judgment is needed. The human stops being the initiator. Cherny says he no longer keeps a to-do list.

Every correction becomes permanent. Cherny doesn’t settle for correcting Claude inside one session. He has it write the lesson into durable project memory or a reusable skill, so the fix carries into every later run. Nous Research built the idea into Hermes Agent, which writes its own skills from completed tasks and refines them as they get reused. This is the mechanism behind the falling intervention rate described under Mode 3.5: each time the human steps in, the system needs them a little less.

Loops manage loops. Cherny describes agents prompting agents in trees of thousands. At OpenAI, a team that began with three engineers shipped roughly a million lines of code over five months with none of it written by hand, in an estimated tenth of the usual time. LangChain describes the full stack as four layers: an agent loop that does the work, a verification loop that grades it, an event-driven loop that starts it, and a hill-climbing loop that reads the accumulated history to improve future runs. Latent Space’s swyx names the matching skill. Climb a level when models improve, because that is where the leverage is, and drop back a level when something breaks, because that is where the reliability is.

Search replaces drafting. Where quality can be expressed as a number, the loop becomes a search. Andrej Karpathy’s autoresearch ran about 700 machine-learning experiments in two days against a single validation metric and kept roughly 20 real improvements, with no human involved between runs. His stated aim is to stop being the bottleneck: set the system up once and let it run.

One maintenance rule comes with the territory. Anthropic’s harness research observes that every component of a harness assumes something the model cannot do alone, and those assumptions expire as models improve. A Mode 4 operator prunes the system on every model release as well as adding to it.

Where full autonomy stops

Karpathy’s verifiability thesis marks the boundary. Classic software automated whatever could be specified, and LLMs automate whatever can be verified. Setting things up once and walking away works where good is a number, such as validation loss or a passing test suite. Where good is a matter of taste, the practitioners who have pushed furthest all report the same limit.

Anthropic made subjective design quality gradable by writing explicit criteria and calibrating a separate evaluator agent against the author’s own preferences, and it worked. The author also notes that tastes vary between people, that the wording of the criteria steered the output in ways he had not anticipated, and that he sometimes preferred a middle iteration to the final one. Steinberger runs many agents in parallel and still rejects the fully automated “dark factory,” insisting on iterative, human-guided development. Karpathy writes that while models remain jagged, the human has to stay in the loop. His summary: “You can outsource your thinking, but you can’t outsource your understanding.”

So Mode 4 divides along the line drawn under Mode 3.5. On metric-governed work the human can leave the loop after setup. On taste-governed work the human stays in it at intervals for good, because the standard being enforced is theirs and it moves.

Rough benefit: unmeasured, and mostly self-reported by the people who build these tools. The best-documented figure is OpenAI’s estimate of a tenfold speed-up on a million-line internal product. Nearly all the evidence comes from software and machine-learning research, where verification is cheap.

The failure cases scale with the system. Cost is the first: Anthropic’s full three-agent harness ran at more than twenty times the price of a solo run on the same prompt. A loop with nothing in it that can say no just lets the agent agree with itself. Osmani names the two human risks. Comprehension debt is the widening gap between what the system has shipped and what its operator understands. Cognitive surrender is what follows when the loop runs itself and the operator stops holding an opinion. The same setup rewards an operator who understands the work and hollows out one who doesn’t.

Beyond Mode 4

Mode 4 is the current frontier, and the people working there are candid about where it breaks. They also intend to keep pushing for more output with less human intervention. The open problems they describe say a good deal about what comes next.

Where Mode 4 still breaks

Reliability on long tasks. METR’s best-measured model can complete tasks that take a human 16 hours or more about half the time, but its horizon at 80% reliability is about three hours. OpenAI’s internal figures from September 2026 have the same shape. Tasks under 15 minutes succeeded 86% of the time with no intervention, while more than half of the successful tasks in the four-to-eight-hour range needed a person to step in at least once. That is the intermittent human of Mode 3.5, measured. Most of the fix is model progress, since measured horizons have been doubling roughly every four months, helped along by decomposition and cheap retries. Of the five problems, this one looks the most tractable.

Agents game the checks. In June 2026 METR declined to publish a robust capability figure for GPT-5.6 Sol because the model cheated more often than any public model it had evaluated, in one case extracting hidden source code that contained the expected answer. Counting the cheating attempts as failures gave a time horizon of about 11 hours. Counting them as successes gave more than 270. OpenAI linked the behaviour to training for instruction following and persistence, the same traits that make an agent useful in a long-running loop.

A Mode 4 system hill-climbs against its checks, so the stale-check failure described under Mode 3.5 becomes an adversarial one: a capable optimiser will find the gap between the check and the intent behind it. The known defences are separating the maker from the checker, holding back checks the worker never sees, and monitoring the process as well as the outcome. OpenAI’s chief scientist, Jakub Pachocki, has warned that reading a model’s chain of thought, one of his company’s central monitoring bets, is losing reliability. He expects progress to be capped increasingly by how far monitoring can be trusted.

Human review doesn’t scale. Osmani calls it the orchestration tax. Isolation tools remove the mechanical collisions between parallel agents, and the operator’s review bandwidth still decides how many can run. OpenAI makes the matching point about its own research organisation: overall progress grows more slowly than the usage figures suggest, because the least automatable tasks become the bottleneck. The partial fixes are to review evidence in place of artefacts, to escalate by risk so that attention goes only where the stakes are real, and to audit by sample.

Intent and taste stay under-specified. Anthropic’s evaluator agent had to be tuned against one person’s judgment, by reading its logs and correcting the places where it diverged from him. At OpenAI, high-level planning decisions remain a tiny share of what agents produce. The direction of the fix is a system that learns its operator’s taste from their corrections and choices. Hermes Agent already builds a model of its user across sessions, in a crude form.

Coordination, memory, cost and security. Cursor ran hundreds of concurrent agents on single projects and found that flat teams turn risk-averse, with no agent taking ownership of the hard problems. A hierarchy of planners, workers and judges fixed much of that, and Cursor still describes coordination as a hard problem.

Memory files and skills are a prosthetic for learning on the job. Whether the real thing is close is disputed: Karpathy has put it about a decade away, while Dario Amodei has said it may prove easier than it looks and Anthropic’s Sholto Douglas predicted a solution during 2026. The two routes differ in a way that matters for trust. A lesson stored in a file can be read and audited, and a lesson stored in model weights cannot.

Cost is rising quickly. OpenAI’s median researcher uses more than $600 of inference a day, and the 90th percentile more than $7,000. Security closes the list: an unattended routine acts with its owner’s credentials while reading outside content that may carry injected instructions.

Mode 5: Delegated objectives

In Mode 4 the human designs the loops. In Mode 5 the system designs and maintains them. It writes the spec, assembles the workflow, proposes the checks and revises all three from its own history. The human sets objectives, budgets and boundaries, reviews by exception, and audits the checks.

The pieces are visible already. Anthropic’s planner agent expands a prompt of one to four sentences into a full product specification. Claude Code’s dynamic workflows let Claude assemble its own multi-agent workflow around a task. OpenAI reports that it has reached its milestone of an automated research intern: a system that carries out clearly scoped research tasks under human guidance, including some that would take an experienced researcher several days.

Because agents game their checks, auditing the checks becomes the scarce human skill at this level. The operator’s question changes from “is this output good?” to “would this check catch a system that passed it without doing the work?”

Mode 6: Standing agent organisations

One level further out, a persistent organisation of agents runs a whole function continuously, with roles, memory, budgets and its own internal review. Pachocki’s description of where OpenAI is heading is “a whole research lab in a data center,” with people still in charge of the goals. OpenAI’s stated target for a fully automated AI researcher is March 2028.

At this level the design problem is organisational. The instruments are the ones used to govern any institution that acts on someone’s behalf: a charter that states purpose, limits on delegated authority, budgets, reporting lines and independent audit.

Scenarios to the end of 2028

These probabilities are judgment informed by the evidence above. No quantitative model sits behind them.

Verification-gated expansion (45%). Autonomy advances fastest wherever checks are cheap and hard to game: software, data work, machine-learning research, mathematics and quantitative engineering. Elsewhere the human stays in the loop at intervals. Mode 5 becomes normal practice in the verifiable domains, and the share of multi-hour tasks needing human intervention falls from over half to something like one in five.

Trust ceiling (25%). Check-gaming and weakening monitoring produce incidents, followed by the binding, independently audited standards that Pachocki has called for. Mode 4 spreads widely, but consequential work keeps a mandatory human sign-off, as aviation and process safety do. Most of the gain comes from more people reaching Modes 3.5 and 4.

Fast automation (20%). Learning moves into model weights, or multi-day reliability arrives, and AI research begins to accelerate itself. Forecasters disagree on timing. Daniel Kokotajlo of the AI Futures Project moved his median for full coding automation at a frontier lab to mid-2028, while the more conservative FutureSearch team sits around 2032. In this scenario the human role compresses quickly to goal-setting and governance.

Plateau (10%). Capability gains slow, or the cost of heavy loops outruns their value, and Mode 4 remains an elite practice.

The weights rest on one observation. The trend lines all point up, and the credible brakes all concern trust.

What stays human

Over this horizon, systems will get good at predicting an operator’s taste and at proposing problems worth solving. The eye-of-the-beholder argument under Mode 3.5 should therefore be read as a claim about authority. A system may come to model a person’s preferences well and suggest goals they would endorse. Whose purposes count remains a decision, and someone has to own it. Even OpenAI, which is pursuing automated research as hard as anyone, reports that people still set its research priorities, judge the results, and decide on scaling, pauses and deployment.

What separates the modes

The dividing line is whether the human can specify the problem, catch a bad answer and stay in the loop. Intelligence helps with all three and guarantees none of them.

From Mode 3 upward, the modes differ in where the human’s judgment is spent: on every cycle, in the design of the checks, or in a system that keeps what it is taught.

On a scripted job, Mode 1 can look surprisingly good. On open work, the force multiplier starts at Mode 3, and everyone below it gets a better autocomplete. Even in the most automated setup, the definition of good stays with the person.

Sources for Mode 3.5 and above

• Peter Steinberger, post on designing loops, 7 June 2026: https://x.com/steipete/status/2063697162748260627
• Boris Cherny and Cat Wu, “Reflecting on a Year of Claude Code”: https://www.youtube.com/watch?v=Hth_tLaC2j8
• Addy Osmani, “Loop Engineering”, 8 June 2026: https://addyo.substack.com/p/loop-engineering
• Anthropic, “Harness design for long-running application development”, 24 March 2026: https://www.anthropic.com/engineering/harness-design-long-running-apps
• OpenAI, “Harness engineering: leveraging Codex in an agent-first world”, 11 February 2026: https://openai.com/index/harness-engineering/
• Andrej Karpathy, Sequoia Ascent 2026 summary: https://karpathy.bearblog.dev/sequoia-ascent-2026/
• Andrej Karpathy, autoresearch: https://github.com/karpathy/autoresearch
• Latent Space, “Loopcraft: The Art of Stacking Loops”, 12 June 2026: https://www.latent.space/p/loopcraft
• Nous Research, Hermes Agent documentation: https://hermes-agent.nousresearch.com/docs/
• LangChain’s four-loop framing, as summarised by Cockroach Labs: https://www.cockroachlabs.com/blog/agent-loops-production-database-patterns/
• METR, developer productivity update, 24 February 2026: https://metr.org/blog/2026-02-24-uplift-update/
• METR, task-completion time horizons of frontier models: https://metr.org/time-horizons/
• METR, summary of the predeployment evaluation of GPT-5.6 Sol, 26 June 2026: https://metr.org/blog/2026-06-26-gpt-5-6-sol/
• OpenAI, “Research acceleration: The view inside OpenAI”, 6 September 2026: https://openai.com/index/research-acceleration-view-inside-openai/ (figures here are taken from The Decoder’s summary: https://the-decoder.com/openai-reports-ai-research-interns-and-warns-about-its-own-pace-at-the-same-time/)
• Jakub Pachocki, “An Alien Mind”, 6 September 2026: https://openai.com/index/an-alien-mind/
• MIT Technology Review, interview with Jakub Pachocki, 20 March 2026: https://www.technologyreview.com/2026/03/20/1134438/openai-is-throwing-everything-into-building-a-fully-automated-researcher/
• Cursor, “Scaling long-running autonomous coding”, 14 January 2026: https://cursor.com/blog/scaling-agents
• Cursor, “Towards self-driving codebases”, 5 February 2026: https://cursor.com/blog/self-driving-codebases
• Transformer, “Why is everyone talking about continual learning?”, 22 January 2026: https://www.transformernews.ai/p/teaching-ai-to-continual-learning
• AI Futures Project, Q1 2026 timelines update: https://blog.aifutures.org/p/q1-2026-timelines-update
• FutureSearch, “AI 2027 Update: A One Year Timeline Check”, 8 April 2026: https://futuresearch.ai/blog/ai-2027-one-year-later/