Dark Factory Patterns: A Developer's Guide

A developer's guide to shipping software with the lights off: one message from a human, then agents plan, build, verify, release and learn, with nobody asking anything until the run reaches its stop line.

Fig. 0 · one run, lights off

The founder's lamp lights once, for the opening message, and stays dark from preflight to retro.

Messages from founder

Questions asked

Decisions logged

Verify round

One illustrative run. Verify fails once and sends the card back to build; round 2 passes. The gate finds the change reversible, so no human is needed. The retro's finding loops round to become a new preflight check.

What they are · Catalog · Authority · Construction · Verification · Release · Learning · Anti-patterns · When to use · Sources

In manufacturing, a dark factory is a plant that can run with the lights off, because nobody works on the floor. FANUC has run lights-out lines since 2001, where robots build robots and can run unsupervised for up to 30 days. Xiaomi's Changping plant automates “100% of the key processes” and turns out about one phone every three seconds, with its people in a “War Room” instead of on the line.

In software the phrase took hold in early 2026. Dan Shapiro's five levels of AI coding end with it:

It's dark, because it's a place where humans are neither needed nor welcome. — Dan Shapiro · The Five Levels · Jan 2026

StrongDM published the rules of their own factory a fortnight later: “Code must not be written by humans” and “Code must not be reviewed by humans.” OpenAI's harness-engineering team reported about a million lines with no manually written code. Stripe and Spotify run large agent fleets with a human still reviewing each change.

At Scry we run a smaller version. The founder sends one message; agents plan, build, review, verify, release and write the retro; and nobody asks the founder anything until the run reaches its stop line. We have finished more than a dozen runs this way so far. This guide collects the patterns that made them work, plus the ones others have published, in the format the Gang of Four used for object-oriented design: a name, an intent, the problem, the solution, and where it has been seen.

solid · running or passedring · a human is neededdashed ring · stop the linehairline · nobody here

Every diagram uses those four lamp states, drawn in ink alone. Diagrams play when they scroll into view; each has a pause button, and with reduced motion each one shows a single still frame instead.

What are dark factory patterns?

A dark factory pattern is a reusable answer to a problem that only shows up when nobody is watching the line. Most of these problems don't exist while a person reviews every diff, because that person is the safety net, the memory and the judge all at once. Take them off the floor and each of those jobs needs a pattern of its own.

Why use them

They move human judgement to where it is cheap: the start (one mandate) and the end (one retro), instead of every step in between.
They make agent work checkable by something other than the agent that did it.
They keep releases reversible, so a mistake costs minutes, not customers.
They turn every run into data for the next one.
They give a team shared names. “Is that a rider?” is quicker than explaining the problem from scratch.

How dark is dark?

Shapiro borrows the levels of driving automation. The human's attention is the lamp: it dims at every level and is off at level 5.

Fig. 1 · five levels

Where the dark factory sits.

Dan Shapiro's levels; the quoted phrases are his car analogies, the roles under them are our summary. Our gate-B runs are level 4: the human checks the result at the end. Prod-mode runs are level 5 for anything that can be undone.

What changed for us

Before we ran dark, our median item took 10 hours from start to production. A run in prod mode now takes about an hour.

10 h

median start → prod before dark runs (54 items, 26 Sep–10 Oct)

36%

of that time was waiting on people; 0.7% was merge, stage and promote

1:06

median message → prod in prod mode

4:00

median to the gate in gate-B mode; 26% of it waiting there

Different cohorts, not a controlled trial. Our dark runs so far are mostly small fixes and features, which flatters them.

How the catalog is organised

The Gang of Four sorted 23 patterns into creational, structural and behavioural. Dark factory patterns sort by the stage of a run they protect:

A · Authority. Who may decide what, once the human has gone home.
C · Construction. How agents split the work, build it and share a repo.
V · Verification. How you know it works when nobody looks at the code.
R · Release. How a change reaches users, and how it comes back.
L · Learning. How each run makes the next one cheaper.

Each pattern has a code and a two-letter symbol, then the fields the GoF used: intent, problem, solution, and where it has been seen, with what to watch out for and related patterns. In our runs is evidence from our own runs; Elsewhere is published work by others.

Fig. 2 · the catalog

25 patterns, in the order a run meets them.

Solid tiles: seen in our runs. Dashed: seen only in others' published work. A ringed lamp means the pattern deliberately involves a human. Select a tile to jump to it.

Authority patterns

Who may decide what, once the human has gone home.

A dark factory still runs on human judgement. The judgement just moves: to the start, where one message sets the mandate, and to a short list of things the factory may never do alone. These patterns decide who may make each call while nobody is watching.

Fig. 3 · authority

One message, a ledger, and a human only for what can't be undone.

The message becomes the mandate. D1 is reversible and ships, logged as auto. D2 is a lit-run question that waited its 15 minutes and took the recommended answer. D3 can't be undone, so it becomes a follow-up for a human instead of a question.

A1 · Single Mandate

founder's words as sole authority · df.py start --quote

One message from the human is the run's whole authority, stored word for word before any work starts.

Problem: Agents that are unsure ask, and every question parks the run until a person answers. Before our dark runs, 36% of elapsed time on 54 items was waiting on people, and the waits had a long tail: 31 waits of 2.5 hours or more made up 156 of 185 founder-wait hours.
Solution: Record the message verbatim as the run's mandate (df.py start --quote). Read it for what it waives and nothing more. Check every later choice against that text, not against a guess about what the human would say.
In our runs: Our trigger text was blunt: “fully build this without any inputs from me until you reach gate b anything short of that would be considered a failure of the experiment”. Asking the founder anything before the stop line counts as a failed run.
Watch out: Leaks come from places nobody thinks of as asking. During one run a spend guard emailed the founder about a $0.35 charge. Audit every outbound channel, not just the chat.
Related: A2 Stop Line · A4 Decision Ledger · L5 Budget Meter

A2 · Stop Line

two modes: gate-b and prod

Decide before the run starts where it stops for a human: at one approval page, or in production.

Problem: “Autonomous” with no end point fails both ways: agents stop too early to be safe, or keep going past anything anyone agreed to.
Solution: Offer exactly two modes. gate-b stops at a single approval page. prod goes through to production and the retro. If the message says both, the more explicit “go to prod” wins. Anything the message does not name stays as it was.
In our runs: Prod-mode runs had a median of 1 h 06 min from message to production. Gate-B runs had a median of 4 h 00 min, and 26% of that wall time was waiting for the founder at the gate.
Watch out: Waiving the gates does not waive everything. Our mandates never waive a non-builder verifier, a security review on auth, data or spend paths, the $5/day spend cap, publishing, or a P0 hotfix call.
Related: A1 Single Mandate · A3 Can't-Undo List · V1 Three Hats

A3 · Can't-Undo List

leave the irreversible out · jidoka for software

Keep a short written list of changes that can't be rolled back, and never ship them in the dark.

Problem: Most bad releases roll back in minutes. A few can't: a table rewritten in place, a sent email, a published package. An agent that judges “is this risky?” case by case will one day judge wrong.
Solution: Write the list down and keep it boring. In prod mode an irreversible item is cut from the release and becomes a follow-up for a human; in gate-b mode it becomes a line on the gate page. The rest of the change still ships.
In our runs: Our list: a destructive “remove old” migration step or data rewritten in place; security rules, auth, keys or permissions; pricing, credits or wallets; desktop app, npm or Figma plugin releases; emails or public posts; removing an API or MCP tool, or a breaking format change. A small CLI tool reached prod in 31 minutes and left its publish decision as a follow-up for a human.
Elsewhere: BCG Platinion puts named humans at stage gates: “an agentic factory does not mean an uncontrolled one”. Toyota's word for the idea is jidoka, “automation with a human touch”.
Watch out: What a waiver covers drifts between runs. Our desktop app's 0.1.9 was published under a standing go on 12 October; a day later desktop releases were on the can't-undo list, and its 0.1.10 stopped for a human. Re-read the list at every run, not once.
Related: A2 Stop Line · R1 Dark Switch · R5 Watch Window

A4 · Decision Ledger

fixed-format decision row · df.py auto

Every judgement call goes into one ledger, in one fixed format, at the moment it is made.

Problem: With nobody asking questions, choices still get made. If they live only in agent transcripts, nobody can review them, and the retro can't tell a lucky guess from a sound rule.
Solution: One command, one row: the choice, the reason, whether it was the recommended default, whether it can be undone. df.py auto <slug> "<choice>" --why "<reason>". The gate page and the retro read the ledger, not the transcripts.
In our runs: The ledger is how the founder reads a whole run in a few minutes afterwards. Three rules came out of it: take the recommended default, prefer the reversible option, and if the irreversible option is better, park it (A3) rather than ask.
Watch out: Free-text rows rot. Keep the format fixed so a script can check that every gate decision has a row.
Related: A5 Default Answer · L1 Retro Done-Gate · A3 Can't-Undo List

A5 · Default Answer

recommended-answer auto-approve

Every question an agent might ask comes with a recommended answer, and the recommendation wins if nobody replies.

Problem: A question with no default is a blocking call. A question with a default is only a notification.
Solution: In a dark run, take the recommended default at once and log it (A4). In a lit run, the same question waits a fixed time and then approves itself. Tell the human with a push notification, not a phone call.
In our runs: Our lit pipeline auto-approves the recommended answer after 15 minutes (it used to be 30). Founder replies to explicit questions had a median of 3 minutes but a p90 of about 3.6 hours; the timeout cuts off that tail.
Elsewhere: Augment's guide calls this keeping humans “over the loop” rather than in it.
Watch out: A default is only as good as whoever wrote it. If a recommendation keeps getting reversed afterwards, the rule behind it is wrong: fix it in the harness (L4).
Related: A1 Single Mandate · A4 Decision Ledger · L4 Harness over Prompt

Construction patterns

How agents split the work, build it and share a repo.

Building was the cheap part of our runs: 25 to 47 minutes each. These patterns keep it cheap as the number of agents grows, by keeping each agent small, briefed before it starts and out of the others' way.

Fig. 4 · construction

Contract first, three fresh agents at once, then the merge train.

The orchestrator writes the contract and briefs three fresh agents in parallel, each in its own worktree. Their PRs join the merge train and reach stage. The dark switch stays off: users see nothing until it is turned on.

C1 · Fresh Narrow Agents

orchestrator–workers

An orchestrator splits the work and starts a new agent, with a narrow brief, for each piece and each round.

Problem: One long-lived agent that builds, fixes, reviews and releases piles up context, carries its own wrong assumptions forward, and gets slower and less careful every round.
Solution: The orchestrator keeps the plan and the ledger. Workers start fresh, get one brief, do one job and finish. Round 2 is a new agent with a delta brief, not the old agent with more to read.
In our runs: Sonnet is our default worker, with at most three Opus builders at once. Building took 25 to 47 minutes in every dark run on 12 October.
Elsewhere: Anthropic's “Building effective agents” names this shape orchestrator–workers and recommends clear stopping conditions.
Watch out: Fresh agents forget the traps. Feed the trap catalog (L2) into every brief, or the same mistake comes back each round.
Related: C2 Pre-Written Brief · C3 Contract-First Fan-Out · V5 Delta Rounds

C2 · Pre-Written Brief

write the next brief during this step

Write the next agent's brief while the current step is still running.

Problem: The gap between “review finished” and “verifier started” is dead time spent writing instructions, and it comes back at every handoff.
Solution: As soon as a step starts, write the brief for the step after it: the reviewer's while the build runs, the verifier's while review runs, the release brief while verification runs. Run the preflight check over the briefs before launching anyone.
In our runs: On our ETA board, pre-written briefs worked in 8 of 8 closed tries and saved a median of 13 minutes a run. On a sidebar flicker fix, writing all three briefs before the builder started took one 15-minute handoff to nothing.
Watch out: A brief written early can go stale. Point it at things resolved at run time (the merge sha, the stage URL) instead of hard-coding them.
Related: C1 Fresh Narrow Agents · R2 Pinned Promote · L2 Trap Catalog

C3 · Contract-First Fan-Out

parallel agents

Fix the interfaces first, then build every piece in parallel against them.

Problem: Agents in sequence are slow. Agents in parallel without a shared contract build pieces that don't fit.
Solution: One short step writes the contract: API shapes, which agent owns which files, test names. Then the builders, the UAT-script writer and the verifier-brief writer all start at once, each in its own worktree.
In our runs: Our most valuable time-saver: worked in 10 of 13 closed tries, median 81 minutes saved. On an admin page, running the round-5 fix and its test update in parallel took a two-hour step to 24 minutes.
Watch out: Not every split pays. Splitting UAT across agents worked in 1 of 2 closed tries and overran by a median of 30.5 minutes.
Related: C1 Fresh Narrow Agents · C5 Clean Origin · V3 Scripted Acceptance

C4 · Digital Twin

Digital Twin Universe (StrongDM)

Run the factory against behavioural clones of the third-party services it depends on.

Problem: Real services have rate limits, costs and side effects. Agents that test against them either test too little or do real damage.
Solution: Build clones that behave like the real APIs closely enough for end-to-end scenarios, then run as many scenarios against them as you like.
Published: StrongDM: “The Digital Twin Universe is our answer: behavioral clones of the third-party services our software depends on.” Simon Willison on why this is new: “Creating a high fidelity clone of a significant SaaS application was always possible, but never economically feasible.”
In our runs: We have not built one. Our stand-in is a stage tier that talks to real third-party services, and that is exactly where our “defects found only on stage” come from.
Watch out: A twin that drifts from the real service turns green tests into false comfort. Check it against the real API on a schedule.
Related: V4 Holdout Scenarios · V3 Scripted Acceptance

C5 · Clean Origin

swarm git hygiene · clean-origin control plane

Every agent works in its own worktree off origin, and release scripts run from a clean checkout, never a shared working tree.

Problem: Many agents on one machine share checkouts, branches and git state. One agent's stash or reset quietly wrecks another's work, and a release script run from a dirty tree ships files nobody reviewed.
Solution: A worktree per agent (git worktree add -b fix/<slug> ../wt-<slug> origin/stage), no bare git stash, and release scripts pointed at a clean origin/main worktree. A guard blocks the dangerous commands.
In our runs: We logged 42 git incidents in 14 classes, and 593 hits of risky commands, 419 of them git stash. Our guard went live in blocking mode on 12 October.
Watch out: Identity leaks too: agents committing under the wrong author. Set the git identity per worktree.
Related: C3 Contract-First Fan-Out · R2 Pinned Promote

Verification patterns

How you know it works when nobody looks at the code.

Take the person who reads every diff off the floor and you need another way to know the code works. Every pattern here has the same root: whoever checks the work must not be whoever did it, and must not be able to see the answers in advance.

Fig. 5 · verification

Three hats and a wall.

The builder proves its test red, green, then red again with the fix reverted. Only a sha passes the reviewer through the wall. The verifier runs scenarios kept from the builder (the ideal; ours are only partly held out, see V4). In round 1, S4 fails and only the verdict goes back; round 2 is a delta fix, and all six pass.

V1 · Three Hats

builder ≠ reviewer ≠ verifier · separation of duty

The agent that builds a change never reviews it, and the agent that reviews it never verifies it.

Problem: An agent grading its own work shares its own blind spots, and can be tempted to make the check pass instead of making the code work.
Solution: Three different agents and three different names on the record. The release step refuses to approve if the verifier is unnamed or is the same agent as the reviewer.
In our runs: df.py approve-prod enforces the names. On a KPI page speed-up the builder's tests passed, but the verifier said “changes”: on stage, staff KPI visits after a gap still took 1.2–1.8 s against about 2.1 s before. Slice 2 cached the whole answer and brought them to 0.4–0.6 s.
Elsewhere: NIST's separation of duty is the same rule for people. METR found reward hacking in 30.4% of runs (39 of 128) on one benchmark, including a model that patched the evaluation function.
Watch out: Three agents on the same model with the same brief share blind spots. Give the verifier different inputs (V4), not just a different name.
Related: V2 Red-First Proof · V4 Holdout Scenarios · V5 Delta Rounds

V2 · Red-First Proof

regression-first fix · revert proof

Prove the test catches the bug: red on the old code, green with the fix, red again with the fix reverted.

Problem: A test written after the fix often passes for the wrong reason, or tests nothing at all. A green test proves nothing on its own.
Solution: Run the new test on the pre-fix sha and record the failure. Apply the fix and record green. Revert only the fix and confirm red again. Write all three counts down.
In our runs: Every dark bug fix records the red count, green count and revert result on its gate page. The UAT script does the same at product level with --expect before on stage.
Elsewhere: Anthropic's harness for long-running agents tells them: “It is unacceptable to remove or edit tests”.
Watch out: Agents under pressure edit the test instead of the code. Lock the test once red is recorded, or have a different agent write it.
Related: V3 Scripted Acceptance · V1 Three Hats

V3 · Scripted Acceptance

script-first UAT · --expect before|after

Every acceptance row is a command, written before the build, that runs against stage.

Problem: “Looks good” from an agent clicking around is not a check anyone can rerun. A row that says “needs founder” puts the human back in the loop.
Solution: Write the UAT script first and run it with --expect before on the pre-fix stage: that run is the reproduction. After merge the verifier runs the same script with --expect after. No row may read “needs founder”.
In our runs: Worked in 10 of 11 closed tries, median 28 minutes saved a run. On the small CLI tool, a script written by a separate agent turned a 30-minute step into about one minute.
Watch out: Scripts check only what they were told to check. Still open the screenshots and look: a script that greps for text misses a broken layout.
Related: V2 Red-First Proof · V4 Holdout Scenarios · C3 Contract-First Fan-Out

V4 · Holdout Scenarios

scenarios + satisfaction (StrongDM)

Keep end-to-end scenarios where the builders can't see them, and measure how many of them pass.

Problem: If builders can read the tests, they will, and the code will fit the tests instead of the user.
Solution: Store user-story scenarios outside the codebase, like a holdout set in model training. Run them against the build and report satisfaction: the share of runs that would satisfy the user.
Published: StrongDM keeps scenarios “often stored outside the codebase (similar to a “holdout” set in model training)” and asks: “of all the observed trajectories through all the scenarios, what fraction of them likely satisfy the user?”
In our runs: Only partly. Our UAT script is written by a separate agent, but it lives in the repo where builders can read it. This is the weakest link in our setup.
Watch out: Holdouts leak. Once a builder sees a failing scenario in full, it is no longer held out. Send back pass/fail and a short reason, not the script.
Related: V3 Scripted Acceptance · C4 Digital Twin · V1 Three Hats

V5 · Delta Rounds

delta verifier · trace before fix · “changes” means slice 2

When a check fails, the next round looks only at what changed, and starts by tracing the cause.

Problem: Full re-reviews every round are slow, and fixes made without a trace tend to fix the symptom.
Solution: Round 2 is a delta: a fresh reviewer gets the diff since round 1 and the list of findings. The builder traces the failure before touching code. NIT-only findings become follow-ups, not another round. A verifier “changes” on a build that already works becomes slice 2 instead of reopening slice 1.
In our runs: One combined code-and-security reviewer worked in 4 of 4 closed tries (median 16.5 minutes saved). On the sidebar flicker fix the orchestrator wrote the round-2 fix and ran every check in parallel: 28 minutes down to 7. That bug still took three rounds and 1 h 58 min.
Watch out: An orchestrator that writes a fix may not review or verify it. Small fixes are where the three hats get quietly dropped.
Related: V1 Three Hats · C1 Fresh Narrow Agents

Release patterns

How a change reaches users, and how it comes back.

In our runs the release tail, not the build, was the biggest cost: 50 minutes to 1.5 hours against 25 to 47 for building. These patterns make shipping exact (the sha you verified, and only what was approved) and make every release easy to take back.

Fig. 6 · release

Claim every rider, pin the sha, trust the probe, watch for 48 hours.

One rider starts unclaimed and blinks as a dashed stop lamp until its approval is found. The promote is pinned to our sha and judged by the health endpoint, not the deploy status. The rollback pointer was written down before the promote. Shas are illustrative.

R1 · Dark Switch

dark launch · release line

Ship new behaviour to production switched off, so the code release and the user-visible release are separate events.

Problem: If merging means users see it, every merge needs a human's eye, and the factory waits.
Solution: Every PR declares one release line: Release: dark <FLAG>, after Gate B, or no behaviour change. The merge train refuses a PR without one. Dark code goes to prod with its switch off; turning it on is a separate, reversible step.
In our runs: Our release-line gate went live on 12 October. The login speed-up shipped behind four switches, turned on one at a time once checked.
Elsewhere: Martin Fowler's DarkLaunching. Pete Hodgson warns that toggles are “inventory which comes with a carrying cost”.
Watch out: Switches pile up. Give each one a removal follow-up with a date when it is created (our aspect-ratio switch is due out on 21 October).
Related: R3 Rider Audit · R5 Watch Window · A3 Can't-Undo List

R2 · Pinned Promote

pin the promote to your own merge sha

Promote the exact commit that was verified, by sha, never “whatever is on the branch now”.

Problem: Branches move. Between your verification and your promote another agent can merge, and promoting the branch head ships their unverified work under your name.
Solution: Verify against the merge sha: the verifier waits until stage /api/healthz reports that sha. Then promote that sha, and refuse if the head has moved.
In our runs: Our verifier briefs poll stage health until it reports the merge sha, then start by themselves. Where our old gh CLI could not pin a merge to a sha, we merged through the API pinned to it.
Watch out: Pinning a sha still ships everything below it on the branch. That is what R3 is for.
Related: R3 Rider Audit · R4 Real Signal · C5 Clean Origin

R3 · Rider Audit

joint promote · df.py riders

Before promoting, list every commit that will ride along with yours and check that each one was approved.

Problem: Promoting your sha promotes every commit below it. An unapproved change from another run rides to production through no gate at all.
Solution: df.py riders <checkout> <merge sha> lists the commits between prod and your sha. Each must be claimed by a run, with its gate or dark switch checked. If one isn't, hold, or promote jointly with its owner's approval.
In our runs: Three small changes went to prod in one joint promote. The lesson from that run was “unapproved rider below merge”, and the rider check became a required step.
Watch out: Joint promotes are convenient, and they grow. Keep releases small and frequent so the rider list stays short.
Related: R2 Pinned Promote · R1 Dark Switch

R4 · Real Signal

health probe over deploy status · cancelled CI is not red

Decide on the signal that measures what you care about, not on the nearest status badge.

Problem: Status badges mislead both ways. A deploy marked “Ready” can still be serving the old build. A CI run “cancelled” because a newer run replaced it looks like a failure and stalls the release.
Solution: Gate on a health endpoint that reports the running sha. Treat cancelled CI as “no result”: find the run for your sha and wait on that one.
In our runs: One release step took 52 minutes against about 20 planned, mostly lost to cancelled CI runs. Its retro named cancelled CI the pipeline's biggest waste, and the fix went into the promote script: cancelled is not red.
Elsewhere: Google's SRE workbook on canarying judges a release by the service's own metrics, compared with a control.
Watch out: Health endpoints mislead too if they report a sha from a file nobody updates. Generate it at build time.
Related: R2 Pinned Promote · R5 Watch Window · L5 Budget Meter

R5 · Watch Window

canary + rollback pointer

Every release gets a timed watch, with its rollback sha written down before the promote.

Problem: Many failures show up hours later, with real users. If nobody is watching, the factory hears about it from a complaint, and rolling back starts with working out what to roll back to.
Solution: Record the current prod sha as the rollback pointer before promoting. Open a watch follow-up due about 48 hours later that checks errors, logs and the feature's own signal. Roll back first, debug second.
In our runs: Every dark release recorded its rollback sha before promoting (the sidebar flicker fix went to prod with its rollback sha written down first) and opened a watch item due two days later.
Elsewhere: Martin Fowler's CanaryRelease and BlueGreenDeployment; Google SRE on canarying.
Watch out: Watch items that nobody closes are noise. Close each one with a line: what was checked and what was seen.
Related: R4 Real Signal · L1 Retro Done-Gate · A3 Can't-Undo List

Learning patterns

How each run makes the next one cheaper.

A factory that does not learn repeats its mistakes at machine speed. These patterns turn each run into data: a retro that has to pass a check, a catalog of traps, and a ledger of predicted against actual time.

Fig. 7 · learning

Each run feeds the next; each estimate gets checked.

Left: a retro adds a trap to the catalog, and the next preflight checks for it. Right: predicted against actual minutes for three whole runs and one release step; hatched bars ran over. The biggest overrun is the copy fix's release step, mostly lost to cancelled CI (R4).

L1 · Retro Done-Gate

standard retro · df.py retro --check

A run is not done until a retro in a fixed format passes a check.

Problem: Free-form retros say “went well” and cite nothing. They can't be compared, and nothing in them turns into a fix.
Solution: A fixed retro: times taken from logs, not memory; shas; what each agent did; what was waited on; which traps fired. A script checks it, and the run stays open until it passes.
In our runs: We surveyed 78 earlier retros: about 19% cited a sha and about 15% gave a time. On a sidebar reorder fix the hand-written time was 75 minutes; the logs said 51. The standard retro with --check became the done-gate on 13 October.
Watch out: A retro can pass the check and still change nothing. Each one must name a trap to add or a check to change, or say “none” on purpose.
Related: L2 Trap Catalog · L3 ETA Ledger · A4 Decision Ledger

L2 · Trap Catalog

poka-yoke for runs

Every trap a run falls into becomes a row in a catalog, and every row becomes a check that runs before the next run starts.

Problem: Fresh agents (C1) don't remember. Without a catalog every run pays for the same lessons again.
Solution: Keep one table: trap, symptom, what to do. The preflight script checks for known traps before any agent launches, and each brief links the rows that apply.
In our runs: Our release-tail catalog has 28 traps, from “cancelled CI looks red” to “browser held by another session”. It opens with: “Building took 25-47 min in every dark run of 2026-10-12. The tail took 50 min to 1.5 h.”
Elsewhere: Lean's poka-yoke: mistake-proofing at the step itself, not inspection afterwards.
Watch out: A preflight that warns about things that are fine trains everyone to ignore it. Fix false positives as fast as real traps.
Related: L1 Retro Done-Gate · L4 Harness over Prompt · C2 Pre-Written Brief

L3 · ETA Ledger

predict, then measure

Every run states an ETA before it starts, and every time-saving idea is logged as an experiment before it is tried.

Problem: Without a prediction you can't tell a fast run from a lucky one, or which change made it fast.
Solution: Set the estimate at the start (estimate.py set), log each speed-up idea with its expected saving (eta-exp.py add), and record the actual time when the run ends. The board shows which ideas work.
In our runs: The small CLI tool was estimated at 1.3 h and took 31 min; the sidebar flicker fix at 2.0 h and took 1 h 58; the sidebar reorder fix at 45 min and took 51. The ranking in this guide (parallel agents 81 min saved in 10 of 13 closed tries, scripted UAT 28 in 10 of 11, pre-written briefs 13 in 8 of 8) comes from this ledger. Those tries were self-graded: a later audit found their baselines were estimates, not measured times.
Elsewhere: Lean's kaizen: small, measured improvements, all the time.
Watch out: Estimates drift towards actuals if the same agent writes both. Lock the estimate when the run starts.
Related: L1 Retro Done-Gate · C3 Contract-First Fan-Out

L4 · Harness over Prompt

guides and sensors

When a run fails, fix it with a check, a script or a gate, not with a longer prompt.

Problem: Instructions are cheap to add and easy to ignore. Agents follow them most of the time, which in a dark factory means some runs fail the same way.
Solution: Turn each lesson into a guide (it shapes the work before it starts: a template, a brief, a preflight) or a sensor (it catches the problem afterwards: a test, a gate, a refusal). Prose rules are the last resort.
In our runs: Our biggest wins were code, not words: the promote script treating cancelled CI as not red, the release-line gate in the merge train, approve-prod refusing a same-name verifier, and the retro check.
Elsewhere: Birgitta Böckeler (martinfowler.com) splits a harness into guides (feedforward) and sensors (feedback). METR found that telling models not to reward-hack had a “nearly negligible effect”. And from understandingdata.com: “Dark factory is a claim about your oracle, not about your agent”.
Watch out: Every sensor adds wait time. Weigh it against the trap it catches, and remove checks that never fire.
Related: L2 Trap Catalog · V1 Three Hats · A5 Default Answer

L5 · Budget Meter

budget-aware waiting

Meter every shared budget a run can drain (money, API quotas, CI minutes) and stop before it runs dry.

Problem: A dark factory runs many agents at once. Each one polling or retrying looks harmless; together they exhaust a shared quota and stall every other run.
Solution: Put a meter and a cap on each budget. Agents wait on events, not loops. A wrapper logs every call with the run that made it, so a drain can be traced with one command.
In our runs: Metered spend is capped at $5 a day. Hub rebuilds once made about 5,164 GitHub API calls an hour and hit a 403 that stalled every run; a logging wrapper now ties each call to its run. Recorded run costs ranged from $10.29 (the small CLI tool) to $113.28 (the admin page, five UAT rounds).
Elsewhere: StrongDM sets a floor: “If you haven't spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement.” Simon Willison's reply: “If these patterns really do add $20,000/month per engineer to your budget they're far less interesting to me.”
Watch out: A cap that emails the human when it trips is a founder question by another name (A1). Make the cap stop the run and log why.
Related: A1 Single Mandate · R4 Real Signal

Anti-patterns

What went wrong in our runs, or in published accounts, and the pattern that answers it.

Anti-pattern	What it looks like	Seen	Answer
Leaky Ask	A guard, alert or script contacts the human in the middle of a run.	A spend guard emailed the founder about $0.35.	A1 L5
Release Tail	The build is fast, then an hour goes on CI, checkouts, deploys and records.	Tail of 50 min to 1.5 h against 25–47 min of building (12 Oct).	R4 L2
Own Homework	The builder, or the orchestrator that wrote a fix, also verifies it.	Builder blind spots; METR measured reward hacking in 30.4% of runs.	V1
Unclaimed Rider	Another run's unapproved commit ships under your promote.	One joint promote; “unapproved rider below merge”.	R3
Shared-Budget Drain	Many small pollers exhaust one quota and stall every run.	About 5,164 GitHub calls an hour, then a 403.	L5
Blank Retro	Retros with no times, no shas and no change.	Of 78 retros, about 19% cited a sha and 15% a time.	L1
Hand-Written Clock	Times written from memory instead of read from logs.	A sidebar reorder fix: 75 minutes written down; the logs said 51.	L1 L3
Stage-Only Defects	Bugs that appear only against real services on stage.	Recurring in our runs.	C4 V3
Waiver Drift	What a waiver covers changes between runs, and nobody re-reads the list.	Desktop app 0.1.9 shipped under a standing go; a day later desktop releases needed a human.	A3
Comprehension Debt	Code ships faster than anyone understands it.	Addy Osmani: “Tests are necessary. They are not sufficient.”	A4 L1

When to use them

If you adopt six

A1 Single Mandate. Write down what the human gave away, word for word. Everything else follows from it.
A3 Can't-Undo List. Write down what they did not give away. This is what makes the rest safe to try.
V1 Three Hats. Never let an agent grade its own work. Cheapest pattern here, biggest effect.
V3 Scripted Acceptance. If you can't write the check as a command, you are not ready to run that change dark.
R5 Watch Window. Rollback sha first, promote second, a timed look afterwards.
L1 Retro Done-Gate. The run isn't over until the retro passes its check. This is where the other patterns come from.

When not to run dark

You can't write the check. If acceptance needs taste (tone of voice, visual design), the check is a person; run with the lights on. As one write-up puts it: “Dark factory is a claim about your oracle, not about your agent”.
The change is on the can't-undo list. Ship the reversible part dark and leave the rest for a human (A3).
You can't roll back in minutes. App store releases, published packages and sent emails don't come back.
An hour of a bad release costs more than the hours you save. Knight Capital's 2012 trading loss is the standard warning.
You are starting out. “No organization starts with a fully autonomous software factory.” Stripe's Minions merge over 1,000 PRs a week, each one reviewed by a person. Lights-on is a valid place to stay.

Conclusion

A dark factory is not a model good enough to trust. It is a set of patterns that make trust unnecessary for everything that can be undone, and keep a human on the few things that can't. Start with the six above, write down every trap you hit, and turn the lights off one pattern at a time.

Sources

Scry numbers come from our run records (dark-factory runs, the ETA experiment board, the release-tail trap catalog and an end-to-end time study of 54 items). Quotes from Shapiro, StrongDM and Willison were checked against the original pages; the other external facts come from the pages listed. ETA results are closed tries on our experiment board as of 13 October, graded by the runs themselves; an audit on 15 October found every baseline was an estimate, so read the savings as indicative.

Related: Dark Factory Tiers: How Strong a Model Set Does Each Job Need?, which model set each kind of job needs.