The Software Factory: When the Lights Go Off
AI coding agents turned engineering into running a software factory. The one thing that decides whether it helps or wrecks you is verification, not generation.
Something quiet happened to the job while everyone was arguing about whether the models can code. The unit of work moved up a level. A year ago a good engineer wrote functions. Now a good engineer builds the thing that writes the functions, points it at a problem, and decides whether to trust what comes back. You are no longer the person typing the change. You are the person running the line that produces the change.
That line has a name that keeps getting reinvented, so let me use the old one. It is a software factory. And a factory has a switch on the wall that almost nobody is thinking about clearly: the lights. You can run parts of it dark, with no human watching the machines, and parts of it lit, with a person at the gate. Get that placement right and the factory multiplies you. Get it wrong and it manufactures a mess faster than you can read it.
Here is the claim this whole piece defends. The thing that decides whether your factory helps or quietly wrecks you is not how good the agents are at generating code. It is how cheaply and reliably you can verify what they produce. The skill of the next decade of engineering is knowing which loops you can safely run in the dark and which ones have to keep the lights on.
The stack: loop, harness, factory
Start from the smallest part and build up, because the words get thrown around loosely and the whole argument depends on keeping them straight.
A loop is one agent doing one job on repeat. It gathers context, takes an action, checks the result, and goes again until some condition says stop. That is the entire shape of an agent. Read the error, edit the file, run the tests, look at what broke, edit again. The loop is the atom.
A harness is the walls around a loop. It is the sandbox the loop runs inside so a bad command cannot touch anything real. It is the set of tools the loop is allowed to reach, the memory that survives from one run to the next, and the gates that define what “done” even means for this job. A raw model is not a harness. A model plus its sandbox, its tools, its memory, and its exit conditions is. Most of the engineering that makes agents actually work lives here, in the walls, not in the model.
A factory is many harnessed loops fed by a queue, drained through a review gate into production, with a human owning the whole arrangement from above. Tickets and signals go in one end. Shipped changes come out the other. Loops run in parallel inside.
The mistake almost everyone makes on first contact is to picture the factory as a bigger, smarter agent. It is not. A factory is an org chart made of loops. The intelligence is not concentrated in one place getting cleverer, it is distributed across many small workers, each doing a narrow job behind its own walls, coordinated by structure. Once you see it as an org chart instead of a brain, the right questions show up on their own. Who does each loop report to? What does it hand off? Who signs off before it ships? Those are management questions, and they are exactly the right ones.
The factory is a closed loop, and one box is expensive
Zoom out to the whole factory and it is itself a loop, just a slower one that runs your product.
Intent goes in. Some of it comes down from leadership as vision and priorities. Some of it comes up from engineers who know where the code hurts. Alongside intent, production sends signals back: an incident fired, a user asked for a thing, a metric moved. Intent plus signals fill a queue. The harness pulls from the queue and builds a change. Automated checks run against it: the test suite, continuous integration, static analysis, security scanning. A review gate decides whether the change is safe to ship. Deploy pushes it out. Monitoring watches production and turns what it sees back into new signals, which refill the queue. Round and round.
Now look at the cost of each box, because this is the crux. Generation is nearly free and getting cheaper every quarter. The queue is free. Running the test suite is cheap. Static analysis is cheap. Deploy is a solved problem. Monitoring is a line item you already pay. Almost every box in the loop is heading toward zero cost and infinite scale.
One box refuses. The review gate. The moment where something has to decide whether this change is actually correct, actually safe, actually the right way to solve the problem. That box is judgment, and judgment does not get cheaper when you add more agents. It is the one part of the factory that does not scale by throwing compute at it.
“Every box in the software factory is racing toward free. One box will not move. That box is judgment, and it is the entire game.
”
Hold that thought, because it is about to explain both the temptation and the trap.
The dark factory feels like breaking the sound barrier
The temptation has a name borrowed from manufacturing. A lights-out factory is one where the robots run in the dark, because robots do not need light to see. FANUC has run plants like this in Japan since 2001, machines building machines with the lights off for weeks at a stretch. Xiaomi opened a heavily automated smartphone plant in 2024 built on the same idea: remove the humans from the line and the line runs faster, longer, and cheaper.
The software version is a dark factory: code that ships to production without any human reading it, verified only by other machines. The tests pass, the scanners are green, the review agent approves, and it deploys. No person ever looked at the diff.
And at first this feels incredible. Here is why, and the reason is subtle enough that it fools smart teams. The review gate was the slowest box in your loop. It was the thing everyone waited on, the bottleneck that made shipping feel like wading through mud. When you remove it, throughput does not just improve. It feels like the line broke the sound barrier. Work that used to take a week clears in an afternoon. The whole team gets high on the velocity.
The costs of running dark are real, but they are buried. They do not show up in the demo, the sprint, or the first month of metrics. They surface later, quiet and late, which is the most dangerous shape a cost can take.
Comprehension debt: the bill nobody logs
Here is the buried cost, and I want to give it a name because naming it is half the battle. Call it comprehension debt: the widening gap between how much code exists in your system and how much any human still understands.
Every codebase carries some. The person who wrote the payments module left, and now three people sort of know how it works. That is normal, and you pay it down slowly by reading, refactoring, and documenting. A dark factory does something different. It does not pay comprehension debt down. It takes it on as fast as the machines can generate, and the tests stay green the entire way, which is exactly what hides the problem. Green tests tell you the code does what the tests check. They tell you nothing about whether anyone alive knows why the code is shaped the way it is.
Whether this kills you depends entirely on what you are building.
A greenfield app survives it. A weekend toy, a fresh prototype, a small tool with no history: a few months of dark generation gets these working, and if the accumulated confusion becomes a problem you throw the thing away and regenerate it. There is nothing precious underneath. Comprehension debt on a disposable codebase is not debt. It is just exhaust.
A decade-old brownfield enterprise system does not survive it. This is the codebase with the load-bearing hack from 2019 that six downstream services quietly depend on, the module whose behavior is its specification, the integration that works for a reason nobody remembers. Point a dark factory at that and three to six months in you are drowning. Not because the agents wrote bad code, but because nobody understands the system anymore, and the moment something breaks in a way the tests did not predict, there is no human left who can hold enough of it in their head to fix it. The velocity you loved was you borrowing against comprehension, and the loan came due.
Back pressure: verification is the constraint, not generation
So what is the actual rule? It is the same rule that governs any pipe, and once you see it in these terms you cannot unsee it.
I call this back pressure, borrowing the plumbing term for what happens when you push more fluid into a pipe than the far end can drain. Picture a funnel. Generation is the wide mouth at the top. Verification is the narrow neck at the bottom. The whole industry spent two years widening the mouth: faster models, cheaper tokens, more agents in parallel. But the neck did not widen. So all that extra generation does is deepen the pile of unverified work stacked up at the neck, waiting for judgment it cannot get.
This reframes the problem most teams think they have. They believe the issue is volume, that they need to generate more, review faster, ship more pull requests. The real issue is a surplus of pull requests nobody can trust. You do not have too little code. You have too much unverified code. Speeding up the mouth makes that worse, not better.
A smarter model does not save you
The obvious objection: fine, but the next model will be good enough that its output does not need much checking. This is where the argument gets interesting, because it will not, and the reason is structural, not temporary.
Think about what a model is trained to be good at. It learns from signals that are crisp and fast. Does this code compile? Do the tests pass? Is this the token a human would have written next? All of these answer in milliseconds and hand back a clean gradient to climb, and the model gets relentlessly better at anything with that shape.
Architectural quality does not have that shape. Whether a design will still be workable in two years, whether a boundary is drawn in the right place, whether this abstraction will help or haunt the next ten engineers: the cost function for these is measured in months and years, not milliseconds. There is no tidy gradient to descend. A system trained for instant, crisp evaluation is, almost by construction, not trained on the thing whose feedback takes a year to arrive. The model can write code that looks right and passes every fast check while making a structural decision that only reveals itself as wrong long after training could ever have caught it.
Which leads to the conclusion that reorganizes everything: the safety net cannot live inside the model. It has to live outside it, in the factory you build around it. You do not get to verification by waiting for a better generator. You get there by building a better net.
The lit factory: move judgment upstream
So keep the lights on. Not everywhere, and not by putting a human back in the slow middle of every loop. The lit factory leaves the lights on precisely where judgment lives, and it does one crucial thing: it moves that judgment upstream.
Most teams put the human at the end, reviewing generated code. That is the worst place for a person to stand. By then two thousand lines exist, and you are reverse-engineering what decision they encode. Move the judgment to the front instead, to product, design, and architecture, before the agent starts a single loop. Reviewing a two-hundred-line plan of what you intend to build is faster, clearer, and catches the expensive mistakes while they are still cheap to fix. You are approving a decision, not archaeology.
And the net that catches the rest is not exotic. It is ordinary software architecture, doing a second job it was never advertised for.
- Good types and honest method signatures mean the compiler catches a whole class of agent mistakes at build time instead of in production. Every constraint you push into the type system is a mistake the agent cannot ship.
- Test seams let you pin behavior in place so a loop can change code without silently changing what it does. The seam is where verification attaches.
- Legible layout means the next reader, human or agent, can find where a thing lives without holding the whole system in their head. Findability is a safety property now.
- Short, legible call stacks keep the blast radius of any single change small and traceable, so when something breaks you can actually follow it.
- Well-defined component boundaries contain damage. A change inside a bounded component cannot reach across the whole system, so a bad loop wrecks one room, not the house.
- Dependency injection lets you swap what a loop touches, isolate it, and test it in a harness, instead of letting it reach directly into everything.
None of this is new. Engineers have preached these for thirty years for human reasons. What changed is the payoff. Every one of these is now a cheap, hard-to-fake net against exactly the kind of mistake an agent will make, at exactly the scale agents operate. Investing in your architecture is no longer just craft. It is you buying back autonomy: the cleaner the net, the more of the factory you can safely run in the dark.
What earns a loop the right to go dark
Now the practical question, the one you actually make money or lose it on. Given a specific loop, do you run it dark or keep it lit?
The answer comes from one property: can the check that gates this loop be trusted without a human? A loop earns the dark only when its verification is cheap, runs at high frequency, and rests on something that cannot be easily faked, and answers immediately without drifting over time.
Run a loop dark only when a machine can tell you, right now and reliably, whether the result is good, using a check the agent cannot game. Keep the lights on wherever a wrong answer is expensive and only a person could catch it.
Layer 2 · Mechanismhow it actually works
The checks that earn the dark share four traits. They are cheap, so you can run them on every step without thinking about cost. They run at high frequency, so drift gets caught in seconds, not sprints. They rest on a hard-to-fake oracle: a green-or-red test, a type gate, a property test that holds for all inputs, or a review agent working from a real rubric rather than a vibe. And they answer immediately, without their meaning eroding over time.
There is a horizon limit that decides loop length. An agent tends to hold the thread for roughly three to ten steps of a task. Push past twenty and it starts losing coherence, because context accumulates and the earlier reasoning gets buried. So short loops are not just easier to verify, they are the only ones that stay reliable long enough to verify at all. A safe dark loop is small on purpose. A clean example: a nightly cron that fixes exactly one anti-pattern across the codebase and opens one small, readable pull request. Narrow input, machine-checkable output, tiny blast radius, done.
Layer 3 · Math & where it breaksgo deeper
Keep the lights on wherever the cost of a wrong answer is high and the error is the kind only a person catches. Three cases demand it. Subtle production bugs that every test passes but that corrupt data or leak state, because the oracle you would need does not exist in your suite. Large blast radius, where a single change can reach across many systems, so being wrong is expensive out of proportion to the diff. And decisions that shape a year of work: authentication, billing, public API contracts, the choices you cannot cheaply reverse once other things depend on them. For these, no machine check is trustworthy enough, and a human at the gate is not a bottleneck to remove. It is the whole point of having a gate.
You can stop after Layer 1 and still be correct about the test for running a loop in the dark, just less complete.
The human moved. The human did not leave.
It is easy to read all this as engineers being automated out of the loop. That gets it exactly backward. The human did not leave the factory. The human moved to a different position on the floor.
Agents run the inner loop: investigate the problem, implement a change, run the tests, report what they did. That is the work that used to be your whole day, and it is now theirs.
Engineers own the outer loop: decide that this is even the right way to address the problem, verify that the diagnosis and the implementation are actually sound, approve the change, and carry the consequences of being wrong when it ships. That last part is the one that does not delegate. The agent does not get fired when the billing bug charges every customer twice. You do.
What connects the two loops is evidence. Not trust, not vibes, evidence: the diff, the tests that prove it, the logs that show it ran, and a short explanation that ties them together into “here is what I changed and why you can believe it works.” Your job at the gate is to read that evidence and decide whether it holds. You are not on the line writing changes anymore. You are at the end of the line designing how the line works, and standing at the one gate that judgment cannot be automated out of.
The light switch is the job now
Here is the failure mode that swallows teams, stated plainly so you can avoid it: flipping every switch to the same setting.
Run the whole factory dark and you feel unstoppable for a quarter, then spend the next one discovering that nobody understands the system you now depend on, and you are tearing it down four months in. Run the whole factory lit and nothing moves, because every trivial change waits on a human review that did not need to happen, and you have rebuilt the exact bottleneck the factory was supposed to remove. All dark burns the house down slowly. All lit is one giant person-shaped traffic jam.
The skilled job, the one worth getting good at, is neither. It is deciding, loop by loop, where each switch goes. Dark where a machine can cheaply and honestly tell good from bad. Lit where a wrong answer is expensive and only a person can see it. That decision is the light switch, and placing switches well is the engineering judgment the next decade actually pays for.
“The old skill was writing the code. The new skill is knowing which loops have earned the dark, and which ones you never let out of the light.
”
I owe the lineage here. The term “software factory” is not new. It goes back to Bob Bemer’s 1968 paper “The economics of program production,” written when the dream was industrializing software the way we industrialized everything else. The sharper framing this piece is built on, the dark and lit factory, comprehension debt, and back pressure, I drew from Dex Horthy of HumanLayer and his talk “Harness Engineering is not Enough: Why Software Factories Fail.” I have rebuilt the ideas in my own terms here, but the shape of the argument is his, and it is the clearest map I have seen of the terrain we are all now standing on.
If you want the next piece of this, it is scale. One factory is a management problem. Many factories, many fleets of loops running across an organization at once, is a coordination problem, and it needs a different tool entirely: a graph, not an org chart. I take that apart in “Graph Engineering for Agent Fleets.” But it starts here, with the wall and the switch. Walk your own factory. Find the loops you are running dark out of habit rather than judgment, and the ones you are keeping lit out of fear rather than need. Move the switches on purpose. That is the job now.
