What Is a Software Factory? AI Agents That Ship Code
A software factory has AI agents write, test and ship code from written specs. Here is how one works, what must be true first, where it breaks and what a small team can adopt.
A software factory is a setup in which AI agents write, test and ship code from written specifications, while people define what to build and judge whether it works. Nobody reads every line. The term sounds like a product you can buy, but it is closer to a discipline: most of the work sits in the specifications, the automated checks and the harness (the scaffolding that runs agents, gives them tools and stops them wandering). The model is the smallest part.
Our position, as a studio that builds with coding agents, is that a small team should adopt the front half of the idea now and leave the "no human reads the code" half alone unless failure is cheap.
What a software factory is, and where the idea comes from
The phrase is older than AI. Proposals for industrialised software production date back to 1968, according to ExplainThis, and they stalled because changing requirements needed human judgement at every step. Agents that can read a codebase, run tests and revise their own work remove some of that bottleneck.
The best-documented example is StrongDM's AI team. Its published charter says code "must not be written by humans" and "must not be reviewed by humans." Humans write specifications, curate end-to-end scenarios and watch a score, as described on StrongDM's factory site. Dan Shapiro's five-level scale, summarised by Simon Willison, calls this top level a "dark software factory": a black box that turns specs into software.
How it differs from using an AI coding assistant
An assistant sits beside a developer, who reads and accepts each suggestion. A factory moves the human upstream to the specification and downstream to the evidence. In between, agents work in loops until a stated completion test passes.
The useful way to see the gap is as three arrangements, not two:
| Arrangement | Who writes code | Who checks it | Sensible when |
|---|---|---|---|
| Assistant | Developer, with suggestions | Developer, line by line | Any team, today |
| Agentic pipeline with review | Agents, from a written spec | Automated checks first, then a human reviews the change | Most product teams with decent tests |
| Full factory | Agents only | Hidden scenarios and a score; no line-by-line review | Failure is cheap, easily reversed, and well simulated |
Most teams who say they run a factory are in the middle row. That is a good place to be.
The parts: specifications, agents, verification and the harness
Specifications come first. An agent given a vague ticket produces confident, plausible work that misses the point. A usable spec states behaviour, edge cases and what must not change, in a form an agent can test itself against.
Verification comes second, and it is where StrongDM put most of its effort. Its scenarios are written like step-by-step customer stories and kept outside the codebase, as a holdout set, so agents cannot memorise them. Success is a fraction of scenarios likely to satisfy a user rather than a green test suite. To run thousands of scenarios an hour, the team built behavioural clones of third-party services such as Okta and Slack, which avoids rate limits and API charges.
The harness comes third. It decides which tools an agent can call, what credentials it holds, when it stops and what it logs. Connecting agents to company systems safely is its own subject; our guide to connecting an AI agent to your company's tools with MCP covers that layer. Splitting work across several agents adds more design choices, which we weigh in single-agent versus multi-agent architecture.
What has to be true before a factory works
A factory amplifies the organisation around it. The 2025 DORA report on AI-assisted development says AI "magnifies the strengths of high-performing organisations and the dysfunctions of struggling ones," and that delivery instability kept rising even as throughput improved, as reported by InfoQ. Three preconditions matter most:
- Tests that already catch real regressions. If your suite is thin, agents will happily break things it does not cover.
- Requirements someone can write down. If the team argues about what the feature should do, an agent will pick an answer for you.
- A codebase an agent can navigate: clear module boundaries, consistent naming, documented commands for build and test.
Where it breaks
Agents optimise for the check they can see. The Stanford CodeX write-up on StrongDM reports an agent that wrote return true, which passes a test and does nothing. Its author also points to circularity: when the same class of model writes and judges the code, both can share the same blind spots.
Security is the second weak point. Veracode tested 80 coding tasks across more than 100 models and found security flaws in 45 per cent of cases, with no improvement from larger or newer models. A factory needs security scanning and threat-aware scenarios built in, because the model will not volunteer them.
Accountability is the third. The CodeX piece asks who is liable when a flaw slips through, and warns that when people stop reading code, the expertise needed to diagnose failures can fade. Novel architecture is the fourth: agents are strong at extending patterns that exist and weak at inventing ones that do not.
Matching the level to the codebase
The same team can sensibly run different levels on different systems.
| Codebase | Recommended level | Why |
|---|---|---|
| A SaaS product's routine feature work (new filters, settings pages, API fields) | Agentic pipeline with human review | Patterns exist, tests exist, a reviewer can read a small diff in minutes |
| An internal reporting tool at a services firm, used by twelve people | Near-full automation | Low blast radius, easy rollback, users report problems quickly |
| A codebase where a defect has legal or physical consequences | Assistant, plus agent-written tests that humans review | Accountability cannot be delegated to a satisfaction score |
What a small team can adopt now, in order
- Write acceptance criteria for every ticket in plain language before any agent touches it.
- Make the test and build commands one-line and reliable, then document them where agents can find them.
- Let an agent open pull requests on small, well-specified tickets, with a human reviewing each one.
- Add a holdout set: a handful of end-to-end scenarios the agent never sees, run before merge.
- Add security scanning and a rule that agents cannot change the checks that judge them.
- Only then ask which review steps can be dropped for which parts of the system, one area at a time.
The practical test for step six is simple: if a wrong merge in this area would be found and reversed within an hour at trivial cost, review can thin out. If not, it stays. Teams building the guardrail side of this will find the groundwork in our post on evals and guardrails for AI agents in production.
What it changes about cost and team shape
The cost moves from salaries to tokens and to the people who write specifications and scenarios. StrongDM's CTO has suggested that a factory spending under $1,000 per engineer per day on tokens "has room for improvement." Over roughly 250 working days that is about $250,000 per engineer a year, the same arithmetic ExplainThis draws. Most small teams should not plan on that scale, and a pipeline with human review costs a small fraction of it.
Team shape changes more than headcount. You need fewer people typing and more who can specify behaviour precisely, design tests and read a diff critically. Those are senior skills, so a factory does not remove the need for experienced engineers; it concentrates the work on them.
If you want to work out which of your systems suit an agentic pipeline and which should stay human-reviewed, we run a free 30-minute workshop on exactly that. Book it here.
Sources: StrongDM; Stanford CodeX; Simon Willison; Veracode (July 2025); InfoQ on DORA 2025; ExplainThis.
