Software Factories

The software factory and its delivery boundary

A software factory is an organized system for repeatable, controlled software production. The production model here is an engineering synthesis: it follows the whole delivery flow, including decisions, waiting, and correction. A work item tracks one requested outcome through that flow; its identity survives individual execution attempts. [2]

An artifact is a retained output, such as code, configuration, documentation, or a build. Delivery requires both suitable artifacts and a recipient prepared to use them. The software delivery lifecycle includes the activities that develop, deliver, and maintain those outputs. [3]

Work item EXP-17 requests CSV export from an existing reporting service so analysts can stop manually copying report rows. Its contracts, identities, and outcomes below form a constructed example. The production map assigns responsibilities, not an agent to every box: one person or program may perform several stages while preserving their distinct acceptance decisions. [4]

Work intake, readiness, and admission

Work intake receives, clarifies, classifies, and selects requests. Triage determines the next disposition and responsible role. Readiness means sufficient information exists to undertake the work; admission commits current production capacity. The Scrum Guide similarly distinguishes refinement and readiness from selection without requiring every organization to adopt a separate readiness document. [8]

For EXP-17, the requirement owner decides intended behavior; the service maintainer owns technical suitability; the release owner authorizes exposure; and the receiving operator accepts ongoing operation. These are responsibilities that need named holders, not necessarily four jobs. The intake record identifies their decisions before execution starts. [9]

Use a locally agreed route based on consequences, uncertainty, and reversibility. A reversible documentation correction can receive lighter oversight than an uncertain access-policy change. Urgency changes priority, not permission. If the required decision exceeds the assigned role’s authority, the request goes to the authorized owner rather than directly to an implementation worker. [9]

Specifications and observable acceptance criteria

A specification records agreed behavior and constraints. An acceptance criterion is an observable condition used to assess a requirement. A revision identifies a particular version. Required behavior states what must happen and under which conditions; a design proposes how to achieve it. [10]

Given/When/Then expresses an initial context, an event, and an expected outcome. For R4: given an authorized report with no matching rows, when export is requested, then the downloaded CSV contains the specified headers and no data rows. Gherkin connects such examples to executable steps while keeping implementation details separate. [13]

Streaming, buffering, and background jobs are design choices constrained by S1. Generated tests provide evidence against its requirements; they cannot silently redefine them. The requirement owner approves S1 and later behavioral changes, while the maintainer reviews feasibility. Missing criteria require a decision, not an implementation guess. [15]

Production records and accepted handoffs

Business progress records accepted work and unresolved obligations. An execution attempt records activity. Several attempts can serve EXP-17 without creating several delivered items. Cases and authoritative state develops this distinction; the factory applies it to specifications, artifacts, checks, and stage decisions. [16]

A handoff transfers work and responsibility through acceptance. For EXP-17, the producer remains accountable while integration acceptance is pending. The recipient needs the candidate, supporting evidence, and authority to act. Accepted handoffs and responsibility explains why sending a link is insufficient; acknowledgment and resolution are distinct events. [20]

Each stage contract therefore names entry conditions, required outputs, an accepting role, and permitted waiting or rejection outcomes. A commit message can describe validation, but the receiving stage still needs the underlying evidence. Repository storage alone does not provide a trustworthy current status or resolve concurrent decisions. [19]

Production routing and bounded worker assignments

Production routing selects workers and stages according to prerequisites, capability, authority, evidence needs, and capacity. A dependency requires another contribution’s output; coupling describes how strongly changes depend on one another. Task boundaries and bounded roles develops these concepts. Shared tooling alone does not establish a dependency, and separate files do not establish independence. [21]

Agree EXP-17’s request parameters, response behavior, and access interface before parallel implementation, documentation, and acceptance-fixture work. Each contribution must remain reviewable and later join a combined candidate. Dependency-aware refactor batching illustrates this approach; directory grouping is only a heuristic for coherent work, not proof that batches cannot interfere. [22]

A coding agent selects investigation, editing, and execution actions using observed results. Its runtime makes those actions executable. Execution feedback and implementation decisions explains the internal loop; the factory supplies its assignment and decides whether the returned package meets the receiving stage’s contract. [23]

Use ordinary code for exact operations, agents for interpretation, and designated people for consequential authority. A worker that discovers an unresolved access decision should return it for clarification. Additional workers cannot manufacture the missing decision. [25]

A Git worktree provides another checked-out working directory with its own HEAD and index. It separates working-file edits, but repository configuration can remain shared. It does not isolate credentials, databases, ports, or network effects, and it leaves semantic integration obligations intact. [26]

Artifact integration and quality acceptance

Integration combines contributions and establishes that they work together. The NASA integration guidance includes interfaces, sequencing, configuration, and the integration environment. For EXP-17, the integration owner assembles identified code, dependencies, configuration, documentation, and build outputs against an identified target base. [27]

A clean merge reconciles text; semantic compatibility requires behavioral evidence. Revision-bound integration explains this boundary. GitHub’s merge queue tests a combination containing the proposed change, current target branch, and earlier queued changes. That combination has a different identity from the original pull request, so its required checks must run on the merge-group candidate. [28]

A quality gate prevents advancement when its required conditions are unmet. Warning-only validation leaves the transition uncontrolled. Verification checks conformity to specified requirements; validation assesses intended use. Passing S1 checks can establish export behavior without establishing that analysts’ reconciliation problem has been solved. [29][14]

A Definition of Done states shared completion-quality conditions. Work failing them does not become a releasable increment merely because implementation ended. Different workers contributing to one product need compatible completion conditions, otherwise local acceptance can conceal gaps at integration. [8]

Acceptance independence depends on evidence as well as reviewer identity. Another agent can share the constructor’s assumptions. Use requirement review and separately derived expected outcomes; reserve accountable judgment for ambiguous criteria. Oracles, rubrics and human judgments explains that design. Narrow judgments expose disagreements more clearly than a single assessment of overall quality. [33]

Bind approvals to the reviewed changes. GitHub can dismiss stale approvals or require approval after the latest push, but bypass settings matter. Required status checks can also accept skipped or neutral results. A locally required executed test therefore needs evidence of execution, not merely a green protection state. [34]

Release responsibility and confirmed delivery

Release responsibility covers authorizing exposure, confirming intended behavior, arranging intervention, and continuing ownership. Merging changes repository history; deployment installs an artifact; exposure makes behavior available to users. Operational acceptance records that the receiving role can take responsibility. Acceptance can occur separately from initial launch. [6]

EXP-17’s release package identifies the exact candidate, configuration and data dependencies, acceptance evidence, authorized target and exposure scope, limitations, recovery conditions, receiving owner, and pending obligations. The recipient receives operating instructions as well as artifacts. A different candidate or configuration requires reassessing the authorization and evidence. [3]

In EXP-17’s chosen route, deployment precedes controlled feature exposure. Confirm exports in the actual target environment, then obtain receiving acceptance. Negotiated follow-up work can remain visible without silently waiving mandatory quality conditions. Production readiness review illustrates checking dependencies, monitoring, response arrangements, documentation, and receiving capability. [6]

A canary exposes bounded production traffic to a candidate while a baseline supplies comparison. Version-specific observations inform promotion or intervention; unrepresentative traffic can miss defects. Controlled rollout and operational intervention covers the mechanics. The factory must supply the candidate, decision authority, stopping conditions, and receiving obligations that those controls enforce. [35]

Exceptions and accountable recovery

An exception is work that cannot advance under normal conditions. Its useful classification identifies the decision needed next. EXP-17’s local policy keeps the current owner accountable until a recipient acknowledges the handoff, records a response expectation, and bounds correction attempts. Escalation is an acceptable result when evidence, capability, or authority is insufficient. [36]

A permitted remedy may still be unavailable. One recovery-agent example proposes a safe schema correction, discovers that the executor cannot perform it, and returns the incident for manual review. Record the proposal as unexecuted. Capability failure must not become a completion claim. [43]

Notification, acknowledgment, and resolution are separate events. PagerDuty’s incident lifecycle illustrates continued escalation before acknowledgment and renewed escalation after an acknowledgment timeout. Each production exception similarly needs a locally defined response expectation and owner while waiting. Exceptions and resolution paths develops the general contract. [20]

Reverting local code cannot erase an already observed external effect. Recovery may require compensating actions that preserve intervening work and can themselves fail. Partial completion and business recovery explains those limits. An uncertain deployment needs reconciliation before repetition; an irreversible disclosure needs an authorized response, not a claim that rollback undid it. [42]

Requirement changes and selective rework

Requirements remain authoritative as the product changes. In EXP-17, approved revision S2 adds explicit export permission to S1’s report-viewing requirement. Access implementation, access fixtures, and access documentation now need revision. A living specification preserves this behavioral change rather than leaving it inside a later implementation conversation. [44]

Pause affected assignments and obtain approval of S2 before resuming them. The earlier acceptance remains historical evidence about S1; it does not authorize S2. For this example, serialization behavior, code, dependencies, and test environment remain unchanged. Its retained evidence can support that narrower claim while access-related obligations reopen. [44]

The repaired worker candidate is not necessarily the intended integrated candidate. If C2 includes a later role-mapping configuration change, S2 access checks must exercise that combination. Passing S2 on an earlier worker branch plus passing S1 on C2 leaves the crucial relationship unchecked. Advance only after the necessary results and authorized acceptance exist. [28]

Selective reuse depends on explicit dependency reasoning. Unchanged files alone are insufficient when their environment, integration base, or criteria changed. Updated review controls can invalidate approvals after changes. If S1 is already exposed, assign a separate operational correction obligation; producing S2 does not by itself contain the earlier behavior. [34][45]

Delivery feedback and controlled process improvement

Repairing one delivery and changing the production process are separate obligations. A useful incident record connects impact, causes, recovery, and concrete follow-up work. Assign each improvement an owner and tracking record; otherwise the immediate repair can finish while the conditions permitting recurrence remain unchanged. [46]

Retain the failed case with relevant specification, code, configuration, model, tool, environment, and evaluator versions. This proposed diagnostic record lets an owner assess a process change against the failure it should prevent. Retrospectives over agent conversations can identify repeated ambiguity and missing capabilities, but those findings are improvement proposals rather than demonstrated gains. [47]

Agents must not weaken protected acceptance rules to improve apparent success. Cases used to revise a process become regression evidence, not untouched assessment data. Test the revised configuration on fresh, representative cases before claiming general improvement. Coverage and independent assessment explains why even aggregate feedback can influence later choices. [49]

Work in progress and bottleneck-aware admission

Work in progress, or WIP, counts admitted but unfinished items. Backlog is accumulated waiting work; work-item age is elapsed time since starting. Throughput counts completions per unit time at a declared boundary. These definitions follow the Kanban Guide. [1]

Pull ready work when capacity opens, unblock aging items, and make priority exceptions explicit. Small, coherent batches help keep acceptance manageable. [1]

Substantive review has finite capacity. More implementation workers can create more outputs than maintainers can inspect. Review capacity and service commitments explains the receiving constraint. Protecting sustained attention can require slowing new implementation while reviewers resolve existing candidates, rather than adding more simultaneous requests for judgment. [50]

Assume a stable admission-to-acceptance system averages six unfinished items and two accepted completions per day, with no cancellations. Average residence time is $6/2=3$ days, including waiting. This is an accounting relationship, not evidence that imposing a six-item limit will produce those results. [51]

The figure separates internal congestion from deferred demand. Holding work before admission may improve admitted cycle time while intake-to-acceptance time remains unchanged. Review, integration, continuous-integration capacity, and feasible experiment volume can each constrain delivery; the limiting stage must be observed rather than inferred from idle agent slots. [52]

Accepted throughput and the limits of automation

Measure delivery at its agreed acceptance boundary. Count EXP-17 once when delivered, while retaining every attempt, rejection, and repair cost. Report canceled and unfinished work separately. A completed agent run is an execution observation, and a suggestion-acceptance event may precede extensive rewriting; neither is a direct productivity measure. [53]

DORA’s delivery metrics cover commit-to-production lead time, deployment frequency, failed-deployment recovery time, change fail rate, and deployment rework rate. Their boundaries are narrower than intake-to-accepted-delivery: earlier clarification and later receiving acceptance fall outside commit-to-deployment time. Deployment rework also excludes some review corrections. Interpret the metrics together for comparable services; they do not directly measure user benefit. [56]

Elapsed time and labor differ when people and agents work concurrently. METR’s experiment update describes concurrency, selective participation, withheld tasks, and incomplete outcomes as obstacles to productivity interpretation. Process time, quality, and total operating work develops the accounting distinction. Comparisons need matched work populations and visible missing outcomes, not only completed tasks. [55]

Maintainability costs can emerge only when later changes encounter architectural damage. Preserving human understanding remains a recovery constraint when agents cannot solve an incident. Automated review can help, but it does not establish that maintainers can safely evolve the system. [58][59]

A practical maintainability check is whether a fresh human or agent session can find the workflow, policy, supporting modules, and memory-maintenance instructions and make a bounded change. Difficulty doing so identifies organizational or documentation work that increased generation would leave unresolved. [60]

Evidence for retaining or simplifying coordination compares coordinating agents with a capable single agent and a prescribed workflow. Expansion requires better accepted outcomes without unacceptable defects, transferred labor, or unowned maintenance. Broader staffing, incentives, and portfolio choices belong to AI Engineering Leadership. [61]

Open questions

1. 01

   Selective evidence reuse needs stronger dependency models. Rechecking everything is costly, but incomplete lineage can preserve invalid approvals. Progress would mean detecting affected claims across requirement, configuration, environment, and base changes while explaining each retained result’s continued applicability. [64]

2. 02

   Useful concurrency limits remain workload-dependent. Coupling and variable review effort make agent-slot counts poor capacity estimates. Progress would require matched comparisons of sequential and concurrent work that include unfinished items, integration repairs, accepted outcomes, and total human effort. [5][55]

3. 03

   Maintainability needs evidence extending beyond immediate acceptance. Later change costs are delayed and difficult to attribute to individual generated decisions. Progress would be a longitudinal evaluation relating accepted changes to subsequent repair effort and maintainers’ ability to modify the system. [59]