Specification-Driven Development

Tom Foster

— 3 min read

Large Language Models (LLMs) are changing how we write software. Their ability to generate code at incredible speeds is a game-changer, but this power comes with a critical challenge: unpredictability. Because LLMs are probabilistic by nature, we can't always guarantee their output is reliable or correct.

The solution isn’t to abandon these powerful tools, but to manage their unpredictability with discipline. This calls for a return to specification-first thinking, reimagined for the AI era as Specification-Driven Development. By carefully engineering the context we provide an LLM, we can turn a static spec document into a dynamic, executable guardrail. This approach harnesses the LLM's speed while enforcing the rigour needed to build robust and maintainable systems.

The Specification as a Living Contract

At its heart, software engineering has always been about turning human intent into reliable machine instructions. The specification (a clear description of what a system must do) is the bridge.

With LLMs, the distinction between specifying the what (requirements, boundaries, outcomes) and the how (the exact implementation) becomes more important than ever. By providing a clear specification of the what, we establish the necessary constraints. The spec becomes a contract that defines the goal, leaving the LLM to write the code.

SDD merges the speed of Agile development with the discipline of the Waterfall model. The key difference is that the specification is no longer a static document written before development begins. Instead, it becomes a living, machine-readable contract that actively guides the creation of the software.

Context Engineering: Building the LLM's Worldview

Getting an LLM to reliably follow a specification isn’t about crafting one perfect prompt. Prompt engineering is tactical; it's about writing a single instruction for a single output. Context engineering is strategic. It’s the architectural discipline of designing the entire information ecosystem that feeds the LLM the right data, tools, and constraints to do its job well.

Most failures in AI coding agents are context failures. An LLM's success depends almost entirely on the quality of the information it receives within its limited context window. The developer's job is to ensure the model gets precisely what it needs.

Ultimately, a well-formed specification is the most powerful context you can provide. It acts as the definitive statement of the problem, a persistent source of truth, and the foundational guardrail for every instruction that follows.

The "Spec-then-Code" Workflow

In this new workflow, the developer’s role evolves from a line-by-line coder to a system architect and verifier. The process is systematic:

  • Create the Specification. Write a detailed technical spec, for example, in a spec.md file. It must define the "what" and, crucially, include explicit verification criteria. How will you prove the task is done?
  • Generate an Implementation Plan. Feed the spec to the LLM and ask for a detailed, step-by-step implementation plan. For complex steps, you can repeat this process recursively, breaking down big problems into smaller ones.
  • Execute the Plan. Guide the LLM through the plan one step at a time. For each functional step, write comprehensive tests first.
  • Verify with Evidence. A task is complete only when there is proof. This means passing tests, a working demo, and satisfying every criterion from the original specification. Trust, but verify with code.

Adopting an SDD workflow offers advantages, but it also introduces new challenges.

Benefits:

  • Quality & Correctness: Building from a spec with automated verification leads to more reliable code.
  • Maintainability: The specification acts as living documentation that stays in sync with the code.
  • Traceability: It creates a clear, auditable trail from requirement to spec to code to test.
  • Liability Shield: If something fails, the specification provides a clear record of intent, helping to distinguish between a flaw in the plan (human) and a flaw in the generation (AI).

Challenges:

  • Non-Determinism: LLMs can still hallucinate or produce inconsistent code, making rigorous, automated verification essential.
  • Security Risks: AI-generated code can contain subtle vulnerabilities, and the models themselves are susceptible to prompt injection attacks.
  • Context Window Limitations: Managing context for large, complex codebases remains a significant engineering hurdle that requires smart retrieval and summarization techniques.

The Engineer, Promoted

This shift doesn't replace the software engineer; it elevates them. The focus moves from the mechanical act of writing code to the higher-level work of:

  • Precise Specification: Deeply understanding a problem and articulating it without ambiguity.
  • Architectural Design: Making the high-level decisions about system structure and technology.
  • Critical Validation: Acting as the ultimate authority on quality, designing the verification strategy, and holding the final responsibility for the system's integrity.

This new era doesn't replace the engineer; it promotes them from a writer of code to an architect of systems.

Read more