Code That Compiles Is Not the Deliverable: What We Check Before the Client Sees It

An LLM can write a function that compiles, passes the tests it wrote for itself, and does exactly what the prompt asked for. None of those three things guarantee it is what the client actually needed. That gap — between "the model answered correctly" and "this is shippable" — is the one we care about, because it is where the most expensive bugs sneak in: the ones that reach production looking finished.

The problem is not model quality

When something AI-generated fails in production, the instinct is to blame the model. That is almost never the real story. The model answered the question it was asked well enough. The problem sits one step earlier: the question did not carry all the context the deliverable needed, and nobody checked the gap between the two before closing the ticket.

A typical case: we ask an assistant to implement an order-cancellation endpoint. The code it produces cancels the order, updates the database status, and returns a 200. It compiles, it passes the tests the model wrote for itself, and a quick glance makes it look done. What it does not do — because nobody asked explicitly, and the model has no way of knowing it exists — is release the reserved stock, notify the billing service, or respect the cancellation window the business requires. None of that is a model failure. It is the result of treating the model answer as the deliverable instead of as a draft.

Why the same LLM cannot be the one that signs off

There is an understandable temptation: if the model wrote the code, have the model review its own work before handing it over. It does not work, and the limitation is structural, not a temporary gap in capability. The model that produced the solution shares the exact blind spot that created the problem. If it did not have the business context to write the stock-release branch, it will not have the context to notice that branch is missing, either.

This mirrors why a developer should not be the sole reviewer of their own pull request: it is not a matter of skill, it is that the person who wrote the code already decided, without noticing, which cases were "obvious" and which were not worth covering. With an LLM the effect is sharper, because the model carries no business memory between sessions and has no implicit grasp of rules that were never written into the prompt.

Four checks we run before calling anything shippable

Mandatory human review, with no exception for how small the change looks. There is no such thing as a change "too small to review." Most of the incidents we have seen with AI-assisted code did not come from large features — they came from one-line changes a human reviewer would have questioned in five seconds: a >= that should have been >, a default value that did not apply to one specific client case.

End-to-end tests written by a person, not by the model that generated the code. A test written by the same assistant that wrote the function tends to validate what the function does, not what it should do. We prefer writing the acceptance criterion first, in plain business language, and deriving the end-to-end test from that — independent of however the solution was implemented. A simplified example with Playwright:

test('canceling an order releases the reserved stock', async ({ page }) => {
  await createOrder(page, { product: 'SKU-123', quantity: 2 });
  const stockBefore = await getReservedStock('SKU-123');

  await cancelOrder(page, lastOrderId);

  const stockAfter = await getReservedStock('SKU-123');
  expect(stockAfter).toBe(stockBefore - 2);
});

That test exists in no prompt. It exists because someone knew the business rule and decided it had to be checked, regardless of which tool wrote the endpoint.

Explicit validation against acceptance criteria, not against "it works." "It works" means the code does not throw an exception. An acceptance criterion describes what has to be true for the business to consider the problem solved. Those are different things, and it is worth not letting the first one stand in for the second just because it is easier to check. Before closing any task that was generated or accelerated with AI, the original ticket acceptance criteria get reviewed one by one, explicitly — not from memory.

A staging demo before production, with the person who requested the feature in the room. Not a screenshot dropped in Slack — a live session where the person who defined the requirement watches the full flow run against data that resembles the real thing. This catches a kind of error that automated tests rarely catch well: a reasonable but wrong reading of an ambiguous requirement. The model interpreted "cancel within the first 24 hours" in a way that was grammatically fine but not what the business actually meant.

What AI changes, and what it does not

Where an AI assistant earns its keep is first-draft speed: sketching the endpoint, generating the trivial unit tests, proposing the migration schema. That compresses hours of work into minutes. None of the four checks above go away because of it. They apply exactly as they would if the code had been written by a junior developer on their first day — because in practice that is the right analogy: fast, capable, without accumulated business context, and without the authority to decide what actually ships.

The real risk is not that a model writes bad code. It writes reasonable code more often than many teams expected a couple of years ago. The risk is treating "reasonable" as a synonym for "done," and skipping the step where someone with full business context decides whether that is what needed to be built in the first place. That step does not automate away, because it is not a technical problem — it is a judgment call, and judgment calls are still made by a person.