Take a payment-retry fix. The agent's one end-to-end test passes, the readiness gate goes green, the pull request merges, and the intent moves to Shipped. Seven days later the retry is firing twice on slow networks. Checkout conversion is down 4%, and no test caught it, because the test only proved the happy path ran.
The spec had a Verification section. It held a test list, not a loop. The gate asked the question it is built to ask, is there at least one check concrete enough to run?, and nothing asked the one that mattered: how will we know, in production, that the intent held?
Key Takeaway: A test list proves the code runs. A feedback loop proves the intent held.
The green-tests trap
Pathmode's readiness gate marks verification as met when the spec has one check specific enough to run: at least three words, and not a placeholder. "Test it" fails. "Run the import twice and confirm no duplicates" passes. That bar is deliberate. The gate decides whether a spec is concrete enough to hand to an agent; it does not grade your verification. So passing the gate is not running the loop.
That tradeoff is the reason for this guide. The thin gate leaves the real work, deciding what is worth proving, to you. A test list lets you off the hook the moment something is green. A loop does not close until the intent holds in the world.
"Did it run" vs. "did the intent hold"
A test answers a narrow question: did this code path execute as written? That is useful, and it belongs to the How layer. The tools own the tests.
A verification loop answers a different question: after the agent walked away, is the user problem you started from solved, and is everything around it still standing? That belongs to the Why layer, and no passing test can answer it for you.
This is the return path in the product intent loop of an AI-native SDLC: production evidence has to travel back to the decision that predicted the outcome, or the system ships without learning.
The payment fix crossed exactly this line. The retry test was green. The intent, users complete payment without being charged twice, had broken. Context isn't judgment: a passing test is context; choosing what is worth proving is judgment.
The five checks of the loop
A complete loop answers five questions. Each one is a kind of check in the spec's verification.checks collection, and every check carries a live status (unknown, passing, or failing) that an agent or a person updates as the work moves:
- The fastest check (
fastest). The quickest signal that the code is broken: a typecheck, a single unit test, a lint rule. - The highest-confidence check (
test). The most reliable signal that it works: an end-to-end or integration test. - The manual fallback (
manual). What a person runs when automation cannot reach: a flow you click through, a query you eyeball. - The observable signal in production (
shipped-signal). The metric, log, or event that confirms it shipped correctly, not just that it shipped: conversion held, error rate flat, the p95 you promised. - What must not regress (
regression-guard). The adjacent behavior this change could quietly break, named specifically.
The first three run before the merge. The last two are about what happens after it, and they are the ones teams skip. The payment fix had a test and nothing else. A shipped-signal on checkout conversion and a regression-guard for "a retry never charges twice" would have put someone on watch for exactly this failure.
💡 Tip: If every check in your loop runs in CI, you've written a test list. A real loop almost always has at least one check that runs after the merge.
The verify-intent skill walks through the five one question at a time, proposes an answer for each from the outcomes and the codebase, and writes them into verification.checks once the loop is complete. Older specs may still carry unitTests, e2eTests, and manualChecks lists; Pathmode reads them as checks, but new work belongs in checks.
Health Metrics back the regression check
Outcomes say what should change. Health Metrics say what must not. The agent optimizes hard for the outcomes you handed it, so name what it could break on the way, or it will not know to protect it.
A regression-guard check says how you will catch the break. The Health Metrics field says what must not degrade at the product level. Write both. The in-app Verification Checkpoint builds its checklist from your outcomes, your Constitution rules, and your health metrics, so a health metric you never wrote is a regression nobody is asked to confirm. (For what each field holds, see The Anatomy of an Agent-Ready Spec. This guide is about operating them.)
Close the loop
A loop that only opens is a wish. It closes in three places:
- The merge. When the branch or pull request names the intent, merging it moves the intent to Shipped. Nobody marks linked work shipped by hand.
- AI verification of the pull request.
verify_implementationtakes the PR link, reads the diff from GitHub, and grades each outcome, constraint, constitution rule, edge case, and verification check against what the code shows, with per-item reasoning and a score. Without a PR, the agent can only describe its own work, and the result says the grade rests on a description and no code was read. It is a reviewer reading code, not a CI run. - After ship. The production signal only counts if someone reads it. Record what happened to an outcome with
record_outcome_measurement: send the measured value together with the expectation, and the server computes whether the target was met. A "met" the agent asserts on its own is stored as self-reported. For work that never goes through a pull request, the human Verification Checkpoint records what held before you mark the intent Shipped or Verified; unchecked items stay visible, so a gap is a decision you can see.
Pathmode does not watch your production metrics for you. What these paths do is write the result back onto the intent, so the next iteration starts from what happened, not from what was planned. That write-back is what makes it a loop and not a checkbox.
Write the loop, not the list
Before you hand a spec to an agent, your verification should answer five questions, not list N tests:
- What's the fastest check that says it's broken?
- What's the highest-confidence check that says it works?
- What's the manual fallback when automation can't reach?
- What's the observable signal in production that proves the intent held?
- What must not regress, and is it also a Health Metric?
If you can answer the first three and go blank on the last two, you've found the checks you were about to skip. That blank is where the 4% hides.
Key Takeaway: The gate asks whether a runnable check exists. The loop asks whether the intent held. Only one of them notices when checkout quietly drops.
Open any shipped intent and look for a shipped-signal check. If there isn't one, that is the check worth writing next.