What AI app builders get wrong: 80 spec gaps from 8 build tests

The short answer

When an AI builder builds from a written spec, the code is rarely the problem. The spec is. We gave Claude eight of our blueprints (an approval workflow, a CRM, a commission calculator, a quote builder, a content calendar, a help desk, an expense report app and a time-off tracker) and had it follow each blueprint's phase prompts in order, then checked every acceptance item in a real browser. All eight builds passed every check in a final run on fresh data: 207 acceptance checks in all. Along the way, and in one item-by-item re-check, they exposed 80 gaps in the specs: places where the builder had to guess because the spec was silent, ambiguous or contradicted itself.

The gaps fall into eight types. The biggest is sections that contradict each other (18 of 80), followed by undefined edge cases (15), missing fields and definitions the rules depend on (13), ambiguous wording (11), formats, schedules and integrations (8), checks that depend on data or timing (8), phases in the wrong order (4), and no screen for managing people (3). Every one has been fixed in the published blueprint. The more a tool talks to the outside world, the more gaps: the help desk, which receives email, had 21.

The builder did not hide these gaps: each phase prompt ends with "list what you built and anything in the spec you had to interpret", and that sentence is what surfaced them.

How we tested

Each blueprint was given to the builder as a single Markdown file, exactly as a reader would download it. The builder sent itself the phase prompts one at a time and built only what each phase asked for. Every acceptance check was written as a browser script (Playwright driving Chromium) that clicks through the app as the named person, and where a check says the server must refuse something, the script also makes that request directly. After the last phase, every check from every phase ran again in one final pass on fresh seed data. Four of the Approval Workflow's gaps came from a later re-check, in which we checked each of its 20 items by name with 65 browser assertions. The full method is on How we test.

The eight kinds of gap

1. Undefined edge cases (15 gaps)

The spec described the normal path and said nothing about the state next to it.

• Unarchiving a contact with no stored follow-up date (CRM). Fix: it gets today.
• Reopening a won or lost deal whose contact has since been archived (CRM). Fix: refused until the contact is unarchived.
• Opening the next commission period when no period exists yet (Commission Calculator). Fix: open the current month.
• Asking for discount approval when every manager owns or requested the quote, so nobody can approve it (Quote Builder). Fix: the request is refused and the quote stays in draft.
• Taking a quote back to draft without saying which fields reset (Quote Builder). Fix: list them, including the expiry-reminder marker.
• Editing a content piece after it was approved, which let changed content go out without review (Content Calendar). Fix: refuse those edits until the piece is withdrawn or reopened.
• A canned reply addressing a requester who gave no name; reopening a ticket whose agent has left; an inbound email with no subject or a very long body (Help Desk, three gaps). Fix: "Hi there", unassign, and trim instead of refuse.
• An approver who is inactive when the request is submitted; escalation handing a step to someone who owns a later step; reporting loops; whether reassigning restarts the reminder clock; and how the approval chain shows a delegated step that later escalates (Approval Workflow, five gaps). Fix: a rule for each.
• Opening last year's balance, which the spec said should create its allowance rows, after which closing the current year was refused because an earlier year was still open (Time-Off Tracker). Fix: viewing a year never writes; rows are created when something is saved.

How to avoid it: for every action in your lifecycle table, ask what happens when the other record involved is archived, inactive, missing or already in the final state.

2. Sections that contradict each other (18 gaps)

• A screen description hid owner-only actions, while the acceptance checks expected a non-owner to try them and see the refusal message (CRM). Fix: show the actions to everyone and refuse on the server.
• "Every action writes exactly one event", next to a Duplicate action that writes an event on both quotes and client views that only record the first (Quote Builder). Fix: state the exceptions.
• An expected payout of $10,060 for August that the build instructions could not produce, because they excluded a deal cancelled after approval (Commission Calculator). Fix: count deals cancelled after the period was approved.
• A CSV check that didn't match the column order in the export rule, and an approver stored as an ID but checked as a name (Commission Calculator). Fix: one definition of the columns.
• A phase prompt referring to refusal messages that didn't exist (Commission Calculator). Fix: write the message.
• A Slack example, "Approved by Maya Chen at 09:14", that clashed with the date format the same section required (Approval Workflow). Fix: show the text as it renders.
• Two sections disagreeing on who may withdraw a piece from review; a screen that hid the Move action while a check expected a writer to try it; a check that skipped a step the workflow requires (Content Calendar).
• Seed data where a ticket was "solved by" a reply sent a day before it was solved; a worked example sitting exactly on the boundary the rule excludes; a rule saying a ticket stays New while another section let agents change its status; threading by email headers with no sender check, against an edge case that says strangers start new tickets (Help Desk).
• Two rules naming different people who can see receipts, and a reminder example saying 3 days where the check expected 4 (Expense Reports).
• A calendar rule that leaves days a person doesn't work grey and blank, against a check expecting "PH" in every row on Christmas Day; an edge case keeping a started pending request "under Waiting for approval" when Who's out only had a next-week section for it; and a rule adding the year to dates outside the current year, against a year-end refusal message checked without it (Time-Off Tracker).

How to avoid it: compute every number in your acceptance checks from your own rules and seed data, and state each fact (a column list, a message, a permission) in exactly one place that others refer to.

3. Missing fields and definitions the rules depend on (13 gaps)

• Sign-in matches a person by email, but the person had no email field (Commission Calculator), or the seed people had no email addresses (Quote Builder).
• The currency was "set once by an admin", with no setting or screen for it (CRM).
• An approval stored the discount in whole basis points while another section said percentages are never rounded (Quote Builder).
• Seed people without email addresses (Content Calendar, Help Desk, Expense Reports).
• Flags with no rule for when they are computed, and an approver change with no event, email or comment rule (Expense Reports).
• No storage for a form's rate limit, a ticket counter that survives deletion, or email retry counts; a rule saying "write the events in section 6" when section 6 named one; a check inspecting an event no screen shows (Help Desk).
• "Every action writes exactly one event", with no event type for changes to people, teams, leave types, holidays or settings (Time-Off Tracker).

How to avoid it: read each business rule and point to the field, list or screen that holds every value it mentions.

4. Ambiguous wording (11 gaps)

• A what-if "split %" could mean the rep's own share or the other rep's (Commission Calculator).
• "Duplicate" didn't say whether the quote discount, tax rate, terms, title and notes come along, and a check depended on it (Quote Builder).
• "An extension must be later than the current date" could mean today or the current expiry date (Quote Builder).
• Whether logging an activity also writes its own "Follow-up moved" entry (CRM).
• "A direct API call is refused" without saying what a refusal looks like. The build refused with an error message and no change, not an HTTP 403 (Approval Workflow). Fix: a rule that defines a refusal.
• "Notifications show as logged" without saying where (Approval Workflow). Fix: they are recorded with the status logged.
• Whether the Solved queue includes closed tickets; how queues sort tickets that already had a reply; what "the last email in the thread" is when there are internal notes; what "tickets solved" counts in a weekly report (Help Desk).
• "Month" in a spending report: the month of the expense, the submission or the payment? (Expense Reports).

How to avoid it: whenever a sentence uses "the current", "same", "copy", "refused" or a bare percentage, say exactly which one or what it looks like.

5. Phases in the wrong order (4 gaps)

• Phase 1 asked for a live commission example on the Plans screen and said not to calculate commissions until Phase 2.
• Phase 1's seed data needed a cancellation and adjustments that only exist from Phase 3 (Commission Calculator).
• Phase 1 asked for archiving rules and checked a deal move before archiving or deals existed (CRM).
• Phase 1 asked for a rule about deactivating approvers before reassigning reports existed in Phase 4 (Expense Reports).

How to avoid it: read your build plan backwards. Nothing in phase N may need a screen, action or rule from a later phase, including the seed data.

6. Formats, schedules and integrations (8 gaps)

• A daily digest "at 08:00" on a scheduler that runs in UTC, so it moves with daylight saving; the cron header wasn't named; the development clock didn't say it runs the job; and a subject could read "0 follow-ups due today" (CRM).
• Export columns without formats: dates, percentages, counts or lists (CRM).
• Slack messages showing times in UTC (Approval Workflow).
• Another daily job "at 08:00 team time" on a UTC scheduler (Content Calendar).
• How to round "Due in 21 h"; which header carries the cron secret and how many retries in all; and an inbound email endpoint that required a Bearer header most email providers can't send (Help Desk). Fix: whole minutes rounded in the reader's favour, named headers, and HTTP Basic auth as an alternative.
• A CSV export that defined dates but not its timestamps or its yes/no column (Time-Off Tracker).

How to avoid it: for every export, email, scheduled job and webhook, write the exact format, time zone, trigger and authentication, and check that the services you name can actually do it.

7. No screen for managing people (3 gaps)

The Commission Calculator, the Quote Builder and the Content Calendar all said "ask an admin to add you", but no screen added, changed or deactivated people. Fix: say who manages people and where, even if the answer is "outside the app".

8. Checks that depend on data or timing (8 gaps)

Some checks changed data that later phases' checks relied on. Adding a $10,000 deal in Phase 1 changed every October total checked in Phase 2 (Commission Calculator), and moving the clock forward in Phase 2 changed the overdue counts checked in Phase 4 (CRM). The Content Calendar's board check expected the seed's counts after an earlier check had added an idea. In the Help Desk, checks quoted durations that are only true within a minute of loading the seed, a report check depended on the weekday the seed was loaded, and a deletion check could not fail on fresh data. The Expense Reports seed loaded relative to today while its checks quoted fixed dates, and only its last phase said to reset. Fix: every blueprint now says to reset to the seed data before each phase's checks, that checks within a phase run in order on the same data, and when timing matters. The order of checks within a phase matters too: in the Time-Off Tracker, an early Phase 1 check deactivated a person whom a later check expected on the allowances list, which showed active people only. Fix: the list now also shows inactive people who still have an allowance that year.

A checklist for your own spec

1. Every rule points to the fields it needs, and the seed data fills them.
2. Every lifecycle action says what happens when the related record is archived, inactive or missing.
3. Every number in an acceptance check can be worked out from the rules and the seed data.
4. Each fact is written once: one column list, one message, one permission table.
5. No phase depends on anything built in a later phase.
6. Each phase's checks start from fresh seed data.
7. Exports, emails and jobs have exact formats, time zones and triggers.
8. The spec says who adds, changes and removes people.
9. Each phase prompt ends with "list what you built and anything in the spec you had to interpret".

For the structure the blueprints use, see the AI app spec template; for the whole method, see How to build internal tools with AI.

Questions people ask

Do AI app builders write bad code?

Not in these tests: the final runs passed every check. The one application bug a builder reported, a crash on the CRM's New deal page, was caught by a check and fixed before the final run. The gaps that mattered were in the specs.

How many acceptance checks does a spec need?

Enough that every rule a person would notice is checked once in the browser. Our blueprints have 14 to 32 checks across four phases, each naming the person, the action and the exact result to see, including server-side refusals.

Why do AI builders guess instead of asking?

Because they are built to finish the task. When the spec is silent they choose the most sensible reading and keep going. Asking each phase to list what it had to interpret turns those silent guesses into a list you can review.