# The follow-up tax: what the first full run found

Run `results/2026-10-22T15-11-57` (22 Oct 2026), the first full run after
the method was committed. Every prompt, answer, judgement and follow-up is
in that report; this page only sums it up. Method: `METHOD.md`.

**What it is:** 20 everyday tasks, each asked 3 times as typed and 3 times
with Perry's free rewrite (no profile). Chat AI and judge: Google
gemini-3.5-flash. Perry: gemini-3.5-flash-lite. It describes THIS test
only; it is not a general measurement of how much time Perry saves.

## The numbers

| | As typed | With Perry |
|---|---|---|
| Needs met by the first answer | 37% | 52% |
| Follow-ups until everything was there (average of 60) | 3.57 | 2.73 |
| Right on the first answer | 3 of 60 | 6 of 60 |
| Never got there in 5 follow-ups | 11 | 7 |
| Words the person typed (average) | 31.7 | 32.1 |

The person typed about the same number of words both ways. With Perry,
some of those words went into his [blanks] before the first answer
instead of into corrections after it.

Per task (average follow-ups, a difference under half a follow-up counted
as the same): **Perry better on 10** (cover letter, declining a wedding,
candle listing, newsletter subject, wedding toast, LinkedIn post, dinner,
CSV script, gift ideas, asking for a raise), **the same on 7**, **worse on 3**
(meeting notes, bedtime story, lost luggage).

## What went wrong, and why

**Meeting notes: a real Perry bug.** The draft was "summarise these meeting
notes:" with the notes pasted below. Perry's rewrite dropped the notes and
ended with "[paste your meeting notes here]", so the AI had nothing to
summarise. As typed, the AI got it right first time in all three repeats.
In the extension, that means the person's pasted notes disappear from the
box. Material the person already pasted must stay in the prompt. (The run
counts it as "never got there"; a real person would notice and paste
again, so that overstates how far it would go, but it is still a failure.)

**Fixed the same day in perry-v21:** the same draft sent to the live
Perry 3 times kept the notes word for word every time (it had lost them 3
times of 3), and the eval has a new case for it (`pasted-text`, 5/5). The
numbers above are from before the fix and stay as they are.

**Bedtime story and lost luggage: a little worse.** The difference is about
one follow-up, partly from Perry choosing a length or tone the person
didn't want. Small, and within what three repeats can tell apart.

**Excel formula: a weakness in our test, not in either way of asking.** The
need "explains in one or two sentences how the formula works" made the
judge fail any explanation longer than two sentences, and once the person
asked "explain briefly", the AI answered without repeating the formula, so
the latest answer missed the formula. Neither way of asking finished
(6 of 6 attempts stuck). The judge only reads the latest answer, which is
right for an email but too strict for a question answered over several
messages. Noted here and left as it is: changing the task after seeing
results would break our own rule.

## Cost

840,440 tokens in and 1,325,352 out for the chat AI, judge and fill step
(gemini-3.5-flash), plus 20 Perry calls. The exact amount is on Google AI
Studio's billing page.

## What this may and may not be used for

- It may be described as what it is: "In our test of 20 everyday tasks,
  with the method and every answer published, ...", linking to all of it.
- Never as "Perry saves X%" or "X fewer follow-ups" in general. The chat AI
  isn't ChatGPT, 20 tasks is a small sample, and we wrote the tasks.
- Any use on the website, in the store or in ads is the founder's call
  first (§ 5 UWG).