Every running platform now writes a paragraph after your run, and the paragraph quotes numbers: a distance, a heart rate, how this month compares with last. Sometimes a number is wrong, and nothing tells you which time. This page explains how debrief checks every figure in a note against the recording before the note is kept, what that catches, what it misses, and the hole our own audit found in it this month.

The figures below come from one production account over 127 notes, because that is the size of the beta today: invite only, one athlete with a full history. They are aggregates from the database, queried read-only on 6 September 2026; no note text was read to produce them.

What the platform notes get wrong

Strava’s Athlete Intelligence went into beta in late 2024. In a Singletrack forum thread on 6 October 2024 a rider wrote: “It’s also factually incorrect, unless ‘the last month’ actually means ‘so far this month’ in AI world, in that I’ve done rides over twice the distance.” When Strava took the feature out of beta on 20 February 2025, its press release said the update brought “more accurate insights with fewer mistakes.”

Garmin’s Active Intelligence launched with Connect+ in March 2025. The first insight DC Rainmaker received, in his walkthrough of 27 March 2025, was “my intensity minutes were the same as the past four weeks”; he wrote “I would struggle to think of a less useful tidbit” and “Garmin’s gonna need to do far better than this.” A year later, the5krunner’s review of 20 April 2026 found that “Active Intelligence continues to restate thin insights.”

The better notes do quote a figure. A sample Rod Trent posted on 10 March 2026 read: “Great run. Your 5k running section was your second best in the last 30 days with a time of 33:35.” That may be exactly right. The reader cannot tell, because the note does not say what it was computed from.

So a post-run paragraph fails in two ways: it says nothing, or it says something checkable that nobody checked. This page is about the second.

How a debrief is written

A debrief is not written from the FIT file. It is written from an evidence package that code builds from the file: the session summary (distance, moving time, heart rate, pace, decoupling, efficiency factor), the laps, the weather at the start point, training load, the athlete’s thresholds and zones, comparable runs from their own history, and the priorities set in earlier notes. Every value has a path: session.summary.decoupling_pct, laps[2].hr, conditions.dewpoint_c.

The writing model is instructed to wrap every number it takes from the package in a tag naming the field it came from:

decoupling <cite src="session.summary.decoupling_pct">6.8%</cite>

That is the whole contract. After the note is generated, a validator with no model in it resolves every tag’s path in the package and checks that the value inside the tag equals the value in the field, at the rounding the package printed. It checks the other direction too: a number outside any tag fails, because a number with no source is the thing we are trying to stop. So does a tag naming a field that does not exist, and a field name leaking into the prose. The one place uncited numbers are allowed is the closing Priorities block, where a coach sets a target instead of reporting a measurement.

If the validator finds anything, the draft goes back to the model once, with every violation listed beside the value the package actually holds, and one instruction: fix exactly these errors and change nothing else. If the second attempt fails, the note is rejected. A rejected note is never shown. The athlete is told the debrief was written and then thrown out because it did not survive its own fact-check, and can ask for another attempt.

When a note passes, the tags are stripped and paces are rewritten into minutes and seconds, and that is what the athlete reads. The tagged version is stored beside it with the exact package it was validated against, so any figure in any note can be traced. The annotated example debrief walks one note this way.

Since 24 August 2026 one more pass runs before validation: a bare number that matches exactly one field in the package receives its tag mechanically. A value two fields share is left alone, because guessing which one the writer meant would be a citation vouching for a claim nobody checked.

The numbers

Since the first production note on 28 July 2026, 127 debriefs have been written for one athlete. 105 were validated (83%) and 22 were rejected (17%). Of the validated notes, 56 passed on the first attempt (44% of all notes) and 49 needed the repair turn (39%).

The format matters. Before 5 August 2026 a citation was a bracketed field path after a claim, matched to numbers by looking back along the line: 74 notes, 17 rejected (23%). Since the tag format shipped: 53 notes, 17 passed first time, 31 were repaired, 5 rejected (9%). The repair turn does more of the work now, and rejection is rarer.

The pipeline logs every first-attempt violation. Since the tag format there have been 78: 58 a number with no citation, 11 a field name in the prose, 4 a malformed tag, 4 a path that does not exist, 1 a cited number that did not match its field. The model almost never puts a wrong number under a right citation. It forgets the citation.

An uncited number is not always an invented one. When the tagging pass was designed in August, a month of first attempts held 61 uncited numbers: 15 had exactly one source in the package, 21 had several, and 25 appeared nowhere in it. Those 25 are the invented numbers, roughly four in ten. The repair turn removes them, or the note is rejected.

Before it shipped, the pass was measured by stripping the bare-number tags from 60 stored notes (389 tags) and asking it to put them back: 274 restored to the exact field, 0 wrong, 115 left alone as ambiguous. A ranking heuristic for the ambiguous ones was tried against the same ground truth and rejected at 38 wrong out of 88. In production the pass has touched 1 of the 8 notes written since it merged, so its effect on the repair rate cannot be measured yet.

The quality of the writing is measured separately, by a golden set: 19 frozen evidence packages, each with written expectations of what a good note must observe. A second model call judges each generated note against them, scoring each item zero, one or two, and scores whether it reads like a coach and keeps heat and fatigue apart when both would lift heart rate the same way. The set has to average at least 1.4; the rerun against the current prompt on 5 September 2026 averaged 1.59, with 3 cases failing and left untuned. The validator itself has 95 tests; the engine’s continuous-integration suite has 454.

What went wrong anyway

On 4 September 2026 an audit of the engine found a hole in the tagging pass. It tagged a bare integer with the single field holding that value, and nothing compared the word after the number with the unit of the field. Take “You ran the 400 m reps hard.” Rep distances have no field in the package, so 400 is a bare number. If exactly one lap had a pace of 400 seconds per kilometre, the pass tagged the distance with the pace field:

You ran the <cite src="laps[1].pace_s_per_km">400</cite> m reps hard

Validation passed, because the value matched the field. Then the presentation pass that rewrites every cited pace into minutes and seconds rewrote this one, and the athlete would have read “You ran the 6:40/km m reps hard.” On a real evaluation package it reproduced: “the 300 m reps” became “the 5:00/km m reps.” For an athlete on miles and feet, a bare 145 beside a lone climb field of 145 m would have shipped as “476 ft.”

The measurement that said 0 wrong out of 274 could not have seen this: rep distances were uncited numbers the repair turn handled before the pass existed, and silently mis-tagged after it. No production note contained the case, and the exposure was estimated at a few percent of interval notes.

The fix shipped on 5 September 2026. A field that a presentation pass rewrites on the strength of its source alone (pace, distance, climb, mass, temperature) may now be the source of a bare integer only when the integer is immediately followed by that field’s own unit word, and the unit word rides inside the tag. “145 m” can cite a climb field; a bare 145 cannot, and stays uncited for the repair turn. Fields no pass rewrites, such as heart rate and cadence, stay open as before. 5 regression tests pin it, one on the evaluation package where it reproduced.

What this does not solve

A cited number is a number the recording supports. It is not necessarily the number worth talking about. The validator says nothing about whether the note picked the right story, judged the intensity correctly, or should have stayed quiet. That is what the golden set is for, and the golden set has 3 open failures today, two of them on reading a run against the athlete’s own heart-rate zones rather than a threshold percentage.

The judgment in a note is not validated. The package fixes what the model may say in numbers; what it makes of them is the model’s, checked only by the prompt and the golden set. There is no plan and no chat. A debrief is a written note after a run, and it stops there.

And the figures above are one athlete’s 127 runs. They will look different when other people’s watches, zones and habits go through the pipeline. This page will be updated when there is enough data to say how.

Where this stands

debrief is an invite-only beta. Suunto watches sync directly; Garmin and COROS runs arrive as a .fit file you export and drop on the upload page. Pricing is on the pricing page, and what the writing model receives is listed on the privacy page. If you would like to be one of the first strangers through it, the waitlist is below.

How often does an AI running note invent a number, and how we stop it