Reliability

The claims, executed rather than asserted

Every scripted case below was executed in full to produce this page: tools called against the dataset, figures computed, the groundedness verifier run over each narrative, 8,679ms end to end. Nothing is a stored score. Click any figure to re-derive it on demand, or take the sandbox at the bottom and try to get a fabricated number past the gate.

Figures verified

1 blocked across 15 cases.

Each one recomputed from source, not compared to a cached value

Groundedness rate

98.1%

Share of cited figures that survived recomputation.

A poisoned case is included deliberately, so 100% would be the wrong answer

Refusals correct

2 / 2

Out-of-coverage questions declined without inventing a figure.

Declining while smuggling in an uncited number scores as a failure

Recomputations

Tool re-executions performed to produce this page.

Plus one per receipt, run again when you open a chip

The poisoned case

One case in the suite has a fabricated figure injected into the narrative on purpose — the tools return the truth, and the number written into the prose is multiplied before it reaches the verifier. This is the exact failure that occurs in production: the model calls the right tools, receives the right numbers, and then writes a different one. It is silent, it survives review because the sentence reads well, and it is the reason club managers stop trusting an insight feed.

Blocked before render

What did the dining room bring in over the last twelve months?

Blocked1 failed of 2

The dining room took over the last twelve months, out of in total club revenue across every category.

Tool trace2 calls · 44ms total · replay

1
revenue_total9ms
from: "2025-09-01", to: "2026-08-31", category: "dining"
2
revenue_total10ms
from: "2025-09-01", to: "2026-08-31"

What the verifier said

$779,811mismatch
source computes 577,638
narrative says $779,811, source computes 577638
evidence 2301851edc15513e

What this proves

The gate is real. The number is not compared against a stored expectation; the tool is re-executed from the parameters on its own evidence record, and the result disagrees.
It fails closed. A blocked narrative does not render as an answer. There is no code path where an unverified figure reaches a manager because the sentence read fluently.
It is specific. The verifier names the figure, the evidence id and the true value, so the failure is debuggable rather than a red banner.
It is rounding-aware. This case is caught on substance, not on formatting — the same gate accepts “$381k” for 381,204, because rejecting honest rounding would just train the writing toward unreadable prose.

Every case in the suite

15 cases · 2 negative controls · 1 poisoned

Scripted assistant cases with verification outcome
Case	Tools called	Figures cited	Verification	Verdict
Why is dining revenue down?Revenue	revenue_totalrevenue_totalrevenue_trendvisit_trendsearch_member_notessearch_member_notes	4 / 4	Verified	Answered, verified
Where is our money actually coming from?Revenue	revenue_by_categoryrevenue_totalrevenue_totalrevenue_totalrevenue_totalrevenue_totalavg_discretionary_spend	6 / 6	Verified	Answered, verified
Are we up or down against last year?Revenue	revenue_totalrevenue_totalrevenue_trendrevenue_trend	4 / 4	Verified	Answered, verified
Guest fees look high this summer. Is that real, and should I be pleased?Revenue	revenue_totalrevenue_totalrevenue_totalrevenue_totalfacility_utilisation	5 / 5	Verified	Answered, verified
Who's about to quit on me?Retention	churn_cohort_sizechurn_cohort_sizechurn_cohort_sizedues_at_riskdues_at_riskmember_count	6 / 6	Verified	Answered, verified
Ryan Cavendish has gone quiet. How worried should I be?Retention	member_churn_scorechurn_cohort_sizedues_at_risk	3 / 3	Verified	Answered, verified
Did the members we signed up in 2025 actually stick?Retention	cohort_retentioncohort_retentioncohort_retentioncohort_retentionmember_count	5 / 5	Verified	Answered, verified
Are the tennis courts earning their keep in the mornings?Utilisation	facility_utilisationfacility_utilisationfacility_utilisationfacility_utilisation	4 / 4	Verified	Answered, verified
Is the club getting quieter?Engagement	visit_trendvisit_trendvisit_trendfacility_utilisationfacility_utilisation	5 / 5	Verified	Answered, verified
Are people actually turning up to what we put on?Engagement	event_attendance_rateevent_attendance_rateevent_attendance_raterevenue_total	4 / 4	Verified	Answered, verified
What are members actually complaining about?Sentiment	search_member_notessearch_member_notessearch_member_notesrevenue_trend	1 / 1	Verified	Answered, verified
What is a member worth to us beyond their dues?Members	avg_discretionary_spendavg_discretionary_spendmember_countmember_counttop_members_by_spend	4 / 4	Verified	Answered, verified
How did dining do in 2022 compared with now?Coverage · negative control	data_coverage	0 / 0	Verified	Refused correctly
How much did the dining staff make in tips last quarter?Coverage · negative control	data_coverage	0 / 0	Verified	Refused correctly
What did the dining room bring in over the last twelve months?Reliability · poisoned	revenue_totalrevenue_total	1 / 2	Blocked	Fabrication blocked

Verifier sandbox

Everything above is the system marking its own homework. Here you get to attack it. The narrative below cites live evidence ids and passes as written. Change any figure inside a citation, or type a bare number into the prose, and press Verify.

Narrative with citations

Citations are written [[e:evidenceId|figure]]. Change the figure inside one — a digit, a magnitude, anything — and press Verify. Numbers typed outside a citation are caught too, as undeclared.

No verdict yet. Press Verify to run the gate over this sentence.