See the evidence for each model.

The five models support different kinds of checks. The verification gradient groups them into three classes; choose a row, then open its evidence below.

Software fidelity and economic validity are separate.

This audit covers the three macro models. Levels are categorical evidence judgements, not a synthetic score: a model can reproduce its reference code perfectly while still lacking independent forecast or counterfactual validation.

Anchored to the March 2026 EFO with OBR add-factors, the emulator reproduces GDP to 0.15% MAPE and consumption to 0.25% over 2025Q1–2027Q4 (CI gate: 1%). GDP remains within 0.29% through 2031Q1, but unemployment becomes unreliable after 2027Q4. This fit is by construction; the independent reform costing below is the stronger test. The headline chart and table now use the live March 2026 vintage; the older November 2025 forecast is retained only inside the historical outturn audit.

The independent check is the reform costing. Raising the basic rate of income tax by 1pp from April 2026 scores at £6.46bn in 2026–27 through the PolicyEngine static-costing bridge, against HMRC's ready-reckoner figure of £6.9bn — a deviation of −6.4%; the costing sits inside the £6–8bn range spanned by recent published vintages. The gap widens in later years (−15.6% by 2028–29) where HMRC's figures embed administrative-data fiscal drag that survey microdata capture less fully.

The spending multiplier is ~1.0 by construction, against the OBR's own published 0.6. Under the demand closure a spending shock lands directly in the GDP identity and the behavioural second round is largely inactive, so a £5bn injection returns almost exactly £5bn of GDP. That is roughly a two-thirds overstatement of the impact multiplier relative to the institution being replicated, and it applies to every spending-side figure this page reports. It is the single most important number for anyone reading a policy score off this model.

Three of the channels that would normally damp such a shock are held exogenous in this configuration — imports (so the usual leakage is switched off), Bank Rate (no monetary offset) and CPI. The first two bias the estimated effect upward; the dead profits→dividends channel biases corporation-tax effects downward. These do not cancel, and their net sign is not established. Reform figures here are best read as relative comparisons between structurally identical runs, where the biases largely difference out, rather than as magnitudes.

working paper (pdf) → obr-macro model page →

Against Brignone & Piffer (2025), the one-year share of UK forecast-error variance from global shocks is 42.1% for GDP and 49.5% for CPI. The paper reports roughly 40% and 50%.

Out-of-sample. Forecasting seven quarters ahead from a frozen 2024Q2 edge, RMSE is 0.3pp for both GDP growth and CPI inflation. Activity tracks well; the median misses the 2025 inflation peak by up to 0.6pp but captures its reversal.

Fourteen of fourteen outturns inside the 68% band is a calibration failure, not a pass. A correctly calibrated 68% interval should contain roughly 9.5 of 14. Containing all of them means the bands are too wide — the intervals are roughly ±0.95pp around a forecast whose RMSE is 0.3pp, about three times wider than the errors warrant. The model is under-confident here, and this figure was previously presented as a validation win. It is reported now as what it is: evidence of over-dispersion.

Neither reading is strong evidence either way. The fourteen points are seven quarters × two variables from a single forecast origin, so they are heavily correlated across horizon and across variables; the effective number of independent observations is perhaps two or three. Coverage cannot be established from one origin in either direction. The rolling-origin machinery could produce a genuine coverage table across 49 origins and eight horizons; until it does, this page makes no coverage claim.

Rolling-origin audit. A stricter expanding-window test now evaluates 49 historical origins without future-data leakage. Against a no-change benchmark, CPI-level relative RMSE is 0.63 at one quarter and 0.67 at eight quarters (lower is better), while UK GDP sits at 1.06. But a no-change benchmark is too weak for a trending price level: against a random walk with drift the CPI figures become 0.83 and 1.03, and the GDP gap is not statistically significant at any horizon. See section 08 for the full benchmark comparison. The defensible forecasting claim is Bank Rate, not inflation, and the single frozen-edge result must not be read as broad GDP forecast superiority.

working paper (pdf) → boe-svar model page →

The baseline reproduces the Fed's LONGBASE database across 284 variables and 20 quarters with a maximum error of 5.6×10⁻¹⁷ — machine precision, against a 10⁻⁸ CI gate.

Shock responses match pyfrbus 1.0.0 to 6.0×10⁻⁹ and 1.1.1 to 1.4×10⁻⁸. The Fed's two releases differ by 1.3×10⁻⁸, so our differences sit at the reference solver's own numerical noise.

Multipliers. Government-purchases and personal-tax shocks of 1% of GDP land inside every published range they are compared against — the last three rows below.

Uncertainty engine. The implementation now supports seeded stochastic simulations that jointly resample the official stochastic-equation residual vector, preserving contemporaneous dependence and reporting failed replications. Model and LONGBASE packages are also verified against separate SHA-256 provenance gates. Published probability results still require a reviewed simulation design; model-consistent expectations remain unavailable.

working paper (pdf) → frb-us model page → frb-us code →

This model does not forecast. It deterministically applies encoded UK and US tax-benefit law to a specified household, so its reference is legislation and official guidance. Coverage depends on the upstream rules and tests.

Population estimates also depend on survey data and calibration. The UK run uses enhanced_frs_2023_24; in the 1pp basic-rate check, it is 6.4% below HMRC in year one. Survey versus administrative data, policy assumptions, and HMRC behavioural elasticities explain that separate aggregate gap.

pe-microsim model page → policyengine.org →

A zero deviation on an imposed quantity is not independent evidence.

working paper (pdf) → psl-og model page →

Published institutional answers sit beside ours; each row states how closely the definitions align.

Three different statistics answer three different questions, and they are not averaged, ranked, or plotted on one axis: our SVAR's pseudo-out-of-sample accuracy against a naive benchmark (computed here, quarterly), the OBR's and external forecasters' own published real-time errors (cited, annual), and our OBR emulator's tracking of the official forecast (replication accuracy, not forecasting accuracy).

boe-svar against a random walk, computed from our own runs

An expanding-window pseudo-out-of-sample exercise re-fits the BVAR at 49 quarterly origins (2012Q1–2024Q1, data sample 1992Q1–2026Q1) and scores forecasts one to eight quarters ahead against three naive benchmarks. A ratio below 1.0 means the model beats the benchmark.

The benchmark decides the answer. A no-change random walk on a trending log level forfeits the whole trend as forecast error, so beating it on a price index is close to uninformative. Against a random walk with drift — the textbook naive for a trending series — the CPI result largely evaporates: 0.63 becomes 0.83 at one quarter and 0.67 becomes 1.03 at eight, where the model is no longer ahead at all. The pattern in the original numbers, wins only on the two trending price series and ties or losses on the four series where a random walk is a genuinely hard benchmark, was an artefact of the benchmark rather than a finding about the model.

What survives is Bank Rate. It is the one series here that does not trend, so no-change is the right naive for it, and it is the one series that improves under the harder benchmark: 0.79 against drift at one quarter (Diebold–Mariano p = 0.018) and 0.85 at eight. On the evidence assembled here that, not inflation, is the model's defensible forecasting claim.

UK GDP is not beaten by the benchmark either. Its ratio of 1.06–1.09 is not statistically distinguishable from a random walk at any horizon (p = 0.38 to 0.67), and excluding the six origins whose target quarter falls in 2020Q1–2021Q2 it becomes 0.77 at one quarter. Under squared loss the 2020Q2 collapse dominates a 49-origin average. Both the full-sample and the excluding-Covid figures are published; neither is the preferred number.

The AR(1) comparison is reported in the source file but is not usable at long horizons: 94.5% of its eight-step squared error for UK GDP comes from a single origin, because an AR(1) extrapolates the Covid collapse. The apparent 3:1 win against it is arithmetic, not evidence. Every benchmark now carries a worst_origin_mse_share diagnostic for exactly this reason.

Bank Rate crosses parity beyond h=3 (source file has all eight variables and horizons). These are quarterly, level-basis, pseudo-out-of-sample ratios computed with hindsight-final data — they are not comparable with the real-time annual-growth errors the OBR publishes, which is why they get separate tables.

What the OBR and external forecasters report about themselves

The OBR's Forecast Evaluation Report (July 2025) publishes its own real-time errors on annual growth rates since 2010, beside the median of external forecasters compiled by HM Treasury. Cited, not recomputed; the statistic is the median absolute error on annual rates, so it cannot be compared with the quarterly RMSE ratios above.

Our own frozen-edge experiment is the nearest in-house analogue: the boe-svar RMSE of 0.32pp on quarterly year-on-year GDP growth and CPI inflation over seven quarters from a 2024Q2 data edge (section 03 above). Different statistic (RMSE vs median absolute error), different frequency (quarterly vs annual), one origin against many — so it sits beside the OBR's numbers, not against them. For the Bank of England, the single matched-vintage episode already cited in section 07 — the August 2024 MPR modal CPI path of about 2.4% for late 2025 against an outturn peak of 3.8% — is the only comparison we can source precisely; the Bank does not publish an RMSE-by-horizon table in the MPR.

Our OBR emulator: replication accuracy, not forecast accuracy

The obr-macro numbers in section 02 measure how closely the anchored emulator reproduces the OBR's published March 2026 path (MAPE 0.15% GDP, 0.25% consumption) — that is replication accuracy. It is not an independent forecast: de-seeded and stripped of add-factors, the same model drifts 5.75% below the EFO path over twelve quarters. Any forecast-accuracy credit for obr-macro therefore belongs to the OBR's own forecast, whose errors are the FER numbers above.