DEFINE-UK: running and gating an unlicensed ecological SFC model

01 — why this model

Climate policy meets the UK financial system.

DEFINE-UK is the ecological stock-flow consistent (E-SFC) model of the UK built by George, Dafermos, Nikolaidi and co-authors (define-model.org/define-uk). It simulates the UK macrofinancial system and its environmental impacts under climate-policy scenarios — a fossil-fuel ban, power-sector subsidies, housing regulation, green public investment, and a mixed package. No other model in the suite answers the question it is here for: who bears a climate-policy scenario. Versions 1.0 and 1.1 were released in April 2026. The specification is the DEFINE-UK Model Manual, Version 1.1 and the accompanying paper “Evaluating climate policy mixes in the UK: an E-SFC approach”.

The suite's standard is that a model earns its place by replication: run the published code or reimplement the published equations, then compare against the published results at recorded tolerances. This report applies that standard, and records where the publications themselves impose a ceiling.

02 — the record at a glance

Every gate, its verdict, and the test that holds it.

Each gate below is enforced by a test in the adapter repository against the cached pinned run (upstream commit 846081a), so drift fails loudly rather than silently. The supporting narrative for each — tolerances, component checks, arithmetic, search history — is one disclosure away.

DEFINE-UK replication gates: verdict, date, and enforcing test
gate	verdict	date	enforced by
Manual Table 4, macro block (growth, unemployment, population 16+, labour force), S1 baseline	PASS — but the 2025 growth miss is −0.30pp against a ±0.31pp gate: passing by 0.01pp is not the same as matching	2026-08-04	`tests/test_replication_baseline.py`
Manual Table 4, total emissions, S1 baseline	DIVERGENCE — pinned at the observed ratios	2026-08-04	`tests/test_replication_baseline.py`
Scenario design — the policy switches each cached scenario toggles	PINNED EXACTLY	2026-08-05	`tests/test_scenario_design.py`
FMM 2023 open-vintage anchors — green public investment GDP peak; baseline 2030 emissions	PASS at wide (pre-1.0) tolerances	2026-08-05	`tests/test_scenario_design.py`
Published numeric scenario results for v1.1	CEILING REACHED — none are published	2026-08-05	`validation/published_targets.json` (search record)
Baseline vs external observations (ONS, DESNZ, OBR)	COMPUTED — gaps are material	2026-08-05	`tests/test_validation.py`
Green public investment cumulative multiplier	COMPUTED — 1.78, ours, not upstream's, and not comparable to any published multiplier	2026-08-05	`tests/test_validation.py`
Clean-room milestone 1 — §2.2 matrices on the §5 initial values	PASS	2026-08-04	`tests/test_accounting.py`
Clean-room milestone 2 — §3.2–3.3 macro and production block	IN PROGRESS — §3.2, §3.3.1 and §3.3.2 landed (118 eqs); §3.3.3 + oracle outstanding, so not passed (section 05)	2026-08-11	`tests/test_macro.py`

Macro block: the tolerances, and what each component hit

Tolerances: population and labour force ±0.01m; growth ±0.31pp; unemployment ±0.2pp. Against Table 4's macro block at 2025/2030/2040 on the S1 baseline, population and labour force are exact to 0.01m; growth matches to 0.06pp at 2030 and 2040 and misses by −0.30pp at 2025; unemployment is within 0.2pp; and the 2025–40 mean and standard deviation of growth and unemployment match the table. The run is calendar-anchored on the population path (1987Q1–2040Q4, 2025 = t153).

The growth gate is not independent of the deviation it admits. ±0.31pp was set after the run, and the only year that uses any of it misses by −0.30pp — the gate clears by 0.01pp. It is doing no work as a test of agreement in 2025; what it does is stop that year drifting further, and the 2030/2040 matches to 0.06pp are the part of this gate that is genuinely informative. The recorded reason for the 2025 gap is that Table 4 never states its annualisation convention (validation/reference_outputs.json), so it is a convention mismatch we could not resolve, not a demonstrated agreement.

Emissions: 393 vs 407 in 2025, 343 vs 382 in 2030, 249 vs 324 by 2040

The pinned code runs below the published Table 4 emissions, and the gap widens with the horizon: 393 vs 407 MtCO2e/yr in 2025 (−3.5%), 343 vs 382 in 2030 (−10.2%), 249 vs 324 by 2040 (−23.3%). The percentages come from the pinned run/table ratios themselves (0.965, 0.898, 0.767 in validation/reference_outputs.json), not from the rounded levels printed here.

Two checks rule out an extraction error on our side: the components EMIS_NELEC + EMIS_ELEC sum exactly to EMIS, and the baseline is identical across every scenario folder. What remains is a vintage/calibration gap between the published table and commit 846081a. The divergence has been queried with the upstream authors, and the tests gate at the observed ratios, so any further drift fails loudly.

Scenario design: the exact switch sets, the two anchors, and the search that closed the target

tests/test_scenario_design.py pins that each cached scenario toggles exactly the policy switches its published description claims: FF_BAN plus capacity/investment ban timings for the ban scenarios; POWER_SUB alone for the subsidy; GVT_INVEST + GREEN_BONDS + GREEN_POWER exactly for green public investment; and the published 40% housing subsidy rate, HOUSING_SUB_RATE = 0.4.

Two coarse numeric anchors survive in the open FMM 2023 conference version of the paper, held at wide tolerances because it is a pre-1.0 vintage. Green public investment peaks “around +1% of GDP” — the cached run peaks at +0.92%, band 0.7–1.3%. The current-policies baseline emits “just under 350 MtCO2e” in 2030 — the cached run gives 342.8, band 330–352. Both anchor.

Read those two as regression pins, not as agreement tests. Both bands were chosen by us to fit a verbal quantity, not published as tolerances, and neither can discriminate much: ±30% around “around +1%”, and an emissions band whose upper edge (352) is above the 350 the phrase “just under” is supposed to bound, so a run at 351 would pass a gate named for a claim it violates. What they establish is that the cached run has not moved since the anchors were recorded.

Nothing finer exists. An exhaustive search, catalogued in validation/published_targets.json, established that no machine-readable numeric scenario results are published for v1.1: the manual's results stop at the baseline (Table 4) plus scenario design parameters, and the George/Dafermos SSRN paper (abstract 6541398) is paywalled with no open mirror. Its scenario set — carbon pricing and subsidies — predates v1.1's regulation policies in any case.

Calibration: the baseline against ONS and DESNZ outturns

Since 2026-08-05 this comparison is an artifact, not a paragraph. define_uk.validation.baseline_calibration() recomputes every row from the cached pinned run against vintage-exact external observations; the result is committed as validation/baseline_vs_external.csv; and tests/test_validation.py fails on any drift between the committed table and a recomputation.

The headline rows: 2025 real GDP growth of 4.66% (annual-mean convention; Table 4 prints 4.96%) against an ONS outturn of 1.31% y/y; 2025 unemployment of 4.39% against an ONS annual mean of 4.88%; population 16+ 55.66m vs ≈56m, consistent; labour force 35.97m, slightly high; and emissions starting above the DESNZ actuals. The full table is on the validation page.

Multiplier: our arithmetic, and why it is not comparable to a published one

The model's cumulative green-public-investment multiplier is 1.78 by our own computation: cumulative ΔGDP of 209.9 over cumulative ΔSPEND_GVT of 118.2, nominal, over the full simulation (define_uk.validation.gpi_multiplier, pinned in the test suite).

It is not like-for-like against the published green-spending multipliers it used to be set beside here, and three properties of our definition all push it the same way. Ours is a cumulative ratio to 2040; the published figures are short-horizon multipliers. Its denominator SPEND_GVT is endogenous — government investment is 12.8% of it, and social benefits fall as the scenario cuts unemployment, so the denominator shrinks and biases the ratio up. And the closure is demand-led with Kaldor–Verdoorn productivity and real GDP not held to the supply ceiling, so a ratio above 1 over fifteen years is what the model implies rather than a finding about the world. The clean comparison — cumulative ΔGDP over cumulative ΔGCF_GVTG, the actual impulse — has not been computed; until it is, 1.78 is an upper-leaning estimate. These caveats travel with the number wherever a delta is served (GPI_MULTIPLIER_CAVEATS, held by a test).

One comparator this report previously quoted has been withdrawn: an IMF green-spending range of 1.1–1.5, attributed to a source that could not be located in either repository. The only IMF reference either repository actually holds is Batini et al. (2014), a different paper with a different quantity — first-year multipliers of 0–1 for advanced economies. No published multiplier is quoted alongside 1.78 here, because none of them measures the same object.

The upstream repository's own tables/Multiplier_Summary.csv is documented unusable: its multiplier columns are identical across all eight scenarios, and its ΔG figure (104.22) matches the cumulative delta of no variable in the scenario file. It is never quoted, and neither is the ≈2.4 once derived from it.

03 — how it is run

Unlicensed upstream, pinned commit, and a run that proves nothing on its own.

The upstream repository is public but carries no licence, so its code is never vendored or hosted. The adapter (github.com/PolicyEngine/define-uk-model) fetches it at pinned commit 846081a at runtime and executes it as published, in R, unmodified. That also fixes the run surface: DEFINE-UK is local-only, never hosted, and the eventual hosting path is the clean-room reimplementation in section 05.

Execution ≠ validation. On 2026-08-01 the full notebook rendered end to end through define_uk.runner.run, producing 151 output files. That demonstrated only that the pinned code executes, not that its outputs mean anything. The adapter's VALIDATION.md holds the gate: nothing built on the repository may present model output as meaningful except by naming the target it rests on — the same bar applied to boe-svar and us-hank. Section 02 records what that gate has since established.

That file's own status line was wrong for a stretch, and the correction belongs in this record because this report cites it as the record. It read “NOT VALIDATED. No published DEFINE-UK result has been reproduced” for some time after target 1a passed, which put it in contradiction with every page that cited it. It now reads PARTIALLY VALIDATED and breaks the claim down target by target: 1a pass, 1b divergence pinned, targets 2 and 3 closed at the achievable ceiling rather than passed, reimplementation milestone 1 pass and milestone 2 outstanding. Nothing there licenses presenting scenario output as a validated result.

The 2026-08-01 run record

R 4.3.0 on macOS, upstream unmodified, an rstudioapi shim the only accommodation. The 151 files are the figures and tables for all four scenario blocks plus tables/Multiplier_Summary.csv.

04 — what the adapter serves

Deltas, never levels — closed at the achievable ceiling.

Baseline levels sit far from official outturns: 2025 growth of 4.66% against an ONS 1.31%, unemployment below the outturn, emissions above the DESNZ actuals they should start from. The manual itself says the baseline “should not be seen as a prediction”; we take that literally. Near-term levels are not competitive with the OBR emulator or the BoE SVAR and are never presented as forecasts.

So the adapter serves annualised scenario-minus-baseline delta paths only — deltas, never levels — framed with mandatory caveats including the demand-led closure, via pe-macro define-scenario (local-only, experimental). The gate stack behind them is the pinned upstream run as oracle, the exact scenario-design pins, and the two FMM 2023 anchors. The upgrade path is equally explicit: the authors publish scenario tables, or the SSRN paper becomes accessible, and the published-figure gate reopens.

05 — the clean-room track

Reimplementing the published equations, milestone by milestone.

Because the upstream code cannot be hosted, the hosting path is a from-scratch Python implementation of the published equations — the same play as the OBR emulator. The protocol is clean-room: the specification sources are the manual, the paper, and Dafermos, Nikolaidi & Galanis (2017); the upstream repository is an output oracle only, its source never read to write equations; and every equation carries a manual_ref naming the manual section and equation number it implements.

Milestone 1: PASS (2026-08-04). The §2.2 transactions and balance-sheet matrices and the full §5 parameter and initial-value tables (Tables 5–6) are transcribed, and every Table 1/2 row and column identity holds on the §5 initial values within the manual's own 4-significant-figure printing precision. tests/test_accounting.py collects 62 tests, of which 44 are the Table 1/2 row and column identities themselves (20 transactions rows, 6 transactions columns, 7 balance-sheet rows, 11 balance-sheet columns); the remainder are the residual-instrument and net-worth definitions and the two that pin the one manual inconsistency below.

Milestone 2: IN PROGRESS, not passed (2026-08-11). §3.2, §3.3.1 and §3.3.2 have landed: model/sectors/macro.py, production.py and power.py implement Eqs. (21)–(138) — 118 equations, contiguous — each solving as a system under its own test module. §3.3.3 input–output and the oracle baseline comparison remain outstanding, so milestone 2 is not passed.

Reimplementation is already earning its keep as a reading of the manual that the R source cannot give: thirteen manual defects are pinned in tests as machine-readable gap records rather than papered over or absorbed into a tolerance, with the wider set written up in the reimplementation notes. An audit of those notes also re-characterised four findings that were reading as the manual contradicting itself — Eqs. (51), (59), (94) and (111). Table 5 describes the parameters concerned as historical means or means of past implied values, and so never claimed to reproduce the manual's own initial period; those are first-period jumps a run started from Table 6 will take in its first quarter, not defects. The same audit withdrew one claim outright, on Eq. (51), where the corroborating inference had ignored a lag.

Three of the pinned manual defects, worked through

Table 6 LENDM_ROW omits the DIVN_ROW term of Eq. (383), so the MFI/RoW transaction columns miss LEND by ∓DIVN_ROW (= 5.44); the manual itself tabulates overall LENDM = 5.44 where it “should equal 0”. Pinned exactly in tests/test_accounting.py.

Table 6's capital block is deflated at the wrong price. Eq. (27), GCF_R = GCF/P_P, gives 111.88 against a tabulated 112.30 — 0.37%, roughly ten times the rounding noise — and Eq. (25) only reproduces the tabulated GDP_R using 112.30. This is one finding, not five: GCF, GCF_NFC, GCF_HH, GCF_GVT, K_P and K_NFC all show nominal/real ratios of 1.0309–1.0312 where the equations prescribe P_P = 1.035, and the same 0.4% reappears in §3.3.2's Eqs. (104), (105), (132) and (133). One systematic property of Table 6's capital rows — most likely a national-accounts investment deflator where the equations want the production deflator — counted once. Pinned in the tests rather than absorbed into a tolerance.

Eq. (31) uses an intercept α₀λ that Table 5 never tabulates — the α₀ block ends at α₀WS. It is defaulted to 0.0, recorded in macro.MANUAL_GAPS, and pinned by a test that fails if a value is later published.

The model design belongs to its authors; the implementation is ours, AGPL-3.0, and independent — not the authors' code, and not endorsed by them.

cite

Cite this work

Ahmadi, Vahid (2026). “DEFINE-UK: running and gating an unlicensed ecological SFC model.” PolicyEngine Macro replication report. https://policyengine-macro.vercel.app/reports/define-uk-replication

@techreport{policyengine2026defineuk,
  title       = {DEFINE-UK: running and gating an unlicensed ecological SFC model},
  author      = {Ahmadi, Vahid},
  year        = {2026},
  institution = {PolicyEngine},
  url         = {https://policyengine-macro.vercel.app/reports/define-uk-replication},
  note        = {PolicyEngine Macro replication report}
}

No DOI is assigned. For an exact version stamp, cite the git commit of the artifact you used from github.com/PolicyEngine/define-uk-model. Full validation record: VALIDATION.md and REIMPLEMENTATION.md in that repository, mirrored on the validation page.