SCPN Phase Orchestrator · four-domain falsification · matched false alarm
We ran the fair test. Nobody passed — including us.
Generic early-warning detection — ours and the classical AR(1)-Kendall baseline of the tipping-point literature — was evaluated on real transitions in four domains at a matched false alarm with permutation significance. No detector reached significance in any domain. That result is sealed, published, and it is the reason this platform certifies detectors instead of overclaiming one.
Transitions led at a matched 10% false alarm — best SCPN member vs the classical baseline
Per domain: the best-performing SCPN suite member against the canonical Dakos et al. 2008 AR(1)-Kendall-τ detector, both calibrated on the same nulls and scored on the same transitions. The tick is the count expected by chance at the detector’s own alarm rate.
What the numbers say
- Palaeoclimate: SCPN 1/6 (p = 0.51); the classical detector 0/6 (p = 1.00) — on the same class of records its own literature made famous, at an operational matched-false-alarm threshold rather than retrospective per-record trends.
- Grid (generic observables): SCPN 3/12 (p = 0.20), classical 1/12 (p = 0.71). The domain-specific modal detector is the separate, positive counterpoint — see the head-to-head demo.
- Cardiac: SCPN 2/6 (p = 0.22), classical 0/6 (p = 1.00).
- EEG: SCPN 2/6 (p = 0.22), classical 3/6 (p = 0.067) — the strongest signal anywhere in the study, and still short of significance.
Why publish a negative result about our own detectors? Because the test is the product. The same harness that demoted our generic suite — matched false-alarm calibration, label-permutation significance, selection-controlled surrogates, hash-sealed byte-reproducible records — is what certified the one detector that does clear the bar, and it is what we run against any detector, anyone’s, on their data. A certification you can trust is one that is allowed to say no.
The sealed record
The full comparison is a committed artefact in the public repository
(examples/real_data/head_to_head_ar1_kendall/), alongside the
publication-form study
(docs/studies/early_warning_matched_false_alarm.md) and the per-domain
sealed evidence directories it draws on.
2c78a7308a183414…- Corpora are small (6–12 transitions per domain) — low statistical power; the honest statement is “no detector demonstrated significance here”, not “early warning is impossible”.
- One classical competitor was implemented; the EEG corpus is single-subject.
- All operating points are matched-false-alarm calibrated; results at other operating points may differ and would need the same protocol to claim anything.