About
Falsify makes an ML evaluation claim verifiable. You pre-register a claim (metric, threshold, dataset, seed) as a SHA-256 manifest before the run, so the number becomes tamper-evident and anyone can re-derive it.
The problem: almost every published AI accuracy number is "trust me." Documentation written after the run cannot prove when the number was decided. Edit the claim later and the hash changes, so the verifier flags it TAMPERED.
What you get:
pip install falsify and npm i -g falsify-js, with byte-identical hashes across Python, JS, Go and Rust.
falsify lock claim.prml.yaml then falsify verify --observed 0.93, a deterministic PASS / FAIL / TAMPERED.
A GitHub Action to block tampered or regressed eval claims in CI.
A public registry: paste a manifest in the browser, get a permalink and badge, no account.
Open spec (CC BY 4.0), MIT code, citable Zenodo DOI.
Why now: the EU AI Act high-risk rules bind on 2 August 2026. Providers must declare accuracy and keep records that can be checked, not just asserted. Falsify is a cryptographic answer to "when was this threshold set relative to seeing the result?"
It is not a reproducibility tool and does not prove a result is true. It proves one narrow thing: the claim was pre-committed and not silently rewritten. The evidence layer underneath a model card.
falsify.dev