The differential corpusalpha
What is compared against what, and what is the current score?
The corpus is moc's own test suite, run against moxzi — a third party's programs, not programs written to make moxzi look good. It lives in a sibling checkout of dfinity/motoko ($MOCTEST, default …/motoko/test/run-drun), so a corpus sweep needs that checkout and roughly fifteen minutes per sweep; it is not part of a plain cargo test.
The two sweeps#
| Sweep | Script | Compares | Oracle |
|---|---|---|---|
| Two-compiler byte parity | scripts/compiler_byte_neutral.sh | raw wasm bytes of each program, built twice | a baseline compiler build vs a candidate build |
| Browser vs native | scripts/web_corpus_gate.sh | observable behaviour: each program's //CALL script replayed | the native moxzid runtime |
The second sweep's oracle chain is what makes it worth running: the native runtime was itself validated against a real IC replica by the differential harness, so browser == native == replica by transitivity.
How a program is selected#
Both sweeps start from the same filter, applied to top-level *.mo in the corpus directory after a perl rewrite of bare actor to persistent actor:
| Rule | Effect |
|---|---|
must contain actor | excludes non-actor programs |
must not reference ic:/canister: URLs | excludes programs needing a live replica |
| (browser sweep) must build under moxzi | build failures are cached as .rejected and skipped |
(browser sweep) must have a //CALL///OR-CALL script | there is nothing to replay otherwise |
(browser sweep) must not be par/non-par | named in CANNOT_PREEMPT |
Counted today: 386 .mo files in run-drun, of which 353 pass the two shared filters, of which 177 carry an author script. The published figures — 221 and 160 — are the programs that actually compared, after toolchain-rejected programs drop out.
The score#
| Claim | Figure | Where it comes from |
|---|---|---|
| Programs byte-identical across two compiler builds | 221 | PAR=6 scripts/compiler_byte_neutral.sh |
| Programs behaving identically browser vs native | 160 | scripts/web_corpus_gate.sh |
| Diverging programs, browser vs native | 0 | same |
compiler_byte_neutral.sh passes only when DIFFER, ONLYNEW and ONLYOLD are all zero. BOTHFAIL — a program neither compiler accepts — is tolerated, because the sweep measures parity, not coverage.
The hollow-pass guard#
A sweep that compares nothing and reports success is the failure mode this project has already lived through once. Both sweeps guard it, on both sides of the harness boundary:
| Guard | Location |
|---|---|
if [ "$same" = "0" ] → FAIL: nothing was compared | compiler_byte_neutral.sh:76 |
total compared = 0 → a hollow run, rc=1 | web_corpus_gate.sh:74 |
missing TOTALS line + nonzero node exit → FAIL | web_corpus_gate.sh:57 |
| harness-side zero-compared trap | test/pic-js/scripts/web-corpus.mjs |
Server gates#
Sixteen scripts, each asserting one property of moxzid, all run by make alpha-check.
| Gate | Asserts |
|---|---|
moxzid_e2e.sh | manifest, registry, front door, and an inter-actor call that mutates the callee |
moxzid_spawn.sh | a runtime-spawned agent survives a restart despite never being in the manifest |
moxzid_install.sh | deploy to a running server, no manifest edit, no restart |
moxzid_upgrade.sh | /upgrade/<actor> works, including for a canister created at runtime |
moxzid_stable_types.sh | an incompatible stable signature is flagged, not silently accepted |
moxzid_lifecycle_limit.sh | an infinite loop in init cannot wedge the actor thread |
moxzid_inspector.sh | /inspect returns stats and the candid interface read from the module itself |
moxzid_crash.sh | survives kill -9 by snapshot and by log replay |
moxzid_cycles.sh | drain, refuse, top up, revive — without a restart |
moxzid_fairness.sh | one actor's 50-message backlog does not delay another actor |
moxzid_timers.sh | a timer fires because time passes, with nobody calling |
moxzid_outcall.sh | a real HTTPS outcall runs, and so does its transform |
moxzid_shutdown.sh | SIGTERM finishes the message, checkpoints, exits 0, restarts from snapshot |
moxzid_backup.sh | the backup commands printed in the operations guide, executed verbatim |
moxzid_web_gate.sh | the actor is a website: gateway, upgrade = ?true re-delivery, SSE |
moxzid_webdav_gate.sh | full WebDAV through http_request, and the drive survives kill -9 |
Browser gates#
| Gate | Asserts |
|---|---|
web_async_gate.sh | a Motoko await runs on the real wasm32 backend, not a JS stand-in |
web_two_actors_gate.sh | two actors exchange a message in a browser engine |
web_reentrancy_gate.sh | A → B → A: an actor at an await is not busy |
web_durable_gate.sh | an actor survives the page, via a synchronous runtime snapshot |
web_timer_gate.sh | a timer fires on an idle actor |
web_readonly_gate.sh | a query's heap writes are discarded, as on the IC |
web_upgrade_gate.sh | pre_upgrade → snapshot → new instance → restore → start → post_upgrade |
web_spawn_gate.sh | create_canister + install_code against a real in-page aaaaa-aa |
web_compile_gate.sh | the compiler compiles in a browser engine, byte-identical |
web_page_gate.sh | a real headless browser, real DOM, real module loading |
web_page_compile_gate.sh | the compiler compiles in a real tab, byte-identical, with no stack flag |
web_worker_gate.sh | a runaway message is terminated and the actor survives |
web_outcall_gate.sh | an outcall mid-message, with the instance borrowed |
web_actor_gate.sh | a dfx generate idlFactory drives the page runtime unmodified |
web_lib_gate.sh | the client library works on the main thread and in a worker |
web_mops_gate.sh | a page compiles a program with mo: imports, byte-identical to the CLI |
The rest of the release gate#
| Gate | Asserts |
|---|---|
moxdb_gate.sh | rows-as-actors: concurrent unique-insert, self-maintained indexes, kill -9 mid-life |
knights_gate.sh | outcall → in-tab compile → in-tab link → in-place upgrade, state riding through |
parse_seam_gate.sh | the resumable parser's seam cannot double-commit a dec |
float_literals_gate.sh | exponents, hex floats and underscores compile to the right value |
parser_depth_gate.sh | deep nesting is a diagnostic, not a stack overflow |
PocketIC job-auth | one customer's job is not visible to another |
PocketIC paid-build | escrow → measure → 3× actual → refund, with the real CLI |
Plus native Rust tests. cargo test --workspace -- --list counts 81 today; the figure quoted as 73 in README.md, CHANGELOG.md and docs/ci.md is stale.
What the corpus does not cover#
Programs the toolchain rejects outright are dropped, not counted as failures — coverage is not what the sweeps measure. Programs with ic:/canister: URLs never run. And a SKIP from a missing Chrome or a missing compiler wasm is loud but still a zero, which is why alpha_check.sh refuses to start without bin/*/compiler.wasm and linker.wasm.
Next#
- Known differences — what the sweeps found and why it stays.
- Language support status — what compiles, and what does not.
- Reporting a bug — how to turn a divergence into a fix.