Sources and packagesalpha
How do sources get to the canister, and how do I upload nothing at all?
The compiler canister holds a virtual filesystem, keyed per caller, and a build compiles against the tree belonging to the principal that started the job. The constraint worth knowing before you upload anything: that tree is canister state, and whoever controls the canister can read it.
The VFS#
moxzi build --remote uploads the whole closure — your sources under /src/<path> and packages under /pkg/<name>/<path> — before it starts a job. The VFS path is not cosmetic: the compiler records source paths in what it emits, so a file uploaded under the wrong name produces different bytes than a local build would. moxzi deps <entry.mo> --paths-only prints exactly the names that will be used, which is the quickest way to check that a local and a remote build will agree.
| method | kind | notes |
|---|---|---|
write(path, data) | update | replaces the file; the bytes it replaced are released from your quota |
appendChunk(path, data) | update | appends, creating the file if absent — how anything past the ~2 MB ingress limit is streamed |
read(path) | query | your own tree only |
list() | query | your own tree only |
remove(path) | update | releases the quota |
vfsStats() | query | used, limit, totalUsed, totalLimit |
The CLI chunks at 1,900,000 bytes: a file at or under that goes in one write, anything larger goes as a write followed by appendChunk calls.
Quotas#
| bound | value | why |
|---|---|---|
| per caller | 64 MiB | stops one principal parking the canister full (evm_engine's whole closure is 16 MB) |
| all callers | 1 GiB | principals are free to mint, so a per-caller cap alone bounds nothing |
| blob store | 256 MiB | the shared content-addressed store is open by design, so it needs its own cap |
Writes past either VFS limit trap rather than truncating — a partial upload that looked like a success would compile the wrong source. The per-caller trap message names the way out: remove files, or use mops tags.
Uploading nothing: the tag layer#
Version-pinned package source is 41% of a large closure (6.56 MB of evm_engine's 16.08 MB), and mops pins are immutable — a published core@2.5.0 is the same bytes forever. So a pin can be named once and reused by every later build and every other tenant, instead of being re-uploaded per compile.
Two layers, deliberately separate:
| layer | key | value |
|---|---|---|
| blobs | sha256(content) | the bytes; dedupes identical files across packages and versions for free |
| tags | "name@version" (mops' own PackageId) | a manifest of (fileId, vfsPath, hash) — not the bytes |
The fileId is mops' own name for the file, "<pkg>@<ver>/<path>", passed through unchanged from mops.lock. Carrying both names lets verification compare named pairs against the registry rather than an anonymous hash set, so a file whose content is genuine but whose path is wrong is caught too.
The protocol is:
| step | call | effect |
|---|---|---|
| 1 | missingBlobs(hashes) | which of these do you not already have? |
| 2 | putBlobs([...]) or putBlobFromFile(path) | upload only those |
| 3 | putTagFrom(tag, entries, source) | register the manifest, recording who registered it and what they claimed the source was |
| 4 | verifyTagAgainstMops(tag) | one inter-canister query to the mops registry |
| 5 | useTags([...]) | materialise into your VFS, zero source bytes uploaded |
putTag refuses unless every referenced blob is already stored, so a tag is never half-registered. useTags rejects unknown tags rather than compiling short, and it accepts only mops-verified tags — re-registering a tag drops its verification, or "verify a good tag, then overwrite it" would defeat the check.
Files materialised by useTags are deliberately not charged against your VFS quota: a tree entry is a reference to the same content-addressed bytes, not a copy, and charging them again would double-count the one thing the tag layer exists to avoid.
Why a tag build is byte-identical#
useTags writes the same files map that write does; nothing in the tag layer touches the compile path. That property is asserted, not assumed: test/pic-js/scripts/mops-tags.mjs uploads a package, builds, deletes the package files from the VFS, rebuilds from the verified tag with zero bytes uploaded, and compares the two artifacts byte for byte. The same script proves the negative cases: a tampered tag is caught by name (mismatched), an unverified tag is refused by useTags, and an unknown tag fails loudly.
Verification against mops#
mops publishes per-file hashes on-chain — getFileHashesQuery(name, version) returning (fileId, sha256) pairs — so authenticity needs no HTTP outcall: one inter-canister query settles "is this really core@2.5.0". verifyTagAgainstMops reports files, matched, mismatched, missing, and extra (lists capped at 20 so the reply stays small), and records a verification timestamp only when all three difference lists are empty.
Content-addressing already proves a client cannot lie about content. Binding a name to a hash set is what the registry check adds. It is a registration-time call, never on the compile path.
Things you cannot do#
| Why | |
|---|---|
| Keep uploaded source private from the operator | The VFS is canister state. There is no encryption; a canister that compiles your code must read your code. |
Drive a tag build from moxzi build --remote today | The CLI has no tag flag; upload always sends the full closure. The protocol above is the canister API, exercised by the pic-js gate, and is what the CLI is expected to speak. |
| Use a tag the operator has not pointed at a registry | useTags fails closed: no setMopsRegistry, or a tag never verified, means no build. |
| Rely on files staying put | Your tree persists until you remove it, but nothing promises retention across an operator's maintenance. Re-upload is cheap; the whole-build cache makes a repeat compile a lookup. |
| Register a tag under a name alone | Tags are keyed by name@version. Three core majors and two base majors coexist in one real closure, which is exactly why. |
If your code is closed and the disclosure is unacceptable, run your own compiler canister. The wasm is in this repo and make compiler-canister-wasm builds it.
Next#
- Building on-chain — the privacy tradeoff in context.
- How a build runs — the closure is also the cache key.
- What a build costs — the escrow is sized from uploaded bytes.