Using kernelsalpha

Opt into a package's native kernel, check what you link, and know what you're trusting.

A native kernel is a small wasm library, built from an existing Rust crate such as sha2 or k256, that backs functions in a Motoko package. When your application opts in, moxzi links the kernel into your canister and calls it in place of those functions' Motoko bodies. Without the opt-in, or under moc, the Motoko bodies run.

This page shows how to opt in, how to check a kernel, and what you trust when you link one. To write your own, see Writing a kernel.

When a kernel is worth it#

Use a kernel for byte-oriented work that costs a lot of instructions in Motoko: hashing, signing and verifying, curve arithmetic, checksums and encodings. One call each, on a local replica:

Across the published packages, kernels cost 2 to over 1,000 times fewer instructions than the Motoko bodies, 8 to 70 times fewer for most functions. Kernels take and return bytes, text and simple values, so code that works on Motoko data structures, such as Candid, maps or JSON, gets nothing from them.

Opt in#

Add the package as a dependency, as usual:

[dependencies]
core = "2.6.0"
moxzi-hash = "0.1.0"

mops install

Then opt into its kernel. moxzi kernel add shows what the kernel is, checks that the kernel file's bytes match its recorded hash, and writes the pin into your mops.toml:

moxzi kernel add moxzi-hash

kernel moxzi-hash 0.1.0
  file     kernel/moxzi-hash.kernel.toml (its sha256 matches its bytes)
  sha256   d9e07eabd5c53e4567f6ce132857b52f7b0d3546f2443b73bdb3b07cf2244a2e
  abi      1
  size     48095 B
  source   https://github.com/icdevsorg/moxzi-kernels/tree/main/kernels/moxzi-hash/rust
  commit   445b558869a5552956229d704011c40480e1854c
  functions (7):
    Blake3.digest -> blake3_hash
    Blake3.keyedHash -> blake3_keyed
    Hmac.sha256 -> hmac_sha256
    Hmac.sha512 -> hmac_sha512
    Ripemd160.digest -> ripemd160
    Sha256.digest -> sha256
    Sha512.digest -> sha512
  tier 1   built, reviewed and pinned by moxzi at these exact bytes
./mops.toml: added [kernels] moxzi-hash = "sha256:d9e07eabd5c53e4567f6ce132857b52f7b0d3546f2443b73bdb3b07cf2244a2e"

The pin is the SHA-256 of the kernel bytes; commit it with mops.toml. moxzi kernel add doesn't install the package, so run mops add first. The application calls the package as usual:

import Array "mo:core/Array";
import Blob "mo:core/Blob";
import IC "mo:core/InternetComputer";
import Nat8 "mo:core/Nat8";
import Sha256 "mo:moxzi-hash/Sha256";

actor {
  // The SHA-256 of `size` bytes, and the instructions the hash took.
  public func hash(size : Nat) : async { digest : Blob; instructions : Nat64 } {
    let data = Blob.fromArray(Array.tabulate<Nat8>(size, func i = Nat8.fromNat(i % 256)));
    var digest : Blob = "";
    let instructions = IC.countInstructions(func() { digest := Sha256.digest(data) });
    { digest; instructions }
  };
};

Build. The build names each kernel it links:

moxzi build main.mo -o kernel.wasm

kernel: moxzi-hash sha256:d9e07eabd5c53e45 (48095 B, 7 function(s)) from https://github.com/icdevsorg/moxzi-kernels/tree/main/kernels/moxzi-hash/rust @ 445b558869a5552956229d704011c40480e1854c [tier-1]
main.mo -> kernel.wasm + kernel.did (296272 bytes, 20 file(s), 199 steps, 3.40s)
compiled by ~/.local/lib/moxzi/compiler.wasm (sha256 9294817c6e9069cbd0e712d0e9dde0ec1b6893d0e7b42d798fa3dd4c8cd772cb), moxzi 0.1.0-alpha.10

[tier-1] marks a kernel published by the moxzi project at one of its released pins; see Published kernels. Any other kernel is third-party. Three rules decide what is linked:

• Only your application opts in. A dependency's mops.toml can't add a kernel, and neither can a dependency of a dependency. A package with a kernel that you haven't listed runs its Motoko body.
• The pin covers the exact bytes. If the installed kernel isn't the one you pinned, for example after a version update, the build stops.
• The Motoko body is the fallback. Without the opt-in, or under moc, the same program builds with the Motoko bodies and gives the same results.
• An opted-in kernel is used, or the build stops. If the package declares a function its Motoko source doesn't have, or declares it with other types, the build fails with M0291 instead of quietly running the Motoko body. A kernel whose modules your program doesn't import isn't used, and the build says so in a note.

Without the opt-in#

Delete the [kernels] line and build again. The build says which package it didn't link, and the Motoko bodies run:

moxzi build main.mo -o motoko.wasm

note: package moxzi-hash provides a native kernel; it runs its Motoko code unless you opt in under [kernels] in mops.toml (`moxzi kernel add moxzi-hash`)
main.mo -> motoko.wasm + motoko.did (504536 bytes, 20 file(s), 204 steps, 4.07s)
compiled by ~/.local/lib/moxzi/compiler.wasm (sha256 9294817c6e9069cbd0e712d0e9dde0ec1b6893d0e7b42d798fa3dd4c8cd772cb), moxzi 0.1.0-alpha.10

To compare the two builds on a local replica, install both with PocketIC and call hash in each:

// Install both builds on a local replica and compare what SHA-256 costs in each.
import { readFileSync } from "node:fs";
import { PocketIc, PocketIcServer } from "@dfinity/pic";

const idlFactory = ({ IDL }) => IDL.Service({
  hash: IDL.Func([IDL.Nat], [IDL.Record({ digest: IDL.Vec(IDL.Nat8), instructions: IDL.Nat64 })], []),
});

const server = await PocketIcServer.start();
const pic = await PocketIc.create(server.getUrl());
const install = async (file) =>
  (await pic.setupCanister({ idlFactory, wasm: readFileSync(file) })).actor;
const kernel = await install("kernel.wasm");
const motoko = await install("motoko.wasm");

const col = (x) => x.toLocaleString("en-US").padStart(13);
console.log("        bytes    with kernel    Motoko body");
for (const size of [64, 1_000, 1_000_000]) {
  const k = await kernel.hash(BigInt(size));
  const m = await motoko.hash(BigInt(size));
  if (Buffer.compare(Buffer.from(k.digest), Buffer.from(m.digest)) !== 0) throw new Error("digests differ");
  console.log(`${col(size)}  ${col(k.instructions)}  ${col(m.instructions)}`);
}
await pic.tearDown();
await server.stop();

npm install @dfinity/pic@0.23.0
node measure.mjs

    bytes    with kernel    Motoko body
       64         10,763         62,164
    1,000         78,770        431,211
1,000,000     75,787,163    405,083,116

Both builds return the same digests.

Update a kernel#

A new version of a package usually carries new kernel bytes. The build then stops instead of linking bytes you haven't reviewed:

Error: kernel moxzi-hash: the installed kernel is sha256:d9e07eabd5c53e4567f6ce132857b52f7b0d3546f2443b73bdb3b07cf2244a2e, but your [kernels] pin is sha256:5be1c0ded5c53e4567f6ce132857b52f7b0d3546f2443b73bdb3b07cf2244a2e. If you reviewed this version, update the pin; otherwise do not build.

Review the new version (its declared source and commit, and its changelog), then run moxzi kernel add PACKAGE again. It replaces the pin and prints the old and new hashes.

Third-party kernels#

Any package can ship a kernel. The first build that links one that isn't tier-1 prints its package, version, sha256, declared source and commit, and what a kernel can do (below), then stops until you accept those exact bytes:

• At a terminal, type yes at the prompt.
• In CI and scripts, pass the pin: --accept-kernel PACKAGE=sha256:HEX. The error message prints the exact flag. --yes doesn't accept kernels.

Acceptances are kept per user, by package and sha256, so later builds of the same bytes don't ask, and a new version asks again. They're stored in accepted-kernels in moxzi's configuration directory (~/Library/Application Support/moxzi/ on macOS, ~/.config/moxzi/ on Linux), one line per kernel. Delete a line to be asked again. Set MOXZI_KERNEL_ACKS to use another file, for example one per CI job.

A package that uses a published package's name at other bytes is third-party too, and the warning says so. Writing a kernel shows the whole exchange.

Check a kernel against its source#

A pin says which bytes run. moxzi kernel verify connects those bytes to source: it rebuilds each kernel your application pins from the source and commit its package declares, with the same toolchain, and compares the result with the package's file and with your pin:

moxzi kernel verify .

It prints MATCH or DIFF for each pin and exits with a non-zero status unless all match. --crate DIR rebuilds from a crate you have on disk instead. You need rustup and the toolchain the CLI reference names. Kernels build to the same bytes on macOS and Linux.

The source of the published kernels is public at icdevsorg/moxzi-kernels (MIT OR Apache-2.0). Each package names the commit it was built from, and the repository's kernels/repro/verify.sh rebuilds every kernel in Docker and compares it with its pin, without moxzi installed.

See what a module contains#

Every linked module lists its kernels in a public moxzi:kernels metadata section, one line each: PACKAGE sha256:HEX abi=1, followed by version=, source= and commit= when the package declares them. With wasm-tools:

wasm-tools print kernel.wasm | grep moxzi:kernels

(@custom "icp:public moxzi:kernels" (after data) "moxzi-hash sha256:d9e07eabd5c53e4567f6ce132857b52f7b0d3546f2443b73bdb3b07cf2244a2e abi=1 version=0.1.0 source=https://github.com/icdevsorg/moxzi-kernels/tree/main/kernels/moxzi-hash/rust commit=445b558869a5552956229d704011c40480e1854c\0a")

Read it from a deployed canister:

dfx canister metadata CANISTER_ID moxzi:kernels --network ic

For an on-chain build, moxzi verify prints the same lines from the build record. See Verifying a build.

On-chain builds#

moxzi build --remote works with kernels:

1. The CLI asks about third-party kernels before it uploads anything.
2. It uploads each kernel the compiler canister doesn't have yet to the canister's shared file store, which computes the hash itself.
3. The canister links the kernels and records their pins and declared provenance in the build record. The result is the same module a local build produces.

What a kernel can do#

A kernel runs inside your canister. The first build that links a third-party kernel prints this:

A native kernel is compiled code that runs inside your canister and shares its memory. It can read and write anything in the canister's heap -- every user's data, keys, and the state that survives upgrades -- and return what it read. The linker keeps the system API away from it (it cannot send messages, spend cycles or set timers) and the glue checks what it returns, but nothing checks what it does to memory. The sha256 pin guarantees which bytes run, not that they are correct or benign. Accept a kernel only if you trust its author or have reviewed its source at the commit it names, and take a canister snapshot before deploying a new one.

The checks on what a kernel returns turn some bugs into traps that roll the message back: a result written past its buffer, a result longer than its memory, Text that isn't UTF-8, or a Bool that is neither 0 nor 1. A kernel that loops hits the instruction limit. The Kernel ABI lists every check and everything the linker refuses.

Troubleshooting#

Limitations#