ArcBox Boot Assets

boot-assets is the single source of truth for ArcBox VM boot artifacts.

Each release publishes per-architecture tarballs plus a unified multi-target manifest:

1. boot-assets-{arch}-v{version}.tar.gz
2. boot-assets-{arch}-v{version}.tar.gz.sha256
3. manifest.json (multi-target)

The tarball contains:

1. kernel — pre-built Linux kernel from arcboxlabs/kernel (all drivers built-in, CONFIG_MODULES=n)
2. rootfs.erofs — minimal read-only rootfs (busybox + Btrfs/ext4 storage tools + iptables-legacy + CA certs)
3. manifest.json — per-arch manifest (merged into unified manifest at release time)

No agent binary in the boot tarball, and no initramfs. Guest runtime binaries are published separately as manifest-listed host-side binaries. The host exposes them as a VirtioFS transfer source; the guest verifies and materializes each generation onto its Btrfs data disk before execution.

Manifest Schema

schema_version equals the major component of asset_version (e.g. 0.2.0 → 0, 1.0.0 → 1).

The manifest supports multiple target architectures and host-side binaries:

{
  "schema_version": 0,
  "asset_version": "0.2.0",
  "built_at": "2026-03-03T12:00:00Z",
  "source_repo": "arcboxlabs/kernel",
  "source_ref": "v0.1.0",
  "source_sha": "abc123",
  "targets": {
    "arm64": {
      "kernel": { "path": "arm64/kernel", "sha256": "...", "version": "6.12.8" },
      "rootfs": { "path": "arm64/rootfs.erofs", "sha256": "..." },
      "kernel_cmdline": "console=hvc0 root=/dev/vda ro rootfstype=erofs earlycon"
    },
    "x86_64": {
      "kernel": { "path": "x86_64/kernel", "sha256": "...", "version": "6.12.8" },
      "rootfs": { "path": "x86_64/rootfs.erofs", "sha256": "..." },
      "kernel_cmdline": "console=ttyS0 root=/dev/vda ro rootfstype=erofs earlycon"
    }
  },
  "binaries": [
    {
      "name": "dockerd",
      "version": "27.5.1",
      "targets": {
        "arm64":  { "path": "bin/arm64/dockerd",  "sha256": "..." },
        "x86_64": { "path": "bin/x86_64/dockerd", "sha256": "..." }
      }
    }
  ]
}

The optional targets.{arch}.runtime field remains accepted only so older manifests stay parseable. New releases do not emit or download the legacy runtime.erofs block image.

boot-assets sync-binaries supports both tarball extraction and direct binary downloads. Use format = "tgz" plus extract = "path/in/archive" for archive sources and format = "binary" for direct executable URLs.

CLI Usage

The tool is built with Rust. Install with cargo build --release.

# Build EROFS rootfs only
boot-assets build rootfs --output build/rootfs.erofs --arch arm64

# Full release build (single arch)
boot-assets build release \
  --version 0.2.0 \
  --kernel build/kernel-arm64 \
  --arch arm64

# With pre-built rootfs
boot-assets build release \
  --version 0.2.0 \
  --kernel build/kernel-arm64 \
  --rootfs build/rootfs.erofs \
  --arch arm64 \
  --source-repo arcboxlabs/kernel \
  --source-ref v0.1.0

# Merge per-arch manifests into unified multi-target manifest
boot-assets merge-manifest dist/arm64/manifest.json dist/x86_64/manifest.json \
  --output dist/manifest.json

Build And Release

Verification

Local checks use the same commands as CI:

cargo fmt --check
cargo clippy --all-features -- -D warnings
cargo test --all-features --lib --bins
cargo build --features build --no-default-features
cargo test --features build --no-default-features --test e2e -- --nocapture

The Rust E2E target in tests/e2e/ is organized as one Cargo integration test crate with submodules. The default local E2E does not require a hypervisor. It validates the published artifact contract: per-arch release tarballs contain the expected files and checksums, the unified manifest points at the expected CDN object paths with matching SHA256 values, and AssetManager can consume the published layout over a real local HTTP server by downloading kernel/rootfs plus runtime binaries into their install locations. This gives a Linux-compatible product smoke test before higher-level VM/HV validation.

To validate a real published environment, run the ignored live E2E test against the CDN. Omit BOOT_ASSETS_LIVE_VERSION to resolve latest.json:

BOOT_ASSETS_LIVE_CDN_BASE_URL=https://boot.arcboxcdn.com \
BOOT_ASSETS_LIVE_VERSION=0.5.1 \
BOOT_ASSETS_LIVE_ARCH=x86_64 \
BOOT_ASSETS_LIVE_PREPARE_BINARIES=true \
cargo test --features download --no-default-features --test e2e live::live_published_boot_assets_are_consumable -- --ignored --nocapture

To prove the kernel and rootfs boot as a real Linux system, run the QEMU boot E2E test. This currently targets x86_64 and waits for the guest to reach Linux userspace (/sbin/init) from the EROFS rootfs. Prefer passing local artifacts from the current build:

# Requires qemu-system-x86_64 on PATH.
BOOT_ASSETS_RELEASE_TARBALL=dist/x86_64/boot-assets-x86_64-v0.5.1.tar.gz \
cargo test --features download --no-default-features --test e2e boot::qemu_boots_x86_64_linux -- --ignored --nocapture

You can also pass unpacked local artifacts directly:

BOOT_ASSETS_BOOT_KERNEL=dist/x86_64/kernel \
BOOT_ASSETS_BOOT_ROOTFS=dist/x86_64/rootfs.erofs \
BOOT_ASSETS_BOOT_CMDLINE="console=ttyS0 root=/dev/vda ro rootfstype=erofs earlycon" \
cargo test --features download --no-default-features --test e2e boot::qemu_boots_x86_64_linux -- --ignored --nocapture

If no local artifacts are provided, the same test falls back to the live CDN:

BOOT_ASSETS_LIVE_CDN_BASE_URL=https://boot.arcboxcdn.com \
BOOT_ASSETS_LIVE_VERSION=0.5.1 \
BOOT_ASSETS_LIVE_ARCH=x86_64 \
cargo test --features download --no-default-features --test e2e boot::qemu_boots_x86_64_linux -- --ignored --nocapture

GitHub Actions also exposes these as the manual Live E2E workflow for testing actual published boot-assets on Linux, including an optional QEMU boot step.

CI release workflow

Workflow file: .github/workflows/release.yml

Trigger:

1. Push tag: v*
2. Manual dispatch with explicit version

Pipeline stages:

1. Download kernel — downloads pre-built ARM64/x86_64 kernels from arcboxlabs/kernel release
2. Build EROFS rootfs — creates minimal rootfs from Alpine static binaries (per-arch)
3. Assemble — packages kernel + rootfs.erofs + manifest.json into tarball (per-arch)
4. Merge — merges per-arch manifests into unified multi-target manifest
5. Release — publishes to GitHub Releases and Cloudflare R2 (served at boot.arcboxcdn.com)

Republishing a released version

.github/workflows/republish.yml (manual dispatch) rebuilds the CDN layout of an existing release and publishes it to R2 without rebuilding: kernel, rootfs and manifest.json come from the GitHub Release, upstream runtime binaries are re-downloaded from the tag's upstream.toml, and FEX plus the patched containerd come from the release run's boot-assets-release-* artifacts (pass that run's id; they expire after 14 days). Every object is checked against the manifest's sha256 before upload, because the daemon pins that manifest by hash and a rebuild would change it.

Local build

Prerequisites:

1. Rust toolchain
2. Docker (for extracting static Alpine binaries and building the EROFS image)
3. Kernel binary from arcboxlabs/kernel release

# Build the CLI
cargo build --release

# Download kernel from arcboxlabs/kernel release
gh release download v0.1.0 --repo arcboxlabs/kernel --pattern "kernel-arm64" --dir build/

# Full release build
./target/release/boot-assets build release \
  --version 0.2.0 \
  --kernel build/kernel-arm64 \
  --arch arm64

Output files are written to dist/.

EROFS Rootfs Contents

/ (EROFS, read-only, LZ4HC compressed)
├── bin/
│   └── busybox          # Static busybox (+ a symlink for every applet)
├── sbin/
│   ├── init             # busybox init: early mounts → mount VirtioFS → run agent init
│   ├── arcbox-storage-recovery # Dedicated recovery PID 1
│   ├── mkfs.btrfs       # Btrfs formatter (first-boot data disk)
│   ├── btrfs            # Btrfs inspection and data recovery tools
│   ├── mkfs.ext4        # ext4 formatter (first-boot metadata disk)
│   ├── e2fsck           # ext4 filesystem checker
│   ├── iptables         # iptables-legacy (Docker bridge networking)
│   └── (symlinks)       # iptables-save, iptables-restore, ip6tables, ...
├── lib/
│   └── *.so*            # musl loader + shared libs for packaged host utilities
├── cacerts/
│   └── ca-certificates.crt
└── (mount points)       # tmp/ run/ proc/ sys/ dev/ mnt/ arcbox/ Users/ etc/ var/

The storage tools run without a mounted data disk. Use btrfs check --readonly and e2fsck -fn on unmounted recovery copies for diagnosis. Preserve the data and metadata images together before attempting repairs; a mounted filesystem or a successful mount alone is not a consistency check.

The rootfs carries an e2fsprogs 1.47.3 patch that preserves errors detected before the main passes when -n disables repairs. Without the patch, e2fsck -fn can report an invalid group descriptor checksum and exit 0. Repair modes keep their upstream behavior.

Every rootfs build runs a read-only regression check with the built tools. A new 64 MiB ext4 image with 4 KiB blocks and metadata checksums must exit 0; a copy with one flipped group descriptor checksum bit must exit 4. Both checks must preserve the full image SHA-256. To rerun the check on Linux, pass the directory containing the produced tools:

sh src/build/scripts/check-e2fsck-readonly.sh /path/to/rootfs/sbin

Recovery VMs must use init=/sbin/arcbox-storage-recovery arcbox.storage_recovery=1. This entry mounts pseudo-filesystems and the agent share, then executes arcbox-agent storage-recovery. The entry rejects an agent without the arcbox-storage-recovery-v1 capability marker before executing the agent. An incompatible image or agent must stop recovery; recovery must not fall back to normal initialization.

FEX runtime

The rootfs does not embed or register FEX. The guest agent copies the manifest-listed binary into the active Btrfs runtime generation, then registers /run/arcbox/runtime/bin/FEX as the x86_64 binfmt_misc interpreter with the upstream POCF flags. Registration therefore never pins a VirtioFS file descriptor. If FEX is absent, boot continues normally and no x86_64 handler is registered.

FEX_ROOTFS is not set; FEX's built-in default RootFS is /, so amd64 OCI containers provide their own amd64 rootfs, loader, and shared libraries. The F flag pins the opened Btrfs-backed interpreter so container processes can invoke it even when the runtime path is not visible inside the container rootfs.

FEX is built from source in the release workflow with boot-assets build fex. The command builds the arm64 FEX interpreter as a static-pie executable (-static-pie) and stages it into the binary manifest — no dynamic library closure, and no FEXServer. Static linking is required: the F flag pins only the interpreter executable's fd into the container namespace, so a dynamic FEX would have the kernel resolve its PT_INTERP against the container's amd64 rootfs (which lacks FEX's arm64 loader) and fail with ENOENT. The build asserts the produced binary carries no PT_INTERP (assert_static_executable).