thaw: snapshotable gVisor for sub-second cold starts

A from-scratch implementation of Modal's snapshot-restore model on top of gVisor, with the full substrate underneath: content-addressed image format, unified blob store, custom lazy FUSE, thin scheduler, and a Modal-shaped Python SDK.

Ryan Clark · May 2026

The thesis, and the shape of the cold-start tax

Modal's pitch (and the reason cold starts on ML serverless got fast) is straightforward: if a function's expensive work is importing torch and building a model, do that work once, checkpoint the warm process, and on every later cold start restore the post-import memory state directly. The model never has to be built again. The Python interpreter never has to re-import.

The natural questions are: how much of that speedup is the snapshot itself, and how much is everything else? Is runsc restore the bottleneck? Is image transfer? Is file system materialization? If snapshot-restore is the magic, what's the irreducible cost? I wanted to know by building it.

thaw is the result: a Go binary (cmd/thaw {snapshot, restore, image, push, pull, mount, deploy, scheduler}) plus a Modal-shaped Python SDK, all sitting on top of gVisor's checkpoint/restore. Everything in the path is custom: image format, on-disk store, lazy FUSE, scheduler. Stdlib-only with one deliberate exception (vendored hanwen/go-fuse; the FUSE wire protocol is a learning-scoped target, not a reimpl-scoped one, and S3 SigV4 went the other way for the same reason).

The headline number, and what it cost to get there

The real workload is examples/torch_app.py: import torch at module load, build resnet50 with weights baked into the image, transform + 1000 ImageNet labels, one warm forward at snapshot time. Per request: a real 661 KB JPEG → real classification. Driven through the actual platform (thaw deploy → thaw scheduler → SDK app.call with JPEG bytes). The metric is the SDK's stat.total_ms: invoke → first usable result, fully synchronous (no scheduler hiding restore time behind a queue).

All numbers in milliseconds. Substrate: Hetzner CPX31 (AMD EPYC-Genoa, 4 shared vCPU, 8 GB RAM, local NVMe), runsc release-20260511.0, --network=none. 0/44 contract violations across the run.

Cold p50 is comfortably sub-second. The cold p99 (806 ms) is about 1% over the 800 ms target for a real heavy inference; lighter workloads (numpy, FastAPI) clear it with margin. I'm calling out the two over-target samples explicitly because "10/12 under 800 ms, 2 at most 806 ms" is more useful than rounding the headline down. Warm 125 ms is where serverless ML actually lives in practice, once a function has been called once.

The row in the middle is the part I want to point at. runsc restore is ~280 ms. It was ~280 ms when the integrated platform measured a cold p50 of 5.7 seconds. It was ~280 ms when the platform measured 1.6 seconds. It was ~280 ms at the headline 774 ms. The snapshot-restore primitive is not the bottleneck and never was. Every other number in the cold path is something happening around restore: file system materialization, page-file reassembly from a content-addressed store, FUSE faults on the critical path, lazy-loading without prefetch. Each of those has its own little story.

The journey, because that's where the reasoning lives

The arm64 dead end

Early in the project, on a local arm64 Lima VM, runsc checkpoint worked fine (~30 ms) but runsc restore SIGILL'd on every real CPython process. Both systrap and ptrace platforms. A trivial /bin/sleep restored cleanly, so core checkpoint/restore worked; the symptom was specific to processes with the kind of state CPython has. My best hypothesis is arm64 PAC/BTI register state not preserved across gVisor restore: the userspace instructions live, but the kernel/CPU pointer-authentication state where the bug lives is untouched by Rosetta (asked and answered separately).

This was the decision-gate moment: thesis-killer, or substrate-specific? I provisioned a Hetzner CPX31 (AMD EPYC-Genoa, x86-64), ran the same matrix, and CPython restored cleanly on both platforms. Substrate decision settled: all subsequent work is x86-64 only. Modal ships this on x86-64; that's not a coincidence. The arm64 issue is a real bug surface, not an architectural property of gVisor, but it ate a half-week and reframed the rest of the project as cloud-only.

The content-addressed store, and a measured negative

The bundle format is content-addressed: rootfs and checkpoint files are decomposed into sha256-named 4 MiB chunks, deduped by construction, reassembled with corruption detection. Tiered storage (NVMe ← S3 origin) is one indirection away, via a hand-rolled SigV4 client because writing it once teaches you what the surface actually looks like and the wire protocol is small enough to be the right scope.

The first cut chunked the flattened docker export tar. Re-importing the identical image deduped at ~98.5% (CAS works). But two images derived from the same base (a numpy-pandas image and the torch image, both FROM python:3.12-slim) shared 0 chunks. The dedup payoff across artifacts was a flat zero.

The cause is exactly what you'd expect once you look at it: torch's extra files shift byte offsets in the tar, so every 4 MiB chunk boundary misaligns versus the simpler image's shared base. File content never coincides with a chunk boundary. CAS is exact; the offsets weren't.

The instinct here is to reach for content-defined chunking (FastCDC). I deliberately didn't. The same root cause (chunking-over-tar) also broke the FUSE that would come next: a lazy FUSE can't serve one file without materializing the whole tar prefix that contains it. So instead of building FastCDC as a standalone thing that would be a throwaway, I went straight to per-file content-addressed entries. Each file becomes its own list of chunks; the image manifest tracks files with their per-file content addresses; the FUSE can seek to any file at any offset.

After the change: re-importing the identical image dedupes to +0 chunks (was +68); bench and torch share 5,612 chunks ≈ 58% of bench reused by torch (was 0); restore got faster not slower (no whole-tar reassemble + untar on the critical path). One refactor solved two problems. The lesson I keep coming back to from this one: when a measured negative has an obvious fix that's also the right move for a downstream system, do the right move instead of patching the negative in isolation.

Lazy without prefetch is a trap

The custom lazy FUSE (internal/thawfs) faults chunks on demand from CAS. find over an entire ~10k-file torch image faults 0 content chunks (directory entries from the manifest are enough); reading two files lands at 8 of 9,617 chunks served = 0.1%. Lazy as advertised.

The question is whether lazy is fast in practice on a heavy workload. Measured: booting the per-file CAS image lazily and running torch init to first result lands at ~18.9 s p50. The naive docker pull cold is ~29 s. The lazy path is barely 1.5× better. A trap, not a win.

What's happening: torch's import working set is huge and synchronous. The FUSE doesn't know in advance which chunks will be needed, so every page fault is a round-trip on the critical path. The catastrophic first-read tail is the cost of not knowing the working set. The same lesson reappeared when I tried to optimize cold restore later: an early prefetch pass that fetched only the checkpoint pages.img performed identically to no prefetch (~5.6 s for restore), and only adding the rootfs profile to the prefetch set collapsed it to 0.5 s. gVisor's C/R re-faults file-backed library mappings (libpython, .so's) on restore, so the snapshot-time profile is exactly the right prefetch target. Measuring caught the wrong call before shipping it; I reverted the half-measure and removed the misleading flag.

Lazy faulting wins when the working set is small and unknown; eager materialization wins when it's large and knowable. For a snapshot-restore platform, the working set is always knowable because the snapshot-time profile is the working set. The platform's default is the eager path; the lazy FUSE is opt-in and retained for the small-workload case it actually wins on.

The cold-path collapse: materialize-once, restore-many

The first integrated build of the scheduler measured 5.7 s p50 cold for the real workload. This was nominally a regression, since the bare thaw restore path was hitting ~240 ms. The integration was paying ~5 seconds I had to find.

The diagnosis: per call, the scheduler was re-extracting the full 1.18 GB rootfs from CAS into a fresh per-sandbox work directory, then runsc restore was reassembling the 365 MB checkpoint pages.img from CAS bytes the same way. Both were repeatable work that could have been done once.

The fix is a symmetric pair of caches, digest-keyed:

The architectural detail that matters: the shared rootfs is a passive cached directory plus a per-sandbox kernel overlay. That's the lazy-FS chain minus the FUSE. The active in-process FUSE was incompatible with the scheduler's lifecycle, because thaw restore exits after the restore call, and an in-process FUSE dies with it, but the scheduler has to keep the sandbox parked for warm-pool reuse. A passive directory + kernel overlay survives thaw restore exiting; the scheduler owns the overlay umount in sandbox.teardown() after runsc delete. Cold / warm / evict / drain are leak-free (0 sandboxes, 0 mounts across the test matrix). This is the moment the lazy-FS work paid off as architecture, not as the FUSE you'd reach for.

The one-time materialize (~5.7 s) is real, per (function, host). I report it separately and prominently rather than folding it into the steady-state distribution. It amortizes over every later cold start and is the same class of cost as Modal materializing at deploy. A later refinement moves the materialization into thaw deploy itself, so the first invocation pays only the genuinely-irreducible import torch + build-resnet50 cost (~13.4 s), gets served, and then checkpoints in the background so call #2 is already steady-state cold at about 0.84 s, roughly 16× faster than call #1. The snapshot is the actual win.

The honest crossover

The benchmark I most want to point at, because it's the one where the result didn't go the way I expected: a 2-way head-to-head against eager-pull-cold (i.e., the naive "just docker pull and run") across three workloads (tiny / numpy-pandas-fastapi / torch-resnet50) × two network profiles (loopback / 25 ms RTT, 100 Mbit). N=30 warm, N=12 cold, 504 runs, 0 failures.

thaw wins warm everywhere, net-independent, by 5–7× (torch warm is 1.9 s vs eager 14 s; this is the case that matters once a function has been invoked once). It wins cold 7–9× on a fast link. But cold over a real-network model for heavy images, thaw loses, with torch by nearly 3×.

The mechanism is visible in the timing structure, though I didn't packet-trace it. thaw ships uncompressed CAS bytes and serializes the full prefetch working set onto the critical path. docker pull ships gzip layers in parallel and is netem-insensitive; its cold wall is extract + import, not transfer. The same metric the head-to-head measures (fully synchronous start to first real op) refuses thaw the scheduler-overlap it doesn't have in this benchmark. The integrated scheduler eventually lets that overlap happen, but the head-to-head deliberately doesn't allow it; the point is to compare substrates with the same discipline.

I report this as a loss and a mechanism, not a footnote. Two named next-steps that would move the constant are recorded but not built: compress CAS chunks in transit (zstd, format-additive), and overlap prefetch with scheduling. Both are out of scope for "understand the substrate", and either would change this crossover. The point of running the benchmark wasn't to win it.

What's actually in the box

Stdlib-only Go, cross-compiles darwin/arm64 → linux/amd64, so the runtime box needs no Go toolchain. One vendored dep: hanwen/go-fuse v2.10.1, the deliberate exception. The FUSE wire protocol is a learning-scoped target, not a reimpl-scoped one; opposite call from S3 SigV4, for the same reason.

Honest caveats

• n=12 cold / n=30 warm on the headline workload. Tight sd (cold 26 ms); the precise statement is "10/12 < 800 ms, 2 ≤ 806 ms" rather than a clean p99. 0/44 contract violations.
• The one-time materialize (~5.7 s) is real, per (function, host). Reported separately rather than folded into the steady-state distribution. Same class of cost as Modal materializing at deploy; the auto-snapshot path moves the rootfs piece into thaw deploy's untimed prepare.
• The no-checkpoint lazy path is a trap for heavy imports (~19 s). Eager materialize-once is the one to use; auto-snapshot-on-first-invocation handles the no-checkpoint-yet case.
• Cold is measured fully synchronous, with no scheduler hiding restore time behind a queue. Deliberately not flattered.
• Shared-vCPU cloud VM, not bare metal. --network=none for the headline run; the head-to-head's netem matrix is the network story.
• Single-node, serialized calls. Distributed scheduling is out of scope by design.
• Cache GC is out of scope. Shared rootfs + checkpoint caches grow per distinct deployment; intended for evaluation, not production.
• No Modal head-to-head. The honest framing is "Modal's model, from scratch on gVisor"; the goal was understanding the stack, not benchmarking against the original.

The visceral demo

Same gVisor, same image, same real photo, same resnet50. The only variable is thaw's snapshot + warm pool. Two contestants race live: contestant A runs without a snapshot (fresh runsc run, import torch, build model, classify), and contestant B runs thaw (SDK → scheduler, one cold restore then warm-pool reuse, keeps classifying until A finishes its single result).

Validated run: dog.jpg → Samoyed on both sides, with the one-time materialize shown as an explicit untimed prepare. In the 5.53 seconds A needed for one inference, B served 16 (1 cold + 15 warm).

What I'd do next

• Zstd-compress CAS chunks in transit. Format-additive: CAS hashes the uncompressed bytes, so only the wire format changes. Moves the constant that decided the network-cold crossover.
• Overlap prefetch with scheduling, which is what Modal actually does. The scheduler exists; what's missing is hiding the cold's prefetch tail behind the request queue's natural latency. That belongs in the scheduler, not the substrate.
• Cache GC. Currently out of scope; for production, the shared rootfs and checkpoint caches grow per distinct deployment digest, so LRU eviction with a configurable cap is the obvious add.
• A Modal head-to-head. The goal was to understand the stack, not benchmark against the reference implementation. Now that the stack is built, the comparison would be worth doing on the same workload, same VM class, same JPEG.

Code & resources

• Repository: github.com/RyanClark2k/thaw
• Real-workload benchmark: scripts/demo.sh; raw docs/data/demo.csv
• No-checkpoint measurement: scripts/nockpt.sh; raw docs/data/nockpt.csv
• Head-to-head vs eager-pull-cold: scripts/headtohead.sh; raw docs/data/headtohead.csv
• Race demo: scripts/race.sh
• Workload: examples/torch_app.py + examples/Dockerfile.torchapp