thaw, snapshotable gVisor for sub-second cold starts

I built Modal's snapshot-restore model from scratch on gVisor, along with the content-addressed image format, blob store, lazy FUSE, scheduler, and Python SDK underneath it.

Ryan Clark · May 2026 · github.com/RyanClark2k/thaw

Modal made ML serverless cold starts fast with one idea. If a function spends its startup importing torch and building a model, do that work once, checkpoint the warm process, and restore the post-import memory state on every later cold start. Nothing re-imports and nothing rebuilds the model.

I wanted to know how much of that speedup comes from the snapshot and how much comes from everything around it. Does runsc restore dominate? Does image transfer? Does file system materialization? So I built the whole stack and measured each layer.

thaw wraps gVisor's checkpoint/restore in a Go binary (cmd/thaw {snapshot, restore, image, push, pull, mount, deploy, scheduler}) and a Modal-shaped Python SDK. I wrote everything in the path, including the image format, the on-disk store, the lazy FUSE, and the scheduler. Stdlib-only Go with one deliberate exception, a vendored hanwen/go-fuse, because I scoped the FUSE wire protocol as something to learn rather than something to reimplement. S3 SigV4 went the other way for the same reason.

What it measures

examples/torch_app.py imports torch at module load, builds resnet50 with weights baked into the image, and holds the transform plus 1000 ImageNet labels. Each request classifies a real 661 KB JPEG. Every sample runs through the actual platform, from thaw deploy to thaw scheduler to an SDK app.call, and the metric counts invoke to first usable result, fully synchronous, with no scheduler hiding restore behind a queue.

All numbers in milliseconds, on a Hetzner CPX31 (AMD EPYC-Genoa, 4 shared vCPU, 8 GB RAM, local NVMe), runsc release-20260511.0, --network=none. 0 of 44 contract violations across the run.

Cold p50 clears a second with room. Cold p99 at 806 ms misses the 800 ms target by about 1% on a heavy real inference, and lighter workloads (numpy, FastAPI) clear it with margin. I'd rather state 10 of 12 samples under 800 ms and 2 at most 806 ms than round the headline down. Warm at 125 ms describes where serverless ML actually lives once you've called a function once.

One row matters more than the headline. runsc restore takes about 280 ms. It took about 280 ms when the integrated platform measured a cold p50 of 5.7 seconds, and again at 1.6 seconds, and again at the final 774 ms. Restore never became the bottleneck at any point in the project. Everything else in the cold path happens around it, and each piece has its own story.

arm64 killed a week

I started on a local arm64 Lima VM. runsc checkpoint worked in about 30 ms, and runsc restore SIGILL'd on every real CPython process, on both the systrap and ptrace platforms. A trivial /bin/sleep restored cleanly, so core checkpoint/restore worked and the symptom tracked the kind of state CPython carries. I suspect arm64 pointer-authentication register state that gVisor doesn't preserve across restore, which Rosetta wouldn't touch either.

I had to decide whether that killed the thesis or only the substrate. I provisioned a Hetzner CPX31 on x86-64, ran the same matrix, and CPython restored cleanly on both platforms. Modal ships this on x86-64 too, which now reads less like a coincidence. The detour cost half a week and made the rest of the project cloud-only.

A measured negative in the content-addressed store

thaw decomposes rootfs and checkpoint files into sha256-named 4 MiB chunks, dedupes them by construction, and reassembles them with corruption detection. Tiered storage sits one indirection away, NVMe in front of an S3 origin through a hand-rolled SigV4 client.

My first cut chunked the flattened docker export tar. Re-importing an identical image deduped at about 98.5%, so the CAS worked. Then I checked two images built from the same base, a numpy-pandas image and the torch image, both FROM python:3.12-slim. They shared zero chunks.

torch's extra files shift byte offsets in the tar, so every 4 MiB boundary misaligns against the simpler image. File content never lines up with a chunk boundary. The CAS hashes exactly what I hand it, and I was handing it the wrong thing.

Content-defined chunking (FastCDC) would patch that, and I skipped it. Chunking over the tar also breaks the lazy FUSE I hadn't written yet, since a lazy FUSE can't serve one file without materializing the whole tar prefix around it. So I went to per-file content-addressed entries instead. Every file carries its own chunk list, the manifest tracks files by content address, and the FUSE can seek to any file at any offset.

Re-importing an identical image now adds 0 chunks, down from 68. The bench and torch images share 5,612 chunks, about 58% of bench reused by torch, up from nothing. Restore got faster too, because nothing reassembles and untars a whole tar on the critical path. When a measured negative has a fix that a downstream system needs anyway, build the fix rather than patching the negative in isolation.

Lazy faulting without prefetch

The lazy FUSE in internal/thawfs faults chunks on demand from the CAS. Running find over a 10k-file torch image faults zero content chunks, since the manifest already holds the directory entries, and reading two files serves 8 of 9,617 chunks. Lazy as advertised.

Then I measured whether lazy runs fast on a heavy workload. Booting the per-file CAS image lazily and running torch init to first result took about 18.9 s p50, against roughly 29 s for a naive docker pull. Barely 1.5×.

torch's import working set is huge and synchronous, and the FUSE can't know in advance which chunks anyone will need, so every page fault costs a round trip on the critical path.

The same thing bit me again later. An early prefetch pass that fetched only the checkpoint pages.img performed identically to no prefetch at all, around 5.6 s for restore. Adding the rootfs profile to the prefetch set collapsed it to 0.5 s, because gVisor re-faults file-backed library mappings (libpython, the .so files) on restore, so the snapshot-time profile names exactly the right prefetch set. Measuring caught the wrong call before it shipped, and I reverted the half-measure and deleted the flag that advertised it.

Lazy faulting wins on a small unknown working set. Eager materialization wins on a large knowable one. A snapshot-restore platform always knows the working set, since the snapshot-time profile is the working set. So thaw defaults to eager and keeps the lazy FUSE opt-in for the small-workload case it does win.

Materialize once, restore many

The first integrated scheduler measured 5.7 s p50 cold on the real workload, against about 240 ms for the bare thaw restore path. I had five seconds to account for.

Per call, the scheduler re-extracted the full 1.18 GB rootfs from the CAS into a fresh per-sandbox work directory, and then runsc restore reassembled the 365 MB checkpoint pages.img from CAS bytes the same way. Both repeat work that one call could do once. Two digest-keyed caches fixed it.

One architectural detail carried that fix. The shared rootfs works as a passive cached directory under a per-sandbox kernel overlay, which gives you the lazy-FS chain with the FUSE removed. An active in-process FUSE can't survive the scheduler's lifecycle, because thaw restore exits after the restore call and takes the FUSE with it, while the scheduler has to keep the sandbox parked for warm-pool reuse. A passive directory plus a kernel overlay outlives that exit, and the scheduler unmounts the overlay in sandbox.teardown() after runsc delete. Cold, warm, evict, and drain all run leak-free, at 0 sandboxes and 0 mounts across the matrix. The lazy-FS work paid off here as architecture rather than as the FUSE I originally reached for.

The one-time materialize costs about 5.7 s per function per host. I report it separately rather than folding it into the steady-state distribution. It amortizes over every later cold start, the same class of cost as Modal materializing at deploy. A later refinement moves it into thaw deploy, so the first invocation pays only the irreducible import torch and build-resnet50 cost of about 13.4 s, serves the request, then checkpoints in the background. Call #2 lands at about 0.84 s, roughly 16× faster than call #1.

Where thaw loses

I ran a two-way head-to-head against eager-pull-cold, meaning the naive docker pull and run, across three workloads (tiny, numpy-pandas-fastapi, torch-resnet50) and two network profiles (loopback, and 25 ms RTT at 100 Mbit). N=30 warm, N=12 cold, 504 runs, 0 failures.

thaw wins warm everywhere by 5 to 7×, independent of the network. torch warm runs 1.9 s against eager's 14 s, and that case decides things once you've invoked a function once. thaw wins cold by 7 to 9× on a fast link, and loses cold over a realistic network on heavy images, by nearly 3× on torch.

I didn't packet-trace it, but the timing structure shows the mechanism. thaw ships uncompressed CAS bytes and serializes the full prefetch working set onto the critical path. docker pull ships gzip layers in parallel and barely notices netem, and its cold wall comes from extract plus import rather than transfer. This benchmark measures a fully synchronous start to first real op, which denies thaw the scheduler overlap it doesn't have here. The integrated scheduler can create that overlap, and I kept it out of the head-to-head so both substrates run under the same discipline.

Two changes would move that constant, and I built neither. Compressing CAS chunks in transit with zstd stays format-additive, since the CAS hashes the uncompressed bytes and only the wire format changes. Overlapping prefetch with scheduling is what Modal actually does, and it belongs in the scheduler rather than the substrate.

What's in the box

The Go side cross-compiles darwin/arm64 to linux/amd64, so the runtime box needs no Go toolchain.

The race

Same gVisor, same image, same photo, same resnet50, with thaw's snapshot and warm pool as the only variable. Contestant A runs without a snapshot, so it does a fresh runsc run, imports torch, builds the model, and classifies. Contestant B goes through the SDK and scheduler, takes one cold restore, then reuses the warm pool and keeps classifying until A finishes its single result.

Both sides label dog.jpg as Samoyed, with the one-time materialize shown as an explicit untimed prepare. In the 5.53 seconds A needed for one inference, B served 16, one cold and 15 warm.

Caveats

• n=12 cold and n=30 warm on the headline workload, with a tight sd of 26 ms on cold. The precise statement reads "10 of 12 under 800 ms, 2 at most 806 ms" rather than a clean p99.
• The one-time materialize of about 5.7 s is real, per function per host, and I keep it out of the steady-state distribution.
• The no-checkpoint lazy path costs about 19 s on heavy imports. Use eager materialize-once, and let auto-snapshot-on-first-invocation handle the no-checkpoint-yet case.
• Cold runs fully synchronous, with no scheduler hiding restore behind a queue. I deliberately didn't flatter it.
• Shared-vCPU cloud VM rather than bare metal, and --network=none for the headline run. The head-to-head's netem matrix carries the network story.
• Single node, serialized calls. Distributed scheduling stays out of scope by design, and so does cache GC, so the shared rootfs and checkpoint caches grow per distinct deployment.
• No Modal head-to-head. I set out to understand the stack rather than benchmark against the original, though now that the stack exists the comparison would be worth running on the same workload and VM class.

Code and data

• Repository at github.com/RyanClark2k/thaw
• Real-workload benchmark scripts/demo.sh, raw docs/data/demo.csv
• No-checkpoint measurement scripts/nockpt.sh, raw docs/data/nockpt.csv
• Head-to-head against eager-pull-cold scripts/headtohead.sh, raw docs/data/headtohead.csv
• Race demo scripts/race.sh, workload examples/torch_app.py and examples/Dockerfile.torchapp