We made our filesystem 47× faster by deleting it
The microsandbox project has significantly improved its filesystem performance by replacing the user-space filesystem with a direct Linux disk image. This change resulted in a geometric mean speedup of 47 times across various file operations, with some operations being over 1,000 times faster. The new approach eliminates the need for FUSE, allowing file operations to remain within the VM, thus enhancing efficiency.
- ▪The previous user-space filesystem caused delays due to multiple round trips between the VM and host.
- ▪The new implementation uses a Linux filesystem image that the VM can mount directly, improving speed.
- ▪The project was able to achieve a substantial reduction in code complexity, with the host filesystem code being about 5,300 lines shorter.
Opening excerpt (first ~120 words) tap to expand
A user in our Discord said microsandbox felt slow. Listing every file in the Python standard library took 5.3 seconds inside a sandbox; in Docker it took milliseconds. We went digging. We fixed it in v0.4: we replaced our user-space filesystem with a Linux disk image that the VM mounts directly. The geometric mean speedup across our mixed guest-visible filesystem suite is 47×, with the worst-case rows more than 1,000× faster, and the host filesystem code is about 5,300 lines shorter. Where this started My first try was monofs: a content-addressed filesystem with block-level dedup, compression, and distributed read replicas. It stored images at 1.3× their original size on disk, and microsandbox is local-first, so the long-tail dedup payoff wasn't worth the up-front cost.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Hacker News (Newest).
Discussion
0 commentsMore from Hacker News (Newest)
