Content-defined chunking in Bazel's remote cache
BuildBuddy has implemented Content-Defined Chunking (CDC) in Bazel's remote cache to reduce data transfer and storage by reusing unchanged portions of build outputs. This approach allows only the changed bytes to be uploaded or downloaded, rather than entire files, improving efficiency for large binaries and packages. The feature is available in Bazel 8.7 or 9.1+ using the --experimental_remote_cache_chunking flag.
- ▪BuildBuddy's Remote Cache uses Content-Defined Chunking to reuse chunks of data that have not changed.
- ▪In benchmarks on BuildBuddy's own repository, CDC reduced upload sizes by 40% and disk cache usage by 40%.
- ▪CDC helps optimize transitive actions like linking, bundling, and packaging, where small input changes result in large output files with mostly identical content.
- ▪The CDC feature requires Bazel 8.7 or 9.1+ and must be enabled with the --experimental_remote_cache_chunking flag.
- ▪Without CDC, remote caches treat slightly modified large files as entirely new blobs, leading to inefficient data transfers and storage duplication.
Opening excerpt (first ~120 words) tap to expand
Remote Cache CDC: Reusing BytesTyler French, Engineer @ BuildBuddyMay 1, 2026 · 15 min read The goal: move the changed bytes, not the whole output. BuildBuddy's Remote Cache uses Content-Defined Chunking (CDC) to make large build outputs behave more incrementally. When a binary, bundle, package, or archive is mostly unchanged, BuildBuddy can reuse chunks it has already seen instead of re-uploading or re-downloading the entire file. In our Bazel chunking implementation PR, we observed 40% less data uploaded and a 40% smaller disk cache when benchmarked on BuildBuddy's own repo. To enable client-side CDC with BuildBuddy, use Bazel 8.7 or 9.1+ and pass --experimental_remote_cache_chunking. Setting the Scene The next frontier for build caching is not just skipping actions.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Buildbuddy.
Discussion
0 commentsMore from Buildbuddy
