ccache Setup

How OctoMY™ uses ccache to make rebuilds and parallel checkouts fast, what you must configure once per machine, and how the build tells you when something is off.

TL;DR for a new build site

1. Install ccache (the qbs build autodetects it on PATH).
2. Run the two ccache --set-config commands under Machine-local setup.
3. That's it. If you skip step 2, every build prints a warning naming the exact commands to run.

Standing up a whole machine, not just ccache? Start from Set up a build site, which covers the checkout layout this page's base_dir assumes.

How it fits together

There are two independent halves, and it's important not to confuse them:

qbs never writes your ccache config. It only prepends the word ccache to the compile command. Everything about where the cache lives and what is shared is ccache's own configuration.

The cache is one machine-wide directory

Every ccache invocation — from any checkout, any branch, any session — reads and writes a single cache directory (default ~/.cache/ccache). There is no per-checkout or per-session cache. That shared directory is the entire basis for "sessions share ccache".

Machine-local setup

These are the settings that are not carried by git and must be applied once per machine. Run them as your build user:

# 1. Cross-checkout sharing: treat the directory that CONTAINS all your
#    checkouts as the hash base, and stop hashing the working directory.
#    Without both, the same file in .../paint and .../comms hashes
#    differently and the two checkouts never share cache entries.
ccache --set-config base_dir=/home/you/Desktop/Projects/octomy/OctoMY
ccache --set-config hash_dir=false

# 2. Cache size cap. One full build puts about 10 GB into the cache for
#    each profile (debug 9.5 GB, measured 2026-10-03). Each branch whose
#    code differs, and each profile, adds its own entries. 40 GB holds a
#    few branches in one profile. With a dedicated build disk, give it room:
#    the reference machine uses 500G.
ccache --set-config max_size=40G

Point base_dir at the directory that contains your checkouts, not at a single checkout. If your trees live under ~/src/octomy/{paint,comms,agent}, use base_dir=~/src/octomy.

Do not set cpp.compilerWrapper in your qbs profile. Older setups added profiles.<profile>.cpp.compilerWrapper: ["ccache"] by hand; that now double-prepends ccache (the repo autodetect already adds it, giving ccache ccache g++). Remove it: qbs config --unset profiles.<profile>.cpp.compilerWrapper.

The build warns you if this is missing

If ccache is installed but base_dir / hash_dir aren't set as above, a resolve-time probe (integration/qbs/imports/CcacheProbe.qbs) prints a warning naming the exact fix commands and linking back to this page. It is warn-only — the build never modifies your global ccache config. The warning fires at resolve time (the same phase where ccache is autodetected), so a fresh checkout surfaces it on the first qbs build. That is deliberate: config that lives outside git is easy to forget on a new machine, and both humans and agents can act on a build warning.

Verifying ccache is actually live

There is one gotcha worth internalizing: cpp.compilerWrapper is evaluated at resolve time. A build graph that was resolved on a machine/PATH without ccache keeps building without ccache until you re-resolve — even after you install ccache. So "is ccache on?" is really "was it present when this tree was last resolved?"

Force a re-resolve, then verify two ways:

# 1. Re-resolve so the compiler wrapper is re-evaluated.
qbs resolve

# 2. Confirm ccache is in the actual compile command.
touch <some-source-file>.cpp
qbs build --command-echo-mode command-line -p <product> 2>&1 | grep -m1 'ccache'
# Expect exactly ONE ccache:  nice -n 19 ccache /usr/bin/...-g++ ...
# Two ('ccache ccache') means the profile compilerWrapper is still set — see above.

# 3. Confirm the cache counter moves on a rebuild.
ccache -s | grep -i 'cacheable calls'   # note the number
# ...rebuild something...
ccache -s | grep -i 'cacheable calls'   # it should have gone up

Known limitation: per-commit cache invalidation

ccache hashes the full compiler command line. Build-metadata -D defines that change on every commit therefore change the hash of every object that carries them, invalidating those cache entries.

OctoMY™ splits these defines by volatility (integration/qbs/BuildInfo.qbs):

• gitStableDefines — OC_GIT_BRANCH, OC_GIT_NON_MAIN_BRANCH, OC_BUILD_VARIANT, OC_BUILD_DEBUG. Applied globally to every product. These change only on a branch or build-variant switch, so busting the cache then is acceptable.
• gitVolatileDefines — OC_GIT_COMMIT, OC_GIT_COMMIT_SHORT, OC_GIT_COMMIT_DATE, OC_GIT_DIRTY, OC_PACKAGE_VERSION. Change on (nearly) every commit, and the version on every nightly. Applied only to the two individual source files that read them — libutil's utility/BuildInfo.cpp and libhelp's help/HelpSearch.cpp — via a dedicated Group carrying cpp.defines: outer.concat(BuildInfo.gitVolatileDefines).

Net effect: a new commit recompiles 2 objects. Measured on the real compile commands — of libhelp's full 5722-object dependency tree, exactly two carry OC_GIT_COMMIT_SHORT: BuildInfo.cpp and HelpSearch.cpp.

This was previously attached at product level, which meant all 36 libutil objects plus 23 in libhelp were re-hashed on every commit to serve four strings that one translation unit each reads. Narrowing it to file scope is also what made adding OC_PACKAGE_VERSION free: the marginal cost of a fifth volatile define is one more string on a command line that was going to be re-hashed anyway.

Adding a file that needs build identity

Two rules, and the second is easy to miss:

1. Read it through BuildInfo (an accessor in libutil) — do not reintroduce a global -DOC_GIT_COMMIT, or you resurrect the churn.
2. If a file genuinely must carry the defines itself, add it to both the Group's files: list and the product's buildIdentitySources: property. The second removes it from the default source enumeration; qbs rejects a file listed by two Groups with Duplicate source file.

Keep the files: list explicit — a wildcard that later matched a second file would silently re-widen the blast radius. Note the failure mode if the Group stops matching: the file still compiles, against the #ifndef placeholder values in BuildInfo.cpp, so nothing in the build breaks. The guard is testBuildInfo::testDefinesArePresentInQbsBuild, which asserts the values are not the placeholders.

A branch switch still changes OC_GIT_BRANCH globally and busts the cache once; that's an accepted, infrequent cost within one checkout. Across parallel checkouts it is not a one-off — see the next section.

Known limitation: the cache is partitioned by branch

base_dir and hash_dir=false remove the path difference between checkouts, so the same file under .../paint and .../comms hashes identically. That half works, and it is what § Machine-local setup buys you.

The other half does not follow from it. OC_GIT_BRANCH is a gitStableDefine, applied globally to every product, and ccache hashes the full compiler command line. So every object compiled in the paint checkout carries -DOC_GIT_BRANCH="paint" and every object in comms carries -DOC_GIT_BRANCH="comms", and the two never collide. Checkouts on different branches share no cache entries at all, for any object. The cache holds one bucket per branch.

What cross-checkout sharing therefore actually delivers:

• A checkout whose default/ tree is deleted and rebuilt — near-total hits.
• A checkout re-created at the same branch, or a tree copied between machines.
• Two checkouts that genuinely sit on the same branch.

What it does not deliver: eight parallel branches sharing compile work. That is an accepted trade rather than a defect — OC_GIT_BRANCH is how a running node reports its provenance, and narrowing it to file scope the way gitVolatileDefines are narrowed would cost every product that reads the branch through BuildInfo. But it is worth knowing before sizing max_size or explaining a hit rate that looks disappointing across a busy build site.

The practical consequence for max_size: budget by branches in flight, not by checkouts. Measured numbers and any future challenge to this trade belong in content/doc/internal/specs/build-performance/.

Maintenance & disk hygiene

ccache -s -v        # stats: hit rate, size, evictions ("cleanups")
ccache -p           # effective config (shows source of each value)
du -sh ~/.cache/ccache

The cache itself is small (~10 GB). The real disk consumers on a busy build machine are the build trees, especially .build/default/ — which is never a legitimate tree (it's drift from a session that overrode the build dir) and is always safe to delete:

# Reclaim illegitimate .build/default drift across all checkouts.
du -sh /path/to/checkouts/*/.build/default 2>/dev/null
# rm -rf each one you've confirmed — the canonical build lives in <checkout>/default/.

Why not hard links?

ccache's hard_link would share inodes between the cache and build outputs to save space, but on the default ext4 root filesystem it's a poor trade: ccache does not hard-link compressed entries (compression is on by default), so you'd have to disable compression (enlarging the cache), and hard links risk cache corruption if any tool rewrites an object in place. It only dedupes the .o slice anyway, not the real hogs (static libs, test binaries). Leave hard_link off. The copy-on-write alternative, file_clone, is not available on ext4 — see the optional chapter below.

Dedicated build disk with XFS

A build site can keep its build trees and this cache on one dedicated XFS disk, mounted at /build. The reference machine does (cache_dir = /build/ccache). Replace or add the build disk is the whole procedure: sector size, partition, mkfs.xfs, mount options, the swap file, and the folders that the lanes link to.

Two settings belong to this page:

• Reflink copies. XFS supports reflink, so ccache can copy a cached object into a build tree without copying its data. Enable it with ccache --set-config file_clone=true. It is safe on a copy-on-write filesystem, and it works only if the cache and the build trees are on the same filesystem.
• base_dir does not move. It points at the directory that contains the lanes' worktrees, not at the build disk. The worktrees stay on the system disk, so a new build disk does not change it.

Why XFS: many compilers write many small files at the same time, and XFS spreads that work over its allocation groups. XFS has no transparent compression. If disk space matters more than speed, btrfs with compress=zstd also supports reflink, at some cost in build speed.