Replace or add the build disk

A build site keeps its build trees, its compiler cache and some large service data on one dedicated disk, mounted at /build. This page tells you how to prepare a new disk for that role: from a disk that is new out of the box to a mounted, tuned XFS filesystem that every lane can use again.

The procedure was written down when the reference machine replaced its 1 TB /build disk with an 8 TB disk (2026-10-03). The values in the examples are from that machine. Replace them with your own.

TL;DR

1. Find out what used the old disk: fstab, the swap file, the ccache config, and every link that points into /build.
2. Identify the new disk by its model and serial number.
3. Set 4 KiB sectors, make one GPT partition, and make XFS with reflink.
4. Mount with noatime,logbsize=256k,nofail, by label.
5. Make the swap file again, and make every folder that a link points to.
6. Enable reflink copies in ccache and the weekly TRIM.
7. Tell the other lanes: their build trees and service data are gone.

What the disk holds

On the reference machine, these items lived on /build. A new disk starts empty, so each one must be made again, or restored from a copy.

Each lane's default and release in its worktree is a symbolic link into /build. Some lanes keep their build tree on / instead (under ~/build/). Those lanes do not use this disk, and a disk swap does not touch them.

Before you start

• Get root access. Every step that writes to the disk or to /etc needs sudo.
• Install the tools: xfsprogs (6.13 or later), gdisk, parted and nvme-cli.
• Stop every build that writes to /build. Do not continue while a qbs process uses a build tree on the old disk. A killed qbs corrupts its build graph.

1. Record how the old disk was used

Do this while the old disk is still in the machine, if you can. If it is gone, the journal of the previous boot still has most of the facts.

cat /etc/fstab
cat ~/.config/ccache/ccache.conf
cat /proc/swaps

# Every link into /build: these targets must exist on the new disk.
find /home/leo -maxdepth 7 -type l -lname "/build*" -printf "%p -> %l\n"

# The old filesystem and swap file, from the previous boot.
sudo journalctl -k -b -1 | grep -iE "xfs|swap"
sudo journalctl -b -1 | grep -iE "build.mount|build/swapfile"

# Services that name /build.
grep -rl "/build" /etc/systemd /etc/docker ~/.config/systemd/user 2>/dev/null
crontab -l | grep /build

On the reference machine, the journal showed one GPT partition (nvme1n1p1) with an XFS V5 filesystem and a 50 GiB swap file.

2. Identify the new disk

lsblk -o NAME,SIZE,TYPE,FSTYPE,LABEL,MOUNTPOINTS,MODEL,SERIAL,TRAN

The new disk has no partitions and no filesystem. Write down its device name, model and serial number. The reference machine showed:

nvme1n1  7,3T disk   PNY CS3140 8TB SSD  PNY24342408220100002  nvme
nvme0n1  1,8T disk   KINGSTON SFYRD2000G ...                   nvme   <- system disk

Do not continue until you can name the new disk without doubt. The steps below destroy all data on the device that you give them.

Make sure that nothing on the new disk is mounted:

grep nvme1n1 /proc/mounts   # must print nothing

3. Set 4 KiB sectors

Many NVMe disks ship with 512-byte sectors and also offer 4 KiB sectors. 4 KiB sectors align with the filesystem block size. Read the formats that the disk offers:

sudo nvme id-ns -H /dev/nvme1n1 | grep "LBA Format"

The reference disk showed:

LBA Format  0 : Metadata Size: 0 bytes - Data Size: 512 bytes  - Relative Performance: 0x2 Good (in use)
LBA Format  1 : Metadata Size: 0 bytes - Data Size: 4096 bytes - Relative Performance: 0x1 Better

If a 4096-byte format exists, select it. This command erases the whole disk. It checks the serial number first, so that it cannot erase a different disk:

sn=$(sudo nvme id-ctrl /dev/nvme1 | awk '/^sn /{print $3}')
[ "$sn" = "PNY24342408220100002" ] && ! grep -q nvme1n1 /proc/mounts \
  && sudo nvme format /dev/nvme1n1 --lbaf=1 --force
cat /sys/block/nvme1n1/queue/logical_block_size   # must print 4096

If the disk offers only 512-byte sectors, skip this step and remove -s size=4096 from the mkfs.xfs command in step 5.

4. Make one GPT partition

The old disk had one partition, and the new disk has the same layout. A partition table tells other tools that the disk is in use.

sudo sgdisk --zap-all /dev/nvme1n1
sudo sgdisk -n 1:0:0 -t 1:8300 -c 1:Y8_BUILD /dev/nvme1n1
sudo partprobe /dev/nvme1n1
sudo sgdisk -p /dev/nvme1n1

Check that the partition starts at 1 MiB: sector 256 with 4 KiB sectors, or sector 2048 with 512-byte sectors.

5. Make the filesystem

XFS suits this workload. Many compilers write many small files at the same time, and XFS spreads that work over its allocation groups. XFS also supports reflink, which lets ccache copy a cached object without copying its data.

Look at the default geometry first. The -N option writes nothing:

sudo mkfs.xfs -N -f -m reflink=1,crc=1 -b size=4096 /dev/nvme1n1p1

mkfs.xfs 6.13 already sets one allocation group per CPU core on an SSD (64 on the reference machine), and it gives an 8 TB filesystem the largest internal log (2038 MiB). Builds change a lot of metadata, so a large log helps. Then make the filesystem:

sudo mkfs.xfs -f -L Y8_BUILD -m reflink=1,crc=1 \
  -b size=4096 -s size=4096 -d agcount=64 /dev/nvme1n1p1
sudo blkid /dev/nvme1n1p1

Set agcount to your core count (nproc). The output must show reflink=1, sectsz=4096 and the agcount that you gave.

6. Mount it

Back up fstab, then replace the old /build line. The old line named the old disk by its UUID. Use the label, as for the other disks of the machine:

sudo cp -a /etc/fstab /etc/fstab.bak-$(date +%F)
sudoedit /etc/fstab

The line:

LABEL=Y8_BUILD  /build  xfs  noatime,logbsize=256k,nofail  0  2

• noatime: no write for each file read. ccache reads much more than it writes.
• logbsize=256k: larger log buffers for a metadata-heavy load.
• nofail: the machine still boots if the disk is missing.

Keep the swap file line (/build/swapfile ... nofail). Then mount through systemd, as at boot:

sudo findmnt --verify --tab-file /etc/fstab
sudo systemctl daemon-reload
sudo systemctl start build.mount
findmnt /build

findmnt --verify warns that /build/swapfile does not exist until you make it in the next step. That warning is correct at this point.

An empty filesystem shows about 2 % "used" (143 GiB of 7.3 TiB on the reference disk). That is the space XFS reserves for its reflink and reverse-map metadata. It is not data.

7. Swap file

The reference machine has two other swap areas (a partition and /swapfile on the system disk). The swap file on /build was 50 GiB, and is 100 GiB on the new disk.

sudo fallocate -l 100G /build/swapfile
sudo chmod 600 /build/swapfile
sudo mkswap /build/swapfile
sudo systemctl start build-swapfile.swap
cat /proc/swaps

Use fallocate, not a reflink copy of another file. A swap file must not share its blocks with another file.

8. Make the folders again

Make every folder that a link points to, and give them to the build user. Without them, every lane's default link points to nothing, and qbs cannot make its build directory there.

dirs=$(find /home/leo -maxdepth 7 -type l -lname "/build*" | xargs -n1 readlink | sort -u)
sudo mkdir -p $dirs /build/qt-build-farm /build/ccache
sudo chown -R leo:leo /build/octomy-builds /build/qt-build-farm /build/ccache /build/monolith-docker-data
sudo chown leo:leo /build

# Every link must resolve. This prints nothing when all of them do.
find /home/leo -maxdepth 7 -type l -lname "/build*" ! -exec test -e {} \; -print

The find with -maxdepth 7 does not reach the Qt build farm link (integration/qt-build-farm/storage in the base worktree), which is why /build/qt-build-farm is in the list by name.

9. ccache and TRIM

ccache --set-config file_clone=true
ccache --set-config max_size=500G
ccache -p | grep -E "cache_dir|file_clone|max_size"

sudo systemctl enable --now fstrim.timer

The disk has room, so the cache gets a large limit. One full build puts about 10 GB into the cache for each profile, and each branch adds its own entries.

file_clone makes ccache copy a cached object with reflink. It is safe on a copy-on-write filesystem, and it works only if the cache and the build trees are on the same filesystem. They are both on /build. Check that reflink works:

head -c 64M /dev/urandom > /build/.reflink-test-a
cp --reflink=always /build/.reflink-test-a /build/.reflink-test-b && echo "reflink OK"
rm -f /build/.reflink-test-a /build/.reflink-test-b

Use the weekly fstrim.timer, not the discard mount option. The timer trims in one batch and does not slow each delete.

10. Tell the other lanes

Every lane with a build tree on /build loses it, and the service data is gone. Write this in the cross-session handoff document (content/doc/internal/handoff/cross-session-state.md), and say what each owner must do:

• Each octomy lane: resolve again, then build. The first build is cold.
• The monolith session: restore the Docker data and the Windows build VM before the next farm or nightly run. Without them, the run fails, and the failure looks like a container or Windows fault.

Verify the disk

Related

• Set up a build site: the layout that this disk serves.
• ccache Setup: the cache configuration.