Test harnesses

The catalog of shared test harnesses: the fakes, fixtures and helpers that a test suite uses instead of writing its own copy. Each section says what one harness gives, the line that includes it, a use from a real suite, when not to use it, and the suite that tests it.

What a shared harness is

A shared harness is test code that more than one suite needs: a fake of an interface, a fixture that builds a chain of real objects, or a helper such as a bounded wait or an image comparison. It lives in one place, so that a defect in it is fixed once, for every suite.

The rules:

• Never write a private copy. Where a harness in this catalog does what you need, use it. Two copies of a fake drift apart, and the next defect gets fixed in one of them only. When the harness lacks something, add it to the harness.
• Add to the catalog. When a second suite needs a helper of yours, move it to a shared place in the same commit as the second use, and add a section to this page.
• Put it beside the library it knows. A harness that knows the types of one library goes into that library's testing/ folder: src/libs/lib<x>/<x>/.../testing/, in the namespace <x>::testing. Only a helper that knows no library goes into libtest, src/libs/libtest/test/.
• A library's testing/ header never includes libtest. libtest does not depend on the libraries, and a library does not depend on libtest. A harness in libconstruct that must wait, for example, uses QTest::qWaitFor and not test::pumpUntil.
• Each harness has a suite of its own, which tests the harness. The sections below name it.
• A testing/ header is test-only by convention. A library exports its include path to every consumer, so production code can include the header and it compiles. The testing/ folder and the testing namespace are the guard. Do not call a harness from shipping code.
• Keep the shared binary in mind. A harness used by a member of the shared test binary must leave no process-wide state behind for the next class: no registry entry, no installed handler, no global counter. Where a harness owns such state, its section says how it gives it back. See Join the shared binary.

The include path is relative to the library folder. A header at src/libs/libconstruct/construct/eval/testing/AnalyticFields.hpp is included as #include "construct/eval/testing/AnalyticFields.hpp", and one at src/libs/libtest/test/CommsParty.hpp as #include "test/CommsParty.hpp". The suite's test.qbs depends on the library that holds the harness.

Contents

libtest

The general helpers, in src/libs/libtest/test/. Every suite built from the Test template links libtest.

Common

The suite entry point: OC_TEST_MAIN and OC_TEST_MAIN_GUILESS make the main of a suite program, or the entry of a shared-binary member, and set up the application, the arguments and the teardown.

• Header: src/libs/libtest/test/Common.hpp
• Include: #include "test/Common.hpp"
• Tested by: every suite, since no suite starts without it.
• Do not use it in a suite outside the Test template that keeps to Qt Core and Qt Test. Such a suite cannot link libtest; see HarnessArguments.

From testCommsParty:

OC_TEST_MAIN(test, TestCommsParty)

TestArgs

Parses the harness options (--headless, --display, --keep-basedir and the others) before QTest sees them, chooses the Qt platform, and answers isHeadless() and projectRoot(). It also selects Qt's PulseAudio backend on Linux, because Qt 6.10.1's PipeWire backend crashes about one first audio device enumeration in 20. --param audio-backend=pipewire selects PipeWire again.

A suite that must see its process before the application object exists registers a before-app probe with TestArgs::registerBeforeAppProbe() and starts its own program with TestArgs::beforeAppProbeArguments(). That start runs the probe at the start of main(), also in the shared binary, and ends with the probe's result. Do not use an environment variable for this: the tree reads none (testNoEnvironmentVariables).

• Header: src/libs/libtest/test/TestArgs.hpp
• Include: #include "test/TestArgs.hpp"
• Tested by: testCwdIsPinned, for the project root and the working folder, and testTestArgs, for the options and the before-app probe.
• Do not call TestArgs::parse() yourself in a suite that uses OC_TEST_MAIN: the macro already calls it. A new option that the runner gives every suite goes into HarnessArguments in the same change. --before-app-probe is not one: only a suite's own second start gets it.

From testPathResolver:

const bool gResolveBeforeAppRegistered = test::utility::TestArgs::registerBeforeAppProbe(
    kResolveBeforeApp, []() -> int {
        const QString root = AppContext::resolveBaseDir(QString());
        std::fputs(root.toLocal8Bit().constData(), stdout);
        std::fputs("\n", stdout);
        return EXIT_SUCCESS;
    });

From testCwdIsPinned:

const QString declared = test::utility::TestArgs::projectRoot();
QVERIFY2(!declared.isEmpty(),
    "libtest was built without OC_TEST_PROJECT_ROOT, so nothing pins the "

HarnessArguments

Removes the harness options from argv, for a suite that is not built from the Test template and so has no TestArgs. It needs Qt Core only.

• Header: src/libs/libtest/test/HarnessArguments.hpp
• Include: #include "test/HarnessArguments.hpp"
• Tested by: testHarnessArguments
• Do not use it in a suite built from the Test template: OC_TEST_MAIN and TestArgs already remove the options.

From testLibserial:

argc = test::harness::stripHarnessArguments(argc, argv);
QCoreApplication app(argc, argv);

SharedMain

The one main of a shared test binary: --list, --run, and the reset of the working folder, the data folder and the application name before each member.

• Header: src/libs/libtest/test/SharedMain.hpp
• Include: #include "test/SharedMain.hpp"
• Tested by: testPathResolver, for enterClass().
• Do not include it in a suite. The build generates the registry that calls it; a suite joins with shared: true in its test.qbs.

From testPathResolver:

const QString saved = QCoreApplication::applicationName();
test::shared::enterClass("testSharedFolderA");
const QString a = PathResolver::dataDir();

Utility

The general helpers of test::utility:

• Header: src/libs/libtest/test/Utility.hpp
• Include: #include "test/Utility.hpp"
• Tested by: testFixturePath, testWriteFile (also appendFile()), testRunProgram, testInstalledPrograms, and testTest (the headless runInteractive() and waits, tempFilePath() and builtProgram()).
• Do not read data through QFINDTESTDATA or __FILE__; use fixturePath(). Wrap writeFile() and appendFile() in QVERIFY: a check inside a helper leaves only the helper. A suite whose only content is runInteractive() asserts nothing without a person, so it belongs in the interactive category.

From testLayeringCheck and testConstructCli:

return test::utility::runProgram(tool, args, {root, 30000, true});

mConstructBin = test::utility::builtProgram(QStringLiteral("construct"));

Pump

test::pumpUntil(step, done, deadlineMs): runs a step and the event loop until a condition holds or a wall-clock deadline passes. test::activateAndSettle(store) activates a store and pumps the loop until its activation callback runs, so that no callback stays pending for a later case.

• Header: src/libs/libtest/test/Pump.hpp
• Include: #include "test/Pump.hpp"
• Tested by: testPump, for pumpUntil() and activateAndSettle(). testSimpleDataStoreLifetime uses activateAndSettle().
• Use it when a timer produces what you wait for. When the assertion counts iterations, bound the loop by the count and keep a clock only as a hang guard. A library's testing/ header cannot use it.

From testKilnBroker:

test::pumpUntil({}, [&]() {
    snapshot = broker.statsSnapshot();
    return snapshot.value(QStringLiteral("now")).toObject().value(QStringLiteral("linksDone")).toInteger() >= 1;
}, 5000);

LoopGapMeter

test::LoopGapMeter: the largest gap between iterations of a thread's event loop, in wall time and less the time the thread waited for a CPU.

• Header: src/libs/libtest/test/LoopGapMeter.hpp
• Include: #include "test/LoopGapMeter.hpp"
• Tested by: testLoopGapMeter.
• Use it for a GUI freeze bound. Assert on maxOwnGapMs(): a busy or blocked GUI thread counts in full, and the wait for a CPU on a loaded machine does not. Put describe() in the failure message. Make the meter and call tick() on the thread whose loop you measure. The wait is known on Linux. Elsewhere the own gap is the wall gap.
• sampleStacksAfter(ms) starts a watchdog thread that captures the measured thread's stack when a gap runs past ms. Print stackSamples() when the bound fails, so the failure names what the thread did.

From testPaintWorkerCanvas:

test::LoopGapMeter gaps;
gaps.sampleStacksAfter(100);
while (gaps.elapsedMs() < ms) {
    QCoreApplication::processEvents(QEventLoop::AllEvents, 20);
    gaps.tick();
}
return reportGaps("pump:", gaps);

ScaleTiming

test::scale: times each gesture of a test at a smaller and a larger size, and names the gestures whose cost grows faster than allowed. sizesFromArgs() reads the two sizes from --param sizes=<smaller>,<larger>, or takes the defaults. measureMs() and bestOfMs() time the work. grown() names a cost that must not depend on the size, and grownPerRow() a cost per row that must not grow: a linear cost passes, and a quadratic one fails.

• Header: src/libs/libtest/test/ScaleTiming.hpp
• Include: #include "test/ScaleTiming.hpp"
• Tested by: testScaleTiming. testConfigScale and seven other suites use it.
• It compares two times measured side by side in one run. Do not use it for a speed budget against a fixed number. Use relativeTime. A time below floorMs counts as floorMs, so that noise in a fast gesture does not fail the case.

From testConfigScale:

const scale::Sizes sizes = scale::sizesFromArgs(100000, 1000000);
// ...
const QStringList grownRows = scale::grownPerRow(small.perRow, sizes, FACTOR);
QVERIFY2(grownRows.isEmpty(), qPrintable(grownRows.join(QStringLiteral(", "))));

SessionProbe

test::utility::canExposeWindows(): whether a shown window becomes exposed in this session, probed once with a real widget.

• Header: src/libs/libtest/test/SessionProbe.hpp
• Include: #include "test/SessionProbe.hpp"
• Tested by: testSessionProbe.
• Do not skip on the platform name, such as offscreen, instead. Skip with QSKIP on false; do not assert on it.

From testViewportSwap:

if (!test::utility::canExposeWindows()) {
    QSKIP("The viewport swap is a layout-and-show defect class, and this "
        "session cannot expose a window, so it runs neither. See the "
        "class docs.");

DeviceRemoval

test::removeD3D12Device(): removes the Direct3D 12 device of a QRhi, or of a native ID3D12Device, on purpose, as a GPU timeout does. The answer says whether the device was removed, and why not: another backend, a build without the Direct3D 12 headers, or a Windows build older than 20348.

• Header: src/libs/libtest/test/DeviceRemoval.hpp
• Include: #include "test/DeviceRemoval.hpp"
• Tested by: testDeviceRemoval. testGpuLossProofs uses it.
• Every object on that device is lost for the rest of the process. Do not use it in a member of the shared binary. Skip with why when removed is false.

From testGpuLossProofs:

const test::DeviceRemoval removal = test::removeD3D12Device(first.get());
if (!removal.removed) {
    QSKIP(qPrintable(removal.why));
}

TemporaryTree

A temporary folder that a test fills with files by relative path, removed with the object.

• Header: src/libs/libtest/test/TemporaryTree.hpp
• Include: #include "test/TemporaryTree.hpp"
• Tested by: testTemporaryTree
• Do not use it for one file; QTemporaryDir and writeFile() do that.

From testKilnGraph (TestKilnImport.cpp):

test::TemporaryTree checkout;
QVERIFY(checkout.dir.isValid());
QVERIFY(checkout.write(QStringLiteral(".kiln"), "variants = debug release\n[path=src/libs/lib*]  type = lib\n"));

Git (libtest)

Runs the git command to build a fixture repository, with a test identity, main as the first branch and no signing.

• Header: src/libs/libtest/test/Git.hpp
• Include: #include "test/Git.hpp"
• Tested by: testGitFixture
• Do not use it for a test of trove's git work; use Git (libworktree), which runs git as trove does.

From testCompression:

if (!test::utility::gitSucceeds(dir.path(), {QStringLiteral("init"), QStringLiteral("-q")})
    || !test::utility::gitSucceeds(dir.path(), {QStringLiteral("add"), QStringLiteral(".")})

ImageCompare

How two images of the same size differ: the fraction of pixels that moved by more than a tolerance, the mean and largest channel difference, and several counts and probes.

• Header: src/libs/libtest/test/ImageCompare.hpp
• Include: #include "test/ImageCompare.hpp"
• Tested by: testImageCompare
• Two images that cannot be compared (different sizes, or a null image) give an empty answer. Check the answer before you read a number; do not replace it with a sentinel such as 1.0 or -1.

From testGizmoScenes:

const auto iconsVsArcCompared = test::image::compare(
    images.value(QStringLiteral("construct-launcher-icons")),
    images.value(QStringLiteral("construct-graded-arc")), 4);
QVERIFY2(iconsVsArcCompared, "the two captures differ in size");

ReferenceImage

test::image::compareWithReference(): compares an image with a PNG reference in the suite's fixtures, pixel for pixel, with no tolerance. When no reference exists, it writes one into test/<suite>/ and answers Created, and a person commits it. When the image differs, it saves the image in the temporary folder for a look. useReferenceFont() maps the family monospace to White Rabbit, a font of the tree, so that a reference with text holds on every machine.

• Header: src/libs/libtest/test/ReferenceImage.hpp
• Include: #include "test/ReferenceImage.hpp"
• Tested by: testReferenceImage. testHeatmapWidget and testStackedAreaChart use it.
• Call releaseReferenceFont() when the case is done: the font substitution is process-wide state, and the shared binary runs other classes after yours. Assert the outcome Same, so that the run that writes a new reference fails until a person commits it. For a comparison with a tolerance, use ImageCompare.

From testHeatmapWidget:

const QFont font = test::image::useReferenceFont();
QCOMPARE(QFontInfo(font).family(), QStringLiteral("White Rabbit"));
std::unique_ptr<HeatmapWidget> w(referenceHeatmap(kind, font, 0));
const auto result = test::image::compareWithReference(renderOnBlack(w.get()),
    QStringLiteral("references/%1.png").arg(kind));
test::image::releaseReferenceFont();
QVERIFY2(result.outcome == test::image::ReferenceResult::Outcome::Same, qPrintable(result.message));

MessageCapture

Records the Qt messages that a scope logs, so that a test asserts a diagnostic. It is the message handler while it lives, and restores the previous one.

• Header: src/libs/libtest/test/MessageCapture.hpp
• Include: #include "test/MessageCapture.hpp"
• Tested by: testMessageCapture
• Do not install a message handler of your own. A capture must end before the capture it is nested in, as a scope does; do not keep one in a member that outlives the case.

From testCrtSafety:

const test::MessageCapture capture;

SourceScan

Walks the project's own source files, without the vendored trees, and reads a file with its comments blanked, for a guard suite that checks the tree's code.

• Header: src/libs/libtest/test/SourceScan.hpp
• Include: #include "test/SourceScan.hpp"
• Tested by: testSourceScan
• An empty walk and a clean one print the same line: assert that the walk found files and print the count. Do not use it to read a fixture; use fixturePath().

From testPathCompliance:

const test::source::SourceTree tree = test::source::sourceFiles();
QStringList out;
if (!tree.error.isEmpty() || tree.files.isEmpty()) {

HelpPages

test::utility::helpPageProblem(): why the help value of a setting does not open a published page and heading, or an empty string when it does. The value is a path under content/doc/published (publishedDocRoot()) that ends in .md, with an optional #anchor. helpAnchorsOf() gives the heading anchors of a page, made as the help browser makes them.

• Header: src/libs/libtest/test/HelpPages.hpp
• Include: #include "test/HelpPages.hpp"
• Tested by: testHelpPages. testNodeSettings and seven other suites use it.
• Count the rows you checked and assert the count: a schema with no rows passes the loop.

From testNodeSettings:

for (const config::SettingDef &d : nodeSchema().defs()) {
    const QString problem = test::utility::helpPageProblem(d.help);
    QVERIFY2(problem.isEmpty(), qPrintable(d.id + QStringLiteral(": ") + problem));
    ++checked;
}

FreePort

test::freeUdpPort() and test::freeTcpPort(): a loopback port that the OS reported free a moment ago, or 0.

• Header: src/libs/libtest/test/FreePort.hpp
• Include: #include "test/FreePort.hpp"
• Tested by: testCommsParty
• The port is free when the probe closes, not after. A suite that needs two ports asks for both before it binds either. Do not use a fixed port range.

From testCommsCarrierRebind:

const quint16 first = test::freeUdpPort();
const quint16 second = test::freeUdpPort();
QVERIFY(first != 0 && second != 0 && first != second);

CommsParty

One party of a comms test: its own key store and address book in a temporary folder of the party's own, a UDP address on the loopback, and a Comms over them, with the Associate that another party keeps of it. Port 0 asks for a free port. Two runs in parallel share no file and no port.

• Header: src/libs/libtest/test/CommsParty.hpp
• Include: #include "test/CommsParty.hpp"
• Tested by: testCommsParty
• Do not drop a party without teardown() or its destructor: an activated key store holds a thread of the global pool until it is deactivated.

From testCourierDataExchange:

test::CommsParty peerA, peerB;
QVERIFY(peerA.setup("courier_exchange_A", 0, policy, {encryptionEnabled}));
QVERIFY(peerB.setup("courier_exchange_B", 0, policy, {encryptionEnabled}));
peerA.know(peerB, TrustLevel::TRUST, ROLE_CONTROL, TYPE_REMOTE);

CommsTester

One party of a UDP send test: a CommsChannel over a CommsCarrierUDP on one port, with a MockCourier for every other port of a range. startSendTest() binds the carrier, and finished() comes when a courier spends its budget of sends. The caller gives it the key store and the address book.

• Header: src/libs/libtest/test/CommsTester.hpp
• Include: #include "test/CommsTester.hpp"
• Tested by: testCommsTesters, which is also its only user.
• For a new two-node comms test, use CommsParty.

From testCommsTesters:

return std::make_unique<CommsTester>(QStringLiteral("party"), kLoopback, myPort, quint16(myPort - 1), range,
    testCount, QSharedPointer<KeyStore>::create(), QSharedPointer<AddressBook>::create());

CommsSignalLogger

Logs the error, client-added and connection-status signals of a comms object.

• Header: src/libs/libtest/test/CommsSignalLogger.hpp
• Include: #include "test/CommsSignalLogger.hpp"
• Tested by: testCommsTesters.
• It logs and asserts nothing. To assert a signal, use QSignalSpy.

Used by testCommsChannel, testCourierDataExchange, testDiscoveryToComms and testTrustGatedComms:

#include "test/CommsSignalLogger.hpp"

CourierTester

Tests a pair of couriers without networking, and simulates bad weather on the wire between them.

• Header: src/libs/libtest/test/CourierTester.hpp
• Include: #include "test/CourierTester.hpp"
• Tested by: testCourierTester.
• Of the network faults it lists, only packet reordering is implemented. For loss, duplication or tampering of BlobCourier frames, use BlobCourierPump.

From testAgentStateCourier:

class AgentStateCourierTester : public CourierTester
{

Corruptifier

Corrupts packets on purpose: bit and byte damage, reordering, duplication, drops and splits, each with a probability, from a seeded random generator.

• Header: src/libs/libtest/test/Corruptifier.hpp
• Include: #include "test/Corruptifier.hpp"
• Tested by: testCorruptifier
• Give it a seeded RNG so that a failure repeats; the constructor without one is the old form.

From testCorruptifier:

Corruptifier c;
QCOMPARE(c.get(None), 1.0);
QCOMPARE(c.get(ScrambleBit), 0.0);

EncryptionTestConfig

The data columns and rows that run a comms case with encryption on and off, for RSA and Ed25519 keys.

• Header: src/libs/libtest/test/EncryptionTestConfig.hpp
• Include: #include "test/EncryptionTestConfig.hpp"
• Tested by: testLibtestKeys.
• Use it in a per-function _data() function. initTestCase_data() with QFETCH_GLOBAL does not iterate the rows.

From testTrustGatedComms:

void TestTrustGatedComms::testBlockedPeerCannotInitiateSession_data() { EncryptionTestConfig::addTestData(); }

Keys (libtest)

TestKeys: a fixed private key, public key and key id, and a key id that is wrong on purpose.

• Header: src/libs/libtest/test/Keys.hpp
• Include: #include "test/Keys.hpp"
• Tested by: testLibtestKeys.
• Do not use it where a case needs a key that it generates itself, such as a key-generation test.

From testKeyStore:

const TestKeys tk;
QVariantMap keyMap;
keyMap["publicKey"]=tk.pubKeyPEM;

SyncTestHelper

A test clock and a TreePair, a local and a remote SyncTree that sync deltas in either direction and assert that they converged. generateRandomOps() applies random sets and removes, drawn from a QRandomGenerator that the caller seeds and prints the seed of.

• Header: src/libs/libtest/test/SyncTestHelper.hpp
• Include: #include "test/SyncTestHelper.hpp"
• Tested by: testSyncTestHelper.
• The clock is one variable for the whole process, which every class of the shared binary sees. Call test::sync::reset() at the start of each case, so that no case starts at another case's time.

From testConfigSyncConvergence:

TreePair pair("test", SyncOwnership::LAST_WRITER_WINS, SyncOwnership::LAST_WRITER_WINS);
advance(10);
pair.syncBoth();
pair.assertConverged();

UDPTester

Sends and receives raw UDP datagrams between two CarrierAddressUDP addresses.

• Header: src/libs/libtest/test/UDPTester.hpp
• Include: #include "test/UDPTester.hpp"
• Tested by: testCommsTesters.
• It opens a real UDP socket. For a comms test that needs no network, use MockCommsCarrier.

From testCommsCarrierRebind:

UDPTester dataSender(fromAddress, address);

MockAddressBook

An AddressBook that adds associates without persistence and sends its change signals when the test says.

• Header: src/libs/libtest/test/mock/MockAddressBook.hpp
• Include: #include "test/mock/MockAddressBook.hpp"
• Tested by: testLibtestMocks.
• Do not use it to test persistence; it can keep everything in memory.

From testAssociateListItemIntegration:

mAddressBook = QSharedPointer<MockAddressBook>::create();
mAddressBook->mockEmitTrustChanged(mAssociate->id());

MockAssociate

Makes Associate objects with test data: trusted, blocked, discovered by camera or by NFC, or with a given channel history.

• Header: src/libs/libtest/test/mock/MockAssociate.hpp
• Include: #include "test/mock/MockAssociate.hpp"
• Tested by: testLibtestMocks.
• For the stored map of an associate, as the serializer reads it, use AssociateMaps.

From testAssociateListItemIntegration:

mAssociate = MockAssociate::createWithChannels(
    "TestAgent",
    TYPE_AGENT,
    TrustLevel::TRUST,

MockCommsSession

A real CommsSession with QObject signals beside it, which the test sends, so that a widget can be tested against a session.

• Header: src/libs/libtest/test/mock/MockCommsSession.hpp
• Include: #include "test/mock/MockCommsSession.hpp"
• Tested by: testLibtestMocks.
• Do not use it to test the session protocol: the test sets the connected and established states with mockSetConnected() and mockSetEstablished().

From testAssociateListItemIntegration:

mSession = QSharedPointer<MockCommsSession>::create();
mSession->mockConfigure();
mSession->mockSetAssociateId(mAssociate->id());

MockCommsCarrier

A CommsCarrier with no socket: the test writes the data it reads and reads the data it writes, and controls the sending-opportunity timer.

• Header: src/libs/libtest/test/mock/MockCommsCarrier.hpp
• Include: #include "test/mock/MockCommsCarrier.hpp"
• Tested by: testLibtestMocks.
• Do not use it to test a real carrier's socket handling.

From testCommsChannel (testCommsMock.cpp):

auto carrierA = QSharedPointer<MockCommsCarrier>::create();
auto commsA = QSharedPointer<Comms>::create();
auto chanA = QSharedPointer<CommsChannel>::create();
carrierA->configure();

MockCommsCarrierLog

Logs the signals of a carrier: ready-read, error, sending opportunity and connection status.

• Header: src/libs/libtest/test/mock/MockCommsCarrierLog.hpp
• Include: #include "test/mock/MockCommsCarrierLog.hpp"
• Tested by: testLibtestMocks.
• It asserts nothing. To assert a signal, use QSignalSpy.

From testCommsCarrierRebind:

MockCommsCarrierLog mocalo;
commsCarrier.setHookCarrierSignals(mocalo, true);

MockCourier

A Courier that sends one datagram a number of times and counts what it sends and receives, within a budget.

• Header: src/libs/libtest/test/mock/MockCourier.hpp
• Include: #include "test/mock/MockCourier.hpp"
• Tested by: testLibtestMocks.
• The courier retires when it reaches its budget. A test that measures a fresh exchange calls reArm(), which resets the counters and the mandate; a reset of the counters alone measures a courier that stopped.

From testDiscoveryToComms:

QSharedPointer<MockCourier> courA(OC_NEW MockCourier(
    "CommsStepCourierA", peerB.id, "Data from Agent", peerA.comms, maxSends, maxRecs
));
chanA->registerCourier(courA, true);

libactivity

Fakes (libactivity)

The activity doubles: Pushable<T> makes the protected pushImpl of an activity reachable, RecordingActivity records what reaches it, FakeActivityContext holds facilities in a hash, and StubHeader records what the stack drives into a header.

• Header: src/libs/libactivity/activity/testing/Fakes.hpp
• Include: #include "activity/testing/Fakes.hpp"
• Tested by: testActivityTesting
• Do not use Pushable<T> to test navigation; push through a real ActivityStack.

From testActivityStack:

auto a = QSharedPointer<RecordingActivity>::create(nullptr, QStringLiteral("A"));
stack.registerActivity(a);
QVERIFY(stack.push("A", QStringList{"first"}));
QCOMPARE(a->reinvokeCount, 0);

libanon

IoRunner

An i2p IOService that runs on a thread of its own, and waitFuture(), a bounded wait for a std::future.

• Header: src/libs/libanon/anon/testing/IoRunner.hpp
• Include: #include "anon/testing/IoRunner.hpp"
• Tested by: testAnonTesting
• Do not wait on a future without a bound; waitFuture() gives one.

From testAnonQtTcp:

IoRunner io;
QtTcpAcceptor acceptor(io.svc);
QtTcpSocket server(io.svc), client(io.svc);

libapp

ShutdownTaskHarness

A synthetic ShutdownLifecycle task with a controllable force-cancel, which deregisters itself when it is destroyed.

• Header: src/libs/libapp/lifecycle/testing/ShutdownTaskHarness.hpp
• Include: #include "lifecycle/testing/ShutdownTaskHarness.hpp"
• Tested by: testShutdownTaskHarness
• Assert wasForceCanceled(); do not spy on the manager's internal state.

From testShutdownLifecycle:

ShutdownTaskHarness h("test.basic");
QVERIFY(h.token() != 0);
QCOMPARE(mgr.activeCount(), 1);

libbahn

FakeLaneSink

A LaneSink that counts, so that every lane rule runs without a sound card. The counting mode keeps the last block and the peaks; the null mode only fills a lane.

• Header: src/libs/libbahn/bahn/testing/FakeLaneSink.hpp
• Include: #include "bahn/testing/FakeLaneSink.hpp"
• Tested by: testFakeLaneSink
• writable is in samples, not frames. Do not use it to test a device output.

From testBahnOutputLanes:

auto sink = QSharedPointer<FakeLaneSink>::create();
sinks.append(sink);
return QSharedPointer<bahn::LaneSink>(sink);

FakeProcessorContext

A ProcessorContext over a buffer of its own, for a test that drives a processor without the engine.

• Header: src/libs/libbahn/bahn/testing/FakeProcessorContext.hpp
• Include: #include "bahn/testing/FakeProcessorContext.hpp"
• Tested by: testFakeProcessorContext
• The default sample rate 0 is a context with no device rate. Set a rate when the processor reads one.

From testDrumMachine:

FakeProcessorContext ctx(bufSize);
proc.process(ctx);
QVERIFY2(hasNonZeroSamples(ctx.buffer(), bufSize),

libbatch

WorkerDriver

Drives a worker's job the way the dispatcher does: driveJob(), runJob(), waitForTerminal() and submitAndWaitTerminal().

• Header: src/libs/libbatch/batch/testing/WorkerDriver.hpp
• Include: #include "batch/testing/WorkerDriver.hpp"
• Tested by: testBatchTesting
• Never call it from shipping code: it builds a stop flag that nothing can set.

From testMeshExportWorker:

batch::testing::driveJob(queue, worker, id);
QVERIFY(worker.canAccept());

Fakes (libbatch)

FixedProvider, a resource of a fixed capacity; jobRequesting(), a job that requests resources; and MockBatchWorker, a worker that completes what it is given.

• Header: src/libs/libbatch/batch/testing/Fakes.hpp
• Include: #include "batch/testing/Fakes.hpp"
• Tested by: testBatchTesting
• For real compute that a test can kill in the middle, use DigestWorker.

From testBatchAdmission:

mgr->registerProvider(QSharedPointer<IResourceProvider>(new FixedProvider(CPU, cpu)));
mgr->setMode(Locality::Local, NodeMode::Drone);
const auto result = mgr->admit(jobRequesting(1, {{CPU, 2000}}), Locality::Local);

FakeRemoteTransport

A remote batch transport whose faults a test injects (drop, refuse, defer) and whose calls it counts, in order.

• Header: src/libs/libbatch/batch/testing/FakeRemoteTransport.hpp
• Include: #include "batch/testing/FakeRemoteTransport.hpp"
• Tested by: testBatchTesting
• A case that asserts it survived a dropped call must also assert that the call was dropped, from the counters; otherwise it passes on a working network and proves nothing.

From testBatchRemoteWorker:

FakeRemoteTransport transport;
RemoteBatchWorker worker{&transport, BatchPolicy::local()};
BatchDispatcher dispatcher{&queue, &registry};

Advertisements

An executor advertisement for a test: a reachable Drone peer with a deterministic, ordered id.

• Header: src/libs/libbatch/batch/testing/Advertisements.hpp
• Include: #include "batch/testing/Advertisements.hpp"
• Tested by: testBatchTesting
• Change only the fields that the case is about on the returned value.

From testBatchExecutorView:

auto ad = executorAdvertisement(1, QStringLiteral("alpha"), {{CPU, 8000}});

DigestWorker

Real compute for a test: a SHA-256 over a generated stream, with expectedDigest(), which the test computes on its own side.

• Header: src/libs/libbatch/batch/testing/DigestWorker.hpp
• Include: #include "batch/testing/DigestWorker.hpp"
• Tested by: testOpalNetTesting, through TwoExecutorFleet.
• Do not use it where the result does not matter; MockBatchWorker is cheaper.

From testOpalNetNodeBatch:

DigestWorker worker(kDigestType);

MemoryArtifactStore

An IArtifactStore in memory that names each artifact by its SHA-256, 64 hex digits, the same shape as the BLAKE3 name of a node's content store, with no dependency on libswarm.

• Header: src/libs/libbatch/batch/testing/MemoryArtifactStore.hpp
• Include: #include "batch/testing/MemoryArtifactStore.hpp"
• Tested by: testBatchTesting. testBatchPortability and the worker suites run on it.
• pathOf() is always empty: no artifact has a file of its own. Do not use it where the code under test reads an artifact by its path.

From testFemSolveJob:

batch::testing::MemoryArtifactStore store;
femjob::FemSolveJobWorker worker;
registry.registerWorker(&worker);
batch::BatchDispatcher dispatcher(&queue, &registry);
dispatcher.setArtifactStore(&store, scratch.path());

PortableConformance

checkPortable(): runs one job of a worker that declares Portable twice, each time through a real BatchDispatcher with an artifact store and a scratch folder, as a node runs a pooled job. It checks the three properties of Portable:

1. No absolute path of this machine is in the job's data or result.
2. Every output is in the result by hash and in the store, and nothing stays in the job's folder.
3. The same inputs give the same outputs.

PortableProjection leaves out measured keys, such as a duration. It compares an output that carries a time or a random id by a projection that the suite gives.

• Header: src/libs/libbatch/batch/testing/PortableConformance.hpp
• Include: #include "batch/testing/PortableConformance.hpp"
• Tested by: testBatchPortability, against workers that break each property. It also checks that every production worker that declares Portable has a suite that calls checkPortable().
• An empty result fails property 2: a job that gives nothing passes the other two properties without proving them. Print report() in the failure.

From testFemSolveJob:

batch::testing::PortableProjection projection;
projection.measuredKeys = {QStringLiteral("solveSeconds")};

const auto verdict = batch::testing::checkPortable(worker, job, store, projection);

QVERIFY2(verdict.kept(), qPrintable(verdict.report()));

libcatalog

FakeWidget

registerFakeWidget(): registers a plain widget of one size and one background color in WidgetRegistry, under test/<id>.

• Header: src/libs/libcatalog/catalog/testing/FakeWidget.hpp
• Include: #include "catalog/testing/FakeWidget.hpp"
• Tested by: testCatalogTesting
• A second call with the same id does nothing, so every case can call it. Do not use it to test a real catalog widget.

From testOpalUIDriver:

registerFakeWidget({QStringLiteral("UIDriverTestWidget"), {QStringLiteral("test")}, {QStringLiteral("default")},
                    {80, 60}, QStringLiteral("#336")});

libcomms

BlobCourierPump

Two BlobCouriers wired in this process and the pumps that move their frames: makeCourierPair(), CourierLink, pumpUntilQuiet(), exchangeFrames(), pumpMeshUntilQuiet(), and PipeOptions for loss, reordering, duplication and tampering.

• Header: src/libs/libcomms/comms/testing/BlobCourierPump.hpp
• Include: #include "comms/testing/BlobCourierPump.hpp"
• Tested by: testBlobCourierPump
• QTRY_VERIFY alone moves no frame; step the link until the condition holds.

From testBlobCourier:

auto p = makeCourierPair();
QSignalSpy received(p.receiver.data(), &BlobCourier::blobReceived);
// ...
pumpUntilQuiet(*p.sender, p.senderId, *p.receiver, p.receiverId);

libconfig

ChainFixture

A small settings schema with every kind of setting, and a resolver over map stores in the shape of Construct's cascade: app is the seed store, and scene, camera and job are the chain. FixtureLevels gives the three chain levels. The header lists the ids of the schema.

• Header: src/libs/libconfig/config/testing/ChainFixture.hpp
• Include: #include "config/testing/ChainFixture.hpp"
• Tested by: testConfigTesting. testConfigResolver and the other libconfig suites use it.
• The schema is built once per process and does not change. Do not add a setting to it for one case. Build a schema in the test.

From testConfigResolver:

ChainFixture c;
c.scene->set(STEP_SCALE, 0.5f);
c.camera->set(STEP_SCALE, 0.5f);
const ResolvedVariant r = c.resolver->resolve(STEP_SCALE);
QCOMPARE(r.value.toFloat(), 0.5f);
QCOMPARE(r.from, FixtureLevels::camera());

libconfigui

The libconfigui harnesses are in the namespace config::testing, as the libconfig one is.

SyntheticSchema

A generated schema of a given number of settings over categories and sections, with every kind, from a seed. expectedMatches() gives the number of settings that a filter text keeps, from a plain scan of the generated strings, separate from the model. makeResolver() gives a resolver over an App and a Project store.

• Header: src/libs/libconfigui/configui/testing/SyntheticSchema.hpp
• Include: #include "configui/testing/SyntheticSchema.hpp"
• Tested by: testConfigUiTesting. testConfigFilter, testConfigScale and the other config window suites use it.
• Use fixedTexts() for a filter test: the counts of those texts are known.

From testConfigFilter:

for (const QString &text : SyntheticSchema::fixedTexts()) {
    s.proxy.setFilterText(text);
    QCOMPARE(visibleSettings(s.proxy), fixture().expectedMatches(text));
}

ShortcutRoundTrip

shortcutRebindSurvivesRestart(): rebinds one shortcut of an app through its shortcut schema, starts again over the same settings file, and checks that the rebind is back. The first start registers the shortcuts before the shortcut settings exist and the second after, so both orders run. It also checks that every registered shortcut has a row, and that the schema has no declaration error.

• Header: src/libs/libconfigui/configui/testing/ShortcutRoundTrip.hpp
• Include: #include "configui/testing/ShortcutRoundTrip.hpp"
• Tested by: testConfigUiTesting. The shortcut suite of each app (testBahnShortcuts, testTroveShortcuts and others) uses it.
• It makes a registry for each start and a settings file in a temporary folder of its own. Register the app's shortcuts in the callback, into the registry that it gets. Assert that the answer is empty.

From testBahnShortcuts:

const QString failure = config::testing::shortcutRebindSurvivesRestart([](ShortcutRegistry &r) {
    registerBahnShortcuts(r);
});
QVERIFY2(failure.isEmpty(), qPrintable(failure));

libconstruct

The construct harnesses are in the testing/ folder beside the code they know.

ScopedDocument

Open documents, published for OPAL tasks, and torn down at the end of the scope, also on an early return or a failed check.

• Header: src/libs/libconstruct/construct/document/testing/ScopedDocument.hpp
• Include: #include "construct/document/testing/ScopedDocument.hpp"
• Tested by: testConstructDocumentTesting
• Do not install an executor or a scene into ConstructServiceLocator by hand.

From testDocumentTabs:

ScopedDocument docs;
DocumentRegistry &registry{docs.registry()};
QList<QUuid> ids{docs.ids()};

OverlayModelFixture

The model half of the scene tree overlay over the production document chain, with no widget: the document, the expression manager and the panel adapter.

• Header: src/libs/libconstruct/construct/ui/testing/OverlayModelFixture.hpp
• Include: #include "construct/ui/testing/OverlayModelFixture.hpp"
• Tested by: testOverlayModelFixture
• For a suite that needs the browser widget, use OverlayTestFixture.

From testWrapSelection:

struct Fixture : construct::ui::testing::OverlayModelFixture
{
    QUuid add(const QString &def, const QUuid &parent = {})

OverlayTestFixture

One scene tree overlay over the production document chain, with the browser, the rebuild nudge and the drag harness.

• Header: src/libs/libconstruct/construct/ui/testing/OverlayTestFixture.hpp
• Include: #include "construct/ui/testing/OverlayTestFixture.hpp"
• Tested by: testConstructUiTesting. Its base is tested by testOverlayModelFixture.
• A unit suite without widgets uses OverlayModelFixture. Do not write a private nudge or drag helper.

From testOverlaySelection:

OverlayTestFixture fx;
auto &scene = fx.scene;
auto &adapter = fx.adapter;

ConstructShellFixture

A ConstructWidget with settings of its own in a temporary folder, configured as the application configures it, with addDocument() to open a tab.

• Header: src/libs/libconstruct/construct/ui/testing/ConstructShellFixture.hpp
• Include: #include "construct/ui/testing/ConstructShellFixture.hpp"
• Tested by: testConstructShellFixture
• Configuration is spread across event-loop turns: wait for configure(), do not assume it finished.

From testDocumentRenderRelease:

mShell = std::make_unique<construct::ui::testing::ConstructShellFixture>();
mConfigured = mShell->configure();
QVERIFY(mConfigured);

SynchronousConfirmHost

A confirm host that answers each request with one token, in the turn that it is asked, as the activity stack does. With no host, a confirm resolves as cancel, so a case that needs Delete, Discard or Save as new installs this host.

• Header: src/libs/libconstruct/construct/ui/testing/SynchronousConfirmHost.hpp
• Include: #include "construct/ui/testing/SynchronousConfirmHost.hpp"
• Tested by: testConfirmController
• Declare the host before the widget that uses it, so that the host lives longer than the widget.

From testDefinitionHomes:

SynchronousConfirmHost confirm(QStringLiteral("delete"));
Fixture f(QStringLiteral("home_probe_delete"));
f.editor.setConfirmController(&confirm);

DragNumberStroke

dragNumberBy(): one drag stroke on a DragNumber, a press, four moves and a release, sent straight to the widget, so that it needs no window. Each move waits, up to 3 seconds, for the value that the move emits, so the stroke writes once per move, as a real drag does. The stroke lands on the center of one number of the box.

• Header: src/libs/libconstruct/construct/ui/testing/DragNumberStroke.hpp
• Include: #include "construct/ui/testing/DragNumberStroke.hpp"
• Tested by: testConstructUiTesting. testOverlayRowStandard and testConstructShellLifecycle use it.
• It returns false when a move emitted nothing in time. Wrap it in QVERIFY2. Do not replace the wait with a fixed sleep: on a loaded machine the sleep ends before the box's throttle timer runs, and a move is lost.

From testOverlayRowStandard:

const int before = f.steps();
QVERIFY2(construct::ui::testing::dragNumberBy(drag, 40), "a move of the stroke emitted no value");
f.pump();

SceneRender

One frame of a scene the way the render export makes it: prepareSceneRender(), renderScene(), and the cameras lookAtOrigin() and sphereCamera().

• Header: src/libs/libconstruct/construct/batch/rendering/testing/SceneRender.hpp
• Include: #include "construct/batch/rendering/testing/SceneRender.hpp"
• Tested by: testConstructRenderTesting
• A host with no usable GPU leaves the renderer uninitialized: skip on that, and fail on a bad snapshot. It creates a GPU device, so a suite that uses it is not a member of the shared binary.

From testRenderQuality:

const SceneRender run = prepareSceneRender(mFixturePath, sphereCamera(), preset, options);
QVERIFY2(run.ready(), qPrintable(run.error));

MarchDiagnostic

marchDiagnosticHit(): whether the debug-view 7 frame says that the ray of a pixel hit something.

• Header: src/libs/libconstruct/construct/batch/rendering/testing/MarchDiagnostic.hpp
• Include: #include "construct/batch/rendering/testing/MarchDiagnostic.hpp"
• Tested by: testConstructRenderTesting
• Read only the blue channel. Do not compare blue with red and green: that inverts for an expensive tracer.

From testTracerOnRealScenes:

if (marchDiagnosticHit(img, x, y)) { ++hits; }

RenderContextFixture

baseRenderContext(): the render context a filename template test expands against, with the session time fixed.

• Header: src/libs/libconstruct/construct/batch/rendering/testing/RenderContextFixture.hpp
• Include: #include "construct/batch/rendering/testing/RenderContextFixture.hpp"
• Tested by: testConstructRenderTesting
• Do not use it for a render; it holds only the values a template expands.

From testFilenameTemplate:

QCOMPARE(t.expand(baseRenderContext(), &err), QStringLiteral("/tmp/no_tokens_here.png"));

AnalyticFields

Signed distance fields with a closed form: SphereField, BoxField, PlaneField, EmptyField and InvalidField.

• Header: src/libs/libconstruct/construct/eval/testing/AnalyticFields.hpp
• Include: #include "construct/eval/testing/AnalyticFields.hpp"
• Tested by: testAnalyticFields
• For the field of a real scene, use SceneEvaluator.

From testDualContouring:

SphereField f(0.8f);
const Grid g = sampleGrid(f, 0.0f, 0.1f, -12, 25); // [-1.2, 1.2]

SceneEvaluator

The chain from a scene to a CPU field, as the bounds worker and the export path run it: sceneNodes(), compileScene(), fieldOf() and evaluatorFor().

• Header: src/libs/libconstruct/construct/eval/testing/SceneEvaluator.hpp
• Include: #include "construct/eval/testing/SceneEvaluator.hpp"
• Tested by: testAnalyticFields
• When the result names a failed stage, report it; do not treat a null evaluator as an empty field.

From testBoundsDetector:

const auto nodes = construct::eval::testing::sceneNodes(scene);
data.nodes = nodes.nodes;
data.rootNodeIndex = nodes.rootNodeIndex;

FieldDelta

The largest difference between two fields over a seeded sample of a cube: maxDelta() and maxDeltaOnGrid().

• Header: src/libs/libconstruct/construct/eval/testing/FieldDelta.hpp
• Include: #include "construct/eval/testing/FieldDelta.hpp"
• Tested by: testAnalyticFields
• Both return -1 when a value is not finite. Assert that the result is at least 0 before you compare it with a tolerance.

From testComponentParity:

float d = maxDeltaOnGrid(*oracle, *live, 140.0f, 16, 11u, &at);
QVERIFY2(d >= 0.0f && d < 1.0e-3f, qPrintable(describeDelta("component vs plain", d, at)));

OneOpKernel

One position-transform op compiled as a kernel in the GPU wrapper's shape (compileOp()), a timed batch call of it (batchCallCost()), and the cost of one drag tick from that call (dragTickMs()), as gate G1 of the leaf probe measures them.

• Header: src/libs/libconstruct/construct/eval/testing/OneOpKernel.hpp
• Include: #include "construct/eval/testing/OneOpKernel.hpp"
• Tested by: testAnalyticFields
• A null program means that the compile failed; report error.
• A cost in wall time is a property of the machine. Do not assert it. Assert a ratio to the reference work (relativeTime), as testEvalPerf does for gate G1 with kBatchCallsPerDragTick, or compare two costs measured side by side, as a unit suite does.

From testLeafProbe:

const OneOpKernel k = compileOp(*def, ::eval::Backend::Native);
const auto cost = construct::eval::testing::batchCallCost(k, *def, kChildren, kCalls);
worstBatchMs = std::max(worstBatchMs, construct::eval::testing::dragTickMs(cost));

DefineComponent

defineComponent() and parameter(): a component definition made from a branch built in a scratch scene, and a component parameter, in one call.

• Header: src/libs/libconstruct/construct/component/testing/DefineComponent.hpp
• Include: #include "construct/component/testing/DefineComponent.hpp"
• Tested by: testComponentTesting
• An empty id means that the registry refused the definition; check it.

From testComponentParity (testComponentScopes.inc.cpp):

return defineComponent(doc.scene, doc.target.definitionSource(), build, params);

ScopedPrimitives

Test primitives in the process-wide registry for as long as the object lives; the destructor removes what it added.

• Header: src/libs/libconstruct/construct/registry/testing/ScopedPrimitives.hpp
• Include: #include "construct/registry/testing/ScopedPrimitives.hpp"
• Tested by: testScopedPrimitives
• Do not add a primitive to the registry without it: in the shared binary, the next class finds it among the built-ins.

From testRapidEditCompileLoop:

ScopedPrimitives prims;
prims.ensure(QStringLiteral("translate"));
prims.ensure(QStringLiteral("rotate"));

CompiledPath

coreNameOf() and pathToCore(): the compiled core of a logical item, and the path to it as the SDF node builder walks the tree.

• Header: src/libs/libconstruct/construct/model/testing/CompiledPath.hpp
• Include: #include "construct/model/testing/CompiledPath.hpp"
• Tested by: testConstructModelTesting.
• The position sum is right only when translates are in play. A case that rotates or scales measures something else.

From testSDFTransform:

QVERIFY2(!pathToCore(s.target, torus).isEmpty(), "the parent is still drawn");
const QList<Instance> path = pathToCore(s.target, sphere);

CameraChain

The scene-and-camera settings chain that the viewport resolves through, and SceneCameraChain, a scene of its own with one camera over it.

• Header: src/libs/libconstruct/construct/settings/testing/CameraChain.hpp
• Include: #include "construct/settings/testing/CameraChain.hpp"
• Tested by: testConstructSettingsTesting
• CameraChain does not own the scene; keep the scene alive longer.

From testSettingsScopesE2E:

CameraChain a(&s.scene, s.camA);

GlslRotations

The GLSL rotations of the rotate primitives, written again in C++ as the shader evaluates them: the independent ground truth for RotationMath.

• Header: src/libs/libconstruct/construct/gizmo/testing/GlslRotations.hpp
• Include: #include "construct/gizmo/testing/GlslRotations.hpp"
• Tested by: testGizmoTesting
• Do not derive them from RotationMath; then the test compares the code with itself.

From testRotationModes:

case 0:  return glslRotateX(p, a);
case 1:  return glslRotateY(p, a);
default: return glslRotateZ(p, a);

RotateRings

The camera, rings and screen points that a gizmo drag test needs; a world point goes to the screen through Camera::worldToScreen.

• Header: src/libs/libconstruct/construct/gizmo/testing/RotateRings.hpp
• Include: #include "construct/gizmo/testing/RotateRings.hpp"
• Tested by: testGizmoTesting
• Do not write a projection of your own; the hit test uses Camera::worldToScreen.

From testGradedArcDrag:

const Camera cam = ringCamera();
const QSize vp = testViewport();
GizmoInteractor gi;
gi.setActiveHandles(rotateRings(), QUuid::createUuid(), QMatrix4x4(), params);

WeldedMesh

weldedMesh(): the triangle soup of a mesher as one indexed mesh, with shared corners merged, so that a test measures it as a surface.

• Header: src/libs/libconstruct/construct/mesh/testing/WeldedMesh.hpp
• Include: #include "construct/mesh/testing/WeldedMesh.hpp"
• Tested by: testConstructMeshTesting
• The weld tolerance is 1e-6. Do not use it to test the welding of the export path.

From testDualContouring:

const auto mesh = weldedMesh(tris);
const MetricReport r = computeMeshMetrics(mesh, f);
QVERIFY(r.closed);

FakeMeshFileWorker

A mesh file worker that runs no pipeline: the real MeshFileWorker (the job construct/mesh_file, a file in and a file out) up to the meshing, which completes or fails at once with the result the test set. makeFileJob() makes a job for it. It was FakeMeshExportWorker until work-placement §WPI-7 retired MeshExportWorker and its snapshot store.

• Header: src/libs/libconstruct/construct/export/testing/FakeMeshFileWorker.hpp
• Include: #include "construct/export/testing/FakeMeshFileWorker.hpp"
• Tested by: testConstructExportTesting
• Do not use it to test meshing; it reads and writes no file.

From testExportUx:

FakeMeshFileWorker worker;
worker.m_shouldSucceed = true;
BatchQueue queue;
BatchWorkerRegistry registry;
registry.registerWorker(&worker);

libcrypt

Hex

fromHex<Array>() and toHex(): hex text to a fixed-size key array and back, for test vectors.

• Header: src/libs/libcrypt/crypt/testing/Hex.hpp
• Include: #include "crypt/testing/Hex.hpp"
• Tested by: testCryptTesting
• The text must decode to exactly the size of the array. A release build copies what fits and leaves the rest zero, so do not rely on it to check a length.

From testCryptCurve25519:

const auto seed = fromHex<Ed25519::Seed>(
    "9d61b19deffd5a60ba844af492ec2cc4"
    "4449c5697b326919703bac031cae7f60");

libdeploy

ElfFixture

ElfFixtureSpec, makeElf() and writeElf(): a minimal 64-bit ELF object with the dynamic facts a closure test needs, made on any host without a compiler.

• Header: src/libs/libdeploy/deploy/testing/ElfFixture.hpp
• Include: #include "deploy/testing/ElfFixture.hpp"
• Tested by: testDeployTesting.
• The file has no code and cannot run. Do not use it where a test loads or executes the library.

From testDeployStaging:

writeElf(lib + QStringLiteral("libQt6Core.so.6"), so(QStringLiteral("libQt6Core.so.6")));

PeFixture

PeFixtureSpec, PeImport, makePe() and writePe(): a minimal PE32+ image with the import, delay-load and export facts that a closure test needs, made on any host, Linux included, without a compiler or a Windows SDK. It is the Windows counterpart of ElfFixture.

• Header: src/libs/libdeploy/deploy/testing/PeFixture.hpp
• Include: #include "deploy/testing/PeFixture.hpp"
• Tested by: testDeployTesting. testDeployClosure, testDeployStaging and testKilnPackage use it.
• The file has no code and cannot run. A name written as # and a number, such as #12, is an import by ordinal.

From testDeployClosure:

PeFixtureSpec s = dll(QStringLiteral("demo.dll"),
    {PeImport{QStringLiteral("KERNEL32.dll"), {QStringLiteral("GetProcAddress"), QStringLiteral("#12")}, false},
        PeImport{QStringLiteral("Qt6Core.dll"), {QStringLiteral("?qVersion@@YAPEBDXZ")}, false},
        PeImport{QStringLiteral("late.dll"), {QStringLiteral("LateCall")}, true}},
    {QStringLiteral("DemoStart"), QStringLiteral("DemoStop")});
const QString path = dir.filePath(QStringLiteral("demo.dll"));
QVERIFY(writePe(path, s));

libfixo

Preset

Presets and their parts for a fixo test: identityPreset(), hexapodVariantsPreset(), idleChannel(), idleStanza() and addIdleChannel().

• Header: src/libs/libfixo/fixo/testing/Preset.hpp
• Include: #include "fixo/testing/Preset.hpp"
• Tested by: testFixoTesting
• Do not use it to test the preset file format; build the file in the test.

From testNativeCompile:

QVERIFY(rt.setPreset(identityPreset(QStringLiteral("NativeCompileTest"), 2, {QStringLiteral("drive")})));

HostCompile

Compiles emitted firmware source for this host with the product JIT (NativeCompiler, the vendored Clang and lld), to a shared library that a test loads. It makes the same request as Runtime::buildNative(), with the fixo pantry bundles mounted.

• Header: src/libs/libfixo/fixo/testing/HostCompile.hpp
• Include: #include "fixo/testing/HostCompile.hpp"
• Tested by: testFixoTesting
• It does not start a compiler process and it does not skip. A failed compile is a finding. Until 2026-10-07 it started g++ or clang++ from PATH, which no machine the product ships to has.
• compileToSo() takes a JIT cache tag as an optional last argument. Put the generator version in it when the source comes from FirmwareGenerator.
• It writes a library to the JIT cache and loads it, so keep a suite that uses it out of the shared binary.

From testCodegenDispatch:

const bool ok = !compileToSo(src, errorOut, cacheTag()).isEmpty();
if (!ok) {
    qWarning().noquote() << "Compiler diagnostics:\n" << errorOut;

libfleet

FakeProvider

An in-memory provider that can be told to fail, time out, reclaim and lie. It answers as a provider answers, and costs nothing.

• Header: src/libs/libfleet/fleet/testing/FakeProvider.hpp
• Include: #include "fleet/testing/FakeProvider.hpp"
• Tested by: testFleetTesting, through FleetFixture.
• It is a fake of the provider contract, not a mock of the fleet's code. Do not use it to test a real provider's wire format; that is LocalHttpServer's job.

From testFleetReconcile:

FakeProvider provider;
MemoryFleetStore store;
const Fleet::Config config = testFleetConfig();

FleetFixture

A Fleet against FakeProvider and a memory store, open and with a rail, with testFleetConfig() and testPurchase().

• Header: src/libs/libfleet/fleet/testing/FleetFixture.hpp
• Include: #include "fleet/testing/FleetFixture.hpp"
• Tested by: testFleetTesting
• The retry does not sleep: it records each wait. A fleet that does not open is a warning and opened is false; check it.
• Options::provider gives the fleet another provider in place of the fake, for example an adapter against a local stub (testFleetAws, testFleetHetzner). The case keeps it alive. provisionAndJoin() needs the fake.

From testFleetReconcile:

struct Harness : fleet::testing::FleetFixture
{
    Harness()
        : FleetFixture(options())

TestJoin

useTestCoordinator(), joinTokenOf() and joinAsMachine(): what a rented machine does at first start, with the token it was handed.

• Header: src/libs/libfleet/fleet/testing/TestJoin.hpp
• Include: #include "fleet/testing/TestJoin.hpp"
• Tested by: testFleetTesting.
• The coordinator identity is for tests; libfleet never parses its key.

From testFleetOpalTasks:

const auto machine = fleet::testing::joinAsMachine(fleet, provider, *i, QDateTime::currentDateTimeUtc(), &err);
if (!machine) {

FakeRunner

An IRunner in memory. Each machine holds staged program digests and runs. A run finishes when the test calls finish(), or at once with the result that scriptProgram() set for its program. A machine set with setUnreachable() answers nothing, as a destroyed one does. It counts the bytes sent and the stage and start calls.

• Header: src/libs/libfleet/fleet/testing/FakeRunner.hpp
• Include: #include "fleet/testing/FakeRunner.hpp"
• Tested by: testFleetTesting. testFleetRun and testFleetOpalTasks use it.
• Use the counters to prove that something was not sent, for example that a second stage of the same digest sent no bytes. Do not use it to test a real runner's transport.

From testFleetRun:

Harness h;
const Instance a = h.up();
const Instance b = h.up();
h.runner.setUnreachable(b.id, true);

libfs

ExFatTestImage

Formatted exFAT images in memory: formatExFat(), writeExFatFile() and formatExFatWithFiles(), the input of the corruptifier and the checker.

• Header: src/libs/libfs/fuzzer/ExFatTestImage.hpp
• Include: #include "fuzzer/ExFatTestImage.hpp"
• Tested by: testExFatTestImage
• It is in the fuzzer folder, not a testing/ folder. Do not use it to test the formatter itself.

From testSemanticCorruptifier:

QVERIFY(formatExFat(device1, QStringLiteral("TEST")));

libglslang

Spirv

GLSL to validated SPIR-V through the vendored glslang and SPIRV-Tools: ensureProcess(), compileToSpirv(), validateSpirv() and compileAndValidate().

• Header: src/libs/libglslang/glslang/testing/Spirv.hpp
• Include: #include "glslang/testing/Spirv.hpp"
• Tested by: testGlslangFrontEnd
• Call ensureProcess() before the first compile: glslang keeps process-wide pools.

From testSpir:

const auto r = octomy::glslang::testing::compileAndValidate(full, lang);
if (!r.ok) {
    qWarning() << "GLSL to validated SPIR-V failed:" << r.log;

libhelp

StubRender

A stand-in for the widget renderer of HelpWidgetImageProvider: a small, deterministic image per widget and mode.

• Header: src/libs/libhelp/help/testing/StubRender.hpp
• Include: #include "help/testing/StubRender.hpp"
• Tested by: testHelpTesting
• Do not use it to test what a real catalog widget looks like.

From testHelpWidgetImageProvider, which counts the renders of its cache cases:

configureStubProvider(p, diskDir, memCap);
p.setRenderFn(countingStubRender(rendered));

libhive

BuildHive

buildHive(): a hive on disk, made headlessly through HiveCreator, the code path that the application runs. The key entropy is random, or comes from a seed that the caller gives.

• Header: src/libs/libhive/hive/testing/BuildHive.hpp
• Include: #include "hive/testing/BuildHive.hpp"
• Tested by: testHiveTesting
• On failure it names the step that failed; print it. When you give a seed, print it too, so that a failure repeats with the same key file.

From testHiveMountRoundTrip:

qInfo().noquote() << QStringLiteral("hive key entropy seed: 0x%1").arg(ENTROPY_SEED, 8, 16, QLatin1Char('0'));
return hive::testing::buildHive(storePath, keyPath, PASSPHRASE, why, ENTROPY_SEED);

ExFatMount

The filesystem callbacks that a hive session mounts a hive with, so that hivecluster-N:/ paths reach the in-memory volume: exFatMount(), and processExFatMount(), one pair for the whole process.

• Header: src/libs/libhive/hive/testing/ExFatMount.hpp
• Include: #include "hive/testing/ExFatMount.hpp"
• Tested by: testHiveTesting
• Use processExFatMount() only when a hive is mounted through one session and unmounted through another; it is shared by every class in the process.

From testExternalProvisioning:

hive->setFilesystemCallbacks(hive::testing::processExFatMount().mount, hive::testing::processExFatMount().unmount);
hive->setDriveNumber(Hive::allocateDriveNumber());

SentientHive

A hive registry and a session in one base folder, with the initial sentient hive mounted as the node mounts it, and an optional sentient identity.

• Header: src/libs/libhive/hive/testing/SentientHive.hpp
• Include: #include "hive/testing/SentientHive.hpp"
• Tested by: testHiveTesting
• A suite that needs a second session over the same registry makes one itself.

From testCaptureTheFlag:

hive::testing::SentientHive fixture(tmpDir.path());
QVERIFY(fixture.withSentient(kShadowPassphrase, QStringLiteral("TestSentient")));
auto session = fixture.session;

libinventory

SyntheticInventory

An InventoryService that publishes the fixed NodeInfo, NodeLoad and NodeHealth of a preset, and reads nothing of the machine. The presets:

• Y8: the values read on y8.
• States: one section for each role.
• Hybrid: a hybrid CPU with a failing drive.

infoOf(), loadOf() and healthOf() give the values of a preset without a service.

• Header: src/libs/libinventory/inventory/testing/SyntheticInventory.hpp
• Include: #include "inventory/testing/SyntheticInventory.hpp"
• Tested by: testInventoryTesting. testMonitorFacts, testMonitorE2E and testMonitorOpalStatus use it.
• Give it to MonitorActivity::setInventory() beside a SyntheticSampler. "Measure now" still runs the real probe. Do not use it to test the inventory probes.

From testMonitorFacts:

MonitorFacts::Input in;
in.info = SyntheticInventory::infoOf(SyntheticInventory::Preset::Y8);
const FactStack s = MonitorFacts::cpu(in);

libkiln

Broker (libkiln)

A kiln broker on a known machine, with shapeMachine(), uniqueEndpoint() and RawClient, a client that writes protocol frames itself.

• Header: src/libs/libkiln/kiln/testing/Broker.hpp
• Include: #include "kiln/testing/Broker.hpp"
• Tested by: testKilnTesting
• Give each case its own endpoint, so that no case meets another's broker.

From testKilnBroker:

shapeMachine(broker, 4);
QString error;
const QString endpoint = uniqueEndpoint();
QVERIFY2(broker.listenOn(endpoint, &error), qPrintable(error));

Faults (libkiln)

kiln::testing::injectFault(): the next answers of a cache technique each change one byte on purpose, so that a test proves that the oracle sees a wrong answer. checkEveryDecision() checks every decision in the Sampled and Trusted states, so that a test does not wait for one hit in 10 or 100. clearFaults() removes every fault not yet taken.

• Header: src/libs/libkiln/kiln/testing/Faults.hpp
• Include: #include "kiln/testing/Faults.hpp"
• Tested by: testKilnTesting. testKilnCompile and testKilnBuildE2E use it.
• The faults are process-wide state. Call clearFaults() in the same case, after the faulty answer. It is compiled into libkiln, but no command, key, flag or message reaches it: only a test's own code can set a fault.

From testKilnCompile:

kiln::testing::injectFault(kiln::Technique::Backend);
const kiln::CompileOutcome wrong = kiln::InProcessCompiler::compile(withBackend(kiln::BackendRequest::Mode::Shadow));
kiln::testing::clearFaults();

libkilnui

SyntheticKilnFeed

Broker snapshots made from a script, for KilnMonitorWidget, so that the lane and job lists show the full range of the load colors with no broker and no build. The modes are those of SyntheticSampler: Calibration, Sweep and Frozen. first() gives a minute of samples, and each next() one more sample a second later. laneShares() and jobShares() give the shares that the list draws for a row.

• Header: src/libs/libkilnui/kilnui/testing/SyntheticKilnFeed.hpp
• Include: #include "kilnui/testing/SyntheticKilnFeed.hpp"
• Tested by: testKilnUiTesting. testKilnActivity and testMonitorInteractive use it.
• The cache's share of a lane is 0 on the first snapshot: the list needs a second snapshot to read a difference. Pass false to laneShares() after next().

From testKilnActivity:

SyntheticKilnFeed feed(SyntheticKilnFeed::Mode::Calibration);
kilnui::KilnMonitorWidget widget;
widget.resize(1200, 900);
widget.applySnapshot(feed.first(1790000000000), true);

libllm

FakeDriver

A driver, its session and completion facets, and the factory that makes it, for a test that drives an LLM consumer without a model. Every action is recorded.

• Header: src/libs/libllm/llm/driver/testing/FakeDriver.hpp
• Include: #include "llm/driver/testing/FakeDriver.hpp"
• Tested by: testFakeDriver
• Call FakeDriver::resetCapture() before each case. Without it, a class of the shared binary sees the last driver of another.

From testTroveSessionManager:

FakeDriver::resetCapture();
DriverManager dm;
dm.registerFactory(std::make_unique<FakeDriverFactory>());

Registry (libllm)

An LLM registry and manager in a known state, without a node: registryFromMap(), managerFor(), emptyManager() and addDefaultModel().

• Header: src/libs/libllm/llm/testing/Registry.hpp
• Include: #include "llm/testing/Registry.hpp"
• Tested by: testLLMTesting
• The providers and models that the bootstrap seeds are dropped. Do not use it to test the seeding.

From testLLMRegistry:

auto reg = registryFromMap();
QSignalSpy addedSpy(reg.data(), &LLMRegistry::providerAdded);

ScriptedClient

An LLMClient with no transport, which answers each request from a script and records every request.

• Header: src/libs/libllm/llm/testing/ScriptedClient.hpp
• Include: #include "llm/testing/ScriptedClient.hpp"
• Tested by: testLLMTesting
• For a consumer that talks to a driver rather than a client, use FakeDriver.
• Set mInputTokens, mOutputTokens and mCostPer1kTokens to give each reply a known spend, as testLLMIntegration does for the budget path of OpalConversationDriver.

From testSessionStore, where a subclass also records the session folders that it gets:

auto client = QSharedPointer<FolderClient>::create();
client->mFreshHandle = QStringLiteral("fresh-handle");

liblocaldoor

Door

postAside(), callAside() and postManyAside(): a call to a door that runs in the test's own thread, made from another thread while the test's event loop spins. FunctionHandler: a call handler made of a function.

• Header: src/libs/liblocaldoor/localdoor/testing/Door.hpp
• Include: #include "localdoor/testing/Door.hpp"
• Tested by: testLocalDoor
• Keep each door in a temporary folder with DoorAddress::inFolder(), so a test never answers for the user's door.
• A status of 0 means that no answer came. Assert the status before the body.

From testLocalDoor:

const auto a = callAside(rig.server(), QStringLiteral("test.door.echo"), {{QStringLiteral("say"), QStringLiteral("hello")}});
QCOMPARE(a.status, 200);

liblockstep

CounterSim

A small deterministic simulation for the lockstep engine: an order-sensitive fold over the command deltas, with a heartbeat each turn.

• Header: src/libs/liblockstep/lockstep/testing/CounterSim.hpp
• Include: #include "lockstep/testing/CounterSim.hpp"
• Tested by: testLockstepTesting
• Do not use it to test a real game's simulation.

From testLockstep:

CounterSim simA;
CounterSim simB;
LockstepEngine<CounterCommand> engineA;
LockstepEngine<CounterCommand> engineB;

libmediacore

Measure

What an audio test measures in a block of samples: binMagnitude(), amplitudeAt(), responseAt(), render(), rms(), peak(), measuredPitch(), centsBetween() and midiToHz().

• Header: src/libs/libmediacore/mediacore/audio/testing/Measure.hpp
• Include: #include "mediacore/audio/testing/Measure.hpp"
• Tested by: testAudioMeasure
• binMagnitude() is not scaled; compare two bins of one window, or use amplitudeAt() for a value in amplitude units.

From testPianoTuning:

const auto got = measuredPitch(out, expected, SR, 12000, 36000);
const auto err = centsBetween(got, expected);

Signal

The test signals that a DSP test feeds a unit: sineWave(), uniformNoise() and addUniformNoise(), the noise from a given seed.

• Header: src/libs/libmediacore/mediacore/audio/testing/Signal.hpp
• Include: #include "mediacore/audio/testing/Signal.hpp"
• Tested by: testAudioMeasure
• Give the noise a constant seed, so that a run repeats.

From testSnrMeasurement:

const std::vector<float> tone = mediacore::audio::testing::sineWave(static_cast<float>(freq),
    static_cast<float>(rate), static_cast<float>(amp), static_cast<std::size_t>(frames));

BufferContext

An AudioContext over a test's own mono buffer, so that a node or a processor renders into it.

• Header: src/libs/libmediacore/mediacore/audio/testing/BufferContext.hpp
• Include: #include "mediacore/audio/testing/BufferContext.hpp"
• Tested by: testAudioMeasure
• The buffer must outlive the context. For a bahn processor, use FakeProcessorContext.

From testGuitarTuning:

BufferContext ctx(out, SR);
g.process(ctx);
return out;

Nodes

The audio test nodes: sources, a gain and sinks with known output, for a schedule, a job or an engine under test.

• Header: src/libs/libmediacore/mediacore/audio/testing/Nodes.hpp
• Include: #include "mediacore/audio/testing/Nodes.hpp"
• Tested by: testAudioTestNodes
• A SineSource that is not channel aware is refused by a wide schedule, on purpose. Use it only to test that refusal.

From testMediacoreGraph:

auto scale = node("scale", QSharedPointer<Gain>::create(0.37));
auto noise = node("noise", QSharedPointer<NoiseSource>::create());
auto sinkA = node("sinkA", QSharedPointer<Sink>::create(SinkKind::Intrinsic));

WavFile

writeToneWav(): a test tone on disk, a 16-bit PCM stereo WAV file that a media player decodes.

• Header: src/libs/libmediacore/mediacore/audio/testing/WavFile.hpp
• Include: #include "mediacore/audio/testing/WavFile.hpp"
• Tested by: testAudioMeasure
• Write it into a temporary folder; check the return value.

From testMediaPlayerSource:

QVERIFY(writeToneWav(mWav, WavTone{3.0, LEFT_HZ, RIGHT_HZ, 12000.0, 44100}));

libmonitor

ProcessSamples

processSample() and kernelThread(): the process samples that the process-list suites share.

• Header: src/libs/libmonitor/monitor/testing/ProcessSamples.hpp
• Include: #include "monitor/testing/ProcessSamples.hpp"
• Tested by: testMonitorTesting
• Do not use it to test the sampler, which reads real processes.

From testProcessGrouping:

const auto p = processSample(101, QStringLiteral("/usr/bin/clang"),
    {QStringLiteral("/usr/bin/clang"), QStringLiteral("-c")});
QCOMPARE(groupKeyFor(p), QStringLiteral("/usr/bin/clang"));

SyntheticSampler

A PlatformSampler that reports a script instead of the machine: cores, a GPU, memory, a disk, an interface, a filesystem and processes, with no read of /proc. Calibration gives fixed rows at known shares and their mixes, Sweep ramps every row from 0 to the machine share and back, and Frozen gives one fixed snapshot for a scene. It also gives the test seams of testResourceMonitor: a set of interfaces, a stall inside sample(), and counts of the requests served.

• Header: src/libs/libmonitor/monitor/testing/SyntheticSampler.hpp
• Include: #include "monitor/testing/SyntheticSampler.hpp"
• Tested by: testMonitorTesting. testMonitorE2E, testResourceMonitor and testMonitorInteractive use it.
• A share is in units of one core's worth. The sampler reports the CPU percent scaled to one core (100 is one core), and memory and VRAM as percents of the whole machine. processOf() gives a row as the sampler reports it.
• Never give it to a running app: a user must not see invented loads as real ones.

From testMonitorE2E:

auto sampler = QSharedPointer<SyntheticSampler>::create(SyntheticSampler::Mode::Calibration, kCores);
ActivityMockRuntime runtime;
auto node = runtime.mockNode();

MonitorDrive

monitor::testing::advanceMonitor(): advances a MonitorActivity by a number of columns with no timer and no resume. It drives the production path: it primes the worker, waits for each delivery, ticks the monitor and lets the activity draw the column.

• Header: src/libs/libmonitor/monitor/testing/MonitorDrive.hpp
• Include: #include "monitor/testing/MonitorDrive.hpp"
• Tested by: testMonitorDrive. testMonitorE2E and testMonitorInteractive use it.
• It returns false when a delivery did not come in time. Assert it. Set a SyntheticSampler first, so that the result is the same on every machine.

From testMonitorE2E:

QVERIFY(activity);
QVERIFY2(monitor::testing::advanceMonitor(*activity, 3), "the synthetic sampler did not deliver");

libnode

CourierShuttle

Moves the datagrams of a LockstepCourier or a MatchmakingCourier into its peer through real QDataStream framing, as CommsChannel does per sending opportunity, without the scheduling and the encryption. shuttle() is a perfect wire. LossyLink is one direction of a wire that drops one datagram in dropEvery and doubles one in dupEvery. It counts the datagrams, so a run repeats exactly.

• Header: src/libs/libnode/node/testing/CourierShuttle.hpp
• Include: #include "node/testing/CourierShuttle.hpp"
• Tested by: testLockstepCourier
• LossyLink counts across every pump(). Use one link per direction.

From testRtsNetworkMatch:

LossyLink aToB{3, 5, 0}; // drop 1/3, duplicate 1/5
LossyLink bToA{4, 6, 0}; // drop 1/4, duplicate 1/6
// ...
aToB.pump(*sessionA.courier(), *sessionB.courier());
bToA.pump(*sessionB.courier(), *sessionA.courier());

NodeConstellation

A zoo server and a number of Agents and Remotes, each made through AppLauncher as the application makes it. Each node and the zoo get a temporary base dir. The last constructor argument, ConstellationMode, says how far a node comes up:

• Configured (the default): the node is configured only. It has no NID, so its id is empty, and it discovers nothing. Use it when a suite needs the node objects, for example to bind widgets to them.
• Live: the node gets a NID (IdentityFactory::provisionNode), it is activated, and it discovers through the constellation's zoo (zooUrl(), http://127.0.0.1:8123/api). An Agent and a Remote find each other and hold each other as associates. useZoo must be set. A Live node needs about two seconds more to start and to stop.

init(true) does not wait. Wait with a bound (QTRY_*) for what you assert. The other channel switches (useNetwork and so on) only log.

ConstellationNode<T> is one such node with its base dir and command line. Use it to bring up a node with a zoo of your own.

• Header: src/libs/libnode/node/testing/NodeConstellation.hpp
• Include: #include "node/testing/NodeConstellation.hpp"
• Tested by: testConnectionUI, testDiscovery
• A derived class with its own hooks must call init(false) in its own destructor; the base destructor reaches only the base hooks.

From testDiscovery:

using DiscoveryTestCase = node::testing::NodeConstellation<Agent, Remote, ZooServer>;
// ...
DiscoveryTestCase dtc(QStringLiteral("DiscoveryTestCase"), 1, 1, true, true, true, true, true, true, node::testing::ConstellationMode::Live);
dtc.init(true);
QVERIFY(dtc.zooOnline());
QCOMPARE(dtc.agents.size(), 1);

liboci

FakeRegistry

An in-process image registry that implements the distribution protocol on 127.0.0.1, with switches for the faults of a real one.

• Header: src/libs/liboci/oci/registry/testing/FakeRegistry.hpp
• Include: #include "oci/registry/testing/FakeRegistry.hpp"
• Tested by: testOciTesting. testOciRegistry asserts the client against it.
• It keeps everything in memory. Do not use it for large images.

From testOciRegistry:

FakeRegistry registry;
QVERIFY(registry.listen());
RegistryClient client(registry.host());

libopal

RunTask

Runs one OPAL task in a test and returns its result: runTask() by name or by task, and runTaskOnThread(), with a context and the log, as a plan runs it.

• Header: src/libs/libopal/opal/testing/RunTask.hpp
• Include: #include "opal/testing/RunTask.hpp"
• Tested by: testOpalTesting
• Do not build an OpalTaskContext by hand to run one task.

From testStudyOpal:

const OpalTaskResult result = runTask(name);
QVERIFY2(!result.success, qPrintable(
    name + QStringLiteral(" answered with no project open")));

WaitForPlan

waitForPlan(): delivers events until the runtime completes a plan, and returns its result; a time-out returns a result that is not a success.

• Header: src/libs/libopal/opal/testing/WaitForPlan.hpp
• Include: #include "opal/testing/WaitForPlan.hpp"
• Tested by: testOpalTesting
• Call it after submit(). A plan that completes inside submit() sends its signal before the wait connects.

From testOpalTrigger:

waitForPlan(runtime);

FakeAuthContext

An OpalAuthContext whose every answer the test sets.

• Header: src/libs/libopal/opal/testing/FakeAuthContext.hpp
• Include: #include "opal/testing/FakeAuthContext.hpp"
• Tested by: testOpalTesting
• The swarm extensions deny, as the base class does. Override them in a subclass when a case needs them.

From testOpalTaskRuntime:

FakeAuthContext auth;
auth.sid = true;
auth.nodeActive = true;

Grants

sessionGrants(): a permission store that allows each of a list of task scopes for the session.

• Header: src/libs/libopal/opal/testing/Grants.hpp
• Include: #include "opal/testing/Grants.hpp"
• Tested by: testOpalTesting
• Do not use it to test the grant prompt; it grants without asking.

From testSpecService:

return ::opal::testing::sessionGrants(octomy::trove::opal::specTaskScopes());

Subject

makeSubject(): a black widget with one white square in its top-left quadrant, so that a crop can be told from a full grab.

• Header: src/libs/libopal/opal/vision/testing/Subject.hpp
• Include: #include "opal/vision/testing/Subject.hpp"
• Tested by: testOpalVisionTesting
• The caller owns the widget.

From testOpalVisionCapture:

QScopedPointer<QWidget> a(makeSubject(40, 40));
QScopedPointer<QWidget> b(makeSubject(40, 40));

libopalnet

InProcessBatchNode

InProcessSwitch, the comms courier reduced to what the batch control plane needs, and BatchNode, one node's batch half, built as a running node builds it.

• Header: src/libs/libopalnet/opalnet/testing/InProcessBatchNode.hpp
• Include: #include "opalnet/testing/InProcessBatchNode.hpp"
• Tested by: testOpalNetTesting, through TwoExecutorFleet.
• A killed node's blobs go nowhere: the sender learns it only from a time-out. The caller owns a node's worker and destroys it after the node.

Used through TwoExecutorFleet, as in testFleetExecutorJoin:

#include "opalnet/testing/TwoExecutorFleet.hpp"

TwoExecutorFleet

A coordinator and two executors on one InProcessSwitch, running DigestWorker jobs, with every placement and every returned job recorded.

• Header: src/libs/libopalnet/opalnet/testing/TwoExecutorFleet.hpp
• Include: #include "opalnet/testing/TwoExecutorFleet.hpp"
• Tested by: testOpalNetTesting
• Submit through submit(), so that wrongResults() can check each result.

From testFleetEndToEnd:

nodes = std::make_unique<TwoExecutorFleet>(kDigestType, BatchPolicy::local(), executorIds.at(0),
    executorIds.at(1));
QVERIFY2(nodes->advertise(10000), "both executors must advertise");

libpaint

Producer

Four helpers that drive a paint engine or a producer with no window:

• HeadlessRhi: an offscreen surface and a QRhi on it.

• initializedProducer(): a PaintEngineProducer initialized on a QRhi.

• makeSettledStroke(): a settled freehand stroke.

• applyWorkerFrame(): applies a full or a damage frame to a canvas image, as RhiPresentWidget does.

• Header: src/libs/libpaint/paint/testing/Producer.hpp

• Include: #include "paint/testing/Producer.hpp"

• Tested by: testPaintProducerHarness. testPaintWorkerCanvas, testPaintBrush and testPaintSwarmE2E use it.

• init() is false, with failure naming the step, on a machine with no offscreen surface or no QRhi backend: skip on it. The stroke is coverage only, so assert on alpha.

From testPaintWorkerCanvas:

paint::testing::HeadlessRhi headless;
if (!headless.init()) {
    QSKIP(qPrintable(headless.failure));
}
QScopedPointer<QRhi> &owner = headless.rhi;

SpectralCompare

How far apart two SpectralPixels are. bandDistance() and bandsClose() compare the 80 spectral bands and alpha, which the engine stores. linearRgbDistance() and linearRgbClose() compare the color in linear RGB, for a blend whose spectra differ while the color does not.

• Header: src/libs/libpaint/paint/testing/SpectralCompare.hpp
• Include: #include "paint/testing/SpectralCompare.hpp"
• Tested by: testPaintTesting. testPaintGradient, testPaintEngineStampArgs and testPaintBrush use it.
• Choose the measure by the question: the bands for what the engine stores, linear RGB for what the pixel looks like.

From testPaintGradient:

QVERIFY2(paint::testing::bandsClose(g.sample(0.0), red(), 1e-3),
         "sample(0) should equal the first stop");

libpair

AssociateMaps

associateMap() and udpAddressList(): the stored shape of an Associate, as a test writes it.

• Header: src/libs/libpair/address/testing/AssociateMaps.hpp
• Include: #include "address/testing/AssociateMaps.hpp"
• Tested by: testPairTesting
• Do not use it in a test of the Associate serializer: testAssociate keeps its own map on purpose, as the independent test vector.

From testAddressBook:

address::testing::AssociateMapSpec spec;
spec.id = TestKeys().keyID;
spec.name = "hogboll";

libreferencework

relativeTime

referencework::relativeTime() times a short slice of fixed reference work and your work in turns, on one thread, and gives the best ratio of three pairs. The ratio is of the thread's CPU times, or of the wall times where the system reports no CPU time for a thread. A speed budget asserted as that ratio holds on a busy machine and on a slower host, because both sides slow down together. A slower algorithm raises the ratio. The reference work is kernelWork(), the kernel of kiln's host score (the kiln reference). The result also carries the wall times, the CPU times, ofCpuTime(), which says which of the two the ratio is of, and describe(), one line for the log.

• Header: src/libs/libreferencework/referencework/ReferenceWork.hpp
• Include: #include "referencework/ReferenceWork.hpp"
• Tested by: testReferenceWork
• Measure the budget on the reference host, y8, and say how in a comment.
• Do not use it for a wait, or for work that waits for a disk, a GPU or a compositor: the kernel is CPU work, and it does not slow down with them.
• It runs your work three times: the work must give the same work each time.

From testOpalWebFetch:

QString text;
const referencework::RelativeTime t = referencework::relativeTime([&]() { text = opal::web::htmlToText(html); });
QVERIFY2(t.ratio < kP2RatioBudget, qPrintable(t.describe()));

libserial

PtyPair

Two connected pseudo-ttys: a serial port and its peer. start() opens two terminals with openpty() and relays bytes between them in a thread of the test, so nothing needs installing.

• Header: src/libs/libserial/serial/testing/PtyPair.hpp
• Include: #include "serial/testing/PtyPair.hpp"
• Tested by: testSerialTesting
• POSIX only, and not compiled into libserial. Skip when start() fails: a restricted environment can refuse a pseudo-terminal.

From testLibserialPty:

mPtyAvailable = mPty.start(&why);
if (!mPtyAvailable) {
    qWarning("%s", qPrintable(why));
    QSKIP("the pty pair did not start; skipping pty tests");

libservice

MockService

A Service whose activation answer the test controls: immediate, manual, delayed, or a deactivation that never comes.

• Header: src/libs/libservice/service/testing/MockService.hpp
• Include: #include "service/testing/MockService.hpp"
• Tested by: testServiceTesting
• In Manual mode, call finishActivation(), or the activation never ends.

From testServiceManager:

QSharedPointer<MockService> a(OC_NEW MockService("Service-A", QStringList(), MockService::Mode::Manual));
QSharedPointer<MockService> b(OC_NEW MockService("Service-B", QStringList{"Service-A"}, MockService::Mode::Manual));

libsing

Devices

Output devices for a test of the audio graph and its settings: outputDevice(), audioBox(), builtInStereo() and boxAndStereo().

• Header: src/libs/libsing/audio/testing/Devices.hpp
• Include: #include "audio/testing/Devices.hpp"
• Tested by: testSingAudioTesting
• Do not use them to test device discovery; they open no device.

From testSingularityPlaybackE2E:

const auto devices = sing::audio::testing::boxAndStereo();

libstandalone

Store

storeIn(), the settings store that a standalone app is handed, and filesWrittenBesideTheProcess(), the check that the app writes nothing in the working folder.

• Header: src/libs/libstandalone/standalone/testing/Store.hpp
• Include: #include "standalone/testing/Store.hpp"
• Tested by: testStandaloneTesting
• filesWrittenBesideTheProcess() changes the working folder while it runs. Do not run it on two threads at once.

From testStandaloneBahn:

const auto strays = filesWrittenBesideTheProcess([&dir] {
    auto settings = storeIn(dir);
    Bahn bahn;

libstudy

OneExpressionStudy

oneExpressionStudy(): the smallest study, one ExpressionComponent bound to one free variable.

• Header: src/libs/libstudy/study/testing/OneExpressionStudy.hpp
• Include: #include "study/testing/OneExpressionStudy.hpp"
• Tested by: testStudyTesting
• For a model graph of more than one component, build the study in the test.

From testStudyStatus:

study::testing::OneExpression spec;
spec.expression = rangeExpression;
return study::testing::oneExpressionStudy(spec);

libswarm

ChunkHash

chunkHashOf(): the BLAKE3 hash of a label, a fixed and readable chunk hash.

• Header: src/libs/libswarm/swarm/testing/ChunkHash.hpp
• Include: #include "swarm/testing/ChunkHash.hpp"
• Tested by: testSwarmNodeTesting
• Do not use it where the chunk's content must match its hash; hash the content.

From testSwarmStorageTasks:

return QString::fromLatin1(chunkHashOf(label).toHex());

Keys (libswarm)

zeroKey(): the all-zero chunk key, for a test that seals and opens its own shards.

• Header: src/libs/libswarm/swarm/testing/Keys.hpp
• Include: #include "swarm/testing/Keys.hpp"
• Tested by: testSwarmNodeTesting
• Never use it for a test of key handling or of secrecy.

From testSwarmNetworkedFetch:

const PublishResult r = net.svcA->publish(hash, /*n*/ 1, /*k*/ 1, zeroKey(), {net.peerB});
QVERIFY(r.allShardsPresent);

L4Nodes

Fully wired L4 nodes for a test that moves shards between peers: L4Node, L4StreamNode, L4NodeOptions, and FakeResourceGate, a scripted resource gate.

• Header: src/libs/libswarm/swarm/testing/L4Nodes.hpp
• Include: #include "swarm/testing/L4Nodes.hpp"
• Tested by: testSwarmNodeTesting
• For whole SwarmService nodes, use Cluster.

From testSwarmL4Driver and testSwarmPublisher:

const L4NodeOptions kPlain{.lazyThreshold = 11};
const L4NodeOptions kGated{.lazyThreshold = 11, .gated = true};
// ...
L4Node pub("pub");

MemoryContentStore

An in-memory IContentStore that keeps what a test puts and binds, and records the calls.

• Header: src/libs/libswarm/swarm/testing/MemoryContentStore.hpp
• Include: #include "swarm/testing/MemoryContentStore.hpp"
• Tested by: testSwarmNodeTesting
• history() and snapshot() return what the test sets. Do not use it to test the history of a real store.

From testSwarmStorageTasks:

auto store = QSharedPointer<MemoryContentStore>::create();
SwarmOpalContext::instance().setStore(store);

StubShardStore

An IShardStore that stores nothing and reports a hosted-bytes figure that the test sets.

• Header: src/libs/libswarm/swarm/testing/StubShardStore.hpp
• Include: #include "swarm/testing/StubShardStore.hpp"
• Tested by: testSwarmNodeTesting
• It keeps the interface's default pin API, so nothing pins. Use a real store for a pin test.

From testSwarmServiceBoot:

auto shardStore = std::make_shared<StubShardStore>(2 * SwarmService::kClaimBytesPerTile);
SwarmService svc(makeStore(dir.path()), makeTransport(QByteArrayLiteral("self-SG")), shardStore);

libswarmnode

Cluster

SwarmCluster: SwarmService nodes over the process-wide fake anonymity transport, which delivers synchronously, each with a content store in a temporary folder of its own.

• Header: src/libs/libswarmnode/swarmnode/testing/Cluster.hpp
• Include: #include "swarmnode/testing/Cluster.hpp"
• Tested by: testSwarmNodeTesting
• For nodes over real BlobCouriers, use TransportLink.

From testSwarmNetworkedFetch:

octomy::swarmnode::testing::SwarmCluster cluster;
PeerId peerA = QByteArrayLiteral("publisher-A");
QSharedPointer<SwarmService> svcA = cluster.add(peerA, false);

TransportLink

Two swarm peers over two BlobCouriers in this process, each with a CommsBlobAnonTransport whose one reachable peer is the other.

• Header: src/libs/libswarmnode/swarmnode/testing/TransportLink.hpp
• Include: #include "swarmnode/testing/TransportLink.hpp"
• Tested by: testSwarmNodeTesting
• A suite that wants its own names for the two sides derives and binds them as references.

From testSwarmCommsTransport:

struct Net : octomy::swarmnode::testing::TransportLink {
    QUuid &uuidA = firstId;
    QUuid &uuidB = secondId;

libtoolchain

ScopedToolchainState

The process-wide toolchain state as it was when the object was made, restored when it goes: the registry's components and toolchains, and the bootstrap service's root folder and compiler.

• Header: src/libs/libtoolchain/toolchain/testing/ScopedToolchainState.hpp
• Include: #include "toolchain/testing/ScopedToolchainState.hpp"
• Tested by: testScopedToolchainState
• Hold one for the whole class, from initTestCase to cleanupTestCase. Do not reset the registry or the service without it.

From testCompilerInterfacePlumbing:

mToolchainState = std::make_unique<toolchain::testing::ScopedToolchainState>();
toolchain::ToolchainBootstrapService::instance().resetForTest();

libtrigger

ScriptedClock

An ITriggerClock whose time the test sets, with an optional calendar.

• Header: src/libs/libtrigger/trigger/testing/ScriptedClock.hpp
• Include: #include "trigger/testing/ScriptedClock.hpp"
• Tested by: testScriptedClock
• Step the clock past an interval; do not wait for wall time.

From testTroveScheduler:

auto clock = QSharedPointer<ScriptedClock>::create();
batch::BatchQueue q;
Scheduler sched(&q, clock);

libwebrequest

LocalHttpServer

A minimal HTTP/1.1 server on 127.0.0.1 that produces wire faults on purpose and keeps a transcript of every request.

• Header: src/libs/libwebrequest/webrequest/testing/LocalHttpServer.hpp
• Include: #include "webrequest/testing/LocalHttpServer.hpp"
• Tested by: testWebRequestTesting. testWebRequestStreaming asserts the client against it.
• It speaks cleartext and trusts every request. Never use it outside a test.

From testWebRequestStreaming:

LocalHttpServer server;
QVERIFY(server.listen());
server.setHandler([](const LocalHttpServer::Request &, LocalHttpServer::Response *r) {
    r->status = 201;

libworktree

Git (libworktree)

Runs git with trove's non-interactive environment, and seedRepo(), a repository with one commit, optionally cloned from a bare origin.

• Header: src/libs/libworktree/worktree/testing/Git.hpp
• Include: #include "worktree/testing/Git.hpp"
• Tested by: testTroveTesting
• For a fixture repository that is not about trove, use Git (libtest). Skip when gitAvailable() is false.

From testWorktree:

const QString repo = seedRepo(tmp.path(), Origin::Bare, QStringLiteral("repo"), nullptr);
QVERIFY2(!repo.isEmpty(), "the seed repository could not be written");

Other shared test support

Test code that is not a header in a testing/ folder, but that a suite uses in place of a copy:

What each entry gives a suite:

• Offscreen QRhi — rhi::createOffscreenRhi(), in src/libs/librhi/rhi/OffscreenBootstrap.hpp.
• Widget and scene screenshots — WidgetScreenshotService and SceneScreenshotService. A widget that the case built is taken with captureLiveWidget(), and a scene widget that the case already built with captureSceneWidget(). Tested by testWidgetScreenshotServiceInMemory and testSceneScreenshotService.
• Block devices — MemoryBlockDevice and FileBlockDevice.
• The exFAT fuzzer — FuzzerEngine, FaultInjectingDevice, SemanticCorruptifier.
• Experiment runs — lab::TestEngine, lab::ParamSet and lab::TestDB. The swarm storage scenarios run on them through swarm/lab/ScenarioRunner (audit QL-650).
• Suite templates — copied when you write a new suite.
• Golden trace — golden-adopt adopts a per-device golden trace that a suite offers with a golden-candidate: line. See Adopt a per-device golden trace.

See also

• Write a test: step 4 chooses the harness.
• Run tests, and the shared binary.