Run Tests

Execute the OctoMY™ test suite to verify builds and catch regressions.

Pro Tip

Run tests with -v2 for verbose output - it shows each check as it runs, making it easier to spot where failures occur. Use -functions to list all available test functions before running.

Test types

OctoMY™ uses several testing approaches:

Running all tests

Build and run

# Build all tests
qbs build -f project.qbs profile:qt6

# Run all tests
qbs run -f project.qbs profile:qt6 -p "*test*"

Using CTest (if configured)

cd default/
ctest --output-on-failure

Running specific tests

Single test

# Run a specific test
qbs run -f project.qbs profile:qt6 -p testCommsChannel

Test pattern

# Run all widget tests
qbs run -f project.qbs profile:qt6 -p "testWidget*"

# Run all integration tests
qbs run -f project.qbs profile:qt6 -p "*Integration*"

Direct execution

# Run test executable directly
./default/testCommsChannel.*/testCommsChannel

# With Qt test options
./default/testCommsChannel.*/testCommsChannel -v2

Qt Test options

Tests use Qt Test framework. Common options:

Examples

# Verbose output
./default/testCommsChannel.*/testCommsChannel -v2

# Output to XML for CI
./default/testCommsChannel.*/testCommsChannel -xml -o results.xml

# Run specific test function
./default/testCommsChannel.*/testCommsChannel testSendReceive

# List available tests
./default/testCommsChannel.*/testCommsChannel -functions

Test output

Successful test

********* Start testing of TestCommsChannel *********
Config: Using QtTest library 6.8.3, Qt 6.8.3
PASS   : TestCommsChannel::initTestCase()
PASS   : TestCommsChannel::testConnect()
PASS   : TestCommsChannel::testSendReceive()
PASS   : TestCommsChannel::testDisconnect()
PASS   : TestCommsChannel::cleanupTestCase()
Totals: 5 passed, 0 failed, 0 skipped
********* Finished testing of TestCommsChannel *********

Failed test

********* Start testing of TestCommsChannel *********
PASS   : TestCommsChannel::initTestCase()
FAIL!  : TestCommsChannel::testSendReceive() Compared values are not the same
   Actual   (received): "hello"
   Expected (expected): "world"
   Loc: [TestCommsChannel.cpp(45)]
PASS   : TestCommsChannel::cleanupTestCase()
Totals: 3 passed, 1 failed, 0 skipped
********* Finished testing of TestCommsChannel *********

Writing tests

Test structure

// test/testMyClass/TestMyClass.hpp
#pragma once

#include <QObject>
#include <QTest>

class TestMyClass : public QObject {
    Q_OBJECT

private slots:
    // Called before all tests
    void initTestCase();

    // Called before each test
    void init();

    // Test functions
    void testConstructor();
    void testMethod();
    void testEdgeCases();

    // Called after each test
    void cleanup();

    // Called after all tests
    void cleanupTestCase();

private:
    MyClass *mInstance{nullptr};
};

Test implementation

// test/testMyClass/TestMyClass.cpp
#include "TestMyClass.hpp"
#include "mylib/MyClass.hpp"

void TestMyClass::initTestCase() {
    // One-time setup
    qDebug() << "Starting TestMyClass";
}

void TestMyClass::init() {
    // Per-test setup
    mInstance = new MyClass();
}

void TestMyClass::testConstructor() {
    QVERIFY(mInstance != nullptr);
    QCOMPARE(mInstance->value(), 0);
}

void TestMyClass::testMethod() {
    mInstance->setValue(42);
    QCOMPARE(mInstance->value(), 42);
}

void TestMyClass::testEdgeCases() {
    mInstance->setValue(-1);
    QVERIFY(mInstance->value() >= 0);  // Should clamp
}

void TestMyClass::cleanup() {
    // Per-test cleanup
    delete mInstance;
    mInstance = nullptr;
}

void TestMyClass::cleanupTestCase() {
    // One-time cleanup
    qDebug() << "Finished TestMyClass";
}

QTEST_MAIN(TestMyClass)
#include "TestMyClass.moc"

Test build file

// test/testMyClass/testMyClass.qbs
import qbs

QtApplication {
    name: "testMyClass"
    type: "application"

    Depends { name: "Qt.test" }
    Depends { name: "libmylib" }

    files: [
        "TestMyClass.cpp",
        "TestMyClass.hpp",
    ]
}

Qt Test macros

Comparison

Control flow

Warnings and info

Data-driven tests

void TestMyClass::testMultipleValues_data() {
    QTest::addColumn<int>("input");
    QTest::addColumn<int>("expected");

    QTest::newRow("zero") << 0 << 0;
    QTest::newRow("positive") << 5 << 10;
    QTest::newRow("negative") << -5 << 0;
}

void TestMyClass::testMultipleValues() {
    QFETCH(int, input);
    QFETCH(int, expected);

    mInstance->setValue(input);
    QCOMPARE(mInstance->doubledValue(), expected);
}

Testing async operations

Signals

void TestComms::testConnect() {
    QSignalSpy spy(mComms, &Comms::connected);

    mComms->connectToHost("localhost", 8124);

    // Wait up to 5 seconds for signal
    QVERIFY(spy.wait(5000));
    QCOMPARE(spy.count(), 1);
}

Timeouts

void TestSlow::testLongOperation() {
    // This test needs more time
    QTest::qWait(100);  // Wait 100ms

    // Or wait for condition
    QTRY_VERIFY_WITH_TIMEOUT(mObject->isReady(), 10000);
}

Test coverage

Generate coverage report

# Build with coverage
qbs build -f project.qbs profile:qt6 \
    qbs.installRoot:coverage \
    cpp.commonCompilerFlags:["--coverage"]

# Run tests
qbs run -f project.qbs profile:qt6 -p "*test*"

# Generate report
lcov --capture --directory . --output-file coverage.info
genhtml coverage.info --output-directory coverage-report

View report

# Open in browser
xdg-open coverage-report/index.html

Continuous integration

GitHub Actions example

name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: recursive

      - name: Install Dependencies
        run: |
          sudo apt-get update
          sudo apt-get install -y qt6-base-dev qbs

      - name: Configure Qbs
        run: |
          qbs setup-toolchains --detect
          qbs setup-qt /usr/bin/qmake6 qt6

      - name: Build
        run: qbs build -f project.qbs profile:qt6

      - name: Run Tests
        run: qbs run -f project.qbs profile:qt6 -p "*test*"

Troubleshooting

Test won't compile

# Check for missing dependencies
qbs resolve -f project.qbs profile:qt6

# Look for Qt.test dependency

Test hangs

# Run with timeout
timeout 60 ./default/testMyClass.*/testMyClass

# Or use Qt Test timeout
./default/testMyClass.*/testMyClass -maxwarnings 100

Flaky tests

For tests that sometimes fail:

void TestUnstable::testUnreliable() {
    // Mark as expected failure if known flaky
    QEXPECT_FAIL("", "Known flaky on CI", Continue);
    QVERIFY(sometimesFailingCondition());
}

Debug failing test

# Run under debugger
gdb ./default/testMyClass.*/testMyClass

# In gdb:
(gdb) run testFailingFunction
(gdb) bt  # Backtrace on crash

Test organization

Current test structure

test/
├── testCommsChannel/      # CommsChannel tests
├── testTrustIconWidget/   # TrustIconWidget tests
├── testPairingActivity/   # PairingActivity tests
├── testAssociate/         # Associate tests
└── tests.qbs              # Test build configuration

Naming conventions

• Test class: TestClassName
• Test file: test/testClassName/TestClassName.cpp
• Test functions: testFeatureName()