Audio Engine

The consolidated audio processing stack. Provides generic processing interfaces (AudioNode, AudioContext), a compiled graph engine (AudioEngine, AudioSchedule), hardware nodes and device subgraphs (AudioOutputNode, AudioInputNode, DeviceSubgraph), channel layouts and channel maps for devices of any channel count, a device management facade (AudioSystem), format negotiation, role-based device assignment, and a shared library of DSP unit templates.

Core interfaces

AudioSample

Type alias for the engine's sample precision.

// mediacore/audio/AudioSample.hpp
using AudioSample = qreal;   // double on most platforms
using ProcessorSample = AudioSample;  // backward-compatible alias

AudioContext

Buffer access for audio processing nodes. The engine assigns a buffer before each node processes via setBuffer().

// mediacore/audio/engine/AudioContext.hpp
class AudioContext {
public:
    size_t sampleCount() const;
    const AudioSample *sourceSamples() const;
    AudioSample *destinationSamples() const;
    void setBuffer(size_t count, AudioSample *samples);
};

Processing is in-place: source and destination point to the same buffer.

AudioNode

Minimum interface for any node in an audio processing graph. Tracker-specific state (parameter schemas, pattern data, serialization) lives in Processor (libbahn), which extends AudioNode.

// mediacore/audio/engine/AudioNode.hpp
class AudioNode {
public:
    virtual void process(AudioContext &context) = 0;
    virtual media::graph::ProcessorCharacter &characteristics() = 0;
    virtual bool prepare(bool ready) = 0;
};

Non-tracker consumers (singularity, voice effects) can implement AudioNode directly without Processor's overhead.

Engine and scheduling

AudioEngine

Holds a compiled AudioSchedule and processes it against an output buffer. Thread pool available for future parallel job execution.

// mediacore/audio/engine/AudioEngine.hpp
class AudioEngine {
public:
    void setSchedule(QSharedPointer<AudioSchedule> schedule);
    QSharedPointer<AudioSchedule> schedule() const;
    void process(AudioContext &context, AudioBuffer &outBuffer);
};

AudioSchedule

Compiled execution plan: an ordered list of AudioJobs created by ScheduleFactory from a node graph, in topological order — a node runs only after every node that feeds it. Owns a unified SIMD-aligned (32-byte) memory block for all intermediate buffers. Output is the average of all ending node buffers. endingNodeIds() names the node at each ending buffer, so a caller that needs one sink's own signal, at its own level, reads that buffer instead of the average. Bahn does this to send each output node to a device of its own.

AudioJob

A single operation in a schedule. Types: ClearBuffer, CopyBuffer, MixBuffer, ProcessBuffer. ProcessBuffer calls AudioNode::process() on each assigned node with the assigned buffer, and records how long each one took.

ComputeGovernor

Turns measured deadline pressure into a render-quality level between 0.15 and 1.0, published on the AudioContext before each block. Pure arithmetic — no Qt type, no allocation, no clock of its own.

// mediacore/audio/engine/ComputeGovernor.hpp
void observe(double elapsedSeconds, size_t frames, double sampleRate);
double quality() const;   // 0.15 .. 1.0

Load is elapsed * sampleRate / frames, so a load of 1.0 means the graph used its whole deadline. Two exponential means drive the response: a fast one decides to give up quality, a slow one decides the overload is over. The asymmetry is deliberate — fall fast, recover slowly — because a governor that reacts to the mean is too slow to prevent a dropout and one that reacts to every spike makes the timbre pump.

Render quality under load

Some nodes can trade detail for time. When the machine is busy, they do.

What you hear. The instruments simplify rather than stutter. A string keeps fewer of its upper partials, a resonant body keeps fewer of its modes, and controls are re-evaluated less often. Brightness goes first, because the top of a partial series is both the most expensive part and the quietest.

What never changes. Pitch and timing. Quality is spent on detail, never on the things that would make the result wrong rather than simpler.

Why you rarely notice it. A partial that leaves the budget is faded out over about 20 ms rather than switched off, because switching off a mode that still holds energy is a step, and a step is a click. Bringing one back needs no fade at all — it holds no energy until the next note.

When a node is paused. If one node alone exceeds the whole block deadline, and quality is already at its floor, and both stay true for twenty consecutive blocks, that node is paused and stops producing sound. It is marked with a red dot in the node graph, and you can restart it from its context menu. This is a last resort: pausing is silence, and every quality reduction before it is only a duller sound.

Audio I/O

AudioOutputNode / AudioInputNode

The only classes that open an audio device. AudioOutputNode is a pure sink: the schedule writes planar samples into its ring, and the node's own I/O thread hands them to QAudioSink as Float or Int16. AudioInputNode does the same in the other direction with QAudioSource. Both rebuild their device after an error through AudioDeviceWatchdog.

// libmediacore/mediacore/audio/engine/AudioOutputNode.hpp
auto out = QSharedPointer<media::audio::AudioOutputNode>::create();
out->configure(device, format);   // the channel count comes from `format`
out->start();

A node configured this way copies a mono block to every channel of its format, drops extra channels and pads missing ones with silence. Code that needs real channel handling uses a DeviceSubgraph instead.

One engine can feed many output nodes. Bahn runs one schedule and gives each output node of the song its own AudioOutputNode, each on its own device.

DeviceSubgraph

One device, one AudioEngine, one schedule and one AudioOutputNode. Configure it with an OutputConfig and it classifies the device, opens it with all of its channels, and converts the schedule's channels to the device's channels through a channel map — see Channel layouts.

media::audio::OutputConfig config;
config.sampleRate  = 48000;
config.graphLayout = media::audio::ChannelLayout::stereo();
config.route       = {2};              // outputs 3–4 on a device with separate outputs

auto subgraph = QSharedPointer<media::audio::DeviceSubgraph>::create();
subgraph->configureOutput(device, config);
subgraph->setSchedule(schedule);       // built with ScheduleFactory(graph, 2)
subgraph->start();
subgraph->processOneBlock(512);        // from a timer

RateBridge

Connects two subgraphs that run at different sample rates or on different clocks. The source endpoint is a sink in the producing schedule, the destination endpoint is a source in the consuming one, and linear interpolation converts the rate.

MediaPlayerSource

Feeds the decoded audio of a QMediaPlayer into a graph. It gives the player a QAudioBufferOutput in the subgraph's rate and a speaker layout, so Qt decodes and resamples the file but never opens a device. node() is the source node for the graph, and playedPositionUs() reports the media time of the audio the graph plays.

Device management facade

AudioSystem

Singleton facade wrapping DeviceManager with audio-specific accessors, format negotiation, and role-based device lookup. Uses deferred initialization to prevent system audio disruption at app startup.

// libaudio/audio/system/AudioSystem.hpp
class AudioSystem : public QObject {
public:
    static AudioSystem& instance();

    // Device enumeration
    QList<QAudioDevice> audioOutputs() const;
    QList<QAudioDevice> audioInputs() const;
    QAudioDevice defaultOutput() const;
    QAudioDevice defaultInput() const;

    // Format negotiation
    QAudioFormat negotiate(const QAudioDevice &device, const AudioFormatHint &hint) const;

    // Role-based device lookup
    QAudioDevice deviceFor(AudioCharacteristic role) const;
    void assign(AudioCharacteristic role, const QAudioDevice &device);

    // Lifecycle
    void initialize();
    bool isInitialized() const;
};

AudioFormatHint / SampleRatePolicy

Consumer specifies preferred format and rate policy:

AudioCharacteristic / AudioCharacteristics

Role-based device assignment (mirrors VisionCharacteristics for video). Persists assignments via QSettings.

Standard roles: PrimaryOutput, PrimaryInput, BanterOutput, BanterInput, IntercomOutput, IntercomInput, MonitorOutput, Custom.

DSP unit templates

Template-based DSP primitives in libmediacore/mediacore/audio/units/. Usable standalone in any hardcoded pipeline or wrapped in Processor nodes for graph-based routing.

Signal generators

Effects

Physical models

All units extend Unit<T> with a single pure virtual: T process().

Channel layouts

A channel layout says how many channels a buffer or a device has, and what each channel is.

ChannelLayout::fromDevice classifies a device. A device whose reported configuration is one of the ten speaker layouts reads as speakers; any other device reads as separate outputs. Separate outputs are only used where the platform sends numbered channels straight to the hardware: Linux with the PipeWire backend, and Windows. On other backends every device reads as speakers, falling back to stereo above 8 channels. The operator can override the result per device with Speakers or SeparateOutputs.

The channel map

ChannelMap::route(graphLayout, deviceLayout, route) builds the gain matrix between a schedule and a device:

A route that does not fit moves down to the last pair. A route change swaps the map without rebuilding the schedule.

Multichannel schedules

ScheduleFactory(graph, channels) gives every buffer in a schedule that many channels. Every node in such a schedule must set ProcessorCharacter::channelAware, which says its process handles all of the context's channels; the factory refuses to build otherwise. A schedule with one channel accepts any node and produces exactly the samples it always did.

Mixer and format conversion

Mixer / MixerFactory

Legacy: no production code uses the Mixer any more. Output formats are converted by AudioOutputNode, and channels by the channel map. The classes remain for reference.

Converts from AudioSample (double, range [-1, 1]) to output hardware format. Supports UInt8, Int16, Int32, Float sample formats and Mono through 7.1 Surround channel configurations.

auto mixer = MixerFactory::createMixer(
    QAudioFormat::Int16,
    QAudioFormat::ChannelConfigStereo,
    MixMode::Expand  // duplicate mono to stereo
);
mixer->mixSamples(engineOutput, hardwareBuffer, byteCount);

SampleConverter

Per-sample format conversion: [-1.0, 1.0] double to target integer/float format.

Processor nodes (libbahn)

Processor extends AudioNode with tracker-specific state: parameter schemas, pattern data delivery (applyRowData), serialization, live parameter binding, pause/performance tracking.

Registered in ProcessorFactory singleton. Current processors:

Generators: Osc, Saw, TB-303, Drum, Sample, Ding, Dribble, Noise, Taiko, KaguraSuzu

Effects: Compressor, Tremolo, Reverb (EAX), Delay, Filter (Biquad), Distortion, Amp, AGC

Utilities: Input, Output, Scope, Average, Recursion, Debug

UI

MediaDeviceSelector

Unified device picker widget replacing all previous device selection UIs. Supports filtering by type (input/output/both), role labels, and live hot-plug refresh via DeviceManager signals.

auto *selector = new MediaDeviceSelector(MediaDeviceSelector::AudioOutputOnly, "Primary Output");
connect(selector, &MediaDeviceSelector::selectedDeviceChanged, this, &MyWidget::onDeviceChanged);

Consumer integration patterns

Graph mode (bahn, singularity)

Build a node graph, compile it to an AudioSchedule with ScheduleFactory, and run it in a DeviceSubgraph. Singularity routes every output — preview, backing track, fanfare, jingles and the settings test tone — through one stereo DeviceSubgraph, with its media players feeding MediaPlayerSource nodes.

Pull mode (banter)

Give an AudioOutputNode a producer callback. Qt's audio thread calls it directly, so there is no ring and no timer. The callback must be real-time safe.

out->setPullCallback([&codec](float *buf, int frames) {
    return codec.fill(buf, frames);
});

Device-only (intercom)

Use AudioSystem for device lookup and format negotiation. Manage QAudioSink/QAudioSource directly.

See Also

• Physical Instruments — the three nodes that spend the quality budget

• Media Architecture -- design philosophy and layer diagram

• Device Management -- DeviceManager, DeviceCache, DeviceMatcher

• Graph Framework -- Graph, GraphNode, Schedule, Buffer

• Video Streaming -- Vision system and streaming protocol