The measurement surface for ALL Pulp DSP and audio-pipeline work — read it BEFORE writing or gating DSP, not only when something already sounds wrong. Covers the C++ harness (signal generators, metrics, assertions, RenderScenario, contracts), the offline Audio Doctor (magnitude/frequency response, THD/THD+N, phase/group delay), and their Python sibling the Audio Quality Lab (tools/audio/quality-lab — null residual + alignment, LTAS log-spectral distance, spectral flux/centroid, HNR, Theil-Sen...
Installs into .claude/skills of the current project.
Are you the author of Audio Harness?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/danielraffel-audio-harness)
---
name: audio-harness
description: The measurement surface for ALL Pulp DSP and audio-pipeline work — read it BEFORE writing or gating DSP, not only when something already sounds wrong. Covers the C++ harness (signal generators, metrics, assertions, RenderScenario, contracts), the offline Audio Doctor (magnitude/frequency response, THD/THD+N, phase/group delay), and their Python sibling the Audio Quality Lab (tools/audio/quality-lab — null residual + alignment, LTAS log-spectral distance, spectral flux/centroid, HNR, Theil-Sen drift slope, Kaiser-sinc resampling, license-guarded corpus, regression-net ratchet). TRIGGER on AUTHORING work — "build/design an oscillator/filter/synth/effect", "add a DSP module", "what should the acceptance gate be", "how do I measure aliasing / anti-aliasing / alias floor", "null against a reference", "is this DSP correct", "choose a tolerance", "golden/regression corpus for audio", "measure drift or jitter", "A/B two renders" — AND on DEBUGGING work — "is there sound / no audio / I hear nothing", "does this filter/compressor/synth/delay produce the right signal", "prove the DSP / prove the contract", "measure the frequency response", "what's the THD / is it distorting", "what's the group delay / phase response / measured latency", "magnitude response curve", "render a test tone and assert", "audio regression", "64-frame works but 128 is silent", "sample-rate change pitch-shifted it", "describe what's in this buffer", "audio doctor", "compare before/after a DSP refactor". Reach for this BEFORE hand-rolling any FFT, null test, alias measurement, pitch tracker, or golden-render script — most of it already exists in one of the two lanes. Test/tool layer over HeadlessHost — deterministic, no audio device, no speakers. Off the realtime thread entirely.
---
# Audio harness (observability + validation)
Pulp's agent-first way to turn "I can't hear it" / "does this sound right?" into
**inspectable, deterministic signal evidence** — without a device, speakers, or a
debugger. You are reading this skill because you need to prove, measure, debug, or
regression-guard the audio a Pulp `Processor` emits.
Everything here is the **test/tool layer**, driven by
`pulp::format::HeadlessHost`. Nothing in this skill runs on the realtime audio
thread — measurements analyze buffers that have already left `process()`. The
realtime probe path is a separate concern (see *Roadmap*).
It lives in **two** places, and the split is load-bearing:
- **`tools/audio/analysis/`** — the `pulp::audio-analysis` lib (headers under
`<pulp/audio/analysis/…>`). The pure buffer/file analysis: metrics,
assertions, artifacts, and the FFT spectrum analyzers. It knows nothing about
`Processor`, so the shipped `pulp audio validate …` CLI links it directly
without pulling in any `test/` library.
- **`test/support/`** — the layers that need a `Processor`: signal generators,
`RenderScenario`, contracts, and the scenario-driven Doctor wiring. Test-only;
links the lib above.
The analysis target also owns the deterministic modal-extraction compiler
(`modal_extraction.hpp`). It is offline-only: validate finite,
sample-rate-qualified input, preserve recipe identity, and gate candidates with
confidence/SNR/T60 evidence before emitting a `ModalSpec`; do not link it into
realtime DSP or treat ranked candidates as host-facing parameter state.
Both use namespace `pulp::test::audio`, so the namespace does NOT tell you which
target a symbol comes from — the include path does.
## When to rope this skill in
- "There's no sound / it sounds wrong" — render the path offline and read the
facts (peak/RMS/silence/frequency/NaN) instead of guessing.
- Building or changing a filter / oscillator / compressor / delay / sampler — state
the contract and prove it (frequency response, delay time, decay, bypass-nulls).
- A regression hunt — "64 frames is fine but 128 is silent", "changing the sample
rate chipmunked it", "the test tone toggle produces nothing".
- A DSP refactor — compare old vs new renders (exact / numeric / spectral).
- "How distorted is this?" — THD / THD+N and a harmonic breakdown.
- "What does this lowpass actually do at 8 kHz?" — a magnitude-response curve.
- "What's the group delay / latency of this filter?" — a phase + group-delay
curve, in samples or seconds, per frequency.
## The layering (each layer builds only on the ones below — no back-edges)
```
── pulp::audio-analysis lib (also linked by the pulp CLI) ──
metrics → assertions → artifacts → spectrum
── test/support (needs a Processor) ──
signal generators → scenarios → contracts → doctor
```
| Layer | Header | Target | What it gives you |
|-------|--------|--------|-------------------|
| Metrics | `<pulp/audio/analysis/audio_metrics.hpp>` | lib | `analyze()` → `BufferMetrics`: peak, RMS, DC, NaN/Inf, clip count, silence-run; `estimate_frequency()` (zero-crossing, documented limits); `to_dbfs`; `summarize()` (agent-readable signal description). |
| Assertions | `<pulp/audio/analysis/audio_assertions.hpp>` | lib | `assert_no_nan_inf / not_clipped / silent / not_silent / peak_between / rms_between / frequency_near / null_near / channels_independent` — each returns `CheckResult{passed,message}` with dBFS/Hz/cents messages, never a bare float. |
| Artifacts | `<pulp/audio/analysis/audio_artifacts.hpp>` | lib | `BufferMetrics` → JSON (`schema_version` + provenance) for failing CI/local runs. |
| Spectrum | `<pulp/audio/analysis/audio_spectrum.hpp>` | lib | The FFT-bearing core of the Doctor, over **already-rendered buffers** (no `Processor`): `response_relative_to_input(input, output, …)`, `magnitude_spectrum_curve()`, `measure_thd(signal, …)`, `measure_group_delay(input, output, …)` (phase + group delay), plus the shared `ResponseCurve` / `ThdResult` / `PhaseCurve` / `Window` types. This is what the `pulp audio validate doctor` CLI runs over decoded WAVs — and what you call directly when your audio came from somewhere other than a scenario (e.g. rendered through a real format adapter). |
| Doctor artifacts | `<pulp/audio/analysis/audio_doctor_artifacts.hpp>` | lib | `write_response_artifact()` / `write_thd_artifact()` / `write_phase_artifact()` + the JSON serializers (`phase_curve_to_json`); `kDoctorCurveSchemaVersion`. |
| Latency evidence | `<pulp/audio/analysis/latency_evidence.hpp>` | lib | Delayed-null + impulse-marker latency policies as pure functions (see *Proving reported latency*). |
| Generators | `"support/audio_test_signals.hpp"`, `"support/audio_signal_generators.hpp"` | test | Deterministic stimulus: sine/square/saw, impulse(+train), step, DC, multi-sine, swept sine, seeded white/pink/brown noise, stepped automation + MIDI note scripts. No clocks, no `random_device`. |
| Reverb | `"support/reverb_metrics.hpp"` | test | `t60_schroeder` / `band_t60` (Schroeder backward integration, T20-extrapolated), `echo_density_curve` / `mixing_time_seconds` (Abel & Huang), `range_rms`, `band_energy`. Header-only; measures a reverberant DECAY, which is why it is not in the analysis lib (see *Measuring a reverb* below). |
| Scenarios | `"support/render_scenario.hpp"` | test | `RenderScenario` builder over HeadlessHost (factory, sample rate, block size, channels, duration, input/MIDI/param scripts); `render()` → `ScenarioResult`. `run_matrix()` (SR × block sweeps) + `assert_block_partition_invariant()`. |
| Contracts | `"support/audio_contracts.hpp"` | test | `AudioContract` — a named claim + scenario + accumulated `CheckResult`s; failures read `contract '<name>': ...`. Family helpers `expect_{passthrough,silence_preserved,tone,finite_and_unclipped}`. |
| Doctor (offline) | `"support/audio_doctor.hpp"` | test | The **scenario-driven** entry points: `response_relative_to_input(scenario, …)`, `measure_thd(scenario, …)`, and `measure_group_delay(scenario, …)` (its results gated by `defined_at(hz)` for group delay and the stricter `phase_defined_at(hz)` for phase) synthesize the stimulus, drive the `Processor`, and delegate the math to `audio_spectrum.hpp`. It re-exports that header's result types, so a test that includes it needs nothing else. |
Two traps this table exists to prevent:
- **`response_relative_to_input` and `measure_thd` are each overloaded across
both targets.** The scenario overloads (`audio_doctor.hpp`) drive a
`Processor` and synthesize their own stimulus, overriding the scenario's
input/duration. The buffer overloads (`audio_spectrum.hpp`) analyze audio you
already rendered and synthesize nothing. Same names, different layers — pick
by where your audio comes from.
- **The heavy FFT lives in `pulp-audio-analysis` and is never linked into a
runtime/plugin build.** That is enforced by a CMake target boundary, so a
violation is a link error rather than a review comment. Do not add FFT code
anywhere else.
Read `test/support/README.md` for the authoritative layering contract.
## Run the proofs
### Shared-memory GPU timing evidence
Use `tools/scripts/gpu_audio_p4_evidence.py` for paired shared/staged timing
captures. Its [evidence contract](../../../docs/validation/gpu-audio-p4-evidence.md)
defines provenance, complete trials, transfer counters, and CPU accounting.
Keep raw JSONL and the exact benchmark binary with every summary. Supply
expanded direct compiler flags; response/configuration files and forwarded
frontend options cannot prove the effective Release settings. Output-name
validation uses hidden probes and must never publish empty final artifacts.
Run `python3 tools/scripts/test_gpu_audio_p4_evidence.py` when changing this
tool. A valid capture or passing schema test is not a physical performance
verdict, and one paired row cannot authorize a product default.
```bash
# Build + run the whole harness (Release — Debug is meaningless for DSP timing/levels)
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
tools/ci/governed-build.sh cmake --build build --target \
pulp-test-audio-support pulp-test-render-scenario pulp-test-audio-contracts \
pulp-test-audio-doctor pulp-test-adapter-audio-parity pulp-test-golden \
pulp-test-audio-matrix pulp-test-audio-tone-regression pulp-test-latency-contract
ctest --test-dir build -R 'audio|golden|render|contract|doctor' --output-on-failure
```
That ctest regex matches on test-case NAMES, so it is deliberately wide — it
sweeps in several hundred unrelated `render`/`contract` cases from other
subsystems. It is the right net when you want "did I break anything audio-
shaped"; when you want just the harness, run the built binaries directly
(`./build/test/pulp-test-audio-doctor`) or filter by tag (`-R` won't do tags —
pass `'[doctor]'` to the binary itself).
The `/audio-harness` slash command wraps this. JSON metric/curve artifacts (on failure or
on demand) land under a temp `pulp-audio-metrics/` dir and are INFO-logged.
### Localize a GPU STFT failure with the bounded recipe
Use `gpu-audio.stft.v1` when the question is whether Pulp's offline Dawn STFT
dispatch, readback, or magnitude output disagrees with an independent CPU FFT
oracle:
```bash
mkdir -p "$PWD/artifacts/gpu/stft"
pulp gpu probe --recipe gpu-audio.stft.v1 \
--artifacts "$PWD/artifacts/gpu/stft" --json
```
This is a deterministic, bounded CLI/MCP child-process probe. It never runs in
the audio callback and does not open an audio device. A pass proves this fixed
GPU STFT recipe completed and matched its CPU oracle on the reported adapter;
it does not prove a plugin's realtime scheduling, arbitrary audio content, or
the wider DSP contract. Keep using `RenderScenario`, Audio Doctor, and the
offline audio-harness checks above for those claims.
Interpret the exit/result contract literally: exit 0 is a completed pass, exit
1 is a completed oracle failure, and exit 2 is unavailable or unverified
evidence, never a skip-pass. Add `--negative-control` when validating the probe
itself; the planted FFT mutation must complete GPU work and return the typed
failure that proves the CPU oracle can catch it.
## Measuring a reverb — two traps that make a working engine look broken
Both of these cost real debugging time on the multirate FDN reverb, and both
produce a *plausible* wrong number rather than an obvious one.
**Measure decay IN A BAND, not broadband.** A broadband Schroeder integration
folds the top two octaves into the number, and any fractional-delay read in a
feedback loop — which is what delay modulation requires — costs real
high-frequency energy per pass. On that engine the cost is 19% of the 10 kHz
decay against 3% at 1 kHz, so the broadband T60 moved 10.6% when modulation was
switched on and modulation looked like a decay control. It is not one. Use
`band_t60(ir, fs, 1000.0)`; state the probe frequency in the test.
**A proportional band's energy is not a level — normalize by bin count.** A
+/-12% band at 8 kHz is 32x wider than the same band at 250 Hz, so summing its
bins builds a ~15 dB tilt into any band-to-band comparison and a perfectly flat
tail reads as 16 dB of spread. `band_energy` returns the RMS across its bins (a
density) for exactly this reason. If you hand-roll a band measure, do the same.
A third, cheaper one: **use a band, not a bin, for anything granular.** A
granular pitch shifter spreads its output into grain-rate sidebands, so a single
880 Hz bin under-reports a working octave-up shifter by an order of magnitude
and can read as zero.
## Proving a freeze / hold — steady input cannot see capture bugs
A hold averages a window of past frames, so every bug in WHICH frames it
averaged is invisible on a steady sine: stale and fresh frames look the same.
`FreezeHold`'s rapid-toggle test passed for months over a real defect (a quick
re-freeze averaged ~7/8 frames from before the previous freeze). Two stimuli
make the window observable:
- **Change the tone mid-stream.** Tone A before the first freeze, tone B from
partway through the hold on; the re-freeze must hold B, judged by spectral
peak. Keep a control in the same test: the first hold must still read A
while the input is already B, or the probe proves nothing.
- **Give every frame its own bin** in a synthetic frame test (frame f → bin
10 + f). The set of non-zero held bins then names exactly which frames were
averaged — an age assertion, not a similarity score
(`test/test_freeze_hold.cpp`).
Two more traps from the same work:
- **A phase accumulated in double and converted to float drifts.** A high bin
advances thousands of radians per hop; unwrapped, the float conversion at mix
time loses the stereo offset and the frame-to-frame advance within seconds.
Assert the inter-channel phase offset and the per-frame advance over a long
hold (thousands of frames at a large hop), not the level.
- **Accumulated `1/N` fade steps do not reach 1 for every N** (N = 12 stops at
0.99999988 in float), so "pure hold" never arrives and a residual live leak
stays. Count fade steps as integers and test several N, including 12.
## Copy-this patterns
Describe / debug a render (the "no sound" workflow):
```cpp
auto m = analyze(rendered, 48000.0);
INFO(summarize(m, estimate_frequency(rendered.channel(0), 48000.0)));
REQUIRE(assert_not_silent(m, -60.0).passed); // which stage went silent?
```
State and prove a DSP contract:
```cpp
auto sc = RenderScenario(create_my_lowpass)
.name("mylp.attenuates_8k").sample_rate(48000.0).block_size(128)
.input(Sine{.hz = 8000.0f, .dbfs = -12.0f}).set_param(kCutoff, 200.0f);
AudioContract c("mylp.attenuates_8k", sc);
c.expect(expect_finite_and_unclipped(c.result()))
.expect(assert_block_partition_invariant(sc, {64,128,256}));
REQUIRE(c.verify().passed); // failure says `contract 'mylp.attenuates_8k': ...`
```
Measure it like Plugin Doctor (offline Doctor, scenario-driven):
```cpp
auto curve = response_relative_to_input(sc, {50.0, 8000.0});
REQUIRE(curve.attenuation_db_at(8000.0) >= 20.0); // "drops ≥20 dB at 8 kHz"
auto thd = measure_thd(sc, /*fundamental_hz=*/999.0); // steady bin-coherent sine
// thd.thd_percent(), thd.thd_plus_n, thd.harmonics[...]
// Phase / group delay. ALWAYS gate first: a stopband has no phase to measure,
// so the analyzer reports it undefined and the accessors return NaN rather than
// a number read out of the noise floor. TWO gates, and they are not the same —
// defined_at() qualifies group delay, phase_defined_at() qualifies phase.
auto gd = measure_group_delay(sc, {100.0, 8000.0});
if (gd.defined_at(100.0))
REQUIRE(gd.group_delay_samples_at(100.0) < 8.0); // "≤ 8 samples latency"
if (gd.phase_defined_at(100.0)) // STRICTER — see below
REQUIRE(gd.phase_radians_at(100.0) < 0.0);
// gd.group_delay_seconds_at(hz)
// gd.magnitude_db_rel_peak_at(hz) — peak-relative, NOT the absolute out/in
// ratio that ResponseCurve::magnitude_db_at returns. The two curves use
// different names for this reason; a +12 dB passband reads +12 there and 0 here.
```
Measure audio that did NOT come from a scenario — e.g. rendered through a real
format adapter, or decoded from a WAV. Drop to the buffer overloads in
`audio_spectrum.hpp`; you supply the stimulus AND the render, so nothing is
synthesized for you:
```cpp
#include <pulp/audio/analysis/audio_spectrum.hpp>
const auto impulse = /* your stimulus */;
const auto rendered = /* what came back */;
const double checkpoints[] = {50.0, 1000.0, 8000.0};
auto curve = response_relative_to_input(impulse.view(), rendered.view(),
48000.0, checkpoints, {.fft_length = 16384});
```
`test/test_adapter_audio_parity.cpp` is the worked example of driving the real
adapter: it renders one Processor through `clap_process()` and through
HeadlessHost and requires the bits to match.
**Measure the path the code under test actually runs.** Driving `clap_process()`
is not by itself proof that you measured the adapter. A `clap_process_t proc{}`
leaves `out_events` null, and the whole output-parameter publication phase sits
behind `if (out_events)` — so a test that never sets it renders audio through the
adapter while silently skipping the param path, and stays green no matter how
badly that path breaks. Before trusting an adapter measurement, check that the
lines you care about are reachable from your fixture, then break them on purpose
and confirm your test goes red.
**Two things that will bite you when measuring an adapter path:**
- **THD is blind to anything non-harmonic.** It sums energy at integer multiples
of the fundamental only. Chop each 256-frame block in half and you amplitude-
modulate the tone at 187.5 Hz; the sidebands land *between* harmonics, so THD
reads a serene −170 dB while the signal is visibly destroyed. `thd_plus_n`
counts every non-fundamental bin and reads ~0 dB (100%). Assert THD+N against
a stated floor, not just `thd_plus_n >= thd` — that comparison is a tautology
and proves nothing.
- **Response and THD are both RATIOS, so they cannot see a broadband gain
error.** A wrong-by-0.5 dB adapter path passes every Doctor check. Level is a
null test's job.
**Group delay is a derivative, so the estimator matters.** `measure_group_delay`
uses the ramped-signal Fourier identity (`τ = Re{X_r·conj(X)}/|X|²`, `X_r =
FFT(n·x[n])`) rather than differencing an unwrapped phase curve: it needs no
unwrapping, has no frequency step to tune, and is exact for any impulse response
that fits inside the rectangular analysis window. Budget `fft_length` so the IR
has decayed within it — a truncated IR is the one real error source, and the
analyzer will faithfully report the delay of the *truncated* signal.
**`phase_defined` is stricter than `defined`, and you want the strict one for
phase.** Reported `phase_rad` IS unwrapped, and unwrapping is a walk from DC
upward — so a bin's phase is only as trustworthy as every bin *below* it, and
past a stopband null or a gap in the reference spectrum the accumulated offset
lands on an unresolved 2πk branch. `defined` cannot see that: it asks only
whether *this* bin has energy, which is the whole precondition for group delay
(estimated per bin, never consulting the unwrap) and not nearly enough for
phase. So a bin can be `defined` — carrying a real, correct group delay — while
its phase is unreachable. `phase_defined` is the gate that says so, and
`phase_radians_at()` is NaN wherever it is false. Expect exactly this on a
band-limited or gapped reference: full group delay across the band, phase only
up to the first gap. (The separate aliasing bound — phase unwraps correctly only
while `group_delay < fft_length / 2` — is *not* gated: per bin the analyzer
cannot tell an aliased phase from a true one, so it is your budgeting duty.)
**`defined` is not only a stopband test — it also rejects a near-silent
output.** The estimator divides by `|X|²`, so it has an energy floor of its own
below which a bin has no phase to differentiate. A processor whose output has
collapsed (a dead gain stage, a degenerate filter) can still produce a
*perfectly shaped* impulse response, just at −160 dBFS: its peak-relative
magnitude looks like a clean 0 dB passband, but every bin is under the
estimator's floor. Those bins are `defined = false`. If a group-delay curve
comes back entirely undefined for a processor you expected to measure, check the
output's absolute level before suspecting the analyzer — an all-undefined curve
usually means the processor emitted (almost) nothing, which is the finding.
## Discipline that keeps it trustworthy
- **Release only.** A Debug DSP/UI build mismeasures levels and timing.
- **Do not assume a polynomial is faster than the platform math library.**
`FastMath::exp2` delegates to the standard float implementation because a
cubic-plus-`ldexp` path measured slower on supported Apple hardware while
losing NaN, infinity, overflow, underflow, and gradual-underflow semantics. In
Pulp callbacks, `ScopedFlushDenormals` still governs the finite subnormal range
and may flush it to zero, matching the platform standard in that FP mode. A
replacement needs a repeatable Release benchmark over pitch and audio-rate FM
domains as well as dense numerical and product-level error proofs. Reproduce
the current comparison with:
```bash
cmake -S . -B build-exp2-bench -DCMAKE_BUILD_TYPE=Release \
-DPULP_BENCHMARK=ON -DPULP_ENABLE_GPU=OFF -DPULP_BUILD_EXAMPLES=OFF
tools/ci/governed-build.sh cmake --build build-exp2-bench \
--target pulp-fast-exp2-benchmark --parallel 4
./build-exp2-bench/test/pulp-fast-exp2-benchmark
```
The optional, non-ctest target covers pitch `[-16,16]`, audio-rate FM
`[-8,8]`, and the normal float exponent range `[-126,127]`. Its
`prior_pulp_cubic_exp2` is explicitly benchmark-only historical Pulp code;
observable checksums prevent either lane from being optimized away. Record
repeated medians from a Release binary (`-O3 -DNDEBUG` on the supported Apple
Clang configuration); timing is evidence, never a CI pass/fail threshold.
Advisory evidence snapshot (2026-08-06, non-portable): three consecutive
executions on `Daniels-Mac-Studio` (Apple M3 Ultra, arm64, macOS 26.5.2,
Apple Clang 21.0.0) from the configuration above produced these
median-nanoseconds-per-call ranges:
| Domain | `FastMath::exp2` (std delegate) | Prior Pulp cubic | Prior / current |
|--------|-------------------------------:|-----------------:|----------------:|
| pitch `[-16,16]` | 1.399-1.401 | 1.678-1.730 | 1.200-1.235x |
| FM `[-8,8]` | 1.427-1.436 | 1.684-1.709 | 1.175-1.191x |
| normal `[-126,127]` | 1.401-1.480 | 1.678-1.798 | 1.195-1.215x |
All three executions reproduced identical per-domain checksums for each lane.
These numbers document this host/compiler decision; rerun before drawing a
conclusion on another architecture or toolchain.
- **Vectorized kernels live in `pulp::simd`, and `pulp::signal` can call
them.** `<pulp/simd/simd.hpp>` (target `pulp-simd`) is linked INTERFACE by
`pulp::signal`; its link items are Highway and, on Apple, Accelerate, so a
DSP header may call it without dragging in `pulp::runtime` and its TLS
stack (configure fails if that ever changes). `<pulp/runtime/simd.hpp>`'s
`simd_*` names are inline wrappers kept for existing callers. Gotchas:
- The unqualified names are a compile-time alias of one backend
(`PULP_SIMD_BACKEND`: `auto` = Accelerate on Apple, Highway elsewhere).
All compiled backends stay callable as `pulp::simd::backend::{scalar,
highway, accelerate}`; `pulp-test-simd-parity` runs every kernel on each
against `scalar`. A new kernel lands in all three backends and in that
suite in the same change, then gets a row in the throughput benchmark.
- Tolerance classes (the parity suite states the derivations): one-rounding
elementwise ops and max/min are bit-exact; multiply-adds get one extra
rounding; reductions and FIR correlation get `2 * n * eps * sum|terms|`
because each backend sums in its own order. Accelerate advances
`vDSP_vrampmul`'s gain by repeated addition, so `ramp_mul` error grows
with the index. The measured float FIR null is -115.6 dBFS worst
(Accelerate, 256 taps), floored at -105 dBFS.
- Under flush-to-zero, a scalar `std::clamp` copies a subnormal through
unchanged while a vector min/max returns zero; parity with subnormal
input needs a `numeric_limits::min()`-scale slack, not bit equality.
- A break-confirm that injects an allocation must let the pointer escape
(store it in a volatile global): clang elides a paired `new`/`delete`,
so the probe sees nothing and the control is void.
- Inside `namespace pulp`, an unqualified `simd::` now names Pulp's
kernel namespace, not Apple's `<simd/simd.h>` vector library. Spell
Apple's as `::simd::` (see `fast_math.hpp`); the collision only shows up
in a translation unit that also includes a header reaching
`<pulp/simd/simd.hpp>`, so it can hide until an unrelated include lands.
- Signal headers that call `pulp::simd` are also compiled by hand-listed
builds (WAM/WebCLAP, VCV Rack via forge-modular, Android, live kernels).
Those add `core/simd/include` to their include list and get the header's
inline scalar fallback (`active_backend_name == "inline-scalar"`), with
no library to link.
- Accelerate's `vDSP_conv`/`vDSP_desamp` are not partition-invariant: the
same output can round differently depending on where it falls in the
call. Highway and scalar `correlate` are (fixed per-output order), so a
block-invariance test is bit-exact there and bounded on Accelerate.
- **Processor-wide throughput lives in one benchmark; extend it, do not
fork it.** `pulp-dsp-throughput-benchmark` (`PULP_BENCHMARK=ON`, Release)
times the heavy processors at 48 kHz x {32,128,512} plus the scalar kernel
shapes they reduce to, and `--json PATH` writes `pulp-bench-sections/1` for
`tools/scripts/bench_diff.py baseline.json current.json`. Add a row there
when a DSP change needs a before/after number. Read it correctly:
- *Mean* is the median over repetitions of whole-run time; *worst-block* p99
and max time each block alone, so they carry one timer read per block and
the clock's tick (Apple silicon: ~41.7 ns, i.e. ~1.3 ns/frame at B=32).
Judge sub-ns rows by the mean, and judge realtime headroom by p99, not
mean: a partitioned convolver's mean hides FFT blocks many times its
average.
- Every case feeds a rolling noise window and publishes output through an
asm memory clobber. A row that reads near zero was hoisted, not fast; the
JSON notes flag kernel rows under 0.005 ns/element as suspect.
- `--smoke` exercises every case in about a second; its numbers mean
nothing. `--filter SUBSTR` runs a subset. The JSON records `optimized`
and `ndebug`; a baseline without both true is not comparable.
- The `dsp-throughput-bench.yml` workflow (weekly + dispatch) uploads the
arm64 macOS and x86-64 Linux JSON as artifacts. It is advisory and must
never gain a threshold.
- **Fast trigonometry is accepted per consumer, not per primitive.** Read
`docs/validation/fast-trigonometry.md` before changing a realtime sine/cosine
path. It records the shipped profiles, licensing pin, Release methodology,
actual FM/additive results, rejected Chebyshev/RLIBM/vForce/recurrence/ASM
alternatives, and exact reopen rules. For Apple additive work, enable
`PULP_BENCHMARK`, build `pulp-fast-trig-apple-bank-benchmark`, verify both
cache and target flags say `-O3 -DNDEBUG`, and repeat the order-balanced
consumer matrix. Primitive throughput cannot authorize a call-site sweep,
default change, global fast-math flag, or a quadrature/GPU/control claim.
For a new local-overlay candidate, run
`pulp-fast-trig-benchmark --qualification-only --inputs 16777216` first; this
emits dense error JSON without timing loops. Exit zero means the measurement
completed, not that the candidate passed; compare its fields with the selected
profile's numerical contract. If it misses, stop rather than interpreting
incidental speed.
- **Grade discontinuity correctors against the shipped baseline.** A minBLEP
table needs a reproducible generator and a freshness check, but table
provenance alone does not prove audio quality. Render free-running and synced
product paths, measure their in-band alias residual with the shared projection
analyzer, and gate the improvement over the current polyBLEP path. Exercise
fractional endpoints, collisions, reset, block partitioning, and near-Nyquist
overload separately; a low ordinary-rate alias number says nothing about
bounded event storage.
- **Analyzer Determinism Contract** — every spectral/estimator assertion declares
its stimulus, window (rectangular for a bin-coherent tone or an impulse; Hann
for broadband — a Hann window annihilates an impulse at n=0), warm-up trim,
estimator, seed, sample rate, and tolerance *class*. State them in the test so a
red is a real DSP change, not an analyzer artifact.
- **Named tolerances**, never magic numbers — see the plan's Threshold Policy.
- **Layering is one-way.** Doctor may use scenarios + spectrum; nothing below
scenarios may include `audio_doctor`/`audio_contracts`, and the
`pulp::audio-analysis` lib may never depend on the test-only layers (it has no
`Processor`/scenario knowledge).
- **Prove the assertion can fail.** A green measurement is not evidence until
you have seen it go red. Mutate the thing under test — drop the param event,
zero the output, route the id wrong — rebuild, and confirm the check you are
relying on actually turns red. This is cheap and it repeatedly catches
tolerances that are too loose to detect the bug they were written for: a
200 Hz lowpass check at 8 kHz looks decisive but still passes with the filter
left at its 1000 Hz default, because 8 kHz is ~36 dB down either way. Only the
1 kHz checkpoint separates them.
- **A fixed sample-rate converter does not anti-alias a variable playback
rate.** Tempo maps, varispeed, and other runtime projections can advance the
source position by more than the asset-rate/timeline-rate ratio prepared at
compile time. Derive the reconstruction cutoff from the effective source
frames per output frame, prepare a bounded kernel bank off the audio thread,
and select from it without allocation. Prove the path with both a passband
control and a bright source that aliases under linear point sampling.
- **Compiling a harness TU by hand? Copy `CXX_DEFINES`, not just `CXX_FLAGS`
and `CXX_INCLUDES`.** Both live in the target's `build/**/flags.make`.
Adapter structs are gated on defines (`PULP_CLAP_GUI` adds members to
`PulpClapPlugin` — the header says so at the top), so a TU built without them
disagrees with the prebuilt `libpulp-format.a` about the struct's size and
`clap_init` writes off the end of your object. The symptom is a
`__stack_chk_fail` abort *after* every assertion has already printed PASSED,
and it appears/disappears with unrelated stack layout changes — it will read
as a product bug and it is not one.
## Live inspection (Audio Inspector window) — landed
For *live* signal inspection while a standalone app / hosted graph runs, there is
a separate developer tool window: `pulp::view::AudioInspectorWindow`
(`core/view/.../audio_inspector_window.hpp`). It is a sibling of the layout
inspector, not a tab — open it via its `CommandRegistry` command
`kToggleAudioInspector` (default Cmd/Ctrl+Shift+A, rebindable) or the
`/audio-inspect` slash command. It shows meters (peak/RMS/clip/NaN-Inf/silence),
the observed probe stage, a copied fixed-capacity waveform, channel balance + an
L/R level-match ratio, and a device/runtime summary — all polled once per UI tick
from a realtime-safe `AudioProbeSnapshot` (it never touches the audio thread). It
honestly shows a "no probe" / "stale" state rather than faking zeros. Live data
requires the standalone output-boundary tap, which is gated behind
`PULP_ENABLE_AUDIO_PROBES`. NOTE: the "L/R match" is a level ratio, not a
phase/Pearson correlation (the RT snapshot carries no inter-sample L*R term yet).
This is for *watching what is currently flowing*; controlled-stimulus measurement
(response/THD/etc.) is the offline Doctor above.
### Launch the Audio Inspector in the standalone host
`StandaloneApp::run_with_editor()` wires the inspector to the host's
output-boundary probe automatically (behind `PULP_ENABLE_AUDIO_PROBES`, default
ON for dev/examples). At runtime, toggle it with **Cmd/Ctrl+Shift+A**, or open it
on launch by setting `PULP_AUDIO_INSPECTOR` in the environment:
```bash
# Live: feed a test signal so output is non-silent, then open the inspector.
PULP_AUDIO_INSPECTOR=1 ./build/examples/<app>/pulp-<app>
```
CLI shortcuts (resolve the standalone binary + set the env vars for you):
```bash
pulp run --audio-inspector # open the live inspector window
pulp run --audio-probe-json /tmp/probe.json # headless: dump probe JSON + exit
pulp run --audio-scope-json /tmp/scope.json # headless: dump live sample-window scope JSON + exit
pulp audio scope --input-wav out.wav --json /tmp/scope.json --png /tmp/scope.png
```
For MCP clients, use the existing `pulp-mcp` tools instead of creating a new
MCP server. `pulp_audio_probe_json` is a one-shot wrapper around
`pulp run --audio-probe-json`, accepts optional `target` and `frames`, and
returns scalar probe counters as `structuredContent`. `pulp_audio_scope`
returns `pulp.audio.scope.v1` sample-window acquisition/measurements; live
target mode may open the audio device, while `input_wav` mode is speakerless
offline and can also write a PNG artifact.
Agent triage pattern:
1. Start with `pulp_audio_probe_json` for the target you are debugging.
2. If the tool errors or no JSON is written, report a launch/probe problem, not
a DSP conclusion.
3. If `callbacks == 0`, increase `frames` or inspect standalone startup/device
lifecycle.
4. If callbacks advance but `peak_max == 0` and `rms_max == 0`, treat the
observed output boundary as truly silent; debug routing, input stimulus,
bypass/mute state, graph wiring, or the processor output branch.
5. If `clip_count`/`clipped_blocks` or `nan_inf_count`/`nan_blocks` are non-zero,
prioritize DSP/gain/state initialization bugs.
6. If the live snapshot is healthy but the user says the sound is wrong, switch
to `pulp_audio_scope` / `pulp audio scope --input-wav` for sample-window
facts, or to an offline render + Audio Doctor for THD/response/residual
checks. The scalar probe cannot prove THD, response, phase, latency, or
perceptual quality.
`--audio-probe-json` is the **programmatic readout** for agents: it writes
`output_probe().latest()` (+ the `AudioStats` subset) as a flat JSON object —
`stage`, `sample_rate`, `block_size`, `channel_count`, `sequence_number`,
`peak_max`/`rms_max`, `peak_dbfs`/`rms_dbfs` (null on true silence),
`clip_count`, `nan_inf_count`, `clipped_blocks`, `nan_blocks`,
`silence_run_blocks`, `callbacks`, then exits. The mapping is the pure
`pulp::audio::audio_probe_snapshot_to_json()` helper
(`pulp/audio/audio_probe_json.hpp`); the frame delay reuses `--frames` /
`PULP_FRAMES`. This is the *live* counterpart to the offline
`pulp audio validate` Doctor below. See `docs/guides/audio-inspector.md`.
`PULP_AUDIO_INSPECTOR` also enables the probe's capture ring (sized to the panel
display width), so the inspector paints a live *waveform* and not just meters —
the default probe config is summary-only. The toggle routes through a
shell-owned `CommandRegistry` via `route_global_keys` (the root view's
`on_global_key`), which is independent of the layout inspector's `on_global_click`
(Cmd/Ctrl+I) — both tools coexist on one window without clobbering.
Headless proof: a screenshot run with the inspector open also captures the
inspector's OWN window surface next to the main screenshot:
```bash
# Writes /tmp/x.png (main) AND /tmp/x.audio-inspector.png (the live panel).
PULP_AUDIO_INSPECTOR=1 ./build/examples/<app>/pulp-<app> \
--screenshot /tmp/x.png --screenshot-frame-delay 90
```
The sibling `<stem>.audio-inspector.png` is only written when the inspector
window is visible and the host has GPU capture (`WindowHost::capture_png()`).
## CLI: `pulp audio validate <verb>` (offline, over WAVs / artifact bundles)
> **Do not hand-roll WAV analysis.** If you are about to `soundfile.read()` two
> WAVs and compare them, these verbs already do it. They are registered in
> **`docs/status/tools.yaml`**, whose digest is generated into CLAUDE.md, and a
> PostToolUse hook names them if you start writing the script anyway. The design
> tools learned this the hard way: guidance that exists but never fires at the
> moment of need is the same as no guidance.
The harness analyzers are also reachable from the shipped CLI, nested under the
existing `pulp audio` command (the `model`/`excerpt-find`/`read-bundle` verbs are
untouched). The CLI is **not** tied to a plugin, so it analyzes captured audio
files and stored `audio-run/` artifacts — not live processor instantiation
(controlled-stimulus render stays the test-side `RenderScenario`). It links the
reusable `pulp::audio-analysis` lib (`tools/audio/analysis`, namespace
`pulp::test::audio`), the same file-analysis code the test harness uses; no
`test/` library is linked into the CLI and no FFT leaks into a runtime build.
```bash
# Agent-readable signal summary (peak/RMS/DC/dominant pitch); --json for machine output
pulp audio validate summarize out.wav [--json]
# Offline Audio Doctor: THD/THD+N and/or spectrum magnitude at checkpoints.
# Writes a schema-versioned JSON curve artifact to the temp dir.
pulp audio validate doctor out.wav --thd [--fundamental 1000]
pulp audio validate doctor out.wav --response 100,1000,8000
# Null/spectral diff verdict (exits nonzero past tolerance)
pulp audio validate compare before.wav after.wav [--mode null|spectral] [--tolerance -120]
# Re-check a stored assertions.json (or an audio-run dir holding one); nonzero on failure
pulp audio validate assert audio-run/assertions.json
```
`assertions.json` schema: `{"schema_version":1,"assertions":[{...}]}` where each
entry has a `check` (`not_silent`, `silent`, `no_nan_inf`, `peak_below`,
`frequency_near`), a `file` (relative to the JSON), and the check's named
tolerance (`min_rms_dbfs`, `ceiling_dbfs`, `expected_hz` + `tolerance_cents`,
...). The `/audio-harness` slash command documents these verbs and points at the
offline render path below.
## Offline plugin render (`pulp audio render`)
`pulp audio render --plugin <bundle> --out <file.wav> (--duration-ms <n> |
--duration-frames <n>)` is the scenario-driven counterpart to `validate`: it
loads an explicit plugin bundle through `pulp::host::PluginSlot` (the generic CLI
has no registered factory, so a bundle is the only offline render source),
drives it block-by-block from declarative flags, writes an int16 WAV, and emits
the same `pulp::audio-analysis` metrics JSON as `validate summarize --json`
(`--manifest <file>` / `--json`). No DAW, no audio device. Drive it with
`--input-signal silence|sine:<hz>[,<dbfs>]` or `--input <file.wav>` (used as-is
at `--sample-rate` — no resampling), `--param <id>=<value>[@frame]`, and
`--midi note:<note>,<vel>,<on>[,<off>]`.
**`--param` values are PLAIN domain** (the parameter's native `min..max`), **not
normalized `[0,1]`** — matching `PluginSlot::set_parameter` /
`ParameterEvent::value`. Parameters are delivered **sample-accurately**: the
stepper windows each block's events with per-block sample offsets and the queue
is forwarded straight to `PluginSlot::process`, which every loader applies at the
event offset (CLAP/VST3/AU sample-accurate; LV2 block-rate by its control-port
contract). We do NOT also call `set_parameter` — that would double-apply each
change (once at offset 0, once at its real offset). MIDI is likewise
sample-accurate. (A plugin that reads its own params once per block still steps
at block boundaries — that is the plugin's rate, not the CLI's.) The block
stepper is a deliberate, callback-driven parallel to `OfflineRenderHost::render`
(PluginSlot has no `ProcessContext`, so it can't reuse the core renderer
directly); a block-partition-invariance test guards the two against drift.
## Live capture-to-WAV — two modes, both LANDED
`pulp run` taps the standalone's output boundary into a WAV the offline `pulp
audio validate` verbs read, then exits. Pick the mode by which window you need:
- **`--audio-capture-wav <file>` (earliest, int16).** Dumps the EARLIEST window
after the stream starts (drop-on-full FIFO). Robust for `validate summarize` /
`assert` (presence / level / clip / NaN); the wrong window for steady-state
`doctor` and quantization-limited for `compare`. `--audio-capture-frames <n>`
sets the window.
- **`--audio-capture-rolling <file>` (last-N, float or int24).** Keeps the LAST
(steady-state) window in a `RollingAudioCaptureBuffer` and writes a **float**
WAV (no int16 floor) — the window `doctor` (THD/response) and `compare`
(sub-−96 dBFS residuals) actually want. `--audio-capture-rolling-frames <n>`
sets the window; `--audio-capture-rolling-format int24` swaps the float WAV for
int24 (≈ −144 dBFS floor, smaller, universal DAW compatibility). Uses the hold
protocol so the off-RT materialize is safe while the audio thread is still
appending. One capture mode per invocation (mutually exclusive with
`--audio-inspector` / `--audio-scope-json` / `--audio-capture-wav`).
WAV writing is `pulp::audio::write_wav_file(path, data, WavBitDepth)` —
`Int16` (default overload), `Int24`, or `Float32`.
For streamed sampler assets, build the production mip sidecar with
`pulp audio sampler-mip build <source.wav|source.aiff>`. It accepts strict
ranged WAV and uncompressed AIFF/AIFF-C, uses the sampler's shared 140 dB
decimator, publishes source-and-payload-hash-addressed float32 WAV levels, and
atomically publishes the `.pulpmip` manifest last. `--levels 1|2`,
`--max-source-bytes`, and `--max-output-bytes` bound offline production.
`--json` emits `ok`, `source`, `manifest`, `payloads`, and on failure `error`;
the command exits 0 on success and 1 on failure. Sidecar production does not
waive PulpSampler's runtime limit of one or two channels and at most 192 kHz.
### In-tree render → WAV bridge (no plugin bundle)
`pulp audio render` and `pulp run --audio-capture-*` both need a built plugin
bundle / standalone. To get an **in-tree** `Processor` (or a raw oscillator
wrapped in one) onto disk for the Python lane WITHOUT a bundle, use the
scenario-side bridge in `"support/wav_bridge.hpp"`:
`write_scenario_wav(result, path, WavBitDepth::Float32)` (or the
`write_buffer_wav(buffer, sample_rate, …)` overload) deinterleaves a
`ScenarioResult` / `Buffer<float>` and calls `write_wav_file`. Float32 is the
default so the file carries the exact rendered samples; the deterministic
harness makes the bytes reproducible. The `pulp-osc-render-wav` test tool
(`test/osc_render_wav.cpp`) is the argv surface: it renders the in-tree
`VcoOscillator` (via `"support/osc_wav_scenario.hpp"`) to a WAV
(`--shape/--freq/--sr/--dur-ms/--channels/--drift-cents/--jitter-cents/--bits`),
so an offline Quality-Lab script can analyze in-tree oscillator output with no
bundle in the loop. The pytest `tools/audio/quality-lab/tests/test_osc_wav_bridge.py`
is the end-to-end readability proof (soundfile loads it, fundamental matches).
## Roadmap
The Phase-7 offline-render and live-capture slices have all landed: `pulp audio
render` (offline plugin render, sample-accurate `--param @frame`), `pulp run
--audio-capture-wav` (earliest-window int16), and `pulp run
--audio-capture-rolling` (last-N, float or int24). The live realtime output tap
they read from is gated behind `PULP_ENABLE_AUDIO_PROBES` (see *Live inspection*
above). The harness's offline/live capture surface is feature-complete; further
work is open-ended (e.g. additional analysis verbs), not a tracked backlog.
## Sibling: the Audio Quality Lab (reference-vs-candidate perceptual artifacts)
This skill covers presence / level / THD / response. For **reference-vs-candidate
perceptual artifact** detection — "did this DSP change make it sound *worse*?"
(transient smear, dulling, metallic fizz, graininess) — there is a separate **opt-in**
developer/CI tool, `tools/audio/quality-lab/` (Python, numpy + soundfile). It is
additive: it does NOT change anything here and is never required to run the basic
harness or `ctest`.
- **Install (managed):** `pulp tool install audio-quality-lab` — provisions an isolated
venv under `~/.pulp/tools/` (the same `pulp tool` lane as `ffmpeg`/`uv`/importers;
`pulp tool list` shows it). Then `pulp tool run audio-quality-lab -- <args>`. A plain
`cd tools/audio/quality-lab && pip install -r requirements.txt` venv works identically.
- **Subcommands** (after `python -m quality_lab.cli` or `pulp tool run audio-quality-lab --`):
`run --case {drum,tonal} --degradation <kind>` (synthetic case + listenable clips),
`engine [--input <wav>] --character <c>` (validate the REAL stretch engine, reference-free
on a dry input), `engine-baseline` (regression gate: did an engine change make it worse?),
`corpus list|add` (versioned, license-guarded corpus).
- **Synth-guidance experiment:** `tools/rack/fidelity.py patch.vcv --wav render.wav`
writes only the speaker-bound audio taps, using one shared peak-normalization gain while
refusing a silent source and content-binding original Rack voltage levels in the adjacent
`.fidelity.json` sidecar. Run `quality-lab analyze`
for one-render features, then `quality-lab catalogue-ab off.wav on.wav --expectations
cited.json --experiment experiment.json --holdout-without off-2.wav --holdout-with
on-2.wav`. The content-bound experiment manifest records prompt, inventory digest, model,
a fixed analysis channel, audibility-gate configuration, matched attempt/seed, distinct
generation identity, guidance state, pair identity, exact WAV digests, source RMS, and
each Fidelity-sidecar digest. Every expectation must
name exactly one target/min/max plus a positive tolerance and cite `source.path` with a
page or locator. The A/B turns those physical expectations into normalized errors and
reuses `goodhart_guard`: a working-pair win is not the headline result until an
independent holdout pair agrees, and any regression on another named metric or a source
level drop over 3 dB refuses the candidate. This proves a bounded guidance tranche before admitting a larger corpus; it
does not make reference-free features universal definitions of quality.
- **`corpus.seed()`** ships FIVE synthetic families, not just the two stretch-oriented ones:
`synthetic_drumbreak` / `synthetic_tonalpad` (time-stretch / tonal) plus three oscillator
families (`family: "oscillator"`, `material_class: "synth"`) built from `osc_fixtures.py`'s
corpus-render helpers (`render_static_shapes`, `render_sync_sweep`, `render_tzfm_grid`) —
`synthetic_osc_static_shapes` (sine/saw/square/triangle at a few pitches), `synthetic_osc_sync_sweep`
(`hard_synced_saw` swept across master frequencies), `synthetic_osc_tzfm_grid` (`tzfm_sine` over a
mod-rate/index grid). They gate on the SAME `regression_net` ratchet as any other family — no
bespoke oscillator ratchet — typically via the `added-hf` axis (aliasing reads as added HF energy)
and `dsp.null_residual_db`'s bit-identical clamp (`-160.0`) for the determinism check. See
`tests/test_corpus_osc.py`.
- Aligns a candidate to a reference (onset-map + local cross-correlation), runs the
detectors (`transient_sharpness`, `spectral_centroid`, `hf_fizz`, `spectral_flux`, `hnr` —
tonal noise/roughness via autocorrelation HNR; plus the standalone `stereo_width` for
image-collapse/phase damage on `(N,2)` stereo arrays; and the EXPERIMENTAL `onset_drift`
— timing/groove drift via event-time residual after common-latency removal, advisory-only
until promoted), and
writes a `report.json` with per-onset localization, coverage/confidence, and provenance.
- Credibility: detectors are validated against an *independent* textbook phase vocoder
(`reference_pv.py`) AND the real product engine, not just their own synthetic degradation.
- **`engine`/`engine-baseline` need a built `stretchcli`.** `engine.resolve()` finds it via,
in order: the `PULP_STRETCHCLI` env-path, a `build/examples/offline-stretch/stretchcli`
found by **walking up from the current directory** (so the engine path works even when the
lab is `pulp tool install`-ed into a managed venv, as long as you run it from a Pulp
checkout that built stretchcli), then the package-relative repo build. Absent → a clean,
actionable `skipped` (build command + env-path), never a failure. Build it with
`cmake --build build --target stretchcli`.
- **Maturity gate:** each detector has `maturity` (`experimental`/`beta`/`stable`). An
`experimental` detector runs and reports under the report's `advisory` block but its
`fired`/`low_coverage` are excluded from the `verdict` AND `engine_baseline` — route
every verdict/gate path through `schema.detectors_for_verdict` /
`detectors_for_engine_baseline`, never re-implement the filter. New detectors ship
`experimental`. (Plan: planning/2026-06-29-audio-quality-lab-postmvp-detectors-maturity-gate.md.)
- **Perceptual models** (`perceptual.py`, opt-in, license-fenced): full-reference,
music/general-audio quality predictors, each behind its OWN env-path and each
`skipped`-independently when absent — set only the ones you want. `evaluate()` consults
all: ViSQOL (`PULP_VISQOL_BIN`, Apache-2.0, MOS-LQO), PEAQ (`PULP_PEAQ_BIN`, GPL, ITU-R
BS.1387 ODG), AQUA-Tk (`PULP_AQUATK_BIN`, GPL-3.0, ODG). Results land under
`report["perceptual"]`, ADVISORY ONLY. Deliberately excludes speech/telephony (PESQ,
POLQA) and no-reference neural speech (DNSMOS, NISQA) metrics — wrong domain/contract.
Three non-obvious facts about the ViSQOL lane, each of which silently breaks a run:
- **ViSQOL reads 16-bit PCM WAV only.** Its own `src/wav_reader.cc` accepts format tags
PCM/extensible and rejects any other bit depth with "Error parsing WAV Header -
Expected 16bit samples." The lab writes **float32** (`audio_io.save_wav`), so the
pipeline's own exports are unreadable by ViSQOL as written; `prepare_for_visqol()`
transcodes to 16-bit PCM first. Do not "simplify" that away.
- **ViSQOL exits 0 when a comparison FAILS.** `main.cc` logs the error and continues the
file-pair loop, returning 0 regardless; only argument-parse and init failures return
-1. So the exit code cannot distinguish scored from unscored, and the score line is
the only evidence a comparison happened.
- **Parse stdout ONLY.** ViSQOL prints `MOS-LQO:\t\t<value>` to stdout (outside the
`--verbose` block — `--verbose` adds paths and per-band tables, it does not gate the
score) and routes diagnostics to stderr. A bare-float fallback applied across stderr
turns an incidental number — a duration, a version, a path segment — into a
confident, fabricated MOS for a run that produced no score.
- **Reachability:** the perceptual layer runs only from `pipeline.run_and_export()`.
`compare` and `regression_net` never touch it, which is where most readers look
first.
- **MIR structural oracle** (`mir.py`, opt-in, license-fenced): aubio (`PULP_AUBIO_BIN`,
GPL-3.0) — a *feature extractor*, NOT a quality metric, so it is a SEPARATE layer from
`perceptual.py`. Gives an independent onset-count cross-check under
`report["advisory"]["mir_oracles"]` to non-circularly validate `onset_drift`. ADVISORY,
never a gate/baseline. **License fence for both layers:** GPL tools (PEAQ/AQUA-Tk/aubio)
are reached ONLY via a developer-set env-path over a process boundary, never bundled,
fetched, linked, or added to `tool-registry.json` — so no copyleft obligation attaches
and no runtime warning is needed (consistent with how the lab already treats PEAQ).
- **Advisory reviewer** (`reviewer.py`, opt-in `PULP_QLAB_REVIEWER_CMD` subprocess, `run
--review`): an LLM/multimodal model names artifacts in plain language under
`advisory.reviewers`. ADVISORY ONLY — never changes the verdict/gate; skip-when-absent;
no network by default. Validate with `reviewer.score_agreement` (precision/recall vs the
synthetic answer key) before trusting it.
- **Tuning loop** (`loop.py`, `quality-lab loop`, EXPERIMENTAL): scores candidates with the
gate-participating detectors (experimental detectors excluded), ranks them, and writes
PROPOSAL-ONLY label updates to `corpus/LABEL_PROPOSALS.json` — never `MANIFEST.json`,
never auto-promotes. `goodhart_guard` refuses a candidate that games one detector while
regressing another (normalized Pareto + held-out slice + NEEDS-EAR).
- **Agent compare report** (`compare.py`, `quality-lab compare before.wav after.wav`): the
agent-facing **measure → compare → judge** surface. Level-matches, runs one curated **axis**
(`--profile`), and emits a typed evidence envelope + an action-oriented verdict
(`regression_suspected` / `material_change_detected` / `no_material_change_detected` /
`inconclusive` / `invalid`). Six axes today — a new axis is one `_AXES` registry entry in
`compare.py`, the shared `_measure`/`_verdict` machinery does the rest:
| `--profile` | axis | measures | bad direction (`bad_sign`) |
|-------------|------|----------|----------------------------|
| `tonal-balance` | `tonal_balance` | LTAS spectral-centroid shift | duller (−1) |
| `added-hf` | `added_hf` | band-relative ≥8 kHz fraction ratio (dB) | added fizz (+1) |
| `noise-roughness` | `noise_roughness` | harmonic-to-noise ratio drop (dB) | rougher/noisier (−1) |
| `graininess` | `graininess` | relative spectral-flux increase | grainier (+1) |
| `stereo-width` | `stereo_width` | RMS(side)/RMS(mid) width + interchannel correlation | narrower/collapsed (−1) |
| `transient-integrity` | `transient_integrity` | per-onset attack-smear deficit (onset-aligned) | softer/smeared (+1) |
`noise-roughness` (HNR, pitch range 40–1000 Hz so bass fundamentals count) and `graininess`
(relative `mean_spectral_flux` increase) are meaningful on tonal/sustained material — a
caller-declared contract (the caller picks the profile), carried as a standing summary caveat +
both per-signal scalars in the payload, NOT an automatic tonal/percussive classifier in the
honesty path (which would just be a tunable threshold on the same statistic). Only mathematically
degenerate inputs go `not_applicable` (a flux reference below an epsilon floor — the div-by-zero
edge). **`stereo-width`** is the one axis fed the ORIGINAL 2-channel signal (via
`audio_io.load_wav_multichannel`, the `_Axis.needs_stereo` flag, and the `_measure_stereo` path)
rather than the mono downmix; mono input on either side is `not_applicable`, its metric is
level-invariant (no level-match), and a candidate whose correlation goes negative is flagged out
of phase (mono-incompatible) in the summary. **`transient-integrity`** (the `_Axis.needs_onsets`
flag) is "did my DSP soften/smear the attacks?" — it detects+matches onsets (`align.detect_onsets`
/ `map_onsets`), locks each attack (`dsp.local_align`), and compares the high-band attack rise
(`dsp.attack_rise`, the SAME primitive the `transient_sharpness` detector uses — extracted to dsp
so there is no forked DSP). One-directional (a softening is the regression; a sharper candidate
reads no change); needs onset-bearing material (< 3 matched onsets → `not_applicable`); self-aligns
per onset so `--align` is a no-op and the global sample-domain advisory is suppressed. That takes
compare to **6 of the lab's 7 detectors** reachable from the agent-facing surface (up from 2).
`compare.STEREO_PROFILES` / `ONSET_PROFILES` mark the capability-specific axes, and
`NET_DEFAULT_PROFILES` (the regression net's default) excludes them so they don't emit spurious
`not_applicable` rows on the wrong material — a suite opts in. `compare.MONO_PROFILES` /
`compare.STEREO_PROFILES` split which axes need 2-channel input — the regression net defaults to
the mono set so a mono comparison doesn't emit a spurious not_applicable stereo-width row.
The `added-hf` axis measures the **ratio of the ≥8 kHz energy fraction** (`10·log10(frac_cand /
frac_ref)`, default ±3 dB), NOT the absolute fraction *delta* — the absolute delta is
signal-dependent and effectively blind on a bass-heavy source (an amp render's HF fraction is
~1e-4, so even a clearly harsh addition barely moves it; only the null-residual corroboration
caught it before). The ratio is invariant to a broadband gain / the level-match, so it flags a
real harshness change on a dark source that the absolute delta missed. Both band fractions are
floor-clamped, so a zero-HF reference (fizz on a dark source) or candidate (brickwall low-pass)
gives a large *finite* dB delta, never `inf` → a false `invalid`. It flags HF *loss* as material
too (a negative dB delta) — only *added* fizz is the bad direction. A large EQ move outside the
band (e.g. a big LF shelf) still leaves some residual sensitivity — far less than the absolute
metric, not zero. The threshold is a dB magnitude with its own per-axis range, so the shipped
CLI / MCP pass any positive threshold through to the Python registry (the single source of truth
for the valid range) rather than a hardcoded fraction bound.
**Intent-safe:** `regression_suspected` needs `--reference-role golden` AND a change in the
axis's bad direction (a duller candidate can't trip the `added-hf` axis; added fizz can't trip
it on a `peer` compare); otherwise the change is the neutral `material_change_detected`.
**Advisory:** exits non-zero only on `invalid` (couldn't measure), never for a judgment — the
gate primitive stays `pulp audio validate compare`. Each axis has its own materiality default
(`--threshold` overrides). Report shape (`quality_lab.compare.v1`) is owned by `schema.py`;
every envelope carries `status`/`applicable`/`materiality`/`provenance`. Use this when an agent
must weigh in on a DSP change with cited evidence rather than a bare pass/fail.
**Corroboration + raw comparators (`report["advisory"]`, off-gate).** When the primary
measurement succeeds, the report also carries an `advisory` namespace: a deterministic
`null_residual` raw comparator (level-matched sample-domain residual RMS relative to the
reference, `db_rel_reference`; lower = more identical) and a `corroboration` block. Corroboration
is a **materiality cross-check — NOT a trust score**: it reports only whether that independent,
algorithm-agnostic residual *also* registers a material change (`corroborated` /
`not_corroborated`), under the same level-matched global contract. Both are `maturity:
experimental`, `participates_in_verdict: false` — they NEVER move the verdict. The valuable
signal is disagreement: `not_corroborated` with `axis_exceeds:false, raw_material:true` means a
real sample-domain difference this axis can't see (e.g. a pure delay — try another profile);
`axis_exceeds:true, raw_material:false` means a marginal/phase-only change to treat with more
caution. **That disagreement is now PROMOTED to the headline** (it used to sit buried in the
advisory): a top-level `report["headline_flags"]` (always present, empty when there's nothing to
flag) carries a structured `{flag, detail, expected_for}` — `uncaptured_material_difference` or
`axis_change_without_residual` — and a matching sentence is appended to `summary`. The flag is
STRUCTURED, not prose, precisely so the known false-alarm class is machine-suppressible: every
time/pitch-variant A/B (chorus, phaser, bendr, any modulated effect) legitimately reads
`not_corroborated` forever because the phase-sensitive residual always disagrees with a tonal
axis, so `uncaptured_material_difference` carries `expected_for:["time_variant_processing"]` — a
caller doing time-variant work filters on that rather than treating the flag as noise. It still
NEVER moves the verdict. Length is handled honestly: the candidate is level-matched over the **common region**
and the shorter signal **zero-padded** to the longer length, so a dropped/added tail *with
content* raises the residual on its own (reads material) while trailing *silence* contributes
~0 (stays immaterial) — a truncated render can never masquerade as a near-identity match, and
the sample counts are reported in the comparator's `detail`. The residual is **phase/delay-sensitive and alignment-free by contract**, so it
corroborates *materiality*, never audibility. Cite it — never treat agreement as a confidence
score (the plan's explicit non-goal: agreement ≠ trust). Deliberately NOT built: the GPL aubio /
AGPL Essentia feature-extractor menu — feature extractors are never verdicts, and an env-path
license fence is disproportionate surface for the value; the pure-numpy residual carries the
corroboration story license-free.
**Time-alignment (`--align latency`).** The axes are alignment-free, so a constant delay/offset
(delay plugin, tempo-sampler start offset) doesn't move the tonal verdict but makes the
phase-sensitive null-residual false-alarm `not_corroborated`. `--align latency` estimates one
constant lag (`dsp.estimate_global_lag` — normalized cross-correlation of the attack envelopes),
trims to a common time base (`dsp.apply_lag_trim`), and measures the aligned pair: a pure delay
then reads `no_material_change` + corroborated, and a real change *behind* a delay is measured
cleanly. The outcome is disclosed on the envelope (`alignment: {policy: "fixed-latency-trim",
lag_samples, confidence, applied}`) and the summary. It **refuses** below a confidence floor
(records `policy: "not_aligned"`, measures unaligned) — a wrong alignment is worse than none.
Default `--align none` = zero behavior change. `--align` threads through `compare_files` → CLI →
MCP (`pulp_audio_compare`'s `align` param) → the `/audio-compare` slash command.
The `--align` grammar + per-mode dispatch live in **`alignment.py`** (extracted from compare.py in
Tier 3 / T3.1): `alignment.parse("mode[:param]") → AlignSpec` and `alignment.apply(spec, ref, cand,
sr, *, needs_stereo, needs_onsets) → (ref, cand, record)`, dispatched through a `_HANDLERS` dict so
a new warp class is one entry. `alignment.py` imports `schema` + `dsp` (never `compare` — capability
facts arrive as booleans; no import cycle); the record shape stays in `schema.py`; the prose lives in
`alignment.describe(record)`. Not to be confused with `align.py` (onset primitives) which the
onset-anchored warp handlers consume.
**`--align varispeed:R`** (Tier 3 / T3.1) — a caller-**declared** tape-style speed change (candidate
= reference resampled by R, pitch+time coupled). A resample is EXACT for this class, so the handler
resamples the candidate back to the reference length (`dsp.resample_to_length` — a pure-numpy
Kaiser-windowed-sinc resampler, no scipy) and the WHOLE alignment-free pipeline including the sample
residual applies (clean varispeed → `no_material_change` + corroborated). It **verifies** the
declared ratio against the observed duration ratio (±2%) and **refuses** (`not_aligned`) otherwise —
never resampling to a wrong length. Anti-masking holds (a defect behind the varispeed still flags).
Honest physical limit: a speed-up (R<1) bandlimits to a lower Nyquist, so near-Nyquist energy is
legitimately lost (correct varispeed behavior, matches an ideal brickwall to ~1% — not a resampler
artifact).
**`--align stretch:R`** (Tier 3 / T3.2) — a declared PITCH-PRESERVING time-stretch (PV/bendr). No
exact inverse exists, so the axes measure the UNWARPED pair directly (LTAS centroid / HF fraction /
HNR / width are time-averages → warp-invariant). Two axes are warp-normalized off the ratio, threaded
via the `AlignSpec`: graininess is `warp_aware` (`_Axis.warp_aware`) and measures the candidate flux
at `mean_spectral_flux(..., hop_scale=R)` (`spectral_flux.v2-warp` — else a clean stretch reads a
false "smoother"); corroboration binds to `dsp.ltas_log_spectral_distance_db` (a phase-blind,
length-independent `ltas_residual` comparator) because the sample residual is invalid across a
stretch (the null residual is still emitted, marked not-a-corroborator). `_align_stretch` returns the
pair UNCHANGED after verifying §6.1 (duration within ±3% of R, `R∈[0.25,4]`) and §6.3 that a single
uniform ratio fits — onset material: MAD of the per-onset residual from the `ref_t·R` prediction
≤ 15 ms with coverage ≥ 0.5; sustained: ratio-mapped `smooth_energy_env` correlation ≥ floor — else
REFUSE (`not_aligned`, "non-uniformly warped"). The `_WARP_MODES` set gates the capability
short-circuit (a warp is applied/routed for onset/stereo axes, not skipped like a constant lag);
`_MEASURE_UNWARPED_MODES` = {stretch, pitch} drives the per-axis warp normalization + corroborator
swap (varispeed is NOT in it — it resamples back, so the standard pipeline applies verbatim).
**`--align pitch:S`** (Tier 3 / T3.3) — a declared DURATION-PRESERVING pitch shift (S semitones,
parsed with an optional `st`/`semitones` suffix; `spec.param` is the signed semitones). Time base
unchanged, so the axes measure the pair directly; **tonal-balance** (also `warp_aware`) compensates
the LTAS centroid via `dsp.pitch_compensated_centroid_shift` (a perfect shift moves the centroid by
the pitch ratio → the axis reports deviation-from-expected, `spectral_centroid.v2-pitch`), and the
corroborator shifts the reference LTAS to its expected position first (`dsp.ltas_logfreq_shift`, via
the `ref_shift_ratio` arg on `ltas_log_spectral_distance_db`). `_align_pitch` verifies `|S| ≤ 24 st`
+ duration preserved (a length change ⇒ not a pure pitch shift) and refuses otherwise. The warp-aware
kernels take the accepted `AlignSpec` (`spec=`) and each reads only its own class off `spec.mode`/
`spec.param` — the generalized threading that replaced T3.2's single `flux_hop_scale` param. Known
advisory imperfection: the log-frequency LTAS compensation carries bin-interpolation error on
discrete partials, so a clean shift can read `not_corroborated` (advisory only, never the verdict).
a non-constant warp / DTW stays on the engine path, not compare.
**`--align ratio:auto`** (Tier 3 / T3.4) — ESTIMATE a uniform stretch ratio, double-gated (§6.2):
two independent estimators — the duration ratio and the onset-time slope (`dsp.theil_sen_slope` over
`map_onsets` pairs, ≥ 6) — must agree within 2%, else REFUSE (`not_aligned`, "ratio estimators
disagree" / "needs ≥ 6 matched onsets"). On agreement `_align_ratio_auto` delegates to
`_align_stretch` with the onset-slope estimate (interior evidence) and tags the record
`estimated=True` + both estimator values. The estimated ratio reaches the warp-aware axes via
`alignment.effective_spec(spec, record)`, which resolves a `ratio:auto` spec to `stretch:<estimated>`
so graininess hop-scales off the ACCEPTED transform, not the "auto" request string. Reliable on
onset-bearing uniform EXPANSIONS; compressions (map_onsets drift) + non-uniform + sustained refuse
(conservative — declare stretch:R). This is the last declared-warp class; a non-constant warp / DTW
stays on the engine path, not compare.
**Honesty disclosures (what compare admits it can't see).** Every axis EXCEPT `stereo-width`
mean-**downmixes** to mono, so on those a stereo/spatial change (widener, panner, M/S) is
invisible. When a file was multichannel the report says so on the mono axes:
`provenance.ref_channels`/`cand_channels`, a `downmix` note on the measurement envelope, and a
one-line summary clause — a stereo-widener whose mid is unchanged reads `no_material_change` on
`tonal-balance` but the report never hides that imaging was discarded, and pointing the same pair
at `--profile stereo-width` catches the change directly (the downmix note is correctly suppressed
there, since that axis DID read the stereo image). The **`added-hf`** axis reports
`not_applicable` (→ `inconclusive`) at low sample rates where the ≥8 kHz band collapses to a
degenerate handful of LTAS bins (e.g. sr = 16 kHz is the single Nyquist bin) instead of a
confident "no change" over nothing. A **DC offset** concentrates energy in LTAS bin 0 and drags
the centroid down — reading as false tonal "dulling" — so the advisory carries a `dc_offset`
raw comparator (per-signal means + `present` flag) and, when present, a summary sentence telling
you to high-pass before comparing rather than chase phantom dullness. All three are
presentation/advisory only — like corroboration, they never move the verdict.
- **Shipped surfaces (same measurement, three entry points):** the Python `quality-lab compare`
is the dev-loop surface; **`pulp audio compare <ref> <cand> [--profile …] [--reference-role …]
[--json …]`** is the shipped CLI verb (thin delegator — `tools/cli/cmd_audio_compare.cpp` locates
the opt-in managed tool and forwards, so no DSP links into the CLI); **`pulp_audio_compare`** is
the MCP tool mirroring it (returns the report JSON). All three run the identical axis logic. The
`/audio-compare` slash command wraps the CLI. Install the tool once: `pulp tool install
audio-quality-lab`.
- **Golden-render regression net (`quality_lab.regression_net`, `quality-lab regression-net
--manifest net.json`).** The daily-driver loop: keep a golden render per plugin/preset, `pulp
audio render --plugin …` the candidate, and `compare` across every wired axis, emitting a table
(plugin | profile | verdict | corroboration | flags). **Fail policy is the contract: the fail
signal keys off axis verdicts only — exit 1 iff an axis reports `regression_suspected`; a pair
that couldn't be measured (missing/corrupt render → `invalid`, or a bad manifest) is a separate
ERROR (exit 2), never greenlit as clean; exit 0 = all measured, none regressed. The corroboration
column is informational and never affects the exit code** — the modulated family
(chorus/phaser/flanger/tremolo/ring-mod) is time-variant
so it reads `not_corroborated` forever (the `expected_for:["time_variant_processing"]` flag). This
is advisory reporting attached to a change; gating proper stays `pulp audio validate compare`. The
module is the reusable REFERENCE — a plugin suite (pulp-classic-effects, bendr, GPU NAM) wires its
own renders into `run_net` and commits a portable manifest (paths resolve relative to it). Read
the change class, not the backend: timbral + modulated changes are valid through the net; bendr
*fixed-ratio* A/Bs are valid, but bendr ratio changes / tempo-sampler offsets are time-misaligned
(the residual false-alarms) → use the reference-free engine path instead. See the guide's
"Golden-render regression net" section.
- **Live plugin before/after (measure a real plugin, then judge the change).** `compare` needs two
WAVs; capture them from a live/hosted plugin with the existing steady-state tap, then judge:
```bash
# capture the KNOWN-GOOD baseline (announce audio + cap duration per the local-dev etiquette)
pulp run --plugin before.clap --audio-capture-rolling before.wav --headless # tear down promptly
# …apply the DSP change, rebuild…
pulp run --plugin after.clap --audio-capture-rolling after.wav --headless
pulp audio compare before.wav after.wav --reference-role golden # base = golden
```
Use `--audio-capture-rolling` (last-N steady-state window — the region `compare` wants), NOT
`--audio-capture-wav` (earliest int16 window). `--audio-capture-rolling` opens the live audio
device: announce before launching, cap the wall-clock, and tear down (see the local-dev audio
etiquette in CLAUDE.md). For a fully offline capture of an explicit bundle, `pulp audio render
--plugin <bundle> --out cand.wav …` writes a WAV without a live device — feed that to `compare`
the same way. This is the "measure a plugin with the scope/inspector and compare findings" path.
- Guide: `docs/guides/audio-quality-lab.md`; module map + deferred-detector status:
`tools/audio/quality-lab/README.md`.
### Compare/measure design rationale & non-goals (why we built it this way / what we skipped)
Durable "why / watch-out-for" notes so this isn't re-litigated (rationale, not workflow history):
- **A vertical agent report before a shared measurement record.** The harness + lab + inspector
already measure; the gap was orchestration + a *defended verdict*. A universal `{ref,cand,delta}`
scalar is too lossy to defend a judgment, so measurements are typed *evidence envelopes* and
curves/THD stay first-class. (Two independent adversarial reviews converged on this.)
- **Cross-tool "agreement = trust" is NOT a signal** and is deliberately absent — different
algorithms disagree for legitimate reasons; treating agreement as trust manufactures
overconfidence. If ever added, it reads as "corroborated under a comparable-axis contract."
- **Doctor `--response`/`--thd` are NOT valid on arbitrary before/after WAVs** — `--response` is a
peak-normalized self-spectrum (not a transfer response); THD is gate-grade only for a steady
bin-coherent sine. They need a controlled stimulus, so they're excluded from `compare` and join
later behind stimulus-supplying profiles.
- **Alignment is `not_required` for both current axes** — global LTAS metrics (centroid shift,
HF fraction) are alignment-free; the envelope records the *policy*, we don't build general
alignment for them. A future transients/timing axis will need a real alignment policy first.
- **Axes are a per-profile registry, added one entry at a time** — each axis declares its
`axis`/`tool`/`bad_sign`/`default_threshold`/`kernel`/`summarize`; the shared `_measure` does
applicability + level-match + materiality and `_verdict` does the intent-safe mapping. `bad_sign`
is the sign of the delta that is a regression against a golden reference (−1 = a drop is bad,
+1 = an increase is bad). Only axes on the *same contract* (global, delta-in-(0,1), self-
applicability) belong here — stimulus/alignment-dependent measures do NOT.
- **`regression_suspected` requires `--reference-role golden`** — "worse" assumes the reference is
the known-good baseline; without that, a change is neutral. It ALSO requires the change to be in
the axis's bad direction (`bad_sign`), so a legitimately-brighter or HF-reduced candidate reads
as `material_change_detected`, not a false regression.
- **Axis thresholds are per-axis and unit-bound to (0, 1)** — every current axis's delta is a
dimensionless ratio in (0, 1), so the threshold guard is `0 < t < 1`. A dB-unit axis (e.g. HNR
drop) is deliberately deferred until the threshold validation is made per-axis; don't shove a
dB metric onto the (0,1) contract.
- **Speech/perceptual & feature-extractor scope is bounded** — PESQ/POLQA/DNSMOS/NISQA are speech/
telephony (out of scope for full-reference music); aubio/librosa/Essentia are feature extractors,
not MOS predictors (advisory raw-comparators behind profiles, never verdicts); scalar-only where
the scalar is the calibrated deliverable (PEAQ/AQUA-Tk ODG). GPL/AGPL tools stay behind the
env-path license fence.
- **`audio_io.load_wav` mono-downmixes** — fine for tonal balance; a future stereo-image axis must
NOT use that loader.
- **`detectors/click.py` refuses a REAL BLEP/BLIT oscillator at a fractional period — do not
read that refusal as a click.** The detector's comb self-reference (delay one period, subtract)
nulls a bandlimited oscillator to the interpolator floor ONLY when successive periods are
identical up to the fractional delay. That holds for an exact additive fixture (sinc-periodic),
which is exactly why a click floor measured on a synthetic fixture does NOT transfer to a real
oscillator: on a polyBLEP saw/square, `sr/f0` is fractional, so the discontinuity lands at a
different sub-sample phase every period and the comb leaves a per-edge *approximation* residual
that reads a false −20 to −30 dB "click" even when the render's alias floor is −55 dB or lower.
`detect()` discriminates that regime with two measurements — `template_novelty_db` and
`edge_concentration` — and REFUSES (`low_coverage`, `fired=False`, a *third* refusal
precondition alongside low period-confidence and pitch drift) rather than false-firing.
`template_novelty_db` aligns the comb residual into per-period frames at the fitted fractional
period, builds the median-frame TEMPLATE, and scores the worst period's departure from it over
the median period's — a one-off seam is the single period that does NOT match the recurring
template (a large outlier), while the smear repeats the same per-edge shape every period so it
barely departs. The departure is scored against a HIGH percentile (90th) of the per-period
departures, NOT the median — because at a HALF-INTEGER period (`sr/f0` frac ~0.5) the BLEP edge
alternates sub-sample phase A/B/A/B (true residual period 2P), so HALF the periods depart from
the per-P template: a median denominator reads that recurring half as a lone outlier and
FALSE-FIRES on a clean oscillator, flipping with period-count parity (render length). A high
percentile puts any recurring cluster over ~10% of periods (2P halves, 3P thirds) INTO the
denominator, so only a genuine one-off seam stays a large ratio. It is a per-period RATIO, so it
is **render-length INDEPENDENT** (a longer clip only adds more template-matching periods — it
moves neither the worst nor the percentile): the same near-floor seam that a global statistic
dropped on a multi-second render now fires at any length, at integer AND half-integer periods.
`edge_concentration` (cosine similarity of `|residual|` to `|diff(y)|`) is kept, but ONLY to
split smear (rides the waveform's own edges) from a block-rate zipper (off the edges) — novelty
alone reads a stationary zipper as low-novelty and would wrongly refuse it. A refusal is NOT a pass:
the honest gate for a discontinuous real oscillator is a **frozen-reference null**
(`dsp.null_residual_db` — render the same patch with the offending parameter held frozen), not
the comb self-reference. When you add a click fixture for a new oscillator, grow a REAL polyBLEP
render (`osc_fixtures.py`), not just a synthetic sum — the synthetic-only fixture is precisely
the blind spot that let the false-fire ship.
## Proving reported latency
`expect_reported_latency(result)` measures the delay actually present in a
render and compares it against `Processor::latency_samples()`. This is the
contract for any processor with nonzero latency: the host slides the whole track
by the number the plugin reports, and nothing else checks it. `RenderScenario`
collects the report facts on **every** render (`ScenarioResult::latency`), so a
contract can ask for the proof without re-rendering.
**Do not reach for this on a processor that has no latency.** Most don't: gain,
EQ, compressors without lookahead, waveshapers, delay *effects*, and ordinary
synth paths correctly report zero, and rendering them to prove `0 == 0` is
ceremony. PulpSampler is the sampler exception: enabling its synthetic heritage
two-leg SRC produces derived, profile-dependent latency, while an all-bypassed
profile reports zero. That heritage path belongs in the same measured-latency
proof category as FFT size, partition scheme, lookahead window, or oversampling
ratio because a refactor can silently change the number. User-facing rationale,
including when NOT to use it: `docs/guides/latency-proof.md`.
## The spectral analyzers fail closed — a refusal is the answer, not a bug
`measure_thd`, `measure_aliasing`, and `magnitude_spectrum_curve` throw rather
than return a number they cannot stand behind: silence (no fundamental to be
relative to), a capture shorter than `fft_length` (zero-padding would measure
the truncation edge), a fundamental at or above Nyquist, and — the one that
surprises people — an alias series that never reaches Nyquist. **Size
`num_harmonics` from the sample rate**: the default 64 models no alias site at
all below ~375 Hz at 48 kHz, and that used to read as *clean* on a maximally
aliased saw, because "no aliases among the zero places I looked" is technically
true. Through the CLI a refusal is exit 2, distinct from a failed check.
Every one of those paths previously returned a confident number that passed a
gate. If a refusal surprises you, the input is usually wrong — not the tool.
## Measuring an oscillator's pitch — `estimate_pitch`, not `estimate_frequency`
`estimate_frequency` (audio_metrics.hpp) is a zero-crossing detector and its own
doc disclaims harmonically-dense material. A real oscillator IS dense: on a
bright tone it locks to a harmonic (measured ~1900 cents high on a formant
waveform), so it cannot measure a VCO's drift or a DCO's few-cent quantization
error. Reach for **`estimate_pitch` / `track_pitch`** (`pitch_track.hpp`)
instead: a coarse `magnitude_spectrum_curve` peak seeds a golden-section refine
over the shipped `fit_tone` projection, so it is leakage-free and sub-cent
(<0.002 cent on a clean tone, coherent or not) and FFT-backend-stable. It
**refuses** silence and noise rather than octave-guessing — honor the confidence
gate. `track_pitch` gives the `f0(t)` trajectory; the drift/jitter statistics over
it (Allan deviation, Theil-Sen slope) stay in the Quality Lab per the C++/Python
split. A steep glide needs the window short relative to the sweep.
**Harmonic-dominant material — don't assume the loudest bin is f0.** A rolled-off
or resonant oscillator can carry MORE energy in an upper partial than in the
fundamental, so the loudest FFT bin lands an octave/twelfth/higher above the true
pitch. `estimate_pitch` recovers f0: it ranks the coarse peak against its
subharmonics (to coarse/8) and adopts the lowest root that BOTH explains the
segment (harmonic-comb energy within a margin of the best) AND carries real energy
at its OWN fundamental tooth. That own-tooth test is load-bearing — explained
energy alone cannot tell f0 from f0/N (a near-pure tone lets any subharmonic refine
one of its harmonics onto the tone), and only the TRUE root has a partial sitting
at the root itself. **The tooth floor is LEAKAGE-AWARE, and it has to be:** a fixed
floor is defeated by rectangular-window LSQ leakage from the loud coarse peak into
the f0/2 fit, and that leakage GROWS as the analysis window shortens (energy ~
`(1/(π·Δf·T))²`). With a fixed floor a plain bass saw over the DEFAULT window holds
only a handful of periods and reads a confident octave (or two) DOWN — the ordinary
case, worse than the octave-up it was meant to fix. So a candidate's tooth must
clear `max(absolute-noise-floor, expected coarse-peak leakage at its Δf over this
window)`; a genuine fundamental far below a loud harmonic (Δf large) or over a long
window (T large) still clears it, a leaky saw subharmonic over a few-period window
does not. Corollary: **a genuinely faint fundamental is not resolvable below the
leakage floor at a short window — use a longer analysis window** (the DCO/VCO
quantization/drift fixtures already do, n≈2^15) rather than trusting a low bass note
over a 4096-sample frame. Do NOT "fix" a wrong octave by lowering the floor — it
re-opens the octave trap. **One honest residual, NOT a refusal:** a genuine
missing-fundamental comb (energy only at 2·f0/3·f0, nothing at f0) has no tooth at
f0 to lock, so the estimate stays on the loudest PRESENT partial (2·f0) — the
frequency actually there, never a fabricated virtual f0; likewise the descent stops
at coarse/8, so a 9th-harmonic-dominant signal (rare) reports a lower harmonic.
Both return a real present frequency, never fail closed. (Regressions fixed
2026-07: octave-UP on harmonic-dominant material — old guard stopped at coarse/4 on
a fixed 10% cliff; then octave-DOWN on a plain short-window saw — fixed-floor
leakage, closed with the leakage-aware floor above.)
## Never gate on `detection_floor_db`
It is a ~2σ bound that assumes a **white** residual. Aliases are discrete tones,
so it is wrong exactly where it is most tempting, and asserting it is the
analyzer grading its own homework. Its one legitimate use is deciding whether a
reading is conclusive at all — a gate means something only while the measured
value clears the floor.
**Prove a floor with a negative control**: a fixture whose alias content is zero
by construction must read collapsed, and injected impurities of known level must
come back at that level. Prefer `tone_residual_db` (least-squares projection) to
reading a windowed spectrum near a loud tone — it sidesteps leakage entirely.
Window floors are measured, not assumed: `flat_top` is amplitude-accurate at
about −93 dB with a flat skirt and **cannot** gate −100 dBc; only `kaiser` at
β=14 resolves it, and even then not at bins 1–3, where the DC-removal pedestal
dominates regardless of window.
The schema and both evaluators live in `pulp::audio-analysis`
(`latency_evidence.hpp`) as pure functions over buffers, so the test harness, the
`pulp` CLI, and MCP all produce the same verdict. Add a policy there, never in
`test/support`, or the surfaces drift.
**Feed it a broadband, aperiodic stimulus.** `make_white_noise` for the
delayed-null policy, `make_impulse` for the marker policy.
**The refusals are the feature, not a limitation.** A stimulus that cannot pin
the delay down returns `not_measurable` / `inconclusive`, and an inconclusive
result that was *asked for* fails the gate — an unprovable claim is a failed
claim, never a silent pass. Expect a refusal when:
- the stimulus is silent, or the output is not a delayed copy of the input (the
delayed-null policy needs the processor in a declared identity / bypass /
fully-dry mode — set that up with `set_param`);
- the stimulus is **periodic on an integer sample count**. This is the one that
bites. Tile-a-cell noise with period 100 against a true delay of 512 makes
delays 12, 112, 212… bit-identical; the sweep's argmin walks upward and would
confidently return **12**, falsely accusing an honest processor of misreporting
by 500 samples. The guard catches competing minima and refuses.
Note this is about *integer-sample* periodicity, not tonality — a 440 Hz sine at
48 kHz has a 109.09-sample period, so its delay IS recoverable and is proven;
- the processor's report *moved* mid-render, including moving away and back. The
harness watches `consume_latency_changed_flag()` as well as polling the value,
because a host notified mid-render has already acted on the intermediate number.
(Consuming the flag is safe here: a headless render has no adapter to steal it
from.)
**Read the confidence numbers, not just the verdict.** The evidence carries
`null_depth_db` (how completely the delayed input explained the output) and
`ambiguity_margin_db` (how much worse the best competing delay scored). Measured
references: a pure delay line nulls to the -200 dB floor with a 203 dB margin;
`SpectralFrameEngine` in identity mode — real STFT, real overlap-add rounding —
nulls to -137 dB with a 140 dB margin. Both are nowhere near the -60 dB floor or
the 12 dB ambiguity bar. A pass with a *small* margin technically cleared the bar
and is still a coin flip waiting to happen — treat it as a finding and say so.
**A pass proves self-consistency, not the right value.** The audio is delayed by
exactly what the processor reports — which is all the host needs, and is why it
sounds correct. It does NOT prove the latency is what it should be: double an FFT
size and the processor honestly reports its new, larger delay, passes the
contract, and has silently added 43 ms to every session. `apply_expected_samples`
(C++) / `--latency-expect` (CLI) / `latency_expect` (MCP) pins the intended value
so that drift fails too. It never masks a real mismatch — an audio-vs-report
disagreement is still reported as *that*.
**Dogfood reference.** `test/test_latency_contract.cpp` proves the contract
against `SpectralFrameEngine` (true latency `fft_size + analysis_hop` = 2560),
not just against a delay line, and a companion case deliberately misreports by
one analysis hop to confirm the catch. Copy that shape when covering real DSP.
## Proving an SOS cascade executor
For a runtime cascade, do not use its own DF2T recurrence as the oracle. Compare
impulses against an independently written high-precision direct-form-I reference,
and compare steady-state sine gain against direct complex evaluation of the SOS
polynomials. Cover every supported design family, float and double execution,
caller ordering/internal headroom, coefficient replacement and reset tails,
non-finite/unstable/over-capacity transaction failures, denormal decay, and an
allocation probe around the prepared runtime surface. Build and time the largest
supported product in Release; Debug timing is not performance evidence.
## Sample-region proof surface
For sample-region DSP, pair the declared scalar-kernel contract with a focused
render proof through the canonical graph executor. Use an explicit `UnitDelay`
for feedback, test regular and irregular callback schedules, and record the
independent scalar oracle, source head, executable, artifact, and log hashes. A
registered or generated proof without an executed receipt does not establish
reachability; retained-state edits also need a positive unchanged-identity case
and a negative changed-identity case.