Files
clipbin-desktop/docs/validation.md
T
harry 3588d5b212 docs: add validation guide, hardware and Windows scripts, CI
Documents what the rig proved and what it found, adds the hardware sign-off
script for the Hyprland machine and an (untested) Windows validation script,
and updates CI to build the cached OBS runtime, run tests and lint the Windows
code paths.
2026-10-08 14:31:54 +02:00

6.4 KiB
Raw Blame History

Recording foundation validation

The first recording milestone proves the OBS capture, encoder and segment pipeline before the UI is built (SPEC.md, "Implementation validation before building out the UI"). clipbin-probe drives the real clipbin-obs engine and prints key=value lines; the scripts in scripts/validate/ turn them into pass/fail results.

Where Script Status
Linux, headless rig (Wayland portal + Xwayland) scripts/validate/linux-rig.sh All 9 scenarios pass on devbox (Intel UHD 770, VA-API)
Linux, real session (CachyOS, Hyprland, RX 9070 XT, 1440p60) scripts/validate/linux-hardware.sh Not run yet
Windows (Game Capture, NVENC/AMF/QSV) scripts/validate/windows.ps1 Not run yet; script untested

Probe commands

cargo run -p clipbin-probe -- encoders --verify          # hardware encoders per codec, proven by encoding
cargo run -p clipbin-probe -- record --target monitor     # records; prints status, health and segments
cargo run -p clipbin-probe -- check                       # keyframes, codec, durations, cross-boundary remux

record targets monitor, window or game (Game Capture on Windows; the same as window elsewhere). On Windows and X11, --window takes OBS's window identifier; on Wayland the portal picker chooses and --restore-token reuses an earlier choice. See the top of crates/clipbin-probe/src/main.rs for all options. check needs FFmpeg's ffprobe and ffmpeg on PATH.

Linux headless rig

scripts/rig/headless.sh [--x11] <command> runs a command inside a private D-Bus session with PipeWire, a virtual audio sink, headless Sway on the GPU, xdg-desktop-portal-wlr (auto-selecting the only output) and a 60 FPS test pattern with a tone. --x11 runs the game and command on Sway's Xwayland.

Arch packages: sway xdg-desktop-portal xdg-desktop-portal-wlr xorg-xwayland xorg-xwininfo pipewire pipewire-pulse wireplumber libpulse ffmpeg plus a VA-API driver (intel-media-driver or libva-mesa-driver).

The suite covers:

  1. Encoder verification: H.264 must be usable; HEVC and AV1 are reported.
  2. Wayland monitor capture in H.264 and HEVC: segments start on keyframes, last 2 s, remux to MP4 across boundaries, decode cleanly, carry the tone.
  3. X11 window capture (Xcomposite).
  4. Failures that must end in status=failed: PipeWire stream lost, captured X11 window closed, segment directory unwritable, free space below the margin.
  5. A portal that grants no usable source never reports Recording.

Linux hardware sign-off

Run scripts/validate/linux-hardware.sh from the Hyprland session. It verifies encoders at 1440p60, records the monitor with each usable codec (the first run shows the portal picker; later runs pass the restore token and ask whether the picker stayed away), records a window, and asks you to close a window mid-recording. Send back report.txt from the output directory.

Windows

Run scripts/validate/windows.ps1 -GameWindow "<title>:<class>:<exe>" with a game running. It assembles the official OBS 32.0.4 runtime, checks the Game Capture hook signatures, builds the probe, verifies NVENC/AMF/QSV encoders and records Monitor, Game and Window Capture.

Windows status:

  • clipbin-core, clipbin-obs and clipbin-probe pass cargo clippy for x86_64-pc-windows-msvc (cross-checked from Linux). Nothing has been linked or run on Windows.
  • The GPUI desktop crate has not been checked for Windows; its ring dependency needs the MSVC C toolchain.
  • The assembled runtime keeps the six Game Capture hook binaries byte-identical to the release, with their Authenticode signatures present; signature validity has not been checked on Windows.
  • Unvalidated: encoder ids and settings for NVENC, AMF and QSV; Game Capture hooking and its loss detection (the engine expects the source size to drop to 0); WASAPI audio; obs-ffmpeg-mux.exe beside the executable; linking dstr_vprintf through the libobs crate's obs.lib.

Findings for the next milestones

  • Plugin loading. libobs always adds default plugin paths: the build-time install prefix on Linux and ../../obs-plugins/64bit relative to the working directory on Windows. Searching would load other OBS installations' plugins or load ours twice, so the engine opens exactly the bundled allowlist (runtime::PLUGINS) by path.
  • Segment timestamps. The FFmpeg muxer restarts each segment's video and audio timestamps at zero independently. Concatenating segments naively leaves 22–43 ms video gaps at boundaries. Video is constant frame rate and AAC packets are continuous, so a segment's exact session offset is the count of earlier frames and audio packets; the A/V offset stays within one AAC packet (≤21 ms) and does not drift. The timeline and export work must place segments this way.
  • Availability. With 1 s keyframes and 2 s segments, splits happen on schedule every 2.0 s, so the newest complete footage is about 2 s behind plus finalisation: inside the spec's 2–5 s window.
  • Honest status needs more than OBS's counters. OBS kept counting bytes into a dead muxer process, and once reported a clean stop after the disk filled. Status therefore requires footage to grow on disk, and recording stops while 1 GB remains so the last segment can be finalised.
  • PipeWire source loss. OBS's PipeWire capture keeps its size and renders its last frame after the stream stops, and only logs the stream state. The engine reads that log line (stable for the pinned OBS version). A more robust follow-up is watching the PipeWire node state directly.
  • Audio capture failures are not detected yet. When the PulseAudio server was unreachable, recording continued with a silent track.
  • Linux runtime dependencies. The Linux OBS build links the system's FFmpeg, PipeWire and other libraries, so it is tied to the distribution it was built on. Packaging (for example Flatpak, or bundling FFmpeg) is an open decision; OBS 32.0 needs mbedtls 3 (mbedtls3 on Arch).
  • OBS objects. Scenes and sources are created private (OBS 31+ otherwise registers scenes with the main canvas, which keeps them alive), unknown source types fail instead of becoming placeholders, and shutdown drains libobs's destroy queue first.
  • Plugin configuration. libobs is started without a module config path; set one under the app's config directory before plugins that save settings are added.