clipbin-desktop/docs/validation.md

Recording foundation validation

The first recording milestone proves the OBS capture, encoder and segment pipeline before the UI is built (SPEC.md, "Implementation validation before building out the UI"). clipbin-probe drives the real clipbin-obs engine and prints key=value lines; the scripts in scripts/validate/ turn them into pass/fail results.

Where	Script	Status
Linux, headless rig (Wayland portal + Xwayland)	scripts/validate/linux-rig.sh	All 9 scenarios pass on devbox (Intel UHD 770, VA-API)
Linux, real session (CachyOS, Hyprland, RX 9070 XT, 1440p60)	scripts/validate/linux-hardware.sh	Passed 2026-10-08 (run 4, dc9cbd6)
Linux desktop automatic sessions (headless Wayland)	scripts/validate/linux-session.sh	Passed 2026-10-08: Recording sidebar, game exit, library reload, window close and active Quit
Windows (Game Capture, NVENC/AMF/QSV)	scripts/validate/windows.ps1	Not run yet; script untested

Probe commands

cargo run -p clipbin-probe -- encoders --verify          # hardware encoders per codec, proven by encoding
cargo run -p clipbin-probe -- record --target monitor     # records; prints status, health and segments
cargo run -p clipbin-probe -- check                       # keyframes, codec, durations, cross-boundary remux

record targets monitor, window or game (Game Capture on Windows; the same as window elsewhere). On Windows and X11, --window takes OBS's window identifier; on Wayland the portal picker chooses and --restore-token reuses an earlier choice. See the top of crates/clipbin-probe/src/main.rs for all options. check needs FFmpeg's ffprobe and ffmpeg on PATH.

Linux headless rig

scripts/rig/headless.sh [--x11] <command> runs a command inside a private D-Bus session with PipeWire, a virtual audio sink, headless Sway on the GPU, xdg-desktop-portal-wlr (auto-selecting the only output) and a 60 FPS test pattern with a tone. --x11 runs the game and command on Sway's Xwayland.

Arch packages: sway xdg-desktop-portal xdg-desktop-portal-wlr xorg-xwayland xorg-xwininfo pipewire pipewire-pulse wireplumber libpulse ffmpeg plus a VA-API driver (intel-media-driver or libva-mesa-driver).

The suite covers:

1. Encoder verification: H.264 must be usable; HEVC and AV1 are reported.
2. Wayland monitor capture in H.264 and HEVC: segments start on keyframes, last 2 s, remux to MP4 across boundaries, decode cleanly, carry the tone.
3. X11 window capture (Xcomposite).
4. Failures that must end in status=failed: PipeWire stream lost, captured X11 window closed, segment directory unwritable, free space below the margin.
5. A portal that refuses the request fails with an actionable reason and never reports Recording.

Segment checks run with --expect-motion, so black or frozen footage (which still makes valid segments) fails.

Linux hardware sign-off

Run scripts/validate/linux-hardware.sh from the Hyprland session. It verifies encoders at 1440p60, records the monitor with each usable codec (the first run shows the portal picker; later runs pass the restore token and ask whether the picker stayed away), records a window, and asks you to close a window mid-recording. Send back report.txt from the output directory.

First hardware run (2026-10-08)

CachyOS 7.2.9, Hyprland, RX 9070 XT, 1440p60, OBS 32.0.4:

• H.264, HEVC and AV1 all encode through VA-API (*_ffmpeg_vaapi_tex). Monitor and window capture recorded keyframe-aligned 2 s segments that remuxed across boundaries with no decode errors.
• The restore token came back on one of three runs, probably because the "Allow a restore token" box in the picker was ticked only once. The script now has a dedicated restore step.
• One run never got a portal selection: the picker either did not appear or was not answered, and the probe stayed in Waiting for capture for 20 s while writing black segments that check passed. check --expect-motion now catches black footage; a refused request now fails at once.
• Closing the captured window (Super+W) is not visible to capture: xdph keeps the PipeWire stream streaming and OBS repeats the window's last frame. See findings.

Second hardware run (2026-10-08)

• The restore token works: the restored run started recording in 2 s without showing the picker.
• Hyprland's portal backend (xdg-desktop-portal-hyprland) wedged in SelectSources during this run and answered nothing afterwards, until restarted; the earlier "no selection" runs were likely the same hang. xdph shows its picker during SelectSources, so a hang there looks the same as a user still choosing. A 10 s timeout tried in afc54b3 would cut users off mid-pick and was removed. Recording stays honestly in Waiting for capture; the app needs a prompt to choose a source (or restart the portal) while it waits.
• The motion check sampled a segment from before capture started (the user took 16 s in the picker); it now uses the segment with the most motion.
• A portrait monitor (Hyprland transform 3) is captured as 1440x2560 and pillarboxed in the 2560x1440 canvas.

Fourth hardware run: pass (2026-10-08, dc9cbd6)

After restarting xdg-desktop-portal-hyprland, every step passed: H.264, HEVC and AV1 through VA-API at 1440p60, monitor and window capture with motion, keyframe-aligned segments that remux across boundaries, and a restored selection that skipped the picker. Closing a window was again not detected (informational, see findings).

Windows

Run scripts/validate/windows.ps1 -GameWindow "<title>:<class>:<exe>" with a game running. It assembles the official OBS 32.0.4 runtime, checks the Game Capture hook signatures, builds the probe, verifies NVENC/AMF/QSV encoders and records Monitor, Game and Window Capture.

Windows status:

• clipbin-core, clipbin-obs and clipbin-probe pass cargo clippy for x86_64-pc-windows-msvc (cross-checked from Linux). Nothing has been linked or run on Windows.
• The GPUI desktop crate has not been checked for Windows; its ring dependency needs the MSVC C toolchain.
• The assembled runtime keeps the six Game Capture hook binaries byte-identical to the release, with their Authenticode signatures present; signature validity has not been checked on Windows.
• The native Steam process bridge (clipbin-core/src/windows_process.rs) uses Toolhelp enumeration, limited process queries and a dedicated thread waiting on the process handle. Registry discovery uses RunningAppID and Apps\<id>\Running; executable paths are matched beneath the app manifest’s installation directory. This code is cross-checked but unvalidated on Windows, including launcher/child-process identity, handle access permissions and exit ordering.
• Unvalidated: atomic metadata replacement and recovery on Windows, settings/data directory paths, desktop close-window survival and clean Quit. GPUI desktop integration cannot be cross-checked from Linux.
• Unvalidated: encoder ids and settings for NVENC, AMF and QSV; Game Capture hooking and its loss detection (the engine expects the source size to drop to 0); WASAPI audio; obs-ffmpeg-mux.exe beside the executable; linking dstr_vprintf through the libobs crate's obs.lib.

Findings for the next milestones

• Plugin loading. libobs always adds default plugin paths: the build-time install prefix on Linux and ../../obs-plugins/64bit relative to the working directory on Windows. Searching would load other OBS installations' plugins or load ours twice, so the engine opens exactly the bundled allowlist (runtime::PLUGINS) by path.
• Segment timestamps. The FFmpeg muxer restarts each segment's video and audio timestamps at zero independently. Concatenating segments naively leaves 22–43 ms video gaps at boundaries. Video is constant frame rate and AAC packets are continuous, so a segment's exact session offset is the count of earlier frames and audio packets; the A/V offset stays within one AAC packet (≤21 ms) and does not drift. The timeline and export work must place segments this way.
• Availability. With 1 s keyframes and 2 s segments, splits happen on schedule every 2.0 s, so the newest complete footage is about 2 s behind plus finalisation: inside the spec's 2–5 s window.
• Honest status needs more than OBS's counters. OBS kept counting bytes into a dead muxer process, and once reported a clean stop after the disk filled. Status therefore requires footage to grow on disk, and recording stops while 1 GB remains so the last segment can be finalised.
• PipeWire source loss. OBS's PipeWire capture keeps its size and renders its last frame after the stream stops, and only logs the stream state. The engine reads that log line (stable for the pinned OBS version). A more robust follow-up is watching the PipeWire node state directly.
• Ending a session when the game closes. With Hyprland's portal, closing a captured window leaves the stream running on its last frame, with no state change or error, so capture cannot tell it from a static window. Monitor capture never sees the game close at all. The session therefore ends from the game's process exiting (SPEC: "Stop/finalise the session when its game exits"); capture loss stays a failure signal, not the normal end:
  • Linux: Steam runs each game under reaper SteamLaunch AppId=<id>, which lives exactly as long as the game (Proton included). Watch that process with a pidfd (pidfd_open + poll) rather than polling /proc.
  • Windows: watch the game's process handle (OpenProcess + WaitForSingleObject), found from Steam's RunningAppID registry value. Game Capture reports size 0 when its game closes; the exit must be seen first so a normal exit finalises the session instead of failing it.
  • Hyprland's IPC does announce window closes, but it is compositor-specific and cannot map a portal stream to a window, so it is not used.
• Nothing is written before capture works. Encoders and the segment output are created only once the source has a picture, so waiting in the picker or for a stalled portal writes no footage.
• Recording size. By default each recording is the captured window's or monitor's own size, set when capture first has a picture (portrait monitors record portrait). If the window is resized mid-session the size stays and the picture is scaled to fit.
• Audio capture failures are not detected yet. When the PulseAudio server was unreachable, recording continued with a silent track.
• Linux runtime dependencies. The Linux OBS build links the system's FFmpeg, PipeWire and other libraries, so it is tied to the distribution it was built on. Packaging (for example Flatpak, or bundling FFmpeg) is an open decision; OBS 32.0 needs mbedtls 3 (mbedtls3 on Arch).
• OBS objects. Scenes and sources are created private (OBS 31+ otherwise registers scenes with the main canvas, which keeps them alive), unknown source types fail instead of becoming placeholders, and shutdown drains libobs's destroy queue first.
• Plugin configuration. libobs is started without a module config path; set one under the app's config directory before plugins that save settings are added.

Automatic desktop sessions

scripts/validate/linux-session.sh [output-dir] runs the desktop and a fake Steam reaper (argv[0]=reaper, SteamLaunch, AppId=480) inside the headless Wayland rig. It verifies Recording status, non-empty segment files and metadata, closes the only window while recording continues, kills the game process, checks finalisation, reopens the app to load its library from disk, captures screenshots and uses Ctrl+Q during a second recording for clean shutdown. The script checks required tools before starting and distinguishes pending status from failed checks. The recovery run also passed with PATH=/usr/bin:/bin without ripgrep installed (2026-10-08); scripts use grep instead. The recording and library screenshots were inspected; storage totals came from real Matroska file sizes. The existing Linux rig also passed all nine scenarios after integration (2026-10-08). Its unwritable-output assertion accepts both OBS Error and EncodeError: the muxer can report either for a permission error.

Session metadata is session.json inside each data-directory session folder; recovery attempts use separate run-NNN folders. Each segment stores its exact start using (total_frames_at_file_changed - 1) / fps. The keyframe triggering the split is already included in OBS’s count but belongs to the new segment: the rig observed 121, 241, 361 at the boundaries of 120-frame segments. The initial file starts at zero. Raw callback counts are preserved in events; the e2e script checks placements exactly against cumulative earlier ffprobe video packet counts within each run. Dropped output frames could make this count drift: each segment records the run’s cumulative dropped-frame count at its completion, and a nonzero count logs an explicit warning. Placement with actual dropped frames still needs validation.

Stopping exactly as a split fires can leave a header-only final Matroska file (the rig observed 747 bytes, no Cluster and no video packets), despite a counted split keyframe and zero reported drops. Finalisation checks the small header prefix for the Cluster ID and deletes an empty tail after output stop. If deletion fails (including a still-open file on Windows), it is marked unavailable in metadata. Footage ends at the last real segment’s end and the remainder is Missing. The offset check verifies every retained segment and explicitly checks any unavailable marker, without treating an empty file as playable. Wall-clock time places each run inside the session and marks capture waiting/recovery as Missing spans. Containers are never summed to derive segment offsets. Metadata is replaced atomically from a synced same-directory temporary file; startup finalises previously active metadata.

Run CLIPBIN_VALIDATE_RECOVERY=1 scripts/validate/linux-session.sh to also kill the portal capture stream and check that bounded automatic recovery resumes into the same session, with a Missing span and a second run directory. This passed in the headless Wayland rig. A deliberately missing runtime path also opened the interface with a persistent failure banner and honoured Quit.

Settings are saved under the config directory, including the portal restore token. Capture selection clears that token and restarts the current session’s capture; automatic recovery retries at five-second intervals, at most three consecutive attempts. A failure stays visible until footage is proven again.

One game is recorded at a time. Startup discovery also records a game already running; simultaneous games are selected by PID order and the next still-live game is detected after the current session ends. Linux Steam names come from library manifests, with App <id> as the fallback. Windows requires a manifest installation path to identify candidate processes and fails honestly if none can be found. X11 desktop session integration and Windows runtime validation remain follow-ups.

Real Steam validated 2026-10-08 on CachyOS/Hyprland (native Steam): Escape from Duckov (AppId 3167020, Proton Experimental under SteamLinuxRuntime_4) runs under reaper SteamLaunch AppId=3167020 -- …/_v2-entry-point … proton waitforexitandrun …/Duckov.exe. Clipbin detected it, named it from its manifest ("Escape from Duckov") and recorded the session. Audio-source failure detection remains limited as described above.

GPUI 0.2.2 ends its Linux event loop when the last window is removed, even without an application close callback. Main keeps the recorder worker alive after that return. Closing the window continues the current game session; when that game exits, it finalises and the process exits. If no session is active, it exits immediately. It stops discovering new games once closed, so there is no indefinitely running invisible app. Reopening the window and persistent background discovery need the future tray integration. Explicit Quit/Ctrl+Q stops and finalises the current recording immediately.

Playback and thumbnails

Playback uses the shared clipbin-media crate (ffmpeg-next 9) without GPUI. One background decoder opens only completed, available segments. Session time comes from Metadata::locate; each file's timestamps are normalised independently. Seeking goes backwards to a keyframe, then decodes forward to the selected frame. A one-frame mailbox and coalesced seeks bound work; overdue frames are decoded but discarded before swscale. Playback threads have lower scheduling priority than recording. Leaving the editor closes its decoder and audio stream.

The editor uploads packed BGRA at its actual pixel size, maintains a 16:9 preview, and releases old GPUI images. Space/K toggle playback; arrows/J/L step one frame. The timeline seeks and drags across files and gaps; completed footage from active sessions becomes browsable on the next 250 ms metadata refresh within the recorder's segment completion delay. Missing/removed footage advances on the wall clock and shows “No footage (not captured / cleaned up)”. There is no live recording preview.

Audio uses cpal and swresample at the output device's rate/channel count, with sample conversion in the device callback. OBS's combined third audio track is selected when three tracks exist; otherwise the first track is used. The queue holds at most 250 ms plus one decoded block. Callback timestamps, including DAC latency, drive the master clock; gaps and mute use the wall clock. Callbacks do no decoding and never wait for the queue lock. Mute and volume are in the editor; an unavailable device is shown explicitly while video remains usable. The Linux rig uses its existing null sink and ALSA's PipeWire plugin as a fallback if cpal's native PulseAudio backend refuses the device. Install/configure pipewire-alsa or an ALSA default routed to PulseAudio on systems without a physical ALSA card.

One low-priority worker produces 320×180 PNG thumbnails beside the first playable session segment and exported MP4s. Its request queue holds 16 paths and duplicate requests coalesce; card discovery and all filesystem work run off the UI thread. clipbin_media::thumbnail::{generate,cache_path,Thumbnails} accept any MP4/MKV. The gradient remains while a thumbnail is pending or footage is unavailable.

scripts/validate/playback-fixture.sh generates independent 1080p60 H.264/AAC segments with one-second keyframes and a three-tone audio fixture. It passed on 2026-10-08: exact frame pixels at frame indices 0/1/59/60/61/119 in every file, real-time boundary crossing, gaps, superseded seeks, pause/seek clock maths, bounded frame placement, resampling 48 kHz to 44.1 kHz and combined-track selection. The ordinary workspace tests skip the generated-media assertions unless CLIPBIN_TEST_MEDIA_DIR points to the fixture; use the script to exercise them.

Windows: the cpal WASAPI dependency cross-check passed, as did the dedicated Windows thread-priority bridge. Core/OBS/probe strict MSVC cross-lint passed. The full media cross-lint is excluded on this Linux machine because the FFmpeg Windows development import libraries are unavailable. On Windows set FFMPEG_DIR to an FFmpeg 7.1 shared development package with MSVC import libraries matching the OBS-bundled avcodec-61/avformat-61/avutil-59/swscale-8/swresample-5 DLLs. Linux links system FFmpeg 8. Actual Windows decoding, WASAPI device latency/seek behaviour, GPUI image presentation and thumbnail caching need a Windows machine. Hardware video decode is not used; HEVC/AV1 software performance and real output devices other than the rig null sink still need measurement.

Final playback validation passed on 2026-10-08, serially with other rigs stopped:

• cargo fmt --all -- --check, strict locked workspace clippy, and locked workspace tests (with CLIPBIN_TEST_MEDIA_DIR=target/playback-fixture).
• env PATH=/usr/bin:/bin:$HOME/.cargo/bin scripts/validate/linux-rig.sh: all nine scenarios. An earlier concurrent run lost its H.264 probe to SIGKILL; the serial rerun passed without timeout or sandbox changes.
• The same PATH with scripts/validate/linux-session.sh: automatic recording, exact segment placement, close-window survival, game exit, library reload and clean Quit. Repeated after the final keyboard integration, also passed.
• scripts/validate/linux-playback.sh target/playback-e2e-final-6: moving preview (11.0% changed pixels inside the preview), active recording with zero reported recording drops, completed-segment availability, real session thumbnail, cross-file drag, exact post-boundary frame/time, cleaned-up gap, zoom, arrow stepping, device audio, half-volume and mute. The audio monitor RMS was 0.08044; the player asserts video time stays within 20 ms of the audio master clock. Screenshots were inspected: sessions-thumbnails.png, editor-playing-1.png, editor-playing-2.png, editor-seek.png, and editor-gap.png in that directory.

The rig runs on an Intel i5-12500 (12 logical CPUs). These are development builds with dependencies at opt-level 2, software video decoding, 1080p60 H.264 source and an 800×450 target (the editor used 740×416). Five serial repetitions of target/debug/examples/playback-check <sessions-root> <id> with scripts/validate/measure-cpu.py, inside the null-sink rig, measured process CPU at 16.7% median / 17.7% p95 of one core (12.2–17.7% range). The filesystem cache was warm; each repetition reopened its decoders. Across presented frames, decode/scale time was 2.39 ms median / 8.80 ms p95; maximum 101.5 ms included opening files and decoding forward on seeks. This is a test pattern with gaps, not a worst-case game or a release-build benchmark. Per-run logs and CPU reports are in target/playback-timing-final; report.json stores the distributions.

During the active-session e2e, the player presented 59.2 frames/s while recording remained at zero reported drops. GPUI upload/construction time was 0.22 ms median / 0.48 ms p95. UI frame intervals (including vsync, not CPU work) were 16.2 ms median / 32.5 ms p95; the one-frame mailbox drops presentation work rather than allowing delay to accumulate. CLIPBIN_TRACE_FRAMES=1 logs decoded video time, audio time, decode/scale work, uploads and actual timeline bounds. The headless e2e restores keyboard focus after virtual-pointer commands and holds virtual keys briefly: instantaneous virtual input can disappear before GPUI/Wayland dispatch. Keyboard actions update the editor entity directly, avoiding a nested window borrow during event dispatch.