500 lines
31 KiB
Markdown
500 lines
31 KiB
Markdown
# Recording foundation validation
|
||
|
||
The first recording milestone proves the OBS capture, encoder and segment
|
||
pipeline before the UI is built (SPEC.md, "Implementation validation before
|
||
building out the UI"). `clipbin-probe` drives the real `clipbin-obs` engine and
|
||
prints `key=value` lines; the scripts in `scripts/validate/` turn them into
|
||
pass/fail results.
|
||
|
||
| Where | Script | Status |
|
||
| --- | --- | --- |
|
||
| Linux, headless rig (Wayland portal + Xwayland) | `scripts/validate/linux-rig.sh` | All 9 scenarios pass on devbox (Intel UHD 770, VA-API) |
|
||
| Linux, real session (CachyOS, Hyprland, RX 9070 XT, 1440p60) | `scripts/validate/linux-hardware.sh` | Passed 2026-10-08 (run 4, dc9cbd6) |
|
||
| Linux desktop automatic sessions (headless Wayland) | `scripts/validate/linux-session.sh` | Passed 2026-10-08: Recording sidebar, game exit, library reload, window close and active Quit |
|
||
| Windows (Game Capture, NVENC/AMF/QSV) | `scripts/validate/windows.ps1` | Not run yet; script untested |
|
||
|
||
## Probe commands
|
||
|
||
```sh
|
||
cargo run -p clipbin-probe -- encoders --verify # hardware encoders per codec, proven by encoding
|
||
cargo run -p clipbin-probe -- record --target monitor # records; prints status, health and segments
|
||
cargo run -p clipbin-probe -- check # keyframes, codec, durations, cross-boundary remux
|
||
```
|
||
|
||
`record` targets `monitor`, `window` or `game` (Game Capture on Windows; the
|
||
same as `window` elsewhere). On Windows and X11, `--window` takes OBS's window
|
||
identifier; on Wayland the portal picker chooses and `--restore-token` reuses
|
||
an earlier choice. See the top of `crates/clipbin-probe/src/main.rs` for all
|
||
options. `check` needs FFmpeg's `ffprobe` and `ffmpeg` on `PATH`.
|
||
|
||
## Linux headless rig
|
||
|
||
`scripts/rig/headless.sh [--x11] <command>` runs a command inside a private
|
||
D-Bus session with PipeWire, a virtual audio sink, headless Sway on the GPU,
|
||
xdg-desktop-portal-wlr (auto-selecting the only output) and a 60 FPS test
|
||
pattern with a tone. `--x11` runs the game and command on Sway's Xwayland.
|
||
|
||
Arch packages: `sway xdg-desktop-portal xdg-desktop-portal-wlr xorg-xwayland
|
||
xorg-xwininfo pipewire pipewire-pulse wireplumber libpulse ffmpeg` plus a VA-API
|
||
driver (`intel-media-driver` or `libva-mesa-driver`).
|
||
|
||
The suite covers:
|
||
|
||
1. Encoder verification: H.264 must be usable; HEVC and AV1 are reported.
|
||
2. Wayland monitor capture in H.264 and HEVC: segments start on keyframes,
|
||
last 2 s, remux to MP4 across boundaries, decode cleanly, carry the tone.
|
||
3. X11 window capture (Xcomposite).
|
||
4. Failures that must end in `status=failed`: PipeWire stream lost, captured
|
||
X11 window closed, segment directory unwritable, free space below the margin.
|
||
5. A portal that refuses the request fails with an actionable reason and
|
||
never reports `Recording`.
|
||
|
||
Segment checks run with `--expect-motion`, so black or frozen footage (which
|
||
still makes valid segments) fails.
|
||
|
||
## Linux hardware sign-off
|
||
|
||
Run `scripts/validate/linux-hardware.sh` from the Hyprland session. It verifies
|
||
encoders at 1440p60, records the monitor with each usable codec (the first run
|
||
shows the portal picker; later runs pass the restore token and ask whether the
|
||
picker stayed away), records a window, and asks you to close a window
|
||
mid-recording. Send back `report.txt` from the output directory.
|
||
|
||
### First hardware run (2026-10-08)
|
||
|
||
CachyOS 7.2.9, Hyprland, RX 9070 XT, 1440p60, OBS 32.0.4:
|
||
|
||
- H.264, HEVC and AV1 all encode through VA-API (`*_ffmpeg_vaapi_tex`).
|
||
Monitor and window capture recorded keyframe-aligned 2 s segments that
|
||
remuxed across boundaries with no decode errors.
|
||
- The restore token came back on one of three runs, probably because the
|
||
"Allow a restore token" box in the picker was ticked only once. The script
|
||
now has a dedicated restore step.
|
||
- One run never got a portal selection: the picker either did not appear or
|
||
was not answered, and the probe stayed in `Waiting for capture` for 20 s
|
||
while writing black segments that `check` passed. `check --expect-motion`
|
||
now catches black footage; a refused request now fails at once.
|
||
- Closing the captured window (Super+W) is not visible to capture: xdph keeps
|
||
the PipeWire stream `streaming` and OBS repeats the window's last frame.
|
||
See findings.
|
||
|
||
### Second hardware run (2026-10-08)
|
||
|
||
- The restore token works: the restored run started recording in 2 s without
|
||
showing the picker.
|
||
- Hyprland's portal backend (xdg-desktop-portal-hyprland) wedged in
|
||
`SelectSources` during this run and answered nothing afterwards, until
|
||
restarted; the earlier "no selection" runs were likely the same hang.
|
||
xdph shows its picker during `SelectSources`, so a hang there looks the same
|
||
as a user still choosing. A 10 s timeout tried in afc54b3 would cut users
|
||
off mid-pick and was removed. Recording stays honestly in `Waiting for
|
||
capture`; the app needs a prompt to choose a source (or restart the portal)
|
||
while it waits.
|
||
- The motion check sampled a segment from before capture started (the user
|
||
took 16 s in the picker); it now uses the segment with the most motion.
|
||
- A portrait monitor (Hyprland transform 3) is captured as 1440x2560 and
|
||
pillarboxed in the 2560x1440 canvas.
|
||
|
||
### Fourth hardware run: pass (2026-10-08, dc9cbd6)
|
||
|
||
After restarting xdg-desktop-portal-hyprland, every step passed: H.264, HEVC
|
||
and AV1 through VA-API at 1440p60, monitor and window capture with motion,
|
||
keyframe-aligned segments that remux across boundaries, and a restored
|
||
selection that skipped the picker. Closing a window was again not detected
|
||
(informational, see findings).
|
||
|
||
## Windows
|
||
|
||
Run `scripts/validate/windows.ps1 -GameWindow "<title>:<class>:<exe>"` with a
|
||
game running. It assembles the official OBS 32.0.4 runtime, checks the Game
|
||
Capture hook signatures, builds the probe, verifies NVENC/AMF/QSV encoders and
|
||
records Monitor, Game and Window Capture.
|
||
|
||
Windows status:
|
||
|
||
- `clipbin-core`, `clipbin-obs` and `clipbin-probe` pass `cargo clippy` for
|
||
`x86_64-pc-windows-msvc` (cross-checked from Linux). Nothing has been linked
|
||
or run on Windows.
|
||
- The GPUI desktop crate has not been checked for Windows; its `ring`
|
||
dependency needs the MSVC C toolchain.
|
||
- The assembled runtime keeps the six Game Capture hook binaries byte-identical
|
||
to the release, with their Authenticode signatures present; signature
|
||
validity has not been checked on Windows.
|
||
- The native Steam process bridge (`clipbin-core/src/windows_process.rs`) uses Toolhelp enumeration, limited process queries and a dedicated thread waiting on the process handle. Registry discovery uses `RunningAppID` and `Apps\<id>\Running`; executable paths are matched beneath the app manifest’s installation directory. This code is cross-checked but unvalidated on Windows, including launcher/child-process identity, handle access permissions and exit ordering.
|
||
- Unvalidated: atomic metadata replacement and recovery on Windows, settings/data directory paths, desktop close-window survival and clean Quit. GPUI desktop integration cannot be cross-checked from Linux.
|
||
- Unvalidated: encoder ids and settings for NVENC, AMF and QSV; Game Capture
|
||
hooking and its loss detection (the engine expects the source size to drop
|
||
to 0); WASAPI audio; `obs-ffmpeg-mux.exe` beside the executable; linking
|
||
`dstr_vprintf` through the libobs crate's `obs.lib`.
|
||
|
||
## Findings for the next milestones
|
||
|
||
- **Plugin loading.** libobs always adds default plugin paths: the build-time
|
||
install prefix on Linux and `../../obs-plugins/64bit` relative to the
|
||
working directory on Windows. Searching would load other OBS installations'
|
||
plugins or load ours twice, so the engine opens exactly the bundled
|
||
allowlist (`runtime::PLUGINS`) by path.
|
||
- **Segment timestamps.** The FFmpeg muxer restarts each segment's video and
|
||
audio timestamps at zero independently. Concatenating segments naively leaves
|
||
22–43 ms video gaps at boundaries. Video is constant frame rate and AAC
|
||
packets are continuous, so a segment's exact session offset is the count of
|
||
earlier frames and audio packets; the A/V offset stays within one AAC packet
|
||
(≤21 ms) and does not drift. The timeline and export work must place
|
||
segments this way.
|
||
- **Availability.** With 1 s keyframes and 2 s segments, splits happen on
|
||
schedule every 2.0 s, so the newest complete footage is about 2 s behind
|
||
plus finalisation: inside the spec's 2–5 s window.
|
||
- **Honest status needs more than OBS's counters.** OBS kept counting bytes
|
||
into a dead muxer process, and once reported a clean stop after the disk
|
||
filled. Status therefore requires footage to grow on disk, and recording
|
||
stops while 1 GB remains so the last segment can be finalised.
|
||
- **PipeWire source loss.** OBS's PipeWire capture keeps its size and renders
|
||
its last frame after the stream stops, and only logs the stream state. The
|
||
engine reads that log line (stable for the pinned OBS version). A more robust
|
||
follow-up is watching the PipeWire node state directly.
|
||
- **Ending a session when the game closes.** With Hyprland's portal, closing a
|
||
captured window leaves the stream running on its last frame, with no state
|
||
change or error, so capture cannot tell it from a static window. Monitor
|
||
capture never sees the game close at all. The session therefore ends from
|
||
the game's process exiting (SPEC: "Stop/finalise the session when its game
|
||
exits"); capture loss stays a failure signal, not the normal end:
|
||
- Linux: Steam runs each game under `reaper SteamLaunch AppId=<id>`, which
|
||
lives exactly as long as the game (Proton included). Watch that process
|
||
with a pidfd (`pidfd_open` + poll) rather than polling `/proc`.
|
||
- Windows: watch the game's process handle (`OpenProcess` +
|
||
`WaitForSingleObject`), found from Steam's `RunningAppID` registry value.
|
||
Game Capture reports size 0 when its game closes; the exit must be seen
|
||
first so a normal exit finalises the session instead of failing it.
|
||
- Hyprland's IPC does announce window closes, but it is compositor-specific
|
||
and cannot map a portal stream to a window, so it is not used.
|
||
- **Nothing is written before capture works.** Encoders and the segment
|
||
output are created only once the source has a picture, so waiting in the
|
||
picker or for a stalled portal writes no footage.
|
||
- **Recording size.** By default each recording is the captured window's or
|
||
monitor's own size, set when capture first has a picture (portrait monitors
|
||
record portrait). If the window is resized mid-session the size stays and
|
||
the picture is scaled to fit.
|
||
- **Audio capture failures are not detected yet.** When the PulseAudio server
|
||
was unreachable, recording continued with a silent track.
|
||
- **Linux runtime dependencies.** The Linux OBS build links the system's
|
||
FFmpeg, PipeWire and other libraries, so it is tied to the distribution it
|
||
was built on. Packaging (for example Flatpak, or bundling FFmpeg) is an open
|
||
decision; OBS 32.0 needs mbedtls 3 (`mbedtls3` on Arch).
|
||
- **OBS objects.** Scenes and sources are created private (OBS 31+ otherwise
|
||
registers scenes with the main canvas, which keeps them alive), unknown
|
||
source types fail instead of becoming placeholders, and shutdown drains
|
||
libobs's destroy queue first.
|
||
- **Plugin configuration.** libobs is started without a module config path;
|
||
set one under the app's config directory before plugins that save settings
|
||
are added.
|
||
|
||
## Automatic desktop sessions
|
||
|
||
`scripts/validate/linux-session.sh [output-dir]` runs the desktop and a fake
|
||
Steam reaper (`argv[0]=reaper`, `SteamLaunch`, `AppId=480`) inside the headless
|
||
Wayland rig. It verifies Recording status, non-empty segment files and metadata,
|
||
closes the only window while recording continues, kills the game process,
|
||
checks finalisation, reopens the app to load its library from disk, captures
|
||
screenshots and uses Ctrl+Q during a second recording for clean shutdown.
|
||
The script checks required tools before starting and distinguishes pending
|
||
status from failed checks. The recovery run also passed with `PATH=/usr/bin:/bin`
|
||
without ripgrep installed (2026-10-08); scripts use grep instead.
|
||
The recording and library screenshots were inspected; storage totals came from
|
||
real Matroska file sizes. The existing Linux rig also passed all nine scenarios
|
||
after integration (2026-10-08). Its unwritable-output assertion accepts both
|
||
OBS Error and EncodeError: the muxer can report either for a permission error.
|
||
|
||
Session metadata is `session.json` inside each data-directory session folder;
|
||
recovery attempts use separate `run-NNN` folders. Each segment stores its exact
|
||
start using `(total_frames_at_file_changed - 1) / fps`. The keyframe triggering
|
||
the split is already included in OBS’s count but belongs to the new segment:
|
||
the rig observed 121, 241, 361 at the boundaries of 120-frame segments. The
|
||
initial file starts at zero. Raw callback counts are preserved in events; the
|
||
e2e script checks placements exactly against cumulative earlier ffprobe video
|
||
packet counts within each run. Dropped output frames could make this count
|
||
drift: each segment records the run’s cumulative dropped-frame count at its
|
||
completion, and a nonzero count logs an explicit warning. Placement with
|
||
actual dropped frames still needs validation.
|
||
|
||
Stopping exactly as a split fires can leave a header-only final Matroska file
|
||
(the rig observed 747 bytes, no Cluster and no video packets), despite a counted
|
||
split keyframe and zero reported drops. Finalisation checks the small header
|
||
prefix for the Cluster ID and deletes an empty tail after output stop. If deletion
|
||
fails (including a still-open file on Windows), it is marked unavailable in
|
||
metadata. Footage ends at the last real segment’s end and the remainder is
|
||
Missing. The offset check verifies every retained segment and explicitly
|
||
checks any unavailable marker, without treating an empty file as playable. Wall-clock time places each run inside the session and
|
||
marks capture waiting/recovery as Missing spans. Containers are never summed
|
||
to derive segment offsets. Metadata is replaced atomically from a synced
|
||
same-directory temporary file; startup finalises previously active metadata.
|
||
|
||
Run `CLIPBIN_VALIDATE_RECOVERY=1 scripts/validate/linux-session.sh` to also
|
||
kill the portal capture stream and check that bounded automatic recovery
|
||
resumes into the same session, with a Missing span and a second run directory.
|
||
This passed in the headless Wayland rig. A deliberately missing runtime path
|
||
also opened the interface with a persistent failure banner and honoured Quit.
|
||
|
||
Settings are saved under the config directory, including the portal restore
|
||
token. Capture selection clears that token and restarts the current session’s
|
||
capture; automatic recovery retries at five-second intervals, at most three
|
||
consecutive attempts. A failure stays visible until footage is proven again.
|
||
|
||
One game is recorded at a time. Startup discovery also records a game already
|
||
running; simultaneous games are selected by PID order and the next still-live
|
||
game is detected after the current session ends. Linux Steam names come from
|
||
library manifests, with `App <id>` as the fallback. Windows requires a manifest
|
||
installation path to identify candidate processes and fails honestly if none
|
||
can be found. X11 desktop session integration and Windows runtime validation
|
||
remain follow-ups.
|
||
|
||
Real Steam validated 2026-10-08 on CachyOS/Hyprland (native Steam): Escape
|
||
from Duckov (AppId 3167020, Proton Experimental under SteamLinuxRuntime_4)
|
||
runs under `reaper SteamLaunch AppId=3167020 -- …/_v2-entry-point … proton
|
||
waitforexitandrun …/Duckov.exe`. Clipbin detected it, named it from its
|
||
manifest ("Escape from Duckov") and recorded the session. Audio-source
|
||
failure detection remains limited as described above.
|
||
|
||
GPUI 0.2.2 ends its Linux event loop when the last window is removed, even
|
||
without an application close callback. Main keeps the recorder worker alive
|
||
after that return. Closing the window continues the current game session;
|
||
when that game exits, it finalises and the process exits. If no session is
|
||
active, it exits immediately. It stops discovering new games once closed,
|
||
so there is no indefinitely running invisible app. Reopening the window and
|
||
persistent background discovery need the future tray integration. Explicit
|
||
Quit/Ctrl+Q stops and finalises the current recording immediately.
|
||
|
||
|
||
## Clips, bookmarks and storage cleanup
|
||
|
||
`clipbin-media::export` copies video and selected AAC tracks to MP4 without
|
||
re-encoding. OBS already writes system, microphone and combined tracks when the
|
||
mic is enabled: exports default to the combined track, with a Settings option
|
||
for the two separate tracks. A session without a mic exports its single track.
|
||
The output is closed and synced under a `.partial.mp4` name, then renamed;
|
||
only the completed MP4 and saved JSON appear in the Clips library.
|
||
|
||
Video timestamps use integer frame clocks, retaining presentation/decode
|
||
ordering for B-frames. Each segment also persists cumulative AAC sample starts
|
||
per stream (`audio_start_samples`, with `#[serde(default)]`). Audio uses its own
|
||
1024-sample AAC packet clock across adjacent pieces, avoiding independently reset/rounded Matroska timestamps.
|
||
Old session metadata defaults these audio counters to empty. Before cleanup,
|
||
the worker reconstructs them from retained finalised files; export can also
|
||
reconstruct an older range while its preceding files still exist. If the earlier
|
||
files are already unavailable and no exact clock was saved, export fails
|
||
honestly rather than guessing the A/V phase.
|
||
Keyframes are anchored to the capture run, including its wall-clock offset.
|
||
Outer unavailable footage is trimmed from completed ranges. Internal Missing
|
||
and Removed spans retain a timestamp jump; players may hold the preceding
|
||
picture or skip to the next picture, but the gap is not collapsed. Export
|
||
ranges spanning codec, size or frame-rate changes fail honestly. Dropped-frame
|
||
placement remains unvalidated, so these ranges fail rather than claiming exact
|
||
CFR output.
|
||
|
||
One bounded export queue runs on a separate thread. Requests retain the action
|
||
receipt timestamp while the current segment finalises; cleanup leases are
|
||
persisted in `clips/export-requests.json`. Failed exports retain their leases
|
||
until retry succeeds or the user releases them from the Clips screen. A crash
|
||
between MP4 publication and JSON persistence can leave an unlisted complete MP4;
|
||
Open folder lets the user recover it. Partial files are never listed as clips.
|
||
|
||
Cleanup checks the byte budget every three seconds and after session end. Its
|
||
pure selector orders completed sessions first, then active footage, oldest
|
||
first. It excludes bookmarks, editor selection and queued/running/failed export
|
||
ranges, and never selects the current file. It persists Removed spans before
|
||
deleting a segment; a deletion failure restores availability and surfaces an
|
||
error. Bookmarks persist in session metadata, protect future post-roll, and can
|
||
be released in the editor. Clips remain outside this budget and can be deleted
|
||
explicitly.
|
||
|
||
Global bindings default to Ctrl+Shift+F9 (clip) and Ctrl+Shift+F10 (bookmark).
|
||
Settings independently offer F11/F12 alternatives; configuration files accept
|
||
F1–F12 (distinct bindings). Wayland uses the GlobalShortcuts portal and displays
|
||
its returned binding descriptions; the desktop handles approval and rebinding.
|
||
A compositor without that portal reports unavailability, with sidebar buttons
|
||
as the fallback. X11 uses XGrabKey with Caps/Num lock variants and detectable
|
||
auto-repeat; Windows uses RegisterHotKey with MOD_NOREPEAT on a message thread.
|
||
Action timestamps are taken when the native event is received and retained
|
||
while the worker queues the export. Real Hyprland first-use approval, portal
|
||
restoration/rebinding and shortcuts while a game has focus remain unvalidated.
|
||
Native X11 desktop behaviour and Windows registration/conflicts also remain
|
||
unvalidated.
|
||
|
||
New checks:
|
||
|
||
- `scripts/validate/linux-clips.sh`: fake Steam session, actual sidebar button
|
||
clicks, cross-segment MP4, codec preservation, zero-error decode, CFR packet
|
||
PTS spacing, continuous audio, original short-clip range, capture continuing
|
||
during export, a persisted bookmark, forced one-byte cleanup with neighbouring
|
||
Removed spans, a new export of the bookmark after earlier files are deleted,
|
||
and a Clips screenshot. Uses only the standard PATH tools,
|
||
`rig-pointer` and `wtype`; no ripgrep dependency.
|
||
- `scripts/validate/check-clip.py <saved-clip.json>`: reusable packet/decode check
|
||
before source cleanup, including selected audio track count, cumulative source
|
||
audio clocks and the original per-track A/V phase to within one audio sample.
|
||
- `scripts/validate/linux-hotkeys.sh`: actual clip/bookmark X11 key grabs under
|
||
Xwayland (`xdotool` injects native X11 keys). The wlr rig has no GlobalShortcuts
|
||
portal, so it cannot validate the Wayland shortcut route.
|
||
|
||
Windows media build setup: use an FFmpeg **7.1 shared development** package
|
||
(headers and MSVC import libraries) matching OBS's bundled avcodec-61 DLLs.
|
||
Set `FFMPEG_DIR` to that package when building `clipbin-media`; distribute the
|
||
matching OBS-bundled FFmpeg DLLs. FFmpeg development/import files for MSVC are
|
||
not available in this Linux environment, so `clipbin-media` is excluded from
|
||
the Windows cross-lint. `clipbin-core` (including the hotkey bridge),
|
||
`clipbin-obs` and `clipbin-probe` are cross-linted. Windows export/remux, the
|
||
default player/folder actions, clip deletion, pending-export leases, bookmark
|
||
release, cleanup of recently closed files and every new GPUI control still
|
||
need linking/running on Windows.
|
||
|
||
Step 2 validation on the Intel UHD 770 Linux rig (2026-10-08): workspace format,
|
||
clippy with `-D warnings`, and all 49 unit tests passed. Windows cross-clippy
|
||
passed for core, OBS and probe with the exclusions above. The end-to-end clips
|
||
script passed for H.264 with one audio track and with separate system/mic
|
||
tracks, and for HEVC with the combined track. The HEVC run requested the default
|
||
120 seconds from a shorter session and correctly saved its available 14 seconds.
|
||
All checked exports retained their codec, decoded without errors, had exact
|
||
1/60-second video spacing and continuous audio, and retained their source A/V
|
||
phase. Forced one-byte cleanup removed neighbouring footage while bookmarks
|
||
and exported MP4s survived; a retained bookmark could still export after its
|
||
preceding files had been deleted. Clips screenshots are produced as `clips.png`
|
||
in each script's output directory.
|
||
|
||
To repeat the extra audio/codec cases, set `CLIPBIN_VALIDATE_MIC=1`, optionally
|
||
`CLIPBIN_VALIDATE_SEPARATE_AUDIO=1`, `CLIPBIN_VALIDATE_CODEC=Hevc`, or
|
||
`CLIPBIN_VALIDATE_CLIP_SECONDS=120` before `linux-clips.sh`. The original nine
|
||
capture scenarios and automatic-session validation also passed using
|
||
`env PATH=/usr/bin:/bin:$HOME/.cargo/bin`. Native X11 hotkey registration and
|
||
both actions passed under Xwayland. These fake Steam tests run sequentially:
|
||
game discovery is machine-wide, although rig teardown only kills its own PIDs.
|
||
AV1 export is unvalidated because this GPU has no usable AV1 encoder. B-frame
|
||
export and ranges containing an internal timestamp gap need additional media
|
||
fixtures; dropped-frame ranges are explicitly rejected. Linux default-player
|
||
and file-manager associations still need testing on a real desktop.
|
||
|
||
## Playback and thumbnails
|
||
|
||
Playback uses the shared `clipbin-media` crate (`ffmpeg-next` 9) without GPUI.
|
||
One background decoder opens only completed, available segments. Session time
|
||
comes from `Metadata::locate`; each file's timestamps are normalised independently.
|
||
Seeking goes backwards to a keyframe, then decodes forward to the selected frame.
|
||
A one-frame mailbox and coalesced seeks bound work; overdue frames are decoded
|
||
but discarded before swscale. Playback threads have lower scheduling priority
|
||
than recording. Leaving the editor closes its decoder and audio stream.
|
||
|
||
The editor uploads packed BGRA at its actual pixel size, maintains a 16:9
|
||
preview, and releases old GPUI images. Space/K toggle playback; arrows/J/L step
|
||
one frame. The timeline seeks and drags across files and gaps; completed footage
|
||
from active sessions becomes browsable on the next 250 ms metadata refresh
|
||
within the recorder's segment completion delay. Missing/removed footage advances
|
||
on the wall clock and shows “No footage (not captured / cleaned up)”. There is
|
||
no live recording preview.
|
||
|
||
Audio uses cpal and swresample at the output device's rate/channel count, with
|
||
sample conversion in the device callback. OBS's combined third audio track is
|
||
selected when three tracks exist; otherwise the first track is used. The queue
|
||
holds at most 250 ms plus one decoded block. Callback timestamps, including DAC
|
||
latency, drive the master clock; gaps and mute use the wall clock. Callbacks do
|
||
no decoding and never wait for the queue lock. Mute and volume are in the editor;
|
||
an unavailable device is shown explicitly while video remains usable. The Linux
|
||
rig uses its existing null sink and ALSA's PipeWire plugin as a fallback if cpal's
|
||
native PulseAudio backend refuses the device. Install/configure `pipewire-alsa`
|
||
or an ALSA default routed to PulseAudio on systems without a physical ALSA card.
|
||
|
||
One low-priority worker produces 320×180 PNG thumbnails beside the first playable
|
||
session segment and exported MP4s. Its request queue holds 16 paths and duplicate
|
||
requests coalesce; card discovery and all filesystem work run off the UI thread.
|
||
`clipbin_media::thumbnail::{generate,cache_path,Thumbnails}` accept any MP4/MKV.
|
||
The gradient remains while a thumbnail is pending or footage is unavailable.
|
||
|
||
`scripts/validate/playback-fixture.sh` generates independent 1080p60 H.264/AAC
|
||
segments with one-second keyframes and a three-tone audio fixture. It passed on
|
||
2026-10-08: exact frame pixels at frame indices 0/1/59/60/61/119 in every file,
|
||
real-time boundary crossing, gaps, superseded seeks, pause/seek clock maths,
|
||
bounded frame placement, resampling 48 kHz to 44.1 kHz and combined-track selection.
|
||
The ordinary workspace tests skip the generated-media assertions unless
|
||
`CLIPBIN_TEST_MEDIA_DIR` points to the fixture; use the script to exercise them.
|
||
|
||
Windows: the cpal WASAPI dependency cross-check passed, as did the dedicated
|
||
Windows thread-priority bridge. Core/OBS/probe strict MSVC cross-lint passed.
|
||
The full media cross-lint is excluded on this Linux machine because the FFmpeg
|
||
Windows development import libraries are unavailable. On Windows set `FFMPEG_DIR`
|
||
to an FFmpeg **7.1 shared development** package with MSVC import libraries matching
|
||
the OBS-bundled avcodec-61/avformat-61/avutil-59/swscale-8/swresample-5 DLLs. Linux
|
||
links system FFmpeg 8. Actual Windows decoding, WASAPI device latency/seek behaviour,
|
||
GPUI image presentation and thumbnail caching need a Windows machine. Hardware
|
||
video decode is not used; HEVC/AV1 software performance and real output devices
|
||
other than the rig null sink still need measurement.
|
||
|
||
Final playback validation passed on 2026-10-08, serially with other rigs stopped:
|
||
|
||
- `cargo fmt --all -- --check`, strict locked workspace clippy, and locked
|
||
workspace tests (with `CLIPBIN_TEST_MEDIA_DIR=target/playback-fixture`).
|
||
- `env PATH=/usr/bin:/bin:$HOME/.cargo/bin scripts/validate/linux-rig.sh`:
|
||
all nine scenarios. An earlier concurrent run lost its H.264 probe to SIGKILL;
|
||
the serial rerun passed without timeout or sandbox changes.
|
||
- The same PATH with `scripts/validate/linux-session.sh`: automatic recording,
|
||
exact segment placement, close-window survival, game exit, library reload and
|
||
clean Quit. Repeated after the final keyboard integration, also passed.
|
||
- `scripts/validate/linux-playback.sh target/playback-e2e-final-6`: moving preview
|
||
(11.0% changed pixels inside the preview), active recording with zero reported
|
||
recording drops, completed-segment availability, real session thumbnail,
|
||
cross-file drag, exact post-boundary frame/time, cleaned-up gap, zoom, arrow
|
||
stepping, device audio, half-volume and mute. The audio monitor RMS was 0.08044;
|
||
the player asserts video time stays within 20 ms of the audio master clock.
|
||
Screenshots were inspected: `sessions-thumbnails.png`, `editor-playing-1.png`,
|
||
`editor-playing-2.png`, `editor-seek.png`, and `editor-gap.png` in that directory.
|
||
|
||
The rig runs on an Intel i5-12500 (12 logical CPUs). These are development builds
|
||
with dependencies at opt-level 2, software video decoding, 1080p60 H.264 source
|
||
and an 800×450 target (the editor used 740×416). Five serial repetitions of
|
||
`target/debug/examples/playback-check <sessions-root> <id>` with
|
||
`scripts/validate/measure-cpu.py`, inside the null-sink rig, measured process CPU
|
||
at **16.7% median / 17.7% p95 of one core** (12.2–17.7% range). The filesystem cache
|
||
was warm; each repetition reopened its decoders. Across presented frames,
|
||
decode/scale time was **2.39 ms median / 8.80 ms p95**; maximum **101.5 ms** included
|
||
opening files and decoding forward on seeks. This is a test pattern with gaps,
|
||
not a worst-case game or a release-build benchmark. Per-run logs and CPU reports
|
||
are in `target/playback-timing-final`; `report.json` stores the distributions.
|
||
|
||
During the active-session e2e, the player presented **59.2 frames/s** while
|
||
recording remained at zero reported drops. GPUI upload/construction time was
|
||
**0.22 ms median / 0.48 ms p95**. UI frame intervals (including vsync, not CPU work)
|
||
were 16.2 ms median / 32.5 ms p95; the one-frame mailbox drops presentation work
|
||
rather than allowing delay to accumulate. `CLIPBIN_TRACE_FRAMES=1` logs decoded
|
||
video time, audio time, decode/scale work, uploads and actual timeline bounds.
|
||
The headless e2e restores keyboard focus after virtual-pointer commands and
|
||
holds virtual keys briefly: instantaneous virtual input can disappear before
|
||
GPUI/Wayland dispatch. Keyboard actions update the editor entity directly,
|
||
avoiding a nested window borrow during event dispatch.
|
||
|
||
After merging clips and playback at `dd016f7`, the clips rig intermittently
|
||
reported “desktop did not honour Quit”. A fresh-window probe on 2026-10-08
|
||
missed 3 of 8 instantaneous `wtype` Ctrl+Q presses; two missed presses occurred
|
||
while Clipbin was focused. Temporary action/GUI-exit/worker-join markers showed
|
||
that the missed presses never reached the Quit handler. All 8 held-key attempts
|
||
reached the handler and completed shutdown. The temporary markers were removed;
|
||
no application shutdown change was needed.
|
||
|
||
Both scripts now source `scripts/rig/keys.sh`: it restores keyboard focus after
|
||
virtual-pointer input, gives the virtual keymap 200 ms to settle, and holds each
|
||
key for 100 ms. Modified keys also give the modifier 100 ms to settle. Delivery
|
||
has a 10-second timeout; each script still requires the actual desktop process
|
||
to exit successfully. Serial reruns passed with
|
||
`env PATH=/usr/bin:/bin:$HOME/.cargo/bin`:
|
||
|
||
- `scripts/validate/linux-clips.sh /tmp/clipbin-clips-quit-fixed`: H.264 retained,
|
||
360 CFR frames over a six-second aligned clip, continuous audio, zero decode
|
||
errors, original press range, bookmark protection, tiny-budget cleanup,
|
||
bookmark re-export after adjacent removal, and both Quit exits.
|
||
- `scripts/validate/linux-playback.sh /tmp/clipbin-playback-quit-fixed`: moving
|
||
preview (4.1% changed pixels), device audio RMS 0.08045, thumbnails, active
|
||
capture, seek/gap/boundary behaviour, volume/mute, frame stepping and both Quit
|
||
exits. Decode median/p95 were 2.578/4.073 ms; GPUI upload median/p95 were
|
||
0.241/0.408 ms.
|
||
|
||
Logs are `/tmp/clipbin-clips-quit-fixed.log` and
|
||
`/tmp/clipbin-playback-quit-fixed.log`; screenshots are in their corresponding
|
||
output directories. `bash -n`, `git diff --check` and `cargo fmt --check` also
|
||
passed. This follow-up changes only the Linux rig scripts and documentation;
|
||
Windows runtime and real Hyprland validation limits above remain unchanged.
|