256 lines
15 KiB
Markdown
256 lines
15 KiB
Markdown
# Recording foundation validation
|
||
|
||
The first recording milestone proves the OBS capture, encoder and segment
|
||
pipeline before the UI is built (SPEC.md, "Implementation validation before
|
||
building out the UI"). `clipbin-probe` drives the real `clipbin-obs` engine and
|
||
prints `key=value` lines; the scripts in `scripts/validate/` turn them into
|
||
pass/fail results.
|
||
|
||
| Where | Script | Status |
|
||
| --- | --- | --- |
|
||
| Linux, headless rig (Wayland portal + Xwayland) | `scripts/validate/linux-rig.sh` | All 9 scenarios pass on devbox (Intel UHD 770, VA-API) |
|
||
| Linux, real session (CachyOS, Hyprland, RX 9070 XT, 1440p60) | `scripts/validate/linux-hardware.sh` | Passed 2026-10-08 (run 4, dc9cbd6) |
|
||
| Linux desktop automatic sessions (headless Wayland) | `scripts/validate/linux-session.sh` | Passed 2026-10-08: Recording sidebar, game exit, library reload, window close and active Quit |
|
||
| Windows (Game Capture, NVENC/AMF/QSV) | `scripts/validate/windows.ps1` | Not run yet; script untested |
|
||
|
||
## Probe commands
|
||
|
||
```sh
|
||
cargo run -p clipbin-probe -- encoders --verify # hardware encoders per codec, proven by encoding
|
||
cargo run -p clipbin-probe -- record --target monitor # records; prints status, health and segments
|
||
cargo run -p clipbin-probe -- check # keyframes, codec, durations, cross-boundary remux
|
||
```
|
||
|
||
`record` targets `monitor`, `window` or `game` (Game Capture on Windows; the
|
||
same as `window` elsewhere). On Windows and X11, `--window` takes OBS's window
|
||
identifier; on Wayland the portal picker chooses and `--restore-token` reuses
|
||
an earlier choice. See the top of `crates/clipbin-probe/src/main.rs` for all
|
||
options. `check` needs FFmpeg's `ffprobe` and `ffmpeg` on `PATH`.
|
||
|
||
## Linux headless rig
|
||
|
||
`scripts/rig/headless.sh [--x11] <command>` runs a command inside a private
|
||
D-Bus session with PipeWire, a virtual audio sink, headless Sway on the GPU,
|
||
xdg-desktop-portal-wlr (auto-selecting the only output) and a 60 FPS test
|
||
pattern with a tone. `--x11` runs the game and command on Sway's Xwayland.
|
||
|
||
Arch packages: `sway xdg-desktop-portal xdg-desktop-portal-wlr xorg-xwayland
|
||
xorg-xwininfo pipewire pipewire-pulse wireplumber libpulse ffmpeg` plus a VA-API
|
||
driver (`intel-media-driver` or `libva-mesa-driver`).
|
||
|
||
The suite covers:
|
||
|
||
1. Encoder verification: H.264 must be usable; HEVC and AV1 are reported.
|
||
2. Wayland monitor capture in H.264 and HEVC: segments start on keyframes,
|
||
last 2 s, remux to MP4 across boundaries, decode cleanly, carry the tone.
|
||
3. X11 window capture (Xcomposite).
|
||
4. Failures that must end in `status=failed`: PipeWire stream lost, captured
|
||
X11 window closed, segment directory unwritable, free space below the margin.
|
||
5. A portal that refuses the request fails with an actionable reason and
|
||
never reports `Recording`.
|
||
|
||
Segment checks run with `--expect-motion`, so black or frozen footage (which
|
||
still makes valid segments) fails.
|
||
|
||
## Linux hardware sign-off
|
||
|
||
Run `scripts/validate/linux-hardware.sh` from the Hyprland session. It verifies
|
||
encoders at 1440p60, records the monitor with each usable codec (the first run
|
||
shows the portal picker; later runs pass the restore token and ask whether the
|
||
picker stayed away), records a window, and asks you to close a window
|
||
mid-recording. Send back `report.txt` from the output directory.
|
||
|
||
### First hardware run (2026-10-08)
|
||
|
||
CachyOS 7.2.9, Hyprland, RX 9070 XT, 1440p60, OBS 32.0.4:
|
||
|
||
- H.264, HEVC and AV1 all encode through VA-API (`*_ffmpeg_vaapi_tex`).
|
||
Monitor and window capture recorded keyframe-aligned 2 s segments that
|
||
remuxed across boundaries with no decode errors.
|
||
- The restore token came back on one of three runs, probably because the
|
||
"Allow a restore token" box in the picker was ticked only once. The script
|
||
now has a dedicated restore step.
|
||
- One run never got a portal selection: the picker either did not appear or
|
||
was not answered, and the probe stayed in `Waiting for capture` for 20 s
|
||
while writing black segments that `check` passed. `check --expect-motion`
|
||
now catches black footage; a refused request now fails at once.
|
||
- Closing the captured window (Super+W) is not visible to capture: xdph keeps
|
||
the PipeWire stream `streaming` and OBS repeats the window's last frame.
|
||
See findings.
|
||
|
||
### Second hardware run (2026-10-08)
|
||
|
||
- The restore token works: the restored run started recording in 2 s without
|
||
showing the picker.
|
||
- Hyprland's portal backend (xdg-desktop-portal-hyprland) wedged in
|
||
`SelectSources` during this run and answered nothing afterwards, until
|
||
restarted; the earlier "no selection" runs were likely the same hang.
|
||
xdph shows its picker during `SelectSources`, so a hang there looks the same
|
||
as a user still choosing. A 10 s timeout tried in afc54b3 would cut users
|
||
off mid-pick and was removed. Recording stays honestly in `Waiting for
|
||
capture`; the app needs a prompt to choose a source (or restart the portal)
|
||
while it waits.
|
||
- The motion check sampled a segment from before capture started (the user
|
||
took 16 s in the picker); it now uses the segment with the most motion.
|
||
- A portrait monitor (Hyprland transform 3) is captured as 1440x2560 and
|
||
pillarboxed in the 2560x1440 canvas.
|
||
|
||
### Fourth hardware run: pass (2026-10-08, dc9cbd6)
|
||
|
||
After restarting xdg-desktop-portal-hyprland, every step passed: H.264, HEVC
|
||
and AV1 through VA-API at 1440p60, monitor and window capture with motion,
|
||
keyframe-aligned segments that remux across boundaries, and a restored
|
||
selection that skipped the picker. Closing a window was again not detected
|
||
(informational, see findings).
|
||
|
||
## Windows
|
||
|
||
Run `scripts/validate/windows.ps1 -GameWindow "<title>:<class>:<exe>"` with a
|
||
game running. It assembles the official OBS 32.0.4 runtime, checks the Game
|
||
Capture hook signatures, builds the probe, verifies NVENC/AMF/QSV encoders and
|
||
records Monitor, Game and Window Capture.
|
||
|
||
Windows status:
|
||
|
||
- `clipbin-core`, `clipbin-obs` and `clipbin-probe` pass `cargo clippy` for
|
||
`x86_64-pc-windows-msvc` (cross-checked from Linux). Nothing has been linked
|
||
or run on Windows.
|
||
- The GPUI desktop crate has not been checked for Windows; its `ring`
|
||
dependency needs the MSVC C toolchain.
|
||
- The assembled runtime keeps the six Game Capture hook binaries byte-identical
|
||
to the release, with their Authenticode signatures present; signature
|
||
validity has not been checked on Windows.
|
||
- The native Steam process bridge (`clipbin-core/src/windows_process.rs`) uses Toolhelp enumeration, limited process queries and a dedicated thread waiting on the process handle. Registry discovery uses `RunningAppID` and `Apps\<id>\Running`; executable paths are matched beneath the app manifest’s installation directory. This code is cross-checked but unvalidated on Windows, including launcher/child-process identity, handle access permissions and exit ordering.
|
||
- Unvalidated: atomic metadata replacement and recovery on Windows, settings/data directory paths, desktop close-window survival and clean Quit. GPUI desktop integration cannot be cross-checked from Linux.
|
||
- Unvalidated: encoder ids and settings for NVENC, AMF and QSV; Game Capture
|
||
hooking and its loss detection (the engine expects the source size to drop
|
||
to 0); WASAPI audio; `obs-ffmpeg-mux.exe` beside the executable; linking
|
||
`dstr_vprintf` through the libobs crate's `obs.lib`.
|
||
|
||
## Findings for the next milestones
|
||
|
||
- **Plugin loading.** libobs always adds default plugin paths: the build-time
|
||
install prefix on Linux and `../../obs-plugins/64bit` relative to the
|
||
working directory on Windows. Searching would load other OBS installations'
|
||
plugins or load ours twice, so the engine opens exactly the bundled
|
||
allowlist (`runtime::PLUGINS`) by path.
|
||
- **Segment timestamps.** The FFmpeg muxer restarts each segment's video and
|
||
audio timestamps at zero independently. Concatenating segments naively leaves
|
||
22–43 ms video gaps at boundaries. Video is constant frame rate and AAC
|
||
packets are continuous, so a segment's exact session offset is the count of
|
||
earlier frames and audio packets; the A/V offset stays within one AAC packet
|
||
(≤21 ms) and does not drift. The timeline and export work must place
|
||
segments this way.
|
||
- **Availability.** With 1 s keyframes and 2 s segments, splits happen on
|
||
schedule every 2.0 s, so the newest complete footage is about 2 s behind
|
||
plus finalisation: inside the spec's 2–5 s window.
|
||
- **Honest status needs more than OBS's counters.** OBS kept counting bytes
|
||
into a dead muxer process, and once reported a clean stop after the disk
|
||
filled. Status therefore requires footage to grow on disk, and recording
|
||
stops while 1 GB remains so the last segment can be finalised.
|
||
- **PipeWire source loss.** OBS's PipeWire capture keeps its size and renders
|
||
its last frame after the stream stops, and only logs the stream state. The
|
||
engine reads that log line (stable for the pinned OBS version). A more robust
|
||
follow-up is watching the PipeWire node state directly.
|
||
- **Ending a session when the game closes.** With Hyprland's portal, closing a
|
||
captured window leaves the stream running on its last frame, with no state
|
||
change or error, so capture cannot tell it from a static window. Monitor
|
||
capture never sees the game close at all. The session therefore ends from
|
||
the game's process exiting (SPEC: "Stop/finalise the session when its game
|
||
exits"); capture loss stays a failure signal, not the normal end:
|
||
- Linux: Steam runs each game under `reaper SteamLaunch AppId=<id>`, which
|
||
lives exactly as long as the game (Proton included). Watch that process
|
||
with a pidfd (`pidfd_open` + poll) rather than polling `/proc`.
|
||
- Windows: watch the game's process handle (`OpenProcess` +
|
||
`WaitForSingleObject`), found from Steam's `RunningAppID` registry value.
|
||
Game Capture reports size 0 when its game closes; the exit must be seen
|
||
first so a normal exit finalises the session instead of failing it.
|
||
- Hyprland's IPC does announce window closes, but it is compositor-specific
|
||
and cannot map a portal stream to a window, so it is not used.
|
||
- **Nothing is written before capture works.** Encoders and the segment
|
||
output are created only once the source has a picture, so waiting in the
|
||
picker or for a stalled portal writes no footage.
|
||
- **Recording size.** By default each recording is the captured window's or
|
||
monitor's own size, set when capture first has a picture (portrait monitors
|
||
record portrait). If the window is resized mid-session the size stays and
|
||
the picture is scaled to fit.
|
||
- **Audio capture failures are not detected yet.** When the PulseAudio server
|
||
was unreachable, recording continued with a silent track.
|
||
- **Linux runtime dependencies.** The Linux OBS build links the system's
|
||
FFmpeg, PipeWire and other libraries, so it is tied to the distribution it
|
||
was built on. Packaging (for example Flatpak, or bundling FFmpeg) is an open
|
||
decision; OBS 32.0 needs mbedtls 3 (`mbedtls3` on Arch).
|
||
- **OBS objects.** Scenes and sources are created private (OBS 31+ otherwise
|
||
registers scenes with the main canvas, which keeps them alive), unknown
|
||
source types fail instead of becoming placeholders, and shutdown drains
|
||
libobs's destroy queue first.
|
||
- **Plugin configuration.** libobs is started without a module config path;
|
||
set one under the app's config directory before plugins that save settings
|
||
are added.
|
||
|
||
## Automatic desktop sessions
|
||
|
||
`scripts/validate/linux-session.sh [output-dir]` runs the desktop and a fake
|
||
Steam reaper (`argv[0]=reaper`, `SteamLaunch`, `AppId=480`) inside the headless
|
||
Wayland rig. It verifies Recording status, non-empty segment files and metadata,
|
||
closes the only window while recording continues, kills the game process,
|
||
checks finalisation, reopens the app to load its library from disk, captures
|
||
screenshots and uses Ctrl+Q during a second recording for clean shutdown.
|
||
The recording and library screenshots were inspected; storage totals came from
|
||
real Matroska file sizes. The existing Linux rig also passed all nine scenarios
|
||
after integration (2026-10-08). Its unwritable-output assertion accepts both
|
||
OBS Error and EncodeError: the muxer can report either for a permission error.
|
||
|
||
Session metadata is `session.json` inside each data-directory session folder;
|
||
recovery attempts use separate `run-NNN` folders. Each segment stores its exact
|
||
start using `(total_frames_at_file_changed - 1) / fps`. The keyframe triggering
|
||
the split is already included in OBS’s count but belongs to the new segment:
|
||
the rig observed 121, 241, 361 at the boundaries of 120-frame segments. The
|
||
initial file starts at zero. Raw callback counts are preserved in events; the
|
||
e2e script checks placements exactly against cumulative earlier ffprobe video
|
||
packet counts within each run. Dropped output frames could make this count
|
||
drift: each segment records the run’s cumulative dropped-frame count at its
|
||
completion, and a nonzero count logs an explicit warning. Placement with
|
||
actual dropped frames still needs validation.
|
||
|
||
Stopping exactly as a split fires can leave a header-only final Matroska file
|
||
(the rig observed 747 bytes, no Cluster and no video packets), despite a counted
|
||
split keyframe and zero reported drops. Finalisation checks the small header
|
||
prefix for the Cluster ID and deletes an empty tail after output stop. If deletion
|
||
fails (including a still-open file on Windows), it is marked unavailable in
|
||
metadata. Footage ends at the last real segment’s end and the remainder is
|
||
Missing. The offset check verifies every retained segment and explicitly
|
||
checks any unavailable marker, without treating an empty file as playable. Wall-clock time places each run inside the session and
|
||
marks capture waiting/recovery as Missing spans. Containers are never summed
|
||
to derive segment offsets. Metadata is replaced atomically from a synced
|
||
same-directory temporary file; startup finalises previously active metadata.
|
||
|
||
Run `CLIPBIN_VALIDATE_RECOVERY=1 scripts/validate/linux-session.sh` to also
|
||
kill the portal capture stream and check that bounded automatic recovery
|
||
resumes into the same session, with a Missing span and a second run directory.
|
||
This passed in the headless Wayland rig. A deliberately missing runtime path
|
||
also opened the interface with a persistent failure banner and honoured Quit.
|
||
|
||
Settings are saved under the config directory, including the portal restore
|
||
token. Capture selection clears that token and restarts the current session’s
|
||
capture; automatic recovery retries at five-second intervals, at most three
|
||
consecutive attempts. A failure stays visible until footage is proven again.
|
||
|
||
One game is recorded at a time. Startup discovery also records a game already
|
||
running; simultaneous games are selected by PID order and the next still-live
|
||
game is detected after the current session ends. Linux Steam names come from
|
||
library manifests, with `App <id>` as the fallback. Windows requires a manifest
|
||
installation path to identify candidate processes and fails honestly if none
|
||
can be found. Real Steam/Proton lifecycle validation, X11 desktop session
|
||
integration and Windows runtime validation remain follow-ups. Audio-source
|
||
failure detection remains limited as described above.
|
||
|
||
GPUI 0.2.2 ends its Linux event loop when the last window is removed, even
|
||
without an application close callback. Main keeps the recorder worker alive
|
||
after that return. Closing the window continues the current game session;
|
||
when that game exits, it finalises and the process exits. If no session is
|
||
active, it exits immediately. It stops discovering new games once closed,
|
||
so there is no indefinitely running invisible app. Reopening the window and
|
||
persistent background discovery need the future tray integration. Explicit
|
||
Quit/Ctrl+Q stops and finalises the current recording immediately.
|