Files
clipbin-desktop/README.md
T
harry dd016f7727 Merge feat/playback: session playback, synced audio and thumbnails
# Conflicts:
#	README.md
#	crates/clipbin-desktop/Cargo.toml
#	crates/clipbin-desktop/src/main.rs
#	crates/clipbin-media/src/lib.rs
#	docs/validation.md
2026-10-08 19:26:04 +02:00

117 lines
9.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Clipbin Desktop
An open-source game recorder for Windows and Linux, inspired by Plays.tv. Rust and GPUI provide the desktop interface; the recording engine uses libobs and official OBS capture components.
[SPEC.md](SPEC.md) defines the agreed v1 product requirements.
## Current state
The recording foundation is being validated (see [docs/validation.md](docs/validation.md)). `clipbin-obs` records official OBS capture sources through one hardware encoder into keyframe-aligned segments and reports honest status; `clipbin-probe` exercises it. It passes the Linux headless rig and the hardware sign-off on CachyOS/Hyprland/RX 9070 XT; all Windows validation is still to do.
The desktop app detects Steam games, owns the recorder on a background thread, persists settings and sessions, and loads recorded sessions from disk. Linux detects Steam’s `reaper` processes and watches their pidfds; Windows reads Steam’s registry state and watches native process handles (cross-checked, not run). Game exit finalises a session; closing the window keeps the current session recording until the game exits, then exits the app. Ctrl+Q or Quit stops the output cleanly.
The desktop interface follows the web Clipbin's look (zinc dark theme, purple accent, Geist Mono): a session library, a session timeline with bookmarks and a clip selection, clips, and settings. It shows sessions from disk; `CLIPBIN_SAMPLE_LIBRARY=1` fills it with example sessions for interface work. Playback and frame seeking work across completed segments, including active sessions and missing footage, with combined audio, mute/volume, Space/K to play/pause and J/L or arrows to step frames. Sessions and clips use cached frame thumbnails. Exports copy the recorded codec to MP4 in a background job, save local clip metadata, and support opening/deleting clips. Ctrl+Shift+F9 clips the last configured interval; Ctrl+Shift+F10 protects a bookmark. Both also have sidebar buttons. Wayland uses the GlobalShortcuts portal (first-use approval, portal rebinding); X11 uses native key grabs and Windows uses RegisterHotKey. Settings offer independent alternate function keys. Storage cleanup keeps unprotected footage within the budget. Autostart and a tray are not implemented yet; after closing the only window, reopening it without a tray is a follow-up.
Windows is a first-class target. Every change must keep the Windows code paths building (`cargo clippy --target x86_64-pc-windows-msvc`) and list what still needs validating on Windows.
## Layout
```text
crates/
clipbin-core/ Settings, Steam detection and session storage; independent of GPUI and OBS.
clipbin-media/ FFmpeg: stream-copy clip export, scaled video/audio playback and cached thumbnails.
clipbin-obs/ Recording engine over the pinned libobs runtime; all unsafe code is in src/ffi.rs.
clipbin-probe/ Validation probe for capture, encoders, segments and status.
clipbin-desktop/ GPUI application and initial session-library view.
scripts/obs/ Assemble the pinned OBS runtime for Linux (built from source) and Windows (official release).
scripts/rig/ Headless Wayland/X11 desktop for capture tests on machines without a display.
scripts/validate/ Validation suites: Linux rig, Linux hardware, Windows.
docs/validation.md Validation status and findings.
.gitea/workflows/ Formatting, lint, test and Windows cross-checks.
SPEC.md Product requirements and acceptance checks.
```
Keep capture and session operations out of the UI thread. Core uses serde/JSON, dirs, tempfile, rustix on Linux and winreg/windows-sys on Windows. Raw OBS calls live in `clipbin-obs/src/ffi.rs`; raw Windows process calls live in the small `clipbin-core/src/windows_process.rs` bridge.
## Development
Use Rust **1.98.0**, as pinned in rust-toolchain.toml. GPUI is pinned to **0.2.2**, with Wayland and X11 enabled. Cargo.lock pins transitive dependencies.
Keep the lockfile checked in and use `--locked`. It currently pins `libc` to 0.2.186 because GPUI’s transitive `xattr` 0.2.3 dependency fails to compile against 0.2.190 on Linux.
On Arch/CachyOS, install the native build dependencies:
```sh
sudo pacman -S --needed base-devel pkgconf fontconfig freetype2 wayland libxcb libxkbcommon libxkbcommon-x11 openssl clang ffmpeg alsa-lib pipewire-alsa libpulse
```
### OBS runtime
Clipbin bundles a pinned OBS **32.0.4** runtime (`scripts/obs/versions.env`) with only the official plugins it uses. The `libobs` crate's headers match that version; the engine refuses to start against any other.
- Linux: `scripts/obs/build-linux.sh` builds libobs and the plugins from source into `target/obs-runtime/linux-x86_64`. Arch/CachyOS build dependencies:
```sh
sudo pacman -S --needed cmake ninja extra-cmake-modules ffmpeg x264 pipewire libpulse libdrm libva jansson curl libxcomposite libxinerama libxrandr wayland-protocols simde uthash nlohmann-json mbedtls3 pciutils srt librist ffnvcodec-headers amf-headers
```
- Windows: `scripts/obs/assemble-windows.sh` (Git Bash) verifies and unpacks the official release zip into `target/obs-runtime/windows-x86_64`, copying files byte-for-byte so the Game Capture hook keeps its signature.
Development builds find the runtime through `.cargo/config.toml`; `clipbin-probe`'s build script places `obs-ffmpeg-mux` (and on Windows the OBS DLLs) next to the executable, where OBS looks for them.
On Debian/Ubuntu:
```sh
sudo apt-get update
sudo apt-get install build-essential pkg-config libfontconfig1-dev libfreetype6-dev libwayland-dev libxcb1-dev libxkbcommon-dev libxkbcommon-x11-dev libssl-dev libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libswresample-dev libasound2-dev libpulse-dev pipewire-alsa
```
Media dynamically links FFmpeg through `ffmpeg-next` 9. Linux uses system FFmpeg 8. On Windows, set `FFMPEG_DIR` to an FFmpeg **7.1 shared development** package with MSVC import libraries, matching the OBS-bundled `avcodec-61`, `avformat-61`, `avutil-59`, `swscale-8` and `swresample-5` DLLs. Keep those OBS DLLs alongside the desktop executable; do not mix FFmpeg major versions. Audio uses cpal (PulseAudio or ALSA/PipeWire on Linux, WASAPI on Windows).
For Windows, use the MSVC Rust toolchain and Visual Studio Build Tools with the Desktop development with C++ workload. Windows support needs validation on a Windows machine; see [docs/validation.md](docs/validation.md#windows).
Run the desktop shell from an active graphical session:
```sh
cargo run --locked -p clipbin-desktop
```
The desktop app needs a working Wayland or X11 session on Linux, or a Windows desktop. It does not start a web server. On this remote development machine, bind any future development server to its tailnet IP.
Use the Quit button or Ctrl+Q to quit.
Checks:
```sh
cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo test --locked --workspace
# Windows code paths, from any platform (rustup target add x86_64-pc-windows-msvc):
cargo clippy --locked --target x86_64-pc-windows-msvc -p clipbin-core -p clipbin-obs -p clipbin-probe --all-targets -- -D warnings
```
Clips/bookmarks/cleanup validation: `scripts/validate/linux-clips.sh`. Native X11 hotkey validation: `scripts/validate/linux-hotkeys.sh`. Both run in the headless rig. Real Hyprland portal hotkeys and Windows runtime behaviour still need validation.
Recording validation on Linux, headless: `scripts/validate/linux-rig.sh`. Automatic desktop sessions: `scripts/validate/linux-session.sh` (fake Steam reaper, recording and library screenshots, close-window survival, game exit and explicit Quit). Playback fixtures: `scripts/validate/playback-fixture.sh`. Playback/audio e2e: `scripts/validate/linux-playback.sh [output-dir]` (moving preview, active recording, gaps, thumbnails, audio monitor, timing and CPU report). Fake Steam rigs must run serially because discovery reads the machine’s `/proc`. Interface screenshots, headless: `scripts/rig/screenshot-ui.sh` (screens: sessions, session, clips, settings, failed, empty). The rig has no input devices; `rig-pointer` drives a virtual pointer (`move`, `click`, `down`/`up`, `scroll`, `sleep` on stdin) and `wtype` types. `CLIPBIN_TRACE_FRAMES=1` logs frame times.
Development builds optimise dependencies (`[profile.dev.package."*"]`): unoptimised GPUI made screens noticeably slow, with first frames of 200–300 ms.
The core can also be checked without native UI dependencies:
```sh
cargo check --locked -p clipbin-core
```
## First recording milestone
1. Validate official OBS capture and recording-health signals on CachyOS, Hyprland and the RX 9070 XT at 1440p60. *Passed on CachyOS/Hyprland/RX 9070 XT (H.264, HEVC, AV1).*
2. Select and pin a tested OBS runtime, Rust bindings, and official plugin set together. Preserve the original signed Windows capture-hook binaries. *Pinned to OBS 32.0.4 and libobs crate 5.0.1; Windows runtime unvalidated.*
3. Prove one continuous hardware encoder can produce short keyframe-aligned segments and remux clips across their boundaries while recording continues. *Proven in the rig and on the RX 9070 XT with VA-API H.264, HEVC and AV1.*
4. Integrate the recorder without blocking GPUI, then add native playback and seeking.
The capture integration uses official OBS components only: Windows Game Capture/Window Capture, X11 Xcomposite, and Wayland PipeWire/desktop-portal capture. Third-party game hooks are excluded.
## License
GPL-3.0-only. See [LICENSE](LICENSE).