Persistent daemon + thin networked client #9

harry · 2026-05-27T14:36:57+01:00

harry commented

2026-05-27 14:36:57 +01:00

Persistent daemon + thin networked client

Turns patterm from a single foreground process into a persistent background daemon that owns all process/project state, plus a thin client that renders and forwards input. A client on another LAN device can attach, navigate projects via the command palette, detach, and reconnect — with child processes surviving client disconnects.

Design source of truth: docs/daemon-client-plan.md (Phases 0–4 implemented).

What's included

Foundation — new internal/protocol (frame taxonomy, Transport interface, JSON-lines ConnTransport, loopback transport that crosses the same Send/Recv boundary); semantic chromeModel, ClientView, and a bounded backpressure-safe clientSubscriber kept separate from daemon-internal listeners; internal/pty now sets cmd.Dir and tears down the whole process group.
Multi-project — daemon-owned ProjectRegistry (implements the full mcp.ToolHost, daemon-wide identity routing); palette Switch/Open project…; two clients can view different projects at once; scratchpad MCP ops route to the caller's project.
Out-of-process daemon — patterm daemon over a unix socket with pidfile/lock + stale cleanup; default patterm [dir] auto-starts the daemon and attaches as the thin client; patterm daemon stop / patterm ls; processes survive client disconnect; reattach repaints from a fresh snapshot. Detach is Ctrl-] (Ctrl-D stays shell EOF). --in-process / PATTERM_NO_DAEMON=1 escape hatch.
LAN — opt-in patterm daemon --listen HOST:PORT with a lightweight bearer token (stored 0600 under $XDG_DATA_HOME/patterm/clients/token, printed for pairing); patterm connect --host HOST:PORT [--token TOKEN]. Unix-socket attaches stay token-exempt. Transport kept pluggable for future TLS.
Per-pane display-owner sizing — when two clients focus the same pane, the first owns its size; others render letterboxed at owner geometry with a status-line indicator; ownership releases on detach. Single-client sizing is unchanged.

Decisions baked in

All phases built; single PR. 2. Human UI clients only — MCP stays a local per-daemon unix socket. 3. Per-client independent view (ClientView). 4. Auto-start on demand. 5. "Persistent" = survive client disconnect while the daemon lives (no resurrection of live PTYs across daemon death). 6. Trusted-network auth: localhost default, opt-in LAN, simple token, no TLS yet (pluggable). 7. Detach via explicit gesture, not Ctrl-D.

Known limitations / deferred

No TLS (transport is pluggable so it can be layered in).
No remote MCP (agents are daemon-local only).
Daemon restart only rehydrates today's persist model (top-level commands, fresh IDs); live PTYs/agents are not resurrected across daemon death.

Testing

go build ./..., full go test ./... (incl. the real-socket harness), and go test -race -run 'Daemon|NetClient|Owner' -count=5 ./internal/app/ all green, verified in an environment with unix sockets/PTYs enabled.
New tests: detach/reattach process survival, TCP token auth + unix exemption, net-client frame loop, two-client same-pane display-owner sizing, plus pty workdir/process-group and project-switch-preserves-trees.
Two concurrency bugs found and fixed during review that the implementor's sandbox masked (it can't open sockets): daemon shutdown hang and a ConnTransport concurrent-send race, plus a pre-existing PTY Read/Close race now hit routinely by daemon shutdown.

Orchestrated build: parallel investigation (sub-claude + sub-codex), synthesized plan, codex peer-review, then phased implementation by codex with the orchestrator verifying every checkpoint against real sockets.

## Persistent daemon + thin networked client Turns patterm from a single foreground process into a persistent background **daemon** that owns all process/project state, plus a thin **client** that renders and forwards input. A client on another LAN device can attach, navigate projects via the command palette, detach, and reconnect — with child processes surviving client disconnects. Design source of truth: `docs/daemon-client-plan.md` (Phases 0–4 implemented). ### What's included - **Foundation** — new `internal/protocol` (frame taxonomy, `Transport` interface, JSON-lines `ConnTransport`, loopback transport that crosses the same `Send`/`Recv` boundary); semantic `chromeModel`, `ClientView`, and a bounded backpressure-safe `clientSubscriber` kept separate from daemon-internal listeners; `internal/pty` now sets `cmd.Dir` and tears down the whole process group. - **Multi-project** — daemon-owned `ProjectRegistry` (implements the full `mcp.ToolHost`, daemon-wide identity routing); palette **Switch/Open project…**; two clients can view different projects at once; scratchpad MCP ops route to the caller's project. - **Out-of-process daemon** — `patterm daemon` over a unix socket with pidfile/lock + stale cleanup; default `patterm [dir]` auto-starts the daemon and attaches as the thin client; `patterm daemon stop` / `patterm ls`; processes survive client disconnect; reattach repaints from a fresh snapshot. Detach is **Ctrl-]** (Ctrl-D stays shell EOF). `--in-process` / `PATTERM_NO_DAEMON=1` escape hatch. - **LAN** — opt-in `patterm daemon --listen HOST:PORT` with a lightweight bearer token (stored `0600` under `$XDG_DATA_HOME/patterm/clients/token`, printed for pairing); `patterm connect --host HOST:PORT [--token TOKEN]`. Unix-socket attaches stay token-exempt. Transport kept pluggable for future TLS. - **Per-pane display-owner sizing** — when two clients focus the same pane, the first owns its size; others render letterboxed at owner geometry with a status-line indicator; ownership releases on detach. Single-client sizing is unchanged. ### Decisions baked in 1. All phases built; single PR. 2. Human UI clients only — MCP stays a local per-daemon unix socket. 3. Per-client independent view (`ClientView`). 4. Auto-start on demand. 5. "Persistent" = survive client disconnect while the daemon lives (no resurrection of live PTYs across daemon death). 6. Trusted-network auth: localhost default, opt-in LAN, simple token, no TLS yet (pluggable). 7. Detach via explicit gesture, not Ctrl-D. ### Known limitations / deferred - No TLS (transport is pluggable so it can be layered in). - No remote MCP (agents are daemon-local only). - Daemon restart only rehydrates today's persist model (top-level commands, fresh IDs); live PTYs/agents are not resurrected across daemon death. ### Testing - `go build ./...`, full `go test ./...` (incl. the real-socket harness), and `go test -race -run 'Daemon|NetClient|Owner' -count=5 ./internal/app/` all green, verified in an environment with unix sockets/PTYs enabled. - New tests: detach/reattach process survival, TCP token auth + unix exemption, net-client frame loop, two-client same-pane display-owner sizing, plus pty workdir/process-group and project-switch-preserves-trees. - Two concurrency bugs found and fixed during review that the implementor's sandbox masked (it can't open sockets): daemon shutdown hang and a `ConnTransport` concurrent-send race, plus a pre-existing `PTY` Read/Close race now hit routinely by daemon shutdown. Orchestrated build: parallel investigation (sub-claude + sub-codex), synthesized plan, codex peer-review, then phased implementation by codex with the orchestrator verifying every checkpoint against real sockets.

harry added 14 commits 2026-05-27 14:36:57 +01:00

Fix PTY workdir and process group teardown b72a32bbc6

Add daemon client protocol frames e63bdad5e1

Scaffold loopback daemon client split 9aecc8b7a2

Update PTY start call sites ec0c148164

docs: add daemon client implementation plan 08c7405c79

app: add loopback multi-project registry 80a14502c4

fix scratchpad routing by caller project c56de27f44

add local daemon socket protocol d07a09d64f

Fix daemon shutdown hang and concurrent-send race 95b1967e9b

- daemon_net: close the client transport on context cancellation so the
  per-connection Recv loop unblocks; otherwise wg.Wait() in the accept loop
  hung on a still-connected client and the daemon never stopped.
- protocol: guard ConnTransport.Send with a mutex so the subscriber pump and
  command handlers can push frames concurrently without racing the bufio.Writer.

Fixes TestDaemonDetachReattachPreservesProcess (now passes under -race).

attach default client to local daemon 5149224000

add tcp daemon listener with token auth 63cb8a4388

add per-pane display ownership 6d15626e05

Fix data race on PTY master between Read and Close 63986e7e00

pumpChild's PTY.Read raced Session.Shutdown's PTY.Close on the master
field (Close set it nil while Read read it). Benign at process exit on
main, but the daemon now runs Shutdown routinely (daemon stop). Guard
the field with a mutex, capturing the fd under the lock and doing the
blocking I/O outside it so Close still unblocks an in-flight Read.

Caught under: go test -race -run 'Daemon|NetClient|Owner' -count=5.

docs: mark daemon/client plan as implemented 4051e7264b

This pull request has changes conflicting with the target branch.

internal/app/host.go
internal/mcp/mcp_test.go
internal/mcp/tools.go

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feat/daemon-client-split:feat/daemon-client-split

git checkout feat/daemon-client-split

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: harry/patterm#9