monoverse

An app halfway between a terminal emulator and a graphical server.

What it is

A rich terminal: a local server runs a child app that paints styled text + true-pixel graphics over one byte stream, and streams input events back. Clean-slate binary protocol, no ANSI on the wire. Zig 0.16, Linux-only.

The plain byte stream means it works over SSH, pipes, and multiplexers with zero special support. Clean-slate — apps target the monoverse protocol directly; there is no ANSI compatibility and no serverless mode.

Design

Dial	Choice	Consequence
Concept	Rich terminal — text + inline graphics	Not a window server; not a plain TTY
Backward compat	None — clean-slate protocol	No ANSI; new apps only
Interop	No fallback; binary always	Apps refuse to run without a server; flattening is mono-cat's job, so there's always one server-side speaker
Topology	Local, like a terminal emulator	Server local, app is a child process
Channel	Single plain byte stream	Survives SSH / pipes / multiplexers
Framing	Length-prefixed binary packets with a type tag	Compact, fast to parse
Primitive	Rich cells — grid cell holds styled text or part of a graphic region	Familiar grid mental model, graphics-capable
Pixels↔cells	Pixels within cells	Graphic reserves a cell rectangle, renders at true pixel res inside it
State model	Immediate mode	App paints each frame; server owns no widget tree
Redraw	Dirty regions only; server retains last framebuffer	App resends only changed cells/rects
Region identity	None — re-declare the rect each frame	State is the framebuffer; coords are the address. RGBA patches a sub-rect, JPEG re-sends the whole rect
Pixel format	RGBA or JPEG, chosen per blit, no negotiation	Server supports both, decodes to one uniform RGBA framebuffer
Input	Structured event stream up the byte stream	Key/mouse/click/resize read like stdin; no server-side widgets in v1
Surfaces	Single live surface, no server scrollback	One screen's framebuffer only; history is the app's job. Multiplexing is tmux's job
Mux seam	Reserve a surface-id field, hardwired to 0 in v1	Forward-compat for multi-surface later, no v1 complexity
Geometry	Live negotiation; app refuses to run without a server	Server reports grid (cols×rows), cell px (w×h), HiDPI scale on init / change / request
Resize	Server emits a resize event; app redraws	Consistent with immediate mode
Language	Zig	Emulator + server + reference client
First app	An interactive shell	Dogfoods the entire stack

Protocol

A bidirectional byte stream of length-prefixed, type-tagged binary frames; every frame carries the v1 surface-id (0).

App → server (paint). text-run (styled run at row,col); blit (pixels — RGBA or JPEG — into a cell-rect, addressing pixels within it); clear (rect); present (commit frame). The app emits paint ops only for cells/rects that changed; the server's retained framebuffer holds the rest. A blit's rect is its identity — re-blit a smaller rect to patch (RGBA), or the whole rect for JPEG.

Server → app (events). handshake, geometry (grid cols×rows, cell px w×h, HiDPI scale — sent on init, on change, and on request), key, mouse (move/button/click at a coordinate), scroll, focus. Events are low-level: a clicked button is just a click coordinate the app resolves against what it painted — no server-side widgets in v1.

The handshake is mandatory; with no server-side speaker (real GUI server or the headless mono-cat flattener), the app errors out. There is no serverless mode. (The wire-level caps/Carriage/memfd refinements are in Design decisions below.)

Build & run

zig build [-Dbackend=rgfw|kms]   # rgfw = desktop X11+GL (default); kms = bare-metal DRM+evdev
zig build test                   # protocol + vt unit tests + smoke (full pipeline via mono-cat)
zig build run | conformance | shell | term | files [-- cmd...]   # GUI server driving an example
zig build x11-adapter [-- xev | urxvt -e htop]           # run a real X11 app via Xvfb+ffmpeg (default: thunar)
zig build <name>-cat             # same example flattened to text via mono-cat (headless)
./zig-out/bin/monoverse <child-app> [args...]            # server spawns the app

The two backends are selected at compile time in server/backend.zig; the unselected one's libs aren't linked. KMS needs DRM master → a VM or a free VT as root (see gotchas). System deps: zig 0.16, libzstd; rgfw: X11/Xrandr/GL; kms: libdrm/xkbcommon.

Layout

Path	Role
protocol/protocol.zig	Wire format: frame types + encode/decode (zero-copy, NeedMore/Malformed). Handshake carries caps; Blit carries a Carriage (@"inline"/memfd); buffer_release is server→app. Has tests.
server/main.zig	Loop: input→event frames to child; child paint frames→framebuffer; present.
server/backend.zig	Seam: platform-agnostic Event types + compile-time Display selection.
server/backend_rgfw.zig	Desktop backend: RGFW window + GL blit of the CPU framebuffer.
server/backend_kms.zig	Bare-metal backend: DRM dumb-buffer scanout + HW cursor + raw evdev input.
server/{framebuffer,font,jpeg}.zig	RGBA8 framebuffer; stb_truetype font (loaded from data/font.tar.zst); JPEG decode.
server/transport.zig	Server↔app channel: socketpair + fork/exec (replaced the PTY — see decisions).
client/pty.zig	Spawn a real shell on a cooked PTY (winsize, $TERM). Used by term, not the server link.
client/vt.zig	VT/ANSI emulator (std-only, unit-tested): PTY bytes → cell grid. All ANSI quarantined here.
examples/term.zig	Real terminal app: PTY shell ↔ vt ↔ protocol. Encodes keys/mouse/wheel → PTY.
examples/{hello,conformance,shell}.zig	Demos: incremental paint; every-frame-type + event echo; self-contained shell v0.
examples/files.zig	VS Code-style file explorer: fold-unfold tree (left) + preview pane (right) — text head, dir listing, or JPEG painted as a real blit. Two foci (tree ↔ preview); all FS access via the 0.16 std.Io interface.
examples/x11-adapter.zig	Runs a real X11 program in headless Xvfb: ffmpeg x11grab→mjpeg pipe → JPEG blits; key/mouse/scroll → XTEST; one-shot window-fit (no WM). Links X11+Xtst; spawns Xvfb+ffmpeg+client.
tools/monocat.zig	Headless pseudo-server (flatten frames to text + inject input — see Testing).
tools/xsim.c	XTest GUI input simulator (⚠ injects into focused window — prefer mono-cat).

Design decisions (the non-obvious "why")

• Server↔app is a socketpair, not a PTY. The protocol self-frames and wants no TTY line discipline (which only mangled it). The socket also carries SCM_RIGHTS fd passing — now used for shared memfd buffers (see carriage decision below). The child's protocol rides stdin/stdout; stderr stays inherited for logging.
• ANSI is quarantined in the terminal app. The server/protocol never see escape sequences; term embeds the VT emulator and hosts the legacy shell on its own PTY. Two transports by design: socketpair (server↔term) + PTY (term↔shell).
• The server resolves platform input to semantics. Apps get codepoint, mods, and KeyName (canonical arrows/F-keys) — never raw scancodes. Backends do the mapping (only they know RGFW vs evdev codes).
• Pixel carriage is negotiated, not fixed. The handshake advertises Caps.fd_passing (true while the link is the local socketpair; a future remote transport says false — Transport.supportsFdPassing). When set, an app may publish a blit{carriage=memfd}: it shares a memfd ring by fd over SCM_RIGHTS (server reads via recvmsg, mmaps it, blitRgbaStrided straight from the buffer — pixels never cross the socket) and the server returns each slot with a buffer_release after the present-upload. Otherwise apps fall back to inline rgba/jpeg bytes. This is CPU shared memory (wl_shm-style); the socket+decode copies go, the framebuffer composite copy stays. dma-buf/GPU zero-copy is a later step. fd↔byte contract: attach the fd on the same sendmsg that begins the registering blit, one un-referenced registration at a time.

Testing

• mono-cat --trace <child> — dump/validate every frame (protocol-layer debugging).
• mono-cat --keys=SPEC | --click=C,R | --scroll=N <child> — inject input headlessly (fired once the app settles). --keys: literal chars + <up> <enter> <c-c> <f5> …. Verify the terminal's byte output against cat -v (use stty raw -echo for mouse), e.g. mono-cat --keys='<up><enter>' term /bin/cat -v → ^[[A. Deterministic; can't leak into your session (unlike xsim). Guarded by smoke_* tests in build.zig.
• VM harness (scripts/vm/): boots the KMS server under QEMU/virtio-gpu, screenshots via QMP (shot.sh), injects input via QMP input-send-event. The only way to test the bare-metal path (can't run under X). MONO_DISPLAY=gtk for a live window.
• Env: maim/xwininfo/ffmpeg for screenshots; MONO_DEBUG=1 enables KMS device/driver logs + a server [n] poll/read/draw loop heartbeat.

Gotchas

1. Zig 0.16 std churn: Init-based main, gutted std.posix (use extern "c"), std.c.getenv, unmanaged ArrayList. Grep /usr/lib/zig/std/ rather than guess.
2. drmModeDirtyFB freezes real GPUs — it's a blocking atomic commit on amdgpu/i915, needed only on virtio-gpu (BO in guest RAM). Gated by driverNeedsDirty.
3. evdev: classify devices — a gamepad/accelerometer's ABS otherwise pegs the cursor and fakes a freeze (loop fine, heartbeat ticking ⇒ suspect input). /dev/input needs root.
4. Touchpad ABS is absolute finger position — convert to relative deltas (BTN_TOOL_FINGER flags it); raw evdev doesn't do this for you like X does.
5. RGFW X11 never sets ev.key.mod (only its Windows path does) → query RGFW_isKeyDown for live modifier state. mono-cat bypasses RGFW, so verify GUI input by hand.
6. PTY raw-mode (historical): server link is socketpair now; still applies in client/pty.zig — set the pts raw in the parent before fork, or a binary stream mangles.

Status & next

The v1 milestones are all done: (1) protocol module — protocol/protocol.zig; (2) server v0 — server/main.zig (RGFW + GL upscale, CPU raster); (3) graphics — RGBA + JPEG via server/jpeg.zig (dirty-region retention is inherent: the CPU framebuffer keeps unchanged pixels, whole thing uploaded per present); (4) mouse — move/click/scroll end-to-end; (5) reference shell — examples/shell.zig (now superseded by the terminal below); (6) mono-cat — tools/monocat.zig; (7) conformance demo — examples/conformance.zig (exercises every frame type, mono-cat as oracle). The reference terminal (examples/term.zig + client/vt.zig) supersedes shell v0: runs $SHELL/any program with colors, modifiers, special keys, mouse, scrollback, and full-screen apps (vim/htop/less). Both backends confirmed on real hardware (rgfw desktop, kms on amdgpu VT).

"Replace X" track — at step 5: run without root. Steps 1–4 (backend seam, KMS out, evdev in + HW cursor, VM harness) done. Step 5: take DRM master + /dev/input via logind/seatd so it runs as a normal user. KMS polish after: double-buffer page-flip for tear-free vsync (single-buffer works on amdgpu, but tears).

Terminal M2 remaining (in client/vt.zig + examples/term.zig; test via mono-cat --keys/--click/--scroll vs cat -v, or render via term-cat):

1. DEC line-drawing charset (ESC ( 0) — most visible gap: charset designator is consumed but not mapped, so ncurses box borders render as lqqqk. Map G0 + SO/SI.
2. Wide/CJK (wcwidth) — putGlyph always advances 1 cell; double-width glyphs overlap.
3. Cursor-key app mode (DECCKM ?1) — emit SS3 arrows when set (normal forms work now).
4. Selection + clipboard + bracketed paste (?2004) — none exists yet; the big daily feature (mouse-drag select → copy/paste path → then bracketed paste).
5. Perf — render re-encodes the whole grid per batch; add dirty-row diffing if heavy.
6. Minor — autowrap ?7, origin ?6, tab stops, true pending-wrap, OSC title/links, scrollback reflow on resize.

x11-adapter (done): examples/x11-adapter.zig boots Xvfb sized to the framebuffer and launches an X client (default dbus-run-session -- thunar). Dual delivery path, chosen from the handshake caps. With fd_passing (local server) it runs ffmpeg … -f rawvideo -pix_fmt rgba, captures each frame into a 3-slot memfd ring (read straight into the mmap, no accumulator memmove), and presents blit{carriage=memfd} — the fd handed over once per slot via sendmsg, slots reused on buffer_release, frames dropped when all slots are in flight, ring re-made on resize (server rebinds on the fresh fd). Idle frames are skipped by byte-compare. Verified live under i3 (thunar renders, correct colors). Without fd_passing (remote) it falls back to the inline mjpeg path below. mjpeg fallback — capture+encode offloaded to ffmpeg x11grab -f mjpeg: read its pipe, split on JPEG SOI/EOI (baseline mjpeg byte-stuffs FF, so a marker scan is safe), and wrap each frame as a blit{jpeg}+present (server stb-decodes it). ~50× less wire than the old RGBA path (~55 KB vs 2.8 MB/frame), no per-pixel work in-process, and x11grab draws the cursor for free. Identical idle frames are byte-equal and skipped. Input (key/mouse/scroll) still replays via XTEST — keys (incl. shift

• named/F-keys) and mouse verified. No WM, so layoutClient does a one-shot resize+focus of the largest-area window; focusClient re-aims keys per keystroke. PR_SET_PDEATHSIG makes Xvfb/ffmpeg/client die with the adapter; the client's stdin/stdout are kept off fd 0/1 (the protocol channel).

Sizing/resize: the server sends an initial geometry at the window's default size, then a second, larger one once the WM sizes the window — so we must not lock the first. This Xvfb can't be live-resized (RandR's RRCreateMode is a no-op here; the max bounding box is pinned to the start size). So x11-adapter boots Xvfb once at MAX_DIM² (4096²) and reconciles size changes in the loop: a resize burst is debounced (~250 ms), then capture re-points — update width/height, respawn ffmpeg with the new -video_size, and re-fit the client (layoutClient resizes the largest viewable top-level to fill, since there's no WM). Geometry past MAX_DIM clamps (overflow stays black). On respawn we close(ff.fd) before waitpid and SIGKILL the old ffmpeg — otherwise it blocks flushing into the pipe we stopped reading and waitpid hangs forever, freezing the adapter so the server's post-resize black framebuffer never repaints (this was the "manual resize → black screen" bug). Verified live under i3: float + repeated resizes keep thunar filling the window, no hang, ffmpeg tracks each settled size.

Known gaps / next: always full-frame mjpeg (no dirty rects — could add -vf mpdecimate to drop idle dups, or XDamage+per-rect JPEG); fixed -q:v 5 quality; no automated smoke (needs an Xvfb+ffmpeg+thunar host).

Other known gaps: no Malformed resync (drainFrames drops the buffer); no app→server geometry-request; single fixed font size (geometry.scale_milli unused); no graceful child-crash UI. Fonts: data/font.tar.zst decompressed to RAM at startup — regenerate per the .gitignore recipe (zstd needs a real file, not a pipe, for known content size).