VP-404 — Live Video Sampler

examples/vp404 is an SP-404-style video sampler: 16 pads, each holding a HAP clip, triggered live from the GUI, MIDI, OSC, or a built-in polyphonic step sequencer, all composited through rustjay-mixer.

cargo run -p vp404                                 # 16 pads, empty
cargo run -p vp404 -- path/to/clip.mov             # seed pad 0 with a clip
cargo run -p vp404 --features capture,api,link     # + live sampling, web pad grid, Ableton Link

The launch clip is optional (or set VP404_CLIP). Non-HAP files dropped onto a pad are converted to HAP5 in the background and cached as a sidecar next to the original.

Pads

  • Trigger modes per pad: Gate (play while held), Latch (toggle), OneShot (play to the out point).
  • Playback modes: Free (a speed control drives the frame counter, like the hardware) or Synced (frame derived from the engine beat clock and a beat division — clips stay locked to tempo).
  • In/out points, speed, mode, and division are engine parameters (pad<i>_trig, ch_pad<i>_speed, …), so MIDI mapping, OSC, and LFO routing all reach them like any other effect parameter.

Live sampling (capture feature)

The ⏺ Record button in the pad grid captures N frames from the engine's active input (webcam, Syphon, NDI, …) straight into the selected pad:

  • Capture is gated on the engine's InputState::frame_seq counter, which the engine bumps once per uploaded source frame — so "record 300 frames" means 300 source frames whatever the source's rate, with no fps configuration.
  • Frames are read back GPU→CPU asynchronously (double-buffered, so a slow map doesn't drop the next frame), encoded to HAP5 on a worker thread, and the finished clip is assigned to the pad.
  • The clip is encoded at the measured capture rate, so playback is always 1:1 wall-clock speed — a 60 fps source sampled for 300 frames records in ~5 s and plays back in ~5 s at 1× speed.

Recordings land in samples/recorded/. VP404_PROBE=1 cargo run -p vp404 -- clip.mov prints any clip's resolution, frame count, and fps without starting the GUI.

Sequencer

A polyphonic step sequencer (own tab) is slaved to the engine beat clock and triggers pads via the same command channel as the UI, with per-step gate lengths and multiple patterns.

Output & control

  • Output tab: NDI / Syphon (macOS) / Spout (Windows) / V4L2 senders and ProRes disk recording of the master output.
  • api feature: REST/WebSocket API plus a browser pad-grid page for triggering pads from another machine.
  • link feature: Ableton Link tempo sync — pick "Ableton Link" as the sync source in the Audio tab and synced pads/sequencer follow the session tempo.

Where to look

AreaFile
Plugin assembly, pad params, trigger edge-detectionsrc/main.rs
Pad state machine (advance is pure + unit-tested)src/pad.rs
Pad bank, PadCmd channel UI → rendersrc/bank.rs
Live sampler (readback → HAP5 encode)src/live_sampler.rs
Pad grid tab + HAP conversion jobssrc/grid_tab.rs
Step sequencer engine / tabsrc/sequencer/, src/sequencer_tab.rs
Output senders tabsrc/output_tab.rs