Hi all,
My Pi 3B+ is a slow small machine so I vibe-coded openflexito, with the idea that the Pi is the relay (and set up once or not at all, so all I need to do is power it up), and the browser is the brain.
The Pi (3B+, Sangaboard v0.5.5, IMX219) streams MJPEG, relays motor/LED commands over UART, blinks the ACT LED and serves the pre-compiled web app, so nothing heavy. Everything else runs in a Svelte 5 + TypeScript app on your laptop: autofocus, camera↔stage calibration, click-to-move and drag-to-pan, scanning + stitching, gallery, measurement, and the image processing.
Some of what that buys, since it’s all client-side:
- RAW: the sensor’s 10-bit Bayer frame over HTTP (/raw.bin), developed in a worker the way the ISP would but losslessly: black level, the tuning file’s ALSC tables or a measured flat field, the frame’s own WB gains and CCM, Malvar-He-Cutler or RCD demosaic → 16-bit PNG. Also writes a proper DNG (correct AsShotNeutral/ColorMatrix1, EXIF, GainMap opcodes) that RawTherapee/darktable open.
- Stacks: focus stacking (Laplacian-pyramid fusion, slices sub-pixel aligned by phase correlation + NCC, Lanczos-3 resampling), LED/exposure stacking, true HDR from a RAW exposure bracket (Debevec), and depth-from-focus.
- Pixel-shift super-resolution using the full 2×2 camera-stage matrix, so the pattern lands right despite the ~4° stage/sensor rotation.
- Scanning with per-tile or sub-grid autofocus and a robust height-map surface fit, stitched with outlier-rejecting position solve and per-tile gain equalisation.
- Time-lapse, organism tracking, video with a stabiliser, measurement/scale bar (defaulting to the Block Stage paper’s 88 nm/step xy, 50 nm/step z), macros, and optional in-browser object detection/image search via Transformers.js.
Crucially, no hardware needed to hack on it: a fake camera renders a stage-coupled specimen (moves scroll it, z defocuses it), so calibration, autofocus and scans run end to end on a laptop. There’s a Playwright e2e suite (34 steps) driving a real browser against it, plus device pytest and webapp vitest suites.
Status / honesty section: the early milestone (stream, stills, RAW fetch, moves, LED, calibration, autofocus, 2×2 scan) is verified on real hardware. The recent capture-quality overhaul (RAW averaging/flat field, HDR brackets, super-resolution, the stacking fixes) is green against the fake device but not yet verified, and I’m going to iterate with the intent to keep my microscope connected over LAN for higher bandwidth. So: works-for-me-on-a-fake-Pi caveats apply to some features for now, and TODO.md lists what’s pending.
This is not a fork of the v3 server but I’ve used it as an algorithm reference (backlash semantics, stage behaviour) and noted divergences. GPL-3.0-or-later, same as upstream.
Repo: GitHub - sdawka/openflexito · GitHub
Would love eyes on it, and I’m curious whether the “Pi as dumb relay” split seems sane to people who know this hardware better than I do.