I’ve hinted above that this is an AI+Human development. It allows for fast development but at the same time needs careful analysis at each stage - even for experimental work like this.
One of the great things is that the AI will maintain a README file as documentation as you’re going along, so for anyone interested:
WebGL Microscope Tile Stitcher
A GPU-accelerated image stitcher structured like OpenCV’s cv::Stitcher / cv::detail::* pipeline, but with the per-pixel stages implemented as WebGL2 shaders instead of CPU/SIMD code.
Files
| File |
Pipeline stage |
OpenCV equivalent |
gl-utils.js |
shared GL plumbing |
cv::Mat / cv::UMat boilerplate |
lens-correction.js |
radial distortion + chromatic aberration |
cv::undistort, per-channel |
feature-detector.js |
corner detection + descriptors |
cv::ORB / cv::AKAZE |
orb-descriptor.js |
optional rotation-invariant binary descriptor |
ORB’s “Oriented FAST + Rotated BRIEF” (rotation half only) |
matcher.js |
descriptor matching |
cv::BFMatcher |
homography.js |
DLT + RANSAC |
cv::findHomography |
matching-graph.js |
pairwise match graph → spanning tree |
BestOf2NearestMatcher + leaveBiggestComponent + HomographyBasedEstimator |
bundle-adjuster.js |
global homography refinement |
cv::detail::BundleAdjuster |
phase-correlation.js |
GPU (WebGL2) FFT-based local translation estimation |
— (used by nonrigid-field.js) |
nonrigid-field.js |
per-tile local displacement field |
NRStitcher’s local correction (see below) |
exposure-compensator.js |
per-tile brightness gain from overlap stats |
cv::detail::GainCompensator |
warper.js |
perspective warp into panorama space |
cv::warpPerspective / PlaneWarper |
blender.js |
feather (weighted) blend |
cv::detail::FeatherBlender |
stitcher.js |
orchestrator |
cv::Stitcher |
demo.html |
file input + canvas UI |
— |
Run it by serving the folder over HTTP (ES modules need a real origin, not file://) and opening demo.html, e.g.:
npx serve .
File format support
Any format a browser’s <img> can decode (PNG, JPEG, WebP, …) works with no extra setup. TIFF is common for microscopy exports but no browser can decode it natively – demo.html loads UTIF.js from a CDN (a dependency- free pure-JS TIFF decoder) specifically for this, and decodes .tif/ .tiff files to a canvas before handing them to WebGLStitcher.stitch() (a canvas has the same .width/.height as an <img> and is an equally valid WebGL texture source, so nothing else in the pipeline needs to know the difference). This is the only external dependency anywhere in this project, and it’s confined to the demo’s file loader – stitcher.js and every pipeline module remain dependency-free.
Notes:
- Multi-page/multi-frame TIFFs (e.g. z-stacks) use only the first page – a console warning is logged if more than one page is found.
- Some exotic TIFF compressions may not be supported by UTIF.js; a failed decode raises a specific error naming the file rather than a generic one.
- If you’d rather not pull from a CDN, pre-convert TIFFs to PNG externally (e.g. via ImageMagick, Fiji/ImageJ, or
tifffile in Python), or vendor UTIF.js locally and change the <script src=...> in demo.html to point at your local copy.
What “spherical” correction means here
Your microscope’s “spherical error” is radial lens distortion (barrel or pincushion), not the equirectangular spherical projection OpenCV’s SphericalWarper uses for wide-FOV panorama photography. For tiled microscopy, a planar homography is the physically correct model between overlapping tiles, so warper.js uses warpPerspective-style planar warping, while lens-correction.js removes the radial distortion per-objective before stitching. If your optics genuinely need a different distortion model (e.g. a strong fisheye), swap the k1/k2 radial model in lens-correction.js for a division or fisheye model — the rest of the pipeline is unaffected.
Chromatic aberration correction
lens-correction.js samples the R, G and B channels at three slightly different radial positions (chromaScale), correcting lateral CA where each wavelength focuses at a different magnification. Calibrate k1, k2, and chromaScale once per objective/condenser setting using a calibration target (e.g. a grid slide), then reuse those constants for every acquisition with that objective.
Pipeline flow (stitcher.js)
- Upload each tile, run
LensCorrector (GPU).
- Run
FeatureDetector (Harris response on GPU, NMS + descriptors on CPU).
- Match tile pairs (
matcher.js) and estimate a homography per pair with RANSAC (homography.js).
matching-graph.js turns those pairwise homographies into one consistent set: it keeps only pairs with enough inliers, drops any tile that isn’t reliably connected to the rest, and builds a maximum spanning tree (weighted by inlier count) from every tile back to the most-connected tile, which becomes the reference frame.- If
bundleAdjustment.enabled, bundle-adjuster.js jointly refines every tile’s homography against every confident pairwise match found (not just the spanning tree) before anything else uses them.
- If
nonRigidRefinement.enabled, nonrigid-field.js estimates a smooth per-tile local displacement field from dense phase-correlation correspondences in each overlap (see below), layered on top of the homographies from step 5.
- If
exposureCompensation.enabled, exposure-compensator.js estimates a per-tile brightness gain (see below) before the real warp/blend pass.
- Compute panorama bounds from all included tiles’ warped corners, warp each into the shared canvas (applying its exposure gain and local displacement field, if any), and additively accumulate weighted color + weight (
warper.js).
- Normalize (divide by accumulated weight) to get the final composite (
blender.js), blit to the visible canvas.
Descriptor options
detector.descriptorType picks how keypoints are described for matching (step 2 above), independent of everything else in the pipeline – matching-graph.js, bundle-adjuster.js, homography.js, warper.js, blender.js, exposure-compensator.js, and nonrigid-field.jsnever touch descriptor internals, only keypoint x/y and the resulting homographies, so switching this doesn’t affect anything downstream.
new WebGLStitcher(canvas, {
detector: { descriptorType: 'orb' }, // default: 'patch'
});
'patch' (default) – a mean-subtracted, normalized square patch (detector.patchRadius, default 15 → 15×15), compared with Euclidean distance. Fast, simple, and robust for the mostly-translational overlaps typical of a tile mosaic – but it samples a fixed, axis-aligned grid, so it is not rotation-invariant: the same physical corner detected in two tiles related by more than a few degrees of rotation produces two descriptors that don’t look alike, and the ratio test in matcher.js will reject the (correct) match.'orb' – orb-descriptor.js, borrowing ORB’s “Oriented FAST + Rotated BRIEF” idea (minus the FAST part, since corners here already come from the GPU Harris response): each keypoint gets an orientation from the intensity centroid of a circular patch around it, then a fixed set of sample-point pairs (a reproducible Gaussian-distributed pattern, generated once by a seeded PRNG – not OpenCV’s specific trained pattern) is rotated by that orientation and compared pairwise into a binary descriptor, matched via Hamming distance. Use this when tiles can be rotated relative to each other by more than a few degrees. Sub-options: detector.orbPatchRadius (default 15, radius for both the orientation estimate and the sample pattern) and detector.orbDescriptorBits (default 256).
Note: Harris corner detection (the response computed in feature-detector.js’s GPU pass) is already rotation-invariant on its own, since it comes from the eigenvalues of the local structure tensor – this option only affects whether the descriptionof each detected corner is rotation-invariant too. Scale invariance is out of scope for both descriptor types; a mosaic mixing magnifications would need a separate image-pyramid feature.
Bundle adjustment
Off by default – turn it on with:
new WebGLStitcher(canvas, {
bundleAdjustment: { enabled: true },
});
What it does: matching-graph.js’s spanning tree gives every tile exactly one path of chained homographies back to the reference frame, using only the tree’s edges – any other pair that matched confidently but wasn’t picked for the tree is otherwise ignored, and small per-pair errors can compound along a long chain (“drift”). Bundle adjustment takes the spanning-tree result as a starting point and refines every included tile’s homography simultaneously (except the reference tile’s, held fixed at identity) to minimize reprojection error across every confident pairwise correspondence at once – so residual error gets distributed and averaged across the whole mosaic instead of accumulating down one path.
This runs as Levenberg-Marquardt on the CPU with a numerically estimated Jacobian (the same “GPU for pixels, CPU for optimization” split RANSAC already uses). One thing worth being explicit about: OpenCV’s cv::detail::BundleAdjusteroptimizes camera intrinsics + rotation for a spherical/cylindrical panorama model; this refines the 8 free parameters of each tile’s planar homography directly, which is the correct model for tiled microscopy (see the “spherical vs planar” note above) – same objective (minimize reprojection error), different parameterization. Sometimes called “global homography alignment” in the mosaic-stitching literature.
Every run with enabled: true logs, via console.log/console.table:
- how many correspondences and edges were used, how many iterations ran, and whether it converged
- reprojection error (RMS, in pixels) before and after refinement, and the percentage improvement
Tunable sub-options (pass any subset under bundleAdjustment):
| Option |
Default |
Effect |
maxIterations |
20 |
Cap on LM iterations. Raise if the log shows it stopping without converging and RMS is still improving; lower to bound worst-case runtime. |
convergenceTolerance |
1e-6 |
Stops early once a step’s relative cost improvement drops below this. Lower = keeps refining longer for smaller gains. |
initialLambda |
1e-3 |
Starting Levenberg-Marquardt damping factor. Rarely needs changing. |
Performance note: this is a dense solve over 8 * (tiles - 1) parameters with a numerically estimated Jacobian, recomputed every iteration – reasonable for up to a few dozen tiles in a browser tab, but the per-iteration cost grows roughly cubically in tile count via the normal-equations solve. For large mosaics, consider a sparse/analytic Jacobian (this module’s residual function stays the same; only the Jacobian and linear solve would need to change), or skip this stage and rely on the spanning tree alone.
Non-rigid refinement
Off by default – turn it on with:
new WebGLStitcher(canvas, {
nonRigidRefinement: { enabled: true },
});
What it’s for: bundle adjustment (and RANSAC before it) fits one rigid projective homography per tile. That’s the right model for lens/camera geometry, but it can’t represent the specimen itself deforming slightly between when two overlapping tiles were captured (drift, radiation damage, drying, live-sample movement). This stage adds a per-tile local correction on top of the homography to absorb exactly that residual – it’s the “local correction” idea from NRStitcher (Miettinen et al. 2019, Bioinformatics 35(24):5290-5297), adapted to run as a refinement layer over our (already more capable, full-projective + bundle-adjusted) global placement rather than replacing it. See the “Why not RANSAC/homography alone” note below for why this division of labor makes sense.
How it works:
- For every overlapping tile pair (from the match graph), lay a regular grid of sample points across the overlap.
- Every grid point in that overlap is correlated in one batched GPU pass: their block-extraction, both forward FFTs, cross-power spectrum, and inverse FFT all run together across a shared “atlas” texture (a grid of NxN cells, one per point) via
phase-correlation.js’s correlateBatch, ending in a single gl.readPixels that retrieves every point’s correlation surface at once – not one full round-trip per point. See “GPU batching” below for why this matters as much as it does.
- Low-confidence points (flat/featureless regions, or blocks with no real match) are discarded. The residual at each accepted point is split evenly between the two tiles – neither is treated as “more correct.”
- Each tile’s scattered correction samples are splatted into a small coarse grid (
fieldGridSize, default 12×12) covering its own pixel extent, smoothed, and uploaded as a texture.
warper.js samples that texture (manual bilinear via texelFetch, so no OES_texture_float_linear extension is needed) and nudges each pixel’s source coordinate by the interpolated correction before the final color sample. Grid cells with no nearby samples stay at exactly zero, so tile interiors far from any overlap are completely unaffected – correction only ever happens near where it was actually measured.
GPU FFT implementation note: phase-correlation.js’s butterfly-stage shader is derived directly from this project’s own (already-verified) CPU radix-2 FFT loop – bit-reversal permutation, then log2(N) in-place butterfly stages – restructured into an out-of-place, ping-ponged form a fragment shader can run per-pixel. It’s the same algorithm, not a different GPU-FFT formulation (e.g. Stockham autosort) that merely agrees with it; see the comment at the top of that file for the full derivation. Runs in single precision (GLSL highp float, effectively 32-bit) rather than the CPU version’s double precision – not a concern at the block sizes used here (peak-finding for a translation estimate doesn’t need double precision; production GPU FFT libraries run single-precision as standard practice).
Why not RANSAC/homography alone, and why not phase correlation alone: phase correlation needs some texture in every grid cell (a flat/blank region contributes nothing) and only cleanly resolves pure translation – if the true local motion includes rotation or scale within a block, the correlation peak degrades. Sparse feature matching handles large rotation/scale/projective effects fine (RANSAC just needs a handful of good corners anywhere in the overlap) but can’t easily give you a dense per-pixel correction. Running this stage after bundle adjustment means the residual it has to find is already small and close to pure translation – exactly phase correlation’s strong suit – while the global model already handled everything phase correlation is bad at.
Every run with enabled: true logs, via console.log/console.table, per-edge grid point / accepted point counts and mean correction magnitude, and warns if every point was rejected (in which case every tile’s field is all-zero – a safe no-op, not an error).
Tunable sub-options (pass any subset under nonRigidRefinement):
| Option |
Default |
Effect |
gridSpacing |
96 |
World-space spacing (px) between sample points in each overlap. Smaller = denser correction, more compute. |
blockSize |
64 |
Phase-correlation block size (px), rounded up to the next power of 2 if needed. Larger blocks give more texture to correlate against but a smaller usable overlap area and higher cost; must be smaller than the tile overlap. |
minConfidence |
2.0 |
Correlation-peak confidence threshold below which a grid point is discarded. Raise to be more conservative (fewer, more trustworthy points). |
maxCorrection |
32(≈blockSize/2) |
Sanity bound on accepted correction magnitude, in px. Corrections larger than this are treated as spurious matches, not real deformation. |
fieldGridSize |
12 |
Resolution of each tile’s output correction grid. Higher = can represent more spatially-varying deformation, at the cost of a noisier fit from the same sample density. |
smoothingPasses |
1 |
Box-blur passes applied to each tile’s correction grid before upload, to reduce cell-to-cell noise. |
MVP scope note (see nonrigid-field.js for the full comment): this is deliberately simplified relative to the paper it’s based on – single-resolution phase correlation only (no coarse-to-fine pyramid; relies on bundle adjustment having already removed large misalignment so the residual is small), and no cross-neighbor region blending at >2-tile overlaps (each edge’s contribution to a tile is averaged in with the others when the field grid is built, but not iteratively reconciled the way the paper’s region-based StL-T is). It’s also a single scalar-per-axis (dx,dy) correction, not a full local affine/rotation model – see the “why not phase correlation alone” note above for why that’s an acceptable trade for a refinement layer specifically.
Side Note - this could be improved by a hybrid pyramid scheme for areas of non-matching.
GPU batching note: the naive version of this (one full extract→FFT→ FFT→cross→invFFT→readPixels round-trip per grid point) turned out to be dominated not by the FFT math but by two fixed costs multiplied by grid point count: dozens of tiny texture/framebuffer allocations per point, and one hard CPU/GPU sync (gl.readPixels drains the whole GPU command queue) per point. Two optimizations were applied in order, each verified against real profiling before moving to the next:
- Buffer pooling – reusing a small fixed set of scratch buffers per block size instead of allocating/freeing ~45 textures+framebuffers per point. Measured impact: ~2.3x (a 5000ms non-rigid stage dropped to ~2170ms on a real tile set), confirming allocation churn was a bit over half the original cost.
- Atlas batching (current implementation) – every grid point in one overlap is packed into a shared “atlas” texture (a grid of NxN cells) and every FFT stage runs once across the whole atlas in parallel, ending in one
readPixels for the entire overlap instead of one per point. This is what actually addresses the sync-stall half of the cost that pooling alone couldn’t touch. correlateBatch (in phase-correlation.js) handles this transparently – it also automatically splits into multiple atlas passes if a single overlap has more grid points than fit under MAX_TEXTURE_SIZE at the chosen blockSize, so there’s no hard cap on overlap size to configure around.
Cost now scales with grid points per overlap (~overlap area / gridSpacing²) times ~6*log2(blockSize) shader passes – but as onebatched pass over however many points fit in an atlas, not as that many separate round-trips. Increase gridSpacing or decrease blockSize if this is still the bottleneck for very large or densely-sampled overlaps.
Exposure/illumination compensation
Off by default – turn it on with:
new WebGLStitcher(canvas, {
exposureCompensation: { enabled: true },
});
If nonRigidRefinement is also enabled, its output is automatically fed into this stage’s own low-resolution statistics pass, so exposure gains are measured against the locally-corrected geometry rather than just the bundle-adjusted one – no extra configuration needed, stitcher.js wires this up for you.
What it does: for every pair of tiles the match graph found overlapping, it warps both tiles at a cheap reduced resolution (workingScale, default 0.25) and compares mean luminance in the region where they actually overlap. Those pairwise comparisons feed a small weighted least-squares solve for one brightness multiplier (“gain”) per tile – tiles are pulled toward agreeing with their neighbors, with a regularization term (regularization, default 0.05) anchoring everything toward gain 1.0 so the solve can’t drift or blow up on weakly-connected tiles. The gain is then applied when each tile is warped into the final composite, so neighboring tiles agree in brightness before the feather blend runs – this is what actually smooths out a visible seam that feather blending alone can leave when tiles were captured under slightly different exposure/illumination.
Every run with enabled: true logs:
- a table of the final gain applied to each tile (and which tile was the reference), via
console.table
- a table of every overlap pair’s stats (pixel count, mean luminance each side) that fed the solve
- a warning if no pairs had enough overlap to compute anything useful (gains all stay at 1.0 in that case – try lowering
exposureCompensation.minOverlapPixels)
This logging happens independently of the debug option, since it directly changes the visible output and you’ll usually want to see it.
Tunable sub-options (pass any subset under exposureCompensation):
| Option |
Default |
Effect |
workingScale |
0.25 |
Resolution fraction used for overlap statistics. Lower = cheaper but noisier; exposure differences are low-frequency so this rarely needs to go above ~0.5. |
minOverlapPixels |
50 |
Minimum overlapping (working-resolution) pixels for a tile pair to be trusted. Raise if you’re getting gains driven by tiny, unreliable overlap slivers. |
regularization |
0.05 |
How strongly each tile’s gain is pulled back toward 1.0, scaled by how much overlap data supports that tile. Raise for more conservative (closer to 1.0) gains; lower to let the data correct brightness more aggressively. |
Note: this estimates a single scalar gain per tile (uniform across R/G/B), which handles exposure/illumination differences but not color-cast differences between tiles. A per-channel variant would only need exposure-compensator.js’s overlap-statistics loop to track sums per channel instead of luminance – the rest of the pipeline is unaffected.
Tile ordering
The input image array is treated as an unordered set by default (pairing: 'all' in WebGLStitcher options) — every pair of tiles is matched, and matching-graph.js figures out how they actually connect regardless of what order they were passed in. This is the right default when you don’t yet have a scan-path planner feeding in acquisition order.
Two ways to speed things up once you do know more about the layout:
pairing: 'sequential' — only match each tile to the next one in the input array. Fast (O(n) pairs instead of O(n^2)), but only correct if the array is already ordered along a single scan line.options.pairs — supply your own list of [i, j] index pairs to try (e.g. only a tile’s raster/serpentine grid neighbors). This is the hook for a future scan-path-aware planner: it only needs to produce a pair list, not touch any matching/warping/blending code.
stitch() returns { canvas, referenceIndex, includedIndices, excludedIndices } — check excludedIndices for any tiles that had no reliable overlap with the rest and were left out of the panorama.
Discovering pairs quickly: test at reduced resolution first
pairing:'all' is O(n²) candidate pairs, and feature detection’s dominant cost (scanning every pixel before non-max suppression) scales with pixel count – both make the full search expensive on large tiles at scale (e.g. 16 tiles at 1200×1200 can mean 30+ seconds before matching even starts). But whether two tiles overlap at all is largely resolution-independent: a real overlap produces a real, if weaker, correspondence signal even at low resolution. demo.html’s “Preview scale” control downsamples every tile (via canvas, before upload) so you can run the expensive pairing:'all' search once at, say, 25% scale – 16x fewer pixels to scan – read off which pairs came back confident in the WebGLStitcher({ debug: true }) attempt table, and reuse exactly those as an explicit options.pairs list for the real run at 100% scale. The full-resolution run then only tests genuine neighbors instead of every non-overlapping pair, which is both much faster and removes the noise pairs that would otherwise compete during matching (see “Rejected homographies” above for how that noise causes real problems, not just wasted time).
This is a discovery tool, not a substitute for full-resolution matching: fewer pixels means fewer/weaker correspondences, so treat a low-res “not confident” as inconclusive (that pair might still be a real overlap with more detail to work with) – only treat a low-res “confident” as reasonably trustworthy evidence of a genuine overlap.
Output canvas size (snapBoundsEpsilon)
Because canvas width/height are derived by projecting every tile corner through its RANSAC-estimated homography, the X and Y extents are two independent floating-point numbers with their own sub-pixel residual – so the output can come out as, say, 1000×1001 rather than a clean 1000×1000 even for tiles that are essentially grid-aligned. This is harmless (nothing is lost or misplaced), but if you want a deterministic size, set snapBoundsEpsilon (default 0, disabled) to round each bound to the nearest integer when it’s within that many pixels of one, e.g.:
new WebGLStitcher(canvas, { snapBoundsEpsilon: 0.5 });
Exhaustive pairing is O(n^2) RANSAC calls, each with its own iteration budget (options.ransac.iterations, default 2000) — fine up to a few dozen tiles in a browser tab, but worth switching to sequential or a custom pairs list for large mosaics.
Troubleshooting “No tile pair produced a reliable homography”
The error message now includes the actual match/inlier counts for the best few pairs tried (and WebGLStitcher({ debug: true }) logs a full table via console.table for every pair, pass or fail). Read those numbers before changing anything:
- 0 matches on pairs that should overlap — the detector isn’t finding keypoints in the overlap region. Try lowering
detector.qualityLevel (more permissive) or raising detector.maxFeatures.
- Matches exist, 0 inliers — the matches are likely false positives. Microscopy images with repetitive texture (fields of similar-looking cells) confuse small descriptor patches; try raising
detector.patchRadius for a more distinctive (larger) patch, or lowering matcher.ratio (stricter ratio test). Also try raising ransac.threshold slightly in case real matches are being rejected by too tight a reprojection tolerance.
- Some inliers, just under
ransac.minInliers — lower ransac.minInliers, or raise detector.maxFeatures/lowerdetector.qualityLevel to get more tentative matches feeding RANSAC.
- Tiles genuinely don’t overlap — check with
pairing: 'sequential' or an explicit pairs list restricted to tiles you know overlap, to rule out the graph matching unrelated pairs.
- Enough inliers, but marked
implausible — the fitted homography had enough inliers but failed a geometric plausibility check — see “Rejected homographies” below for the two distinct ways this can happen.
Rejected homographies (implausible geometry)
Inlier count alone doesn’t guarantee a homography makes sense, or that applying it won’t visibly warp a tile. estimateHomographyRANSAC (homography.js) runs two independent plausibility checks after fitting a homography with enough inliers – each catches a different failure mode, and the attempt table (WebGLStitcher({ debug: true })) and the “no reliable homography” error both report which one fired via a reason field, so you can tell them apart instead of guessing:
'degenerate-near-data' — with enough candidate pairs (e.g. pairing:'all' on a large tile set, which tries every pair, most of which don’t actually overlap), repetitive texture can produce enough spurious matches between two tiles that don’t overlap at all to clear ransac.minInliers with a nonsensical transform. This check maps the fitted (inlier) points’ own bounding box through the homography and rejects it if a corner lands at/near infinity, the mapped shape self-intersects, or the area is absurdly rescaled relative to the source (ransac.minAreaRatio/maxAreaRatio, default 0.05/20).
'warps-full-tile' — a homography can pass the check above (it looks perfectly sane right where the correspondence data is) and still produce visible warping once applied across the entire tile during warping. This happens because RANSAC/DLT fit from a handful of points confined to a narrow overlap strip is inherently underconstrained outside that strip – the fit can pick up spurious curvature that only becomes visible far from the data. This second check maps the tile’s actual full extent (not just the inlier bounding box) through the homography, with tighter bounds appropriate for what a real adjacent-tile transform should look like (close to a similarity transform): area ratio withinransac.fullTileMinAreaRatio/fullTileMaxAreaRatio (default 0.3/3.0), edge-length ratio (catches trapezoidal/keystone distortion specifically, which can pass an area-ratio check while still looking clearly wrong) within ransac.fullTileMaxEdgeRatio(default 3.0), and the mapped quadrilateral must stay convex (non-self-intersecting).
If you’re seeing either on a large tile set, both point at the same underlying cause: correspondence data that’s too sparse or too spatially concentrated to constrain a full 8-DOF projective homography reliably. Try, in order:
- Widen the correspondence spread: raise
detector.maxFeatures and/or lower detector.qualityLevel to find more keypoints across more of each tile, not just its strongest few corners.
- Loosen
matcher.ratio to admit more tentative matches for RANSAC to work with (raises noise too, but the plausibility checks filter that back out).
- If you know roughly how tiles are laid out, switch from
pairing:'all' to pairing:'sequential' or an explicit pairs list – this removes the ~most candidate pairs that are noise (don’t actually overlap) in the first place, which both directly prevents 'degenerate-near-data' rejections and stops that noise from crowding out genuine pairs’ matches during the ratio test.
'warps-full-tile' specifically may also mean ransac.minInliers is too low for how sparse the real correspondence data is for this tile set – raising it discards fits before they even reach the plausibility check, at the cost of needing stronger matches to accept a pair at all.
A side note here - the AI keeps suggestions for further development but for now the focus is on the non-rigid refinement as that should deform the image to match.
I may create a ghosting detector for debugging - an inverse phase correlation on the proposed match. The result will be a spike in the input where the matching fails and ghosting occurs. This could then trigger a rethink/deeper processing in the pipe for that match. It becomes complex because the image deformation changes global edges, so the alternative is then to resample the tile to fit. Data transformation has implications on ‘accuracy’ of the dataset.
123667 ms — output 2771x2444px
