Homography
RANSAC Problem
HomographyProblem is a type alias for MinimalSolveProblem{HomographyMat{T}, 4, T, ...} — a 2D-2D correspondence problem with the 4-point minimal solver and, for LO refinement, Levenberg–Marquardt on the minimal 8-dof sl(3) chart about the current H (Sampson-whitened transfer error).
VisualGeometryToolkitEstimation.HomographyProblem — Type
HomographyProblem(pairs; sampler=nothing, backend=NativeSolver(),
src_map=I, dst_map=I, model_frames=nothing)Construct a HomographyProblem from 2D-2D point correspondences.
pairs: any correspondence container satisfying thesources/targetsprotocol (Correspondences,StructArray{<:Pair}, plain Pair vector). The PROSAC prior is its:scoreattribute column whenpairsis aCorrespondencesthat has one (VGCwithattrs(cs; score = s), the descriptor matcher's output): sampling ordered by descending score; uniform otherwise. Endpoints carry their observation noise (VisualGeometryToolkitCoreconventions/observation-noise.md): plain = unit isotropic in the point's own frame,Uncertain(p, CovMat(Σ))= its covariance,exact(p)= none (not both endpoints). The constraint is whitened byG₁Σ₁G₁ᵀ + G₂Σ₂G₂ᵀin scoring, the refit and the lifts alike; a board or world plane isexact, so its residual is the transfer error in pixels.sampler: override the auto-built sampler (advanced).src_map/dst_map: per-view observation maps, data frame → model frame. Identity (the default) means the data frame IS the model frame: pixels in, a pixel-frameHout. Non-identity maps declare a calibration instead of baking it into the data — supply PIXELS andK⁻¹maps, and the returnedHis in the model frame the maps point at, verified to 3.7e-13 against a known calibratedH. Scoring, the local optimizations and the consensus gate all treat thatHconsistently: residuals are measured in the data frame (pixels), the returned model stays in the model frame.model_frames: the(source, destination)frame tags of the model frame, checked against LABELED maps by VGC's junction law at construction. Defaultnothingmeans(ImageFrame{:src}, ImageFrame{:dst}).
Which frame is which: declared, then checked against the maps
The model frame of a mapped homography is not a property of HomographyMat — and it is NOT read off the maps: the maps are what gets checked, so the reference they are checked against is declared separately. Raw-matrix maps (and unlabeled HomographyMaps) are the unchecked wildcard and need no declaration. Labeled maps must land in the declared frames, or construction is a MethodError; a calibrated H whose K⁻¹ maps are labeled into the z = 1 chart therefore declares it:
HomographyProblem(pixel_cs; src_map = C₁, dst_map = C₂,
model_frames = (GnomonicFrame{:src}, GnomonicFrame{:dst}))Limit: mapped homographies are PINHOLE by construction
An observation map is a MATRIX. A general camera's back-projection is not — it is nonlinear, and cannot be expressed as a map at all. So anything routed through src_map/dst_map is restricted to pinhole cameras, and this path is a pinhole island for the same structural reason as the relative-pose bundle's K matrices.
Worth being exact about where the restriction comes from: a calibrated homography is perfectly well defined on bearings — f₂ ∼ H f₁ needs no affine plane — so the limit is imposed by the MAP MECHANISM, not by the geometry. Camera-general calibrated H would take back-projection in place of K⁻¹, which is a separate decision alongside the relative-pose bundle carrying camera models. Nothing is lost for a calibrated pinhole benchmark.
HomographyProblem(pair::PoolPair{<:HomographyMat}; kw...) -> HomographyProblemThe pool-frame homography problem of a pool record: its correspondences (NormalizedImageFrame) verbatim, no observation maps. Its sample gate reads the scale-free tolerance on its normalized view, so it takes no coincident_tol. kw go to HomographyProblem.
Fitting Pipeline
VisualGeometryToolkitEstimation.find_homography — Function
find_homography(correspondences; scoring,
extent=nothing, workspace=nothing,
max_trials, min_trials,
local_optimization_strategy, config=nothing)Robust homography estimation via RANSAC with MSAC scoring and least-squares-on-the-inliers local optimization. Every default is configs/default.toml's ([ransac], [pipelines.homography]); a keyword overrides.
Returns a RansacEstimate, the :no_model result (iterations 0) for fewer than 4 correspondences. iterations(result) is the number of trials (hypotheses drawn); test found_model(result) before reading the model (see RansacEstimate).
Arguments
correspondences: 2D-2D correspondences —Correspondences,StructArray{<:Pair}or a Pair vector (Point2storage canonicalized internally). ACorrespondences:scoreattribute column is the per-match prior (PROSAC, descending score); uniform otherwise. Endpoints carry their OBSERVATION NOISE (VisualGeometryToolkitCoreconventions/observation-noise.md): a plain point is unit isotropic noise in its own frame,Uncertain(p, CovMat(Σ))its own covariance,exact(p)none. Image↔image matches stay plain; a board, model or world plane isexact—find_homography(exact.(board) .=> pixels)scores the transfer error in pixels whatever the board's units (a plain metre board would be scored in metres, its threshold silently scaled by the board→image scale).scoring: the RANSAC scorer, an object of VisualGeometryToolkitCore's scoring contract carrying its own width in PIXELS — the unit of this problem's residuals (every pixel-datafind_*pipeline states its width by reference to this paragraph): a threshold scorer'sthreshold(MsacScoring(HomographyProblem; threshold = 2.0),RansacScoring), the Student-t + uniform scorer'soutlier_halfwidth(StudentTUniformScoring(HomographyProblem, 5.0; scale_prior, label_prior)), a paper's scorer (RANSACScoringDoneRight's families). Default: the class's scorer of record,configs/default.toml[pipelines.homography] scoring(OpenCV's convention, not a tuned value), read byscoring_from_toml. A barethresholdoroutlier_halfwidthkeyword is refused (ArgumentError): a width belongs to its scorer.extent: optionalHyperRectangle{2}world-plane region; when given the problem is wrapped inBoundedProblemso the fitted H is validated over the rectangle.workspace: optional pre-allocatedRansacWorkspaceto reuse —RansacWorkspace([T = Float32,] HomographyProblem{Float64}, n_data),Tthe per-point storage precision, holds the DEFAULT (MSAC) scorer's workspace; for another scoring or local optimization build it from them,RansacWorkspace(problem, scoring, local_optimization_strategy). Each call checks the workspace against its scoring (VisualGeometryToolkitCorecheck_workspace).max_trials,min_trials: RANSAC trial bounds ([ransac]).local_optimization_strategy: LO-RANSAC strategy (default: the class's[pipelines.<class>]strategy — here least squares (Irlswith the TLS loss) on the inliers with its truncation at the scorer's decision point: a threshold scorer's threshold, the Student-t scorer's MAP crossover at the consensus partition's scale; any otherAbstractOptimizationStrategychanges mechanism — LocalOptimizationDoneRight's half-quadratic lifts among them).config: OverrideRansacConfig(ignoresmax_trials/min_trials).rng: the sampler's random number generator.
These keywords are shared by every find_* pipeline; the others document theirs by reference to this list, stating only their class's defaults.
Examples
# Bare pairs, no prior (uniform sampling)
result = find_homography(pairs)
# Descriptor-similarity prior as a `:score` column (PROSAC sampling) — the
# matcher's `Correspondences(...; score, ratio)` output carries it already; from bare
# pairs, build the columnar container (`withattrs` takes a `Correspondences`)
result = find_homography(Correspondences(sources(pairs), targets(pairs); score = similarities))Units: the source points are plain numbers. A homography maps from the source plane, so its entries carry that plane's unit, which a matrix cannot record: physical (unitful) source points are an ArgumentError — strip them in the unit the rest of your plain data uses — metres, as the stack's boundaries strip (ustrip.(m, X)).
VisualGeometryToolkitEstimation.refine_homography — Function
refine_homography(H, correspondences; optimization_strategy, scale | width, workspace = nothing)
-> FitResult{HomographyMat{Float64}, RefineMetadata}The homography H refined against correspondences from H itself — a refine: the guess is required and leads, the model family is kept, and nothing is estimated from scratch. It runs NO scorer: it estimates no scale, forms no consensus set and drops no datum. Its robust loss runs over every datum at the caller's scale, frozen. (The RANSAC local-optimization step from a seed — a scorer, its consensus set, the strategy anchored on it — is locally_optimize, not a refine.)
Returns a FitResult with RefineMetadata: the refined model, its termination record (retcode, stop_rule: the LM's, whose retcode_symbol is :converged, :max_iterations, :stalled or :solve_failed), iterations (the LM's accepted iterations), rms the weighted residual rms √(Σ wᵢrᵢ² / Σ wᵢ) and n the support (count(w > 0)), and in the metadata the per-datum weights and residuals at the estimate and the scale record (noise_scale(result), scale_source(result), loss_width). There is no inlier mask: a loss weight has no cut of its own (inlier_mask(result, cut) states one). Fewer than 4 correspondences: the guess back unrefined, record (ReturnCode.InitialFailure, NoRule) (retcode_symbol:degenerate), n 0.
Keywords — shared by every refine_*
optimization_strategy:Triggs{Shape}(…)— ONE robustified chart LM over every datum (Core'sloss_robustifier, handed to NLLSsolver); the default,Triggs{WelschShape}(), is the class'sconfigs/default.toml [refine.<class>] optimization_strategy— ornothing, plain least squares over every datum (one chart LM). AnIrls{Shape}(…)is anArgumentErrornamingTriggs{Shape}(a refine runs one LM, no reweighting cycles); any other strategy needs a consensus set: anArgumentErrornaminglocally_optimize. Least squares is spellednothing, not a Gaussian shape.scaleorwidth: EXACTLY ONE for a robust strategy, neither for least squares; anything else is anArgumentError(Core'sscale_input).scale = σis the residual's noise scale (here PIXELS), or an upstream robust result, whosenoise_scaleit takes (scale = find_homography(…)). The loss width is then the strategy's efficiency constant times σ: Core'sefficiency_constant(Shape, d)at the problem's codimensiond, or the constant the strategy states (Triggs{K}(efficiency_constant = c), the plain multiple; TLS has none and needs one, orwidth).width = wis the loss width itself; the result then records no scale (ScaleNone), so a downstream stage needs its own.workspace: aRansacWorkspace(its chart-LM buffers are reused) or a chart-LM workspace;nothingdraws the task's own.
Refused, each with the way to say it (ArgumentError): scoring, threshold, outlier_halfwidth (a refine runs no scorer), loss, loss_width, efficiency_constant (the loss is the strategy, its width scale/width), local_optimization, local_optimization_strategy and robust (the strategy is optimization_strategy), extent (an extent check validates a consensus set: locally_optimize(BoundedProblem(problem, extent), θ₀; …)) and the sampling keywords of find_* (max_trials, min_trials, config, rng).