Model Manifolds and Uncertainty API
The minimal parameterizations the estimators refine on (the Manifold contract and its instances), first-order covariance transport and solver covariance, and the image-domain confidence regions that display it. A model's ambient_coordinates are what a manifold's default retract_jacobian differentiates; a model type defined outside this package adds its method. Split from the Robust Estimation API page. See the Robust Estimation and Stereo Rigs and Uncertainty guides.
Model Manifolds
VisualGeometryToolkitCore.Manifold — Type
ManifoldLocal coordinate system anchored at a model iterate. Contract:
tangent_size(ch)::Int— manifold dimension D;model(ch)— the ambient model at the anchor (=retract(ch, 0));retract(ch, v)— ambient model at local coordinatesv::SVector{D};recenter(ch, v)— the same manifold family re-anchored atretract(ch, v), with ONE normalization policy: a rotation part re-projected onto SO(3) (reproject_so3), the homography chart's anchor its unit-Frobenius representative of the same model (a first-order retraction lets the scale drift);retractitself never normalizes. Its element type ispromote_typeof the manifold's andv's, as arithmetic's is: a step of the manifold's own float type returns the manifold's own type (what a solver stepping a manifold variable holds), and a step of ForwardDiff Duals returns a Dual-valued manifold, so automatic differentiation throughrecenterworks (NLLSsolver's, throughupdate). A step that does not widen the manifold's element type keeps its storage types exactly — a rotation's representation and each part's own element type — so the type a solver holds never changes under an update;retract_jacobian(ch, v = 0)— T(v) = ∂vec(retract(ch, v))/∂v (9 × D for 3×3 models): at the anchor by default, at anyvfor an optimizer that iterates in the manifold instead of re-anchoring. Named after the function it differentiates, as every<thing>_jacobianis: it is Ceres'Manifold::PlusJacobianand GTSAM's derivative ofretract. The inverse form, if one is ever needed, islocal_coordinates_jacobian(Ceres'MinusJacobian);action_jacobian(ch, x, v = 0)— ∂(retract(ch, v)(x))/∂v, the manifold's action on a point or ray differentiated (at the anchor by default, at anyvfor an optimizer that iterates in the manifold): generically the chain rule of the model's owntransform_deriv_paramsthrough T, closed form where a manifold has one;local_coordinates(ch, target)— the inverse retraction (closed form for the rigid and lens manifolds, Gauss–Newton for the projective ones);ambient_coordinates(model)— the model's ambient coordinate vector, which the defaultretract_jacobian(and so the defaultaction_jacobianandtransform_covariance) differentiates; provided for static arrays and tuples of them, and added by every other model a manifold retracts to;- covariance transport through the retraction, first order:
transform_covariance(ch, EuclideanManifold(ch), Σv)to the ambient space and back, or to another chart of the same model (see header).
NAMES, against the established libraries: retract and local_coordinates are GTSAM's retract / localCoordinates, which are Ceres' Manifold::Plus and Manifold::Minus. action_jacobian is the optional Jacobian GTSAM's transformFrom fills. The two constraint verbs have NO counterpart in either — Ceres and GTSAM both frame a problem as residuals of an explicit cost, while constraint_with_jacobians and constraint_jacobians are the IMPLICIT constraint g(θ, x) = 0 with Förstner's A = ∂g/∂θ and B = ∂g/∂x, which is what an implicit-function covariance and a Gauss-Helmert step need.
A manifold is the MODEL's (minimal_manifold(model)), and the free/fixed choice a fitter makes is a manifold operation (restrict) — so a residual composed with a manifold is differentiated once, here, not per consumer.
VisualGeometryToolkitCore.action_jacobian — Method
action_jacobian(ch::Manifold, x) -> SMatrix
action_jacobian(ch::Manifold, x, v) -> SMatrixThe Jacobian of the manifold's ACTION on x — ∂(retract(ch, v)(x))/∂v at v = 0, or at the chart coordinates v — what every residual through a posed or parameterized model needs. Generic: the chain rule of the model's own parameter derivative (transform_deriv_params(retract(ch, v), x), ∂model(x)/∂ambient) through the retraction differential (retract_jacobian(ch, v), ∂ambient/∂v). A manifold with a closed form overrides it (RigidManifold6: [−4·skew(R·x) I₃] at the anchor); every override is pinned against this default in the manifold tests.
VisualGeometryToolkitCore.ambient_coordinates — Function
ambient_coordinates(model) -> SVectorThe model's AMBIENT coordinates: the vector its manifolds' default retract_jacobian differentiates (so the default action_jacobian and transform_covariance too), and the anchor of its ambient chart EuclideanManifold(ch). A static matrix is its column-major entries, a static vector itself, a tuple the concatenation of its parts'. Every model a manifold retracts to adds its method beside its manifold — a downstream model that is not a static array or a tuple of them adds one too, rather than overriding retract_jacobian.
Public, not exported; a contract generic.
VisualGeometryToolkitCore.model — Method
model(ch::Manifold)The ambient model at the chart's anchor: retract(ch, zero_tangent(ch)). A chart that stores its anchor model overrides it with a read.
VisualGeometryToolkitCore.recenter — Function
recenter(ch::Manifold, v) -> ManifoldThe same manifold family re-anchored at retract(ch, v), with the family's one normalization policy (a rotation re-projected onto SO(3), a homography's unit Frobenius representative). Part of the Manifold contract.
VisualGeometryToolkitCore.restrict — Function
restrict(ch::Manifold, free::SVector{D, Bool}) -> SubsetManifold
restrict(ch::Manifold, idx::SVector{M, Int})The manifold ch on the coordinates free marks (or the ascending, distinct indices idx): tangent_size is the number of free coordinates, and retract, recenter, retract_jacobian and action_jacobian act through the parent with the fixed coordinates at zero. restrict(ch, ()) — no coordinate — is the pinned model. See SubsetManifold; a lens manifold also restricts by coefficient name.
VisualGeometryToolkitCore.retract — Function
retract(ch::Manifold, v::StaticVector{D}) -> modelThe ambient model at local coordinates v (v = 0 is the anchor, model(ch)); it never normalizes. GTSAM's retract, Ceres' Manifold::Plus; its inverse is local_coordinates and its differential retract_jacobian. Part of the Manifold contract.
VisualGeometryToolkitCore.retract_directions — Method
retract_directions(ch::Manifold, v[, Val(Q)]) -> NTuple{Q, SMatrix{3, 3}}The first Q (default: all) columns of retract_jacobian(ch, v) — ∂M/∂vq for a chart whose ambient model is a 3×3 matrix (its first nine ambient coordinates, column-major) — each reshaped to a 3×3 matrix: the dM tuple the row Jacobians take (sampson_jacobian_rows!, whitened_jacobian_rows!, radial_transfer_jacobian_rows!). For a radial homography's [vec H; k] chart, Val(8) gives the eight sl(3) directions of H. In the Jacobian's element type.
VisualGeometryToolkitCore.retract_jacobian — Method
retract_jacobian(ch::Manifold) -> SMatrix
retract_jacobian(ch::Manifold, v::AbstractVector) -> SMatrixThe RETRACTION DIFFERENTIAL: ∂vec(retract(ch, v))/∂v, the ambient model's derivative with respect to the chart coordinates — at the anchor (v = 0) in the one-argument form, at any v for an optimizer that iterates in the manifold instead of re-anchoring.
Named after the function it differentiates, as every <thing>_jacobian is. Against the established libraries it is Ceres' Manifold::PlusJacobian and GTSAM's derivative of retract; the inverse form, if one is ever needed, is local_coordinates_jacobian (Ceres' MinusJacobian).
This is the ∂ambient/∂v factor of action_jacobian, which chains it with the model's own transform_deriv_params. The default for a chart that does not override it is _retract_jacobian_fd, ForwardDiff through retract — exact to first order with no hand-derivation risk, and every closed-form override is pinned against it in the manifold contract tests. A linear chart (LensManifold) returns the identity. The default needs only retract and the model's ambient_coordinates. A chart overrides the two-argument form for any AbstractVectorv (one method; a constant differential ignores v), and the one-argument form when its anchor's differential is cheaper than evaluating at zero_tangent(ch).
Public, not exported; VisualGeometryToolkitEstimation builds its LM and covariance steps on it.
VisualGeometryToolkitCore.tangent_part — Function
tangent_part(ch::Manifold, v, offset = 0) -> SVector{tangent_size(ch)}The coordinates of ch in a longer tangent vector v: the tangent_size(ch) entries after the first offset, statically sized and in v's element type (ForwardDiff Duals included), since a manifold's tangent_size is a constant of its type. A product manifold cuts its parts' coordinates with it, and so does a solver that stacks several manifolds' coordinates in one step vector.
VisualGeometryToolkitCore.tangent_size — Function
tangent_size(ch::Manifold) -> IntThe manifold's dimension D: the length of the local coordinates v that retract takes. Part of the Manifold contract.
VisualGeometryToolkitCore.transform_covariance — Method
transform_covariance(from::Manifold, to::Manifold, Σ) -> MatrixA covariance in from's coordinates re-expressed in to's, the two charts anchored at the same model and sharing its ambient vectorization, to first order: through the ambient space, T_to⁺·(T_from·Σ·T_fromᵀ)·T_to⁺ᵀ (T the charts' retract_jacobians). The ambient space is a chart as well, EuclideanManifold(ch): TO it the covariance is pushed, T·Σ·Tᵀ; FROM it pulled, T⁺·Σ·T⁺ᵀ (a gauge-deficient chart projects its gauge out). Exact where to is full rank. A rigid pose's covariance between the MRP (RigidManifold6), left-perturbation (LeftRigidManifold6) and decoupled (RigidExpManifold6) charts, for instance — the MRP's factor 4 at the anchor and the translation coupling done by the charts themselves, not by hand. A change of coordinates at one model, not a parallel transport between points.
GTSAM equivalent: TransformCovariance<T> (a group-frame change), and the Local/Retract Jacobians a chart change composes.
VisualGeometryToolkitCore.zero_tangent — Method
zero_tangent(ch::Manifold) -> SVector{tangent_size(ch)}The chart coordinates of the anchor, v = 0, in the chart's float type (Float64 unless the chart stores another): what model(ch) and the one-argument retract_jacobian evaluate at. A chart generic in its float type overrides it.
VisualGeometryToolkitCore.EssentialManifold — Type
EssentialManifold{G <: EssentialTranslation, T} <: ManifoldThe essential-matrix chart anchored at (R, t), |t| = 1: R(v) = RotMatrix(MRP(ω))·R, t(v) the translation kind's increment, E(v) = skew(t(v))·R(v), stored in the float type T. The stack's two are aliases: EssentialManifold5 (minimal, production) and EssentialManifold6 (one gauge direction, eval). See Manifold.
VisualGeometryToolkitCore.EssentialManifold — Method
EssentialManifold5(E::EssentialMat)
EssentialManifold6(E::EssentialMat)The chart anchored at the essential matrix E: the pose (R, t) of the FIRST of motion(E)'s four candidates, with t normalized (|t| = 1), so model(ch) = skew(t)·R is E up to scale and sign. The Sampson residuals are invariant to both and to the candidate, which is why any one anchors a refit.
VisualGeometryToolkitCore.EssentialManifold5 — Type
EssentialManifold5(R, t) <: ManifoldThe PRODUCTION 5-dof essential-matrix lift chart, EssentialManifold{<:S2TangentT}: so(3) ⊕ T_tS², gauge-fixed. Anchored at (R, t) with |t| = 1 and an orthonormal tangent basis (b1, b2) of S² at t frozen at the anchor; coordinates v = (ω₁, ω₂, ω₃, τ₁, τ₂) act as R(v) = RotMatrix(MRP(ω))·R, t(v) = normalize(t + τ₁·b1 + τ₂·b2), E(v) = skew(t(v))·R(v). Minimal (rank 5, no gauge nullity). See EssentialManifold6 for the deliberately non-minimal eval companion.
VisualGeometryToolkitCore.EssentialManifold6 — Type
EssentialManifold6(R, t) <: ManifoldThe EVAL 6-dof essential-matrix chart, EssentialManifold{RenormalizedT}: MRP ⊕ unconstrained-t-then-normalize. Coordinates v = (ω₁..₃, τ₁..₃) act as R(v) = RotMatrix(MRP(ω))·R, t(v) = normalize(t + τ), E = skew(t)·R. DELIBERATELY NOT MINIMAL: the radial direction τ ∥ t is a gauge nullity (rank 5) absorbed by the engines' GN damping — the historical eval solver chart kept for solver-parity with the GNC/scheduled arms; a covariance transported into it (pinv) has the gauge projected out. The production minimal chart is EssentialManifold5.
VisualGeometryToolkitCore.EssentialTranslation — Type
EssentialTranslationThe translation increment of an EssentialManifold: S2TangentT (2 dof, minimal) or RenormalizedT (3 dof, one gauge direction).
VisualGeometryToolkitCore.RenormalizedT — Type
t(v) = normalize(t + τ), τ ∈ R³: one gauge direction (τ ∥ t).
VisualGeometryToolkitCore.S2TangentT — Type
t(v) = normalize(t + τ₁·b1 + τ₂·b2): the tangent basis (b1, b2) of S² at the anchor's t, frozen.
VisualGeometryToolkitCore.pose_action_jacobian — Function
pose_action_jacobian(ch::EssentialManifold, x, v = zero_tangent(ch)) -> SMatrix{3, tangent_size(ch)}The Jacobian ∂(R(v)·x + t(v))/∂v of the chart's POSE acting on a point x, with (R(v), t(v)) = recenter_pose(ch, v): 3 × 5 for EssentialManifold5, 3 × 6 for EssentialManifold6. Closed form: the rotation columns are −[R(v)·x]×·L(ω) (L the MRP's left Jacobian, 4·I₃ at ω = 0, as for RigidManifold6), the translation columns (I − t̂t̂ᵀ)/‖s‖ times the increment's basis ([b1 b2], or I₃), s the translation before normalization and t̂ = s/‖s‖. Where action_jacobian differentiates the model E(v) acting on a point, this differentiates the rigid motion the chart carries: a relative pose with unit translation, the two-view bundle's pose block. Neither Ceres nor GTSAM has the counterpart (neither has a gauge-fixed relative-pose chart).
VisualGeometryToolkitCore.s2_tangent_basis — Method
Tangent basis of S² at t — the production _setup_tangent_basis code, VERBATIM (PoseLib relative.h port; branch structure and the b2 = normalize(cross(b1, t)) orientation preserved bit-for-bit for the consumer byte-identity gates).
VisualGeometryToolkitCore.s2_tangent_basis_matrix — Method
s2_tangent_basis_matrix([T,] f) -> SMatrix{3, 2, T}The tangent basis of S² at f, s2_tangent_basis's pair (b1, b2), as the columns of one 3 × 2 matrix B (Bᵀ is the 2 × 3 projection onto the tangent plane), its elements converted to T (default: the basis's own type, promote_type(eltype(f), Float64)). The one packing of the basis: a bearing residual's coordinates, a back-projection Jacobian's tangent rows, a transfer form's projection.
VisualGeometryToolkitCore.FundMatManifold7 — Method
Manifold from a fundamental matrix anchor via its SVD (the _orthonormal_manifold convention).
VisualGeometryToolkitCore.anchor_svd — Method
anchor_svd(ch::FundMatManifold7) -> (U, σ, V)The anchor of the Bartoli–Sturm chart: the orthonormal factors U, V and the normalized second singular value σ = s₂/s₁ it is centred on.
VisualGeometryToolkitCore.SL3_BASIS — Constant
SL3_BASISThe eight traceless generators G₁…G₈ of sl(3) used by HomographyManifold8: the six off-diagonal elementary matrices plus diag(1, −1, 0) and diag(0, 1, −1).
VisualGeometryToolkitCore.HomographyManifold8 — Type
HomographyManifold8(H0) <: ManifoldThe minimal 8-dof sl(3) homography manifold about an anchor: H(v) = H₀·(I + Σ vᵢ·Gᵢ) with G₁…G₈ the traceless sl(3) basis (SL3_BASIS). Tracelessness deletes the homogeneity gauge direction H₀·εI, so the manifold is minimal (rank 8) with no ‖H‖ = 1 constraint and no fixed-entry singularity; ∂H/∂vᵢ = H₀·Gᵢ is exact and constant (first-order manifold, no dexp) — the LM refit's Jacobian convention. The anchor is stored in its float type (Float64 from a Float64 or integer H0). See Manifold.
VisualGeometryToolkitCore.CarriedT — Type
t(v) = ΔR(ω)·t + τ: the rotation carries the translation (x ↦ ΔR·(R·x + t) + τ).
VisualGeometryToolkitCore.ExpParam — Type
ΔR(ω) = exp_so3(ω): the rotation vector (radians).
VisualGeometryToolkitCore.FreeT — Type
t(v) = t + τ: the translation is a free R³ update.
VisualGeometryToolkitCore.LeftRigidManifold6 — Type
LeftRigidManifold6(R, t) <: Manifold
LeftRigidManifold6(pose::EuclideanMap{3})The LEFT-perturbation pose chart, RigidChart{ExpParam, CarriedT}: coordinates v = (δω₁..₃, δt₁..₃) act on the whole map, x ↦ exp([δω]×)·(R·x + t) + δt, so retract(ch, v) is (exp_so3(δω)·R, exp_so3(δω)·t + δt).
It is the perturbation constraint_with_jacobians differentiates for a pose (G_θ = Jπ·[−[X_cam]× I₃]): action_jacobian(ch, x) = [−[R·x + t]× I₃], so a PnP refit — and the PnP lift, which is the refit's rows at v = 0 — that solves in these coordinates re-anchors (recenter) with exactly the perturbation its Jacobian assumed. local_coordinates is the closed-form inverse (δω = log_so3(R·R₀ᵀ), δt = t − exp_so3(δω)·t₀). Float-typed storage (SMatrix{3,3,T}, SVector{3,T}). See RigidChart.
VisualGeometryToolkitCore.MRPParam — Type
ΔR(ω) = RotMatrix(MRP(ω)): Rotations' modified Rodrigues parameters, 4× the rotation vector at the identity.
VisualGeometryToolkitCore.RigidChart — Type
RigidChart{P <: RotationParam, C <: TranslationCoupling, Rot, Tr} <: ManifoldThe 6-dof rigid pose chart family, anchored at (R::Rot, t::Tr): coordinates v = (ω₁..₃, τ₁..₃) act as R(v) = ΔR(ω)·R and t(v) = t + τ (FreeT) or ΔR(ω)·t + τ (CarriedT), ΔR the increment of P (MRPParam, ExpParam). Full rank (no gauge). retract returns a EuclideanMap; the ambient vectorization is [vec(R); t] ∈ R¹²; recenter re-projects the rotation onto SO(3) (the one policy) and keeps the anchor's storage types, in the element type the contract's rule gives (the anchor's for a step of its own float type); local_coordinates is closed form. The stack's named charts are aliases: RigidManifold6, LeftRigidManifold6, RigidExpManifold6.
VisualGeometryToolkitCore.RigidExpManifold6 — Type
RigidExpManifold6(R, t) <: Manifold
RigidExpManifold6(pose::EuclideanMap{3})The DECOUPLED pose chart, RigidChart{ExpParam, FreeT}: R(v) = exp_so3(δω)·R, t(v) = t + δt. Unlike LeftRigidManifold6 the rotation does not carry the translation; unlike RigidManifold6 the rotation coordinate is the rotation vector (radians), not the MRP. The chart of the hand-eye constraint's G_θ: action_jacobian(ch, x) = [−[R·x]× I₃]. Float-typed storage. Move a covariance between it and the other rigid charts with transform_covariance. See RigidChart.
VisualGeometryToolkitCore.RigidManifold6 — Type
RigidManifold6(R, t) <: ManifoldThe MRP rigid pose chart, RigidChart{MRPParam, FreeT}: R(v) = RotMatrix(MRP(ω))·R, t(v) = t + τ. Parametric in the anchor's storage types (a RotMatrix or an SMatrix, any static 3-vector), which recenter keeps — the rigid-pose retraction of P3P, the relpose-bundle pose block and BlobBoards' bundle. Its action Jacobian at the anchor is [−4·[R·x]× I₃] (the MRP's factor 4). Distinct from EssentialManifold6: the translation update is free, so there is no radial-t gauge. See RigidChart.
VisualGeometryToolkitCore.RotationChart — Type
RotationChart{P <: RotationParam, Rot} <: ManifoldThe 3-dof rotation chart of the rigid family, anchored at R::Rot: coordinates ω act on the left, R(ω) = ΔR(ω)·R, ΔR the increment of P (MRPParam, ExpParam) — the rotation block of RigidChart, written over the same per-parameterization facts (the increment, its logarithm, its left Jacobian, the anchor scale). retract returns a RotMatrix{3}, the ambient vectorization is vec(R) ∈ R⁹, recenter re-projects onto SO(3) and keeps the anchor's storage type, local_coordinates is closed form (ω the coordinates of R·R₀ᵀ), the retraction differential is closed form: ∂vec(R(ω))/∂ω_q = vec([L(ω)e_q]×·R(ω)). Full rank, no gauge. SO3Manifold is the rotation-vector member, minimal_manifold(R) of a rotation.
VisualGeometryToolkitCore.RotationParam — Type
RotationParamHow a RigidChart parameterizes its rotation increment ΔR(ω): MRPParam or ExpParam.
VisualGeometryToolkitCore.SO3Manifold — Type
SO3Manifold(R) <: ManifoldThe rotation-vector chart of SO(3), RotationChart{ExpParam}: R(ω) = exp_so3(ω)·R (the left perturbation, ω in radians) — the rotation block of LeftRigidManifold6 and RigidExpManifold6, and minimal_manifold of a rotation. Float-typed storage (SMatrix{3, 3, T}).
VisualGeometryToolkitCore.TranslationCoupling — Type
TranslationCouplingWhether a RigidChart's rotation increment carries the translation: FreeT (t(v) = t + τ) or CarriedT (t(v) = ΔR·t + τ, the whole map perturbed on the left).
VisualGeometryToolkitCore.local_coordinates — Method
local_coordinates(ch::RigidChart, pose::EuclideanMap{3}) -> SVector{6}The rigid chart's inverse retraction: the v with retract(ch, v) == pose — ω the coordinates of R·R₀ᵀ (its MRP — the principal set, |ω| < 1 for rotations under π — or log_so3), τ = t − t₀ (or t − ΔR(ω)·t₀ when the rotation carries the translation). Closed form and exact.
VisualGeometryToolkitCore.local_coordinates — Method
local_coordinates(ch::RotationChart, R) -> SVector{3}The rotation chart's inverse retraction: the coordinates of R·R₀ᵀ (its MRP or log_so3). Closed form and exact.
VisualGeometryToolkitCore.LinearSubspaceManifold — Type
LinearSubspaceManifold{C, F} <: Manifoldrestrict(ch, Z)'s manifold: the parent manifold::C of tangent size D on the M coordinates y of the subspace spanned by the columns of Z::Matrix{Float64} (D × M), the parent's coordinates v = Z y. At the anchor the model is the parent's, so the parameters move as θ = θ₀ + Z y through the parent's chart (exactly, for an unbounded parent). retract, recenter and the Jacobians are the parent's composed with Z (retract_jacobian(rc) = retract_jacobian(ch) * Z); local_coordinates is the least-squares y of the parent's coordinates, (ZᵀZ)⁻¹ Zᵀ v by the Cholesky factor F of ZᵀZ, computed once at construction; exact for a model in the subspace.
VisualGeometryToolkitCore.SubsetManifold — Type
SubsetManifold{C, D, I} <: Manifoldrestrict's manifold: the parent manifold::C of tangent size D on its free coordinates idx::I (ascending), an SVector{M, Int}: a tangent size that is a constant of the type. retract(rc, v) is the parent's at v lifted into D coordinates (zeros elsewhere), the Jacobians are the parent's columns idx, recenter re-anchors the parent.
VisualGeometryToolkitCore.restrict — Method
restrict(ch::Manifold, Z::AbstractMatrix) -> LinearSubspaceManifoldThe manifold ch on the subspace of its coordinates spanned by the columns of Z (tangent_size(ch) rows, any number of columns): its coordinates y move the parent's as v = Z y. A fit whose parameters are vec(P) = vec(P₀) + Z y — a vignetting's control points on a regular subspace (a radial profile's pole_regular_basis, VisualRadiometryToolkitCore's), a target's deformation basis — is the parameter manifold of P₀ restricted to Z. Z is stored as a plain Matrix{Float64} (a control-point basis has hundreds to thousands of entries) with the Cholesky factor of ZᵀZ; its columns must be linearly independent (ArgumentError).
VisualGeometryToolkitCore.SphereManifold — Type
SphereManifold(q::StaticVector{N}, embed = identity) <: ManifoldThe unit sphere at q (normalized on construction): coordinates v ∈ R^{N−1} along an orthonormal basis B of q's complement — the last N − 1 columns of the Householder reflection taking e₁ to ∓q, built from q + sign(q₁)·e₁ so it is stable for every q — and retract(ch, v) = embed(normalize(q + B·v)).
embed turns the unit vector into the model: the identity for a bare vector, a RadialCamera with its centre for minimal_manifold(::RadialCamera). The ambient space is the vector's: retract_jacobian(ch, v) is (I − ŷŷᵀ)·B/‖q + B·v‖ (N × (N − 1), B at the anchor). local_coordinates(ch, y) is closed form and invariant to the scale and sign of y: Bᵀy/(qᵀy). recenter re-anchors at the retracted point, with a new basis.
VisualGeometryToolkitCore.EuclideanManifold — Type
EuclideanManifold(x::StaticVector{N}) <: ManifoldRᴺ at x: retract(ch, v) = x + v, the identity differential, local_coordinates(ch, y) = y − x. The chart of a model's plain-vector part — a division coefficient, a 3D point — in a ProductManifold.
VisualGeometryToolkitCore.EuclideanManifold — Method
EuclideanManifold(ch::Manifold)The AMBIENT space of the chart ch as a chart: Rᴺ anchored at the ambient coordinates of model(ch) (vec(M) for a 3×3 model, [vec R; t] for a pose, a lens's parameters, …) — where transform_covariance pushes a covariance to, transform_covariance(ch, EuclideanManifold(ch), Σv), and pulls it from, transform_covariance(EuclideanManifold(ch), ch, Σ).
VisualGeometryToolkitCore.ProductManifold — Type
ProductManifold(charts::Manifold...; embed = identity) <: ManifoldThe chart of a model made of independent parts, one chart each: coordinates the concatenation of the parts' (in order), retract(ch, v) = embed(parts) with parts the tuple of the parts' retractions, recenter part by part, retract_jacobian block diagonal over the parts' ambient spaces (an SMatrix when the parts' are static and the whole stays small, a dense Matrix otherwise — products run to many parts: a rig of poses, a pose and its points), and local_coordinates(ch, parts::Tuple) the concatenation. embed assembles the model from its parts: minimal_manifold(::RadialHomography) is sl(3) ⊕ the division lens's chart, assembled into the RadialHomography with its centre.
VisualGeometryToolkitCore.Line2DManifold — Type
Line2DManifold(l::Line2D) <: ManifoldThe 2-dof chart of a 2D line n·x + d = 0 (|n| = 1): v = (φ, δd) rotates the normal by φ and moves the offset, retract(ch, v) = Line2D(R(φ)·n, d + δd). The ambient space is the line's coefficients (n₁, n₂, d), retract_jacobian[R(φ)·t 0; 0 1] with t = (−n₂, n₁) the line's direction. A line acts on a point by its signed distance (constraint_with_jacobians(l, x)), so action_jacobian(ch, x) = (t·x, 1) — the point-on-line constraint's chart row. local_coordinates is closed form and reads (n, d) and (−n, −d) as the same line. minimal_manifold(l::Line2D) is this chart.
VisualGeometryToolkitCore.CircleManifold — Type
CircleManifold(c::Circle) <: ManifoldThe 3-dof chart of a 2D circle with centre c and radius r > 0: v = (δc₁, δc₂, δlog r) moves the centre and scales the radius, retract(ch, v) = Circle(c + (δc₁, δc₂), r·exp(δlog r)), so every point of the chart is a circle with a positive radius. The ambient space is (c₁, c₂, r) (ambient_coordinates); retract_jacobian is diag(1, 1, r·exp(δlog r)). A circle acts on a point by the point-on-circle constraint (constraint_with_jacobians(c, x)), so action_jacobian(ch, x) = (−2(x − c)ᵀ, −2r²) is that constraint's chart row. local_coordinates is closed form. minimal_manifold(c::Circle) is this chart.
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(c::Circle, x::StaticVector{2}) -> (g, G_x::SVector{2}, G_θ::SVector{3})The point-on-circle constraint g = ‖x − c‖² − r² with G_x = 2(x − c) and G_θ = ∂g/∂(c₁, c₂, r) = (−2(x − c), −2r), the circle's ambient coordinates (CircleManifold's). Its Sampson distance g/‖G_x‖ = (d² − r²)/(2d), d = ‖x − c‖, is the geometric distance d − r to first order (the ratio is (d + r)/(2d)). A point at the centre has G_x = 0 — the one Sampson degeneracy (base/sampson.jl), judged at the circle's constraint_rounding_scale.
VisualGeometryToolkitCore.EllipseManifold — Type
EllipseManifold(e::Ellipse) <: ManifoldThe 5-dof chart of a 2D ellipse: VisualGeometryToolkitCore's log-shape chart (ellipse_log_shape) taken as local coordinates, v = (δc₁, δc₂, δS₁₁, δS₁₂, δS₂₂), retract(ch, v) = ellipse_from_log_shape(ellipse_log_shape(e) + v), so every point of the chart is a real ellipse (the exponential of every symmetric S is positive definite) and the chart is smooth at a circle. The ambient space is the centre and the quadratic form Q = exp(−2S) = R·diag(a⁻², b⁻²)·Rᵀ of the ellipse, (c₁, c₂, Q₁₁, Q₁₂, Q₂₂) (ambient_coordinates), in which the point-on-ellipse constraint (constraint_with_jacobians(e, x)) is polynomial; retract_jacobian is diag(I₂, ∂Q/∂S), the derivative of the symmetric matrix exponential in closed form (Daleckii–Krein). action_jacobian(ch, x) is that constraint's chart row. local_coordinates is closed form. minimal_manifold(e::Ellipse) is this chart.
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(c::Conic, x::StaticVector{2}) -> (g, G_x::SVector{2}, G_θ::SVector{6})The point-on-conic constraint of a general conic C (SMatrix{3, 3}(c), symmetric 3×3): g = x̃ᵀ C x̃, x̃ = (x, 1), with G_x = 2(C x̃)₁,₂ and G_θ over the conic's six distinct entries in upper-triangle column-major order, (C₁₁, C₁₂, C₂₂, C₁₃, C₂₃, C₃₃): (x², 2xy, y², 2x, 2y, 1). Homogeneous: a scaled conic is the same curve and has the same sampson_distance (the constraint and its gradient scale together), which is the first-order distance to the curve — for an ellipse's conic, its constraint_with_jacobians(::Ellipse, x)'s. A point where C x̃ has no image part (the centre of a central conic) has G_x = 0: the one Sampson degeneracy.
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(e::Ellipse, x::StaticVector{2}) -> (g, G_x::SVector{2}, G_θ::SVector{5})The point-on-ellipse constraint, the implicit conic g = (x − c)ᵀ Q (x − c) − 1 (Q = R·diag(a⁻², b⁻²)·Rᵀ, so g = x̃ᵀ C x̃ for the ellipse's conic C scaled to −1 at the centre), with G_x = 2Q(x − c) and G_θ = ∂g/∂(c₁, c₂, Q₁₁, Q₁₂, Q₂₂) = (−2Q(x − c), d₁², 2d₁d₂, d₂²), d = x − c, the ellipse's ambient coordinates (EllipseManifold's). Its Sampson distance g/‖G_x‖ is the geometric distance to the ellipse to first order, and at a circle of radius r the circle's (‖d‖² − r²)/(2‖d‖) (VisualGeometryToolkitCore's constraint_with_jacobians(::Circle, x)). A point at the centre has G_x = 0 — the one Sampson degeneracy (base/sampson.jl), at the ellipse's constraint_rounding_scale.
VisualGeometryToolkitCore.constraint_dim — Method
constraint_dim(model) -> IntNumber of scalar algebraic constraints per correspondence for model's class: 1 for the epipolar models (FundMat, EssentialMat), 2 for HomographyMat and for a pose (EuclideanMap{3}, the reprojection constraint). Matches the row count of constraint_with_jacobians.
VisualGeometryToolkitCore.constraint_jacobians — Method
constraint_jacobians(ch::Manifold, v, cs) -> (∂_v g, ∂_x g)The Jacobians of manifold_constraints at chart coordinates v: with respect to v (G_θ·retract_jacobian(ch, v), dg·k × D) and to the observations (block diagonal, dg·k × nₓk, each pair's G_x over (x₁, y₁, x₂, y₂), or over the image point of a pose correspondence, nₓ = 2). At v = 0 these are the two factors of the implicit-function covariance (implicit_covariance); at v ≠ 0, the Newton matrix of a chart-Newton solve that iterates in the manifold.
NO Ceres or GTSAM counterpart: both frame a problem as residuals of an EXPLICIT cost, whereas this is the IMPLICIT constraint g(θ, x) = 0 in Förstner's form — A = ∂g/∂θ, B = ∂g/∂x — which is what an implicit-function covariance and a Gauss-Helmert step consume. The value-and-Jacobians sibling is constraint_with_jacobians.
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(l::Line2D, x::StaticVector{2}) -> (g, G_x::SVector{2}, G_θ::SVector{3})The point-on-line constraint g = n·x + d — the point's signed distance (the normal is unit) — with G_x = n and G_θ = ∂g/∂(n₁, n₂, d) = (x₁, x₂, 1), the line's coefficients (Line2DManifold's ambient space). Single-point data: the covariance machinery takes a vector of points for a line chart.
VisualGeometryToolkitCore.implicit_covariance — Method
implicit_covariance(ch::Manifold, cs; Σx = nothing) -> Matrix (D×D) or nothing
implicit_covariance(block, indices; noise_weighted = false) -> Matrix or nothing
implicit_covariance(A::AbstractMatrix, B::AbstractMatrix; Σx = nothing) -> Matrix or nothingThe first-order covariance of the solution v(x) of an implicitly defined estimation problem (Haralick's framing): g(v, x) = 0 for a square system (a minimal solve), ∂_v(½‖W₀^{½}·g‖²) = 0 — least-squares stationarity of the whitened constraints — for an overdetermined one. With A = ∂_v g, B = ∂_x g and the estimator's own isotropic whitening W₀ = blockdiag((BᵢBᵢᵀ)⁻¹) over the correspondences' row blocks,
Σ̃_v = (AᵀW₀A)⁻¹ · AᵀW₀·(B·Σx·Bᵀ)·W₀A · (AᵀW₀A)⁻¹,which is A⁻¹·B·Σx·Bᵀ·A⁻ᵀ when A is square (the whitening cancels — the implicit-function theorem proper) and (AᵀW₀A)⁻¹ for unit isotropic Σx. Unit scale: Σx defaults to unit isotropic observation noise; the physical covariance is s²·Σ̃_v with s the consumer's. Invariant to any invertible per-correspondence recombination of the constraint rows (the algebraic and the transfer forms of a homography give the same Σ̃_v).
- Chart first: the constraints of the correspondences
cs(u₁ => u₂pairs;X => ufor a pose chart; single points for a line) differentiated at the anchor ofch(constraint_jacobians), in the coordinates ofch, which must be anchored at the estimate.Σx: one covariance shape per correspondence (4×4 over(x₁, y₁, x₂, y₂); 2×2 for a pose's or a line's point), in the coordinates ofcs— data mapped to another frame needs its noise transported there (the map'stransform_deriv) before a pixels²means anything. A minimal sample (k·d_g = D) takes the square solve. - Blocks, never materializing
B(O(D²)memory, for large inlier sets):block(i)returns(Aᵢ, Bᵢ)or(Aᵢ, Bᵢ, Σxᵢ)for eachiinindices— the correspondence's rows of∂_v gand∂_x g(aStaticVectoris one row) and its covariance shape (unit isotropic when absent). Function first, as inmap(f, xs). Withnoise_weighted = truethe estimator is instead the one weighted by each block's OWN noise,W₀ᵢ = (BᵢΣxᵢBᵢᵀ)⁻¹(the Σxᵢ-whitened residuals a scorer uses), whose covariance is(AᵀW₀A)⁻¹—Σxᵢfolded intoBᵢasBᵢ·chol(Σxᵢ).L. The default sandwichesΣxᵢaround the isotropically weighted estimator: a different estimator, a different covariance whenever theΣxᵢare not isotropic. - Dense square factors:
A = ∂_v g(D×D) andB = ∂_x g(D×nₓk) of a square systemg(v, x) = 0whose rows are not per-correspondence constraints — e.g. the stationarity conditions of a staged estimator, one row block per stage — givesA⁻¹·B·Σx·Bᵀ·A⁻ᵀdirectly.Σx: one covariance shape per column block ofB(nₓcolumns each,kblocks); unit when absent. The minimal-sample path of the chart form above is this method.
nothing when the system is singular (A for a square one, AᵀW₀A otherwise — a degenerate sample, the same the solvers' own gates reject, or ill-posed geometry) or when there are no blocks; a block whose Bᵢ is degenerate (whitening_factor's rule) drops out, as it does from the estimator.
GTSAM equivalent: Marginals::jointMarginalCovariance at the optimum, when the factor rows are correctly whitened. This one also covers the square minimal solve and the sandwich for a mismatched weighting.
VisualGeometryToolkitCore.local_coordinates — Method
local_coordinates(ch::Manifold, Mtarget; iters=25, tol=1e-13)
-> SVector{tangent_size(ch)}Inverse retraction ("manifold log"): the local coordinates v such that retract(ch, v) ∝ Mtarget (projective alignment — scale AND sign of the target are matched to the manifold family at each Gauss–Newton step, so any solver normalization convention may be supplied). This is the harness primitive for expressing an estimate IN THE MANIFOLD RECENTERED AT TRUTH.
Gauge-deficient manifolds (e.g. EssentialManifold6, near-essential FundMatManifold7 anchors) resolve the step through pinv, projecting the gauge direction out — the returned coordinates are the minimum-norm representative. Converges quadratically for targets in the manifold's injectivity neighborhood (MC-perturbation scale); for far targets check the residual.
VisualGeometryToolkitCore.manifold_constraints — Method
manifold_constraints(ch::Manifold, v, cs) -> VectorThe stacked algebraic constraints g(v, x) of the correspondences cs (u₁ => u₂ pairs; X => u for a pose chart, u calibrated; single points for a line) at chart coordinates v: constraint_with_jacobians' value for retract(ch, v), constraint_dim rows per pair. Zero on the constraint variety; the residual a chart-Newton solve drives to zero.
VisualGeometryToolkitCore.minimal_manifold — Method
minimal_manifold(model) -> ManifoldThe minimal (gauge-free) manifold anchored at model: FundMat → FundMatManifold7 (via the anchor SVD), HomographyMat → HomographyManifold8, EssentialMat → EssentialManifold5 (via the first motion(E) candidate — the tangent-domain covariance and its ambient push are candidate-invariant, see the header gauge note), EuclideanMap{3} → RigidManifold6, RadialHomography → sl(3) ⊕ its division lens's chart (a ProductManifold), RadialCamera → S⁵ (a SphereManifold, cameras/calibration.jl), a lens → its LensManifold (cameras/lens_chart.jl).
Confidence Regions
VisualGeometryToolkitCore.LineConfidenceBand — Type
LineConfidenceBandConfidence region of an uncertain homogeneous 2D line: nominal line l, line covariance Σl (3×3, from pushing a model covariance through the line construction), and squared Mahalanobis level k2. Region: {x : (x̃ᵀl)² ≤ k2·x̃ᵀΣl x̃}; boundary conic C = l·lᵀ − k2·Σl. Plot with lines!(ax, band, rect) (Makie layer) or sample the boundary with band_boundary.
VisualGeometryToolkitCore.band_boundary — Method
band_boundary(b::LineConfidenceBand, rect::HyperRectangle{2}; n=201)
-> (upper::Vector{Point2{Float64}}, lower::Vector{Point2{Float64}})Sample the two boundary branches of the band inside rect, as offsets along the nominal line's normal over n stations of the clipped nominal segment. Stations where the band is UNBOUNDED (the near-degeneracy blow-up: the normal-section quadratic loses its positive curvature or its real roots) emit (NaN, NaN) so polylines break there instead of lying. Returns empty vectors when the nominal line misses rect.
VisualGeometryToolkitCore.band_conic — Method
Boundary point conic C = l·lᵀ − k²·Σ_l of the band.
VisualGeometryToolkitCore.confidence_ellipse — Method
confidence_ellipse(μ, Σ; level=0.95) -> EllipseThe level confidence ellipse of a 2D Gaussian N(μ, Σ): {x : (x−μ)ᵀ Σ⁻¹ (x−μ) = χ²₂(level)}, as the standard Ellipse primitive (so it plots through the existing poly!/convert_arguments layer).
It is Ellipse(μ, Symmetric(χ²₂(level)·Σ)), so a Σ that is not positive definite (singular, indefinite, or NaN) is refused with that constructor's ArgumentError, never clamped into a degenerate ellipse.
VisualGeometryToolkitCore.epipolar_band — Method
epipolar_band(F::AbstractFundMat, Σθ::AbstractMatrix(9×9), u₁;
level=0.95, dof=2) -> LineConfidenceBandConfidence band of the epipolar line l = F·ũ₁ in image 2 under the ambient model covariance Σθ (vec(F) column-major). dof=2 gives the whole-line (H–Z envelope) convention, dof=1 the pointwise convention — see the file header. Uncertainty of u₁ itself is not included (model-only band; fold a 4×4 point covariance into Σ_l externally if needed).
VisualGeometryToolkitCore.in_band — Method
Signed band membership: x̃ᵀCx̃ ≤ 0 ⇔ inside.
Index
VisualGeometryToolkitCore.SL3_BASISVisualGeometryToolkitCore.CarriedTVisualGeometryToolkitCore.CircleManifoldVisualGeometryToolkitCore.EllipseManifoldVisualGeometryToolkitCore.EssentialManifoldVisualGeometryToolkitCore.EssentialManifoldVisualGeometryToolkitCore.EssentialManifold5VisualGeometryToolkitCore.EssentialManifold6VisualGeometryToolkitCore.EssentialTranslationVisualGeometryToolkitCore.EuclideanManifoldVisualGeometryToolkitCore.EuclideanManifoldVisualGeometryToolkitCore.ExpParamVisualGeometryToolkitCore.FreeTVisualGeometryToolkitCore.FundMatManifold7VisualGeometryToolkitCore.HomographyManifold8VisualGeometryToolkitCore.LeftRigidManifold6VisualGeometryToolkitCore.Line2DManifoldVisualGeometryToolkitCore.LineConfidenceBandVisualGeometryToolkitCore.LinearSubspaceManifoldVisualGeometryToolkitCore.MRPParamVisualGeometryToolkitCore.ManifoldVisualGeometryToolkitCore.ProductManifoldVisualGeometryToolkitCore.RenormalizedTVisualGeometryToolkitCore.RigidChartVisualGeometryToolkitCore.RigidExpManifold6VisualGeometryToolkitCore.RigidManifold6VisualGeometryToolkitCore.RotationChartVisualGeometryToolkitCore.RotationParamVisualGeometryToolkitCore.S2TangentTVisualGeometryToolkitCore.SO3ManifoldVisualGeometryToolkitCore.SphereManifoldVisualGeometryToolkitCore.SubsetManifoldVisualGeometryToolkitCore.TranslationCouplingVisualGeometryToolkitCore.action_jacobianVisualGeometryToolkitCore.ambient_coordinatesVisualGeometryToolkitCore.anchor_svdVisualGeometryToolkitCore.band_boundaryVisualGeometryToolkitCore.band_conicVisualGeometryToolkitCore.confidence_ellipseVisualGeometryToolkitCore.constraint_dimVisualGeometryToolkitCore.constraint_jacobiansVisualGeometryToolkitCore.constraint_with_jacobiansVisualGeometryToolkitCore.constraint_with_jacobiansVisualGeometryToolkitCore.constraint_with_jacobiansVisualGeometryToolkitCore.constraint_with_jacobiansVisualGeometryToolkitCore.epipolar_bandVisualGeometryToolkitCore.implicit_covarianceVisualGeometryToolkitCore.in_bandVisualGeometryToolkitCore.local_coordinatesVisualGeometryToolkitCore.local_coordinatesVisualGeometryToolkitCore.local_coordinatesVisualGeometryToolkitCore.manifold_constraintsVisualGeometryToolkitCore.minimal_manifoldVisualGeometryToolkitCore.modelVisualGeometryToolkitCore.pose_action_jacobianVisualGeometryToolkitCore.recenterVisualGeometryToolkitCore.restrictVisualGeometryToolkitCore.restrictVisualGeometryToolkitCore.retractVisualGeometryToolkitCore.retract_directionsVisualGeometryToolkitCore.retract_jacobianVisualGeometryToolkitCore.s2_tangent_basisVisualGeometryToolkitCore.s2_tangent_basis_matrixVisualGeometryToolkitCore.tangent_partVisualGeometryToolkitCore.tangent_sizeVisualGeometryToolkitCore.transform_covarianceVisualGeometryToolkitCore.zero_tangent