Transforms API

Framed maps — the geometric objects, whose type is their transform class — the dense homography matrix, coordinate conversions, uncertainty propagation, and the SO(3)/SE(3) helpers. Region maps and image warping are on the Warping API page. See the Transforms and Coordinate Frames guides.

Framed Maps

A map acts on a primitive in its class: an EuclideanMap or a SimilarityMap keeps a circle a Circle (and a feature frame its type), an AffineMap gives a HomEllipseMat, a HomographyMap a Conic (the actions are on the Warping API page).

VisualGeometryToolkitCore.CameraExtrinsics — Type
CameraExtrinsics

Frame-pair alias for camera EXTRINSICS (world-to-camera transform): CameraExtrinsics{T, R, V} = EuclideanMap{3, T, R, V, WorldFrame, CameraFrame} (monogram X_CW). The typed counterpart of CameraPose; inv(::CameraExtrinsics) -> CameraPose via the frame swap. Not a wrapper — a CameraExtrinsics IS an EuclideanMap.

CameraExtrinsics(m::EuclideanMap{3}) labels m as extrinsics (an explicit relabel, overriding whatever frames m carried). PosedCamera(model, ::CameraExtrinsics) stores it directly in the extrinsics slot.

source
VisualGeometryToolkitCore.CameraPose — Type
CameraPose

Frame-pair alias for a camera POSE (camera-to-world transform): CameraPose{T, R, V} = EuclideanMap{3, T, R, V, CameraFrame, WorldFrame} (monogram X_WC). Not a wrapper — a CameraPose IS an EuclideanMap, so .R/.t, point application, and composition all apply directly.

CameraPose(m::EuclideanMap{3}) labels m as a pose (an explicit relabel, overriding whatever frames m carried).

The typed algebra subsumes the former wrapper machinery: inv(::CameraPose) -> CameraExtrinsics (and back) via the frame swap, and PosedCamera(model, ::CameraPose) inverts internally into the extrinsics slot, so handing a pose where extrinsics are expected is unrepresentable.

See also CameraExtrinsics, lookat_pose, Frame.

source
VisualGeometryToolkitCore.EuclideanMap — Type
EuclideanMap{N,T,R<:Rotation{N,T},V<:StaticVector{N},Src<:Frame{N},Dst<:Frame{N}} <: Transformation

Rigid transformation x ↦ R*x + t in N dimensions, combining rotation and translation.

This represents a Euclidean (rigid body) transformation that preserves distances and angles. It consists of a rotation R followed by a translation t.

The translation's element type is free. A PHYSICAL map — a camera or target pose, a rig member — holds FlexUnits' base-form quantities, Quantity{T, Dimensions{FixRat32}} in metres (UnitfulEuclideanMap); an inner-loop map holds plain numbers — Float64, Float32, a ForwardDiff Dual — beside a rotation of its own eltype T (PlainEuclideanMap). has_units tells them apart, ustrip(u, E) gives the plain map in the unit u, and attaching a unit is construction, EuclideanMap(rotation(E), translation(E) .* u). A unitful map meeting plain numbers — applied to a plain point, composed with a plain map — is FlexUnits' DimensionError, never a promotion.

The trailing Src/Dst parameters are coordinate-Frame tags: the map consumes Src-coordinates and produces Dst-coordinates (Drake monogram: X_AB ≙ Src = B, Dst = A). All constructors that do not state frames produce Src = Dst = UnlabeledFrame{N} — labels are opt-in and never viral. inv swaps the frames; ∘ checks the junction frame between two labeled maps (see the truth table at that method) and throws a MethodError on a labeled mismatch. Use relabel/unlabel as explicit escape hatches.

Fields

  • R::Rotation{N,T}: Rotation component (from Rotations.jl)
  • t::StaticVector{N}: Translation vector — plain numbers, or lengths in metres

Constructors

# From rotation and translation vector
R = RotMatrix{2}(π/4)  # 45° rotation
t = [2.0, 1.0]         # translation
euclidean = EuclideanMap(R, t)

# From MRP (Modified Rodrigues Parameters) for 3D
mrp = MRP(0.1, 0.2, 0.3)
euclidean_3d = EuclideanMap(mrp, [1.0, 2.0, 3.0])

# A physical pose: the translation a length (stored in metres)
pose = EuclideanMap(mrp, SVector(62.8, 0.0, 1200.0) .* mm)

# From tuple
euclidean = EuclideanMap((R, t))

Usage

# Apply transformation to points
point = Point2f(1.0, 0.0)
transformed = euclidean(point)

# Compose transformations
combined = euclidean2 ∘ euclidean1

# Inverse transformation
inverse_euclidean = inv(euclidean)
source
VisualGeometryToolkitCore.EuclideanMap — Method
EuclideanMap(qw, qx, qy, qz, tx, ty, tz) -> EuclideanMap{3,Float64,…}
EuclideanMap(quat::AbstractVector, t::AbstractVector) -> EuclideanMap{3,Float64,…}

COLMAP-style 3D rigid transform from a unit quaternion (qw, qx, qy, qz) and a 3-vector translation (tx, ty, tz). Used by SfM-format readers (ETH3D, IMC PhotoTourism, ScanNet++, custom COLMAP exports) where pose data arrives as raw scalars on a per-line basis.

Rotations.QuatRotation auto-normalizes the quaternion, so callers don't need to pre-normalize — the resulting rotation is always a valid SO(3) element.

Examples

# Per-scalar form (typical when parsing line-by-line)
T = EuclideanMap(0.998, 0.0, 0.064, 0.0, 1.5, 2.5, 3.0)

# Vector form (typical when receiving npz arrays)
T = EuclideanMap([qw, qx, qy, qz], [tx, ty, tz])
source
VisualGeometryToolkitCore.EuclideanMap — Method
EuclideanMap(I, t) -> EuclideanMap

The pure translation by t: the identity rotation (LinearAlgebra.I, a UniformScaling) and the translation t, plain or a vector of lengths, as EuclideanMap(R, t) takes them. EuclideanMap(I, [0m, 0m, 0m]) is a physical identity pose, EuclideanMap(I, zeros(3)) a plain one.

source
VisualGeometryToolkitCore.EuclideanMap — Method
EuclideanMap(R::Rotation{N}, t::AbstractVector{<:Quantity}; check = true)

A PHYSICAL map: every component of t a length, in any length unit (a quantity of another dimension is a DimensionError), stored as FlexUnits' base-form quantities — metres (base_quantity). A RotMatrix, QuatRotation or Angle2d rotation is stored as given; any other Rotation as a RotMatrix of its own eltype.

source
VisualGeometryToolkitCore.EuclideanMap — Method
EuclideanMap(R::Rotation{N,T}, t::StaticVector{N,<:Real}; check=true) -> EuclideanMap{N,T,...}

Element-type-preserving constructor that PRESERVES the input static-vector subtype and eltype. Vec3, Point3, SVector, MVector, FieldVector are all bit-identical at the data level — accept whatever the caller hands in. The translation's eltype is independent of the rotation's: a Float32 or ForwardDiff Dual translation beside a Float64 rotation stays what it is (an integer translation becomes its float type).

If a typed container (e.g. MVector{4, Extrinsics3} or FixedModels{4, Extrinsics3}) needs a specific translation type, the implicit Base.convert defined below performs the (zero-cost) bitcast at the storage boundary.

source
VisualGeometryToolkitCore.EuclideanMap — Method
EuclideanMap(R::SMatrix{3,3,T,9}, t::StaticVector{3};
             check=true, project=false) -> EuclideanMap

Construct a rigid-body transform directly from a 3×3 SMatrix (not already a Rotation type). This is the boundary constructor — the entry point at which raw 3×3 numerical data crosses into the typed EuclideanMap. All other constructors take a typed Rotation{N,T} whose invariants are enforced by Rotations.jl.

Validation policy

checkprojectsemanticstypical caller
truefalsestrict — must already be a rotationmath you trust
truetruesnap-to-nearest then verifydataset loaders
falsefalsetrusted — skip everythinginternal algebra
falsetruesnap, no verifyvery dirty inputs

project=true snaps R to the nearest proper rotation via SVD (U·Vᵀ, with a sign flip if the result has determinant −1). Floating-point drift from text-file roundtrip or naive inv(T_B)·T_A composition typically sits at 1e-6-ish, well within the snap-then-verify regime.

Narrowed to SMatrix{3,3,T,9} (rather than StaticMatrix{3,3,T}) to avoid dispatch ambiguity with the Rotation-typed constructor: RotMatrix{3,T} is a subtype of both Rotation and StaticMatrix{3,3,T}, but is not an SMatrix, so narrowing here keeps the two paths disjoint.

source
VisualGeometryToolkitCore.Extrinsics3 — Type
Extrinsics3

Concrete LABELED CameraExtrinsics alias: CameraExtrinsics{Float64, RotMatrix{3,Float64,9}, Vec{3,Float64}}, i.e. EuclideanMap{3, Float64, …, Src = WorldFrame, Dst = CameraFrame} (monogram X_CW, applies as x_cam = R·X_world + t).

The model currency of the absolute-pose solvers: p3p returns FixedModels{4, Extrinsics3} and P3P <: MinimalSolver{Extrinsics3}. isbitstype(Extrinsics3) == true — stored inline in FixedModels and RansacWorkspace, zero heap allocation on update.

Replaces the deprecated Pose3 alias (removed in 0.7): same layout and element types, but the world-to-camera direction is now a dispatch fact instead of a docstring caveat — an Extrinsics3 IS a CameraExtrinsics, so inv yields a CameraPose and labeled composition is frame-checked. Solver code that composes models against other labeled maps must have the junction frames agree (or unlabel explicitly).

source
VisualGeometryToolkitCore.PlainEuclideanMap — Type
PlainEuclideanMap{N}
PlainEuclideanMap{N, T}

An EuclideanMap whose translation is plain numbers (V <: StaticVector{N, <:Real}: Float64, Float32, a ForwardDiff Dual) — the map the inner loop takes: minimal kernels, charts and manifolds, solvers. A unitful map handed to a function typed on it is a MethodError. T is the rotation's eltype.

source
FlexUnits.ustrip — Method
ustrip(u::Units, X::Uncertain{<:EuclideanMap{3}}) -> Uncertain

The plain form of an uncertain physical pose: the map stripped in u (ustrip(u, E)) and its 6×6 covariance (or precision) — rotation first, the RigidExpManifold6 chart — rescaled with it: the translation block in u², the rotation–translation block in rad·u. The inverse is with_unit(X, u).

source
FlexUnits.ustrip — Method
ustrip(u::Units, E::EuclideanMap) -> EuclideanMap

The plain map of a physical E: its translation in the length unit u, the rotation and the frames unchanged. An ArgumentError on a map that is already plain (has_units); with_unit(E, u) attaches one.

source
VisualGeometryToolkitCore.angle_between — Method
angle_between(u, v) -> Real

The angle between two vectors of 2 or 3 entries, in radians in [0, π]: θ = atan(‖u × v‖, u · v) (for 2-vectors ‖u × v‖ = |u₁v₂ − u₂v₁|). Neither vector need be unit: the form is scale-free. Accurate at every angle, where the acos of the normalized dot product loses half the digits near 0 and π (its derivative is unbounded there): two unit vectors 1e-8 rad apart have a dot product that rounds to 1, so acos returns 0 (or 1.5e-8 a few ulps away), and this form returns the angle to full precision. 0 when either vector is zero; NaN in, NaN out. Generic in the element type (ForwardDiff duals flow through away from θ = 0, where ‖u × v‖ has no derivative). The angle between two bearings, two rotation axes, a predicted and an observed ray; for two translations up to scale, lengths included, translation_angle.

source
VisualGeometryToolkitCore.attach_point — Function
strip_point(u::Units, p) -> p′
attach_point(u::Units, p) -> p′

A point's lengths as plain numbers in the unit u, and the inverse: a static vector of lengths (an Uncertain one with its covariance in the unit's square). A point that is plain already (already has units) is an ArgumentError. The moves a carrier of points makes in its own ustrip(u, x) and with_unit(x, u).

source
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(pose::EuclideanMap{3}, X::SVector{3}, ray::SVector{3})
    -> (g::SVector{2}, G_x::SMatrix{2,5}, G_θ::SMatrix{2,6})

Compute the reprojection constraint and its derivatives for a 3D rigid pose.

The constraint is the perspective-projected residual in calibrated coordinates: g = π(R·X + t) − π(ray)

where π is perspective division [x,y,z] → [x/z, y/z] and ray is the backprojected observation (encodes camera calibration).

The model Jacobian G_θ uses the left-perturbation convention with a 6-DOF parameterization δ = (δω₁, δω₂, δω₃, δt₁, δt₂, δt₃): ∂g/∂δ = Jπ · [−[X_cam]× I₃]

where Jπ is the 2×3 perspective division Jacobian and [v]× is the skew-symmetric (cross-product) matrix of v.

Returns:

  • g::SVector{2,T} — reprojection residual (calibrated coordinates)
  • G_x::SMatrix{2,5,T} — ∂g/∂(X, ray₁, ray₂) observation Jacobian
  • G_θ::SMatrix{2,6,T} — ∂g/∂(δω, δt) pose perturbation Jacobian
PINHOLE ONLY — this residual lives in the gnomonic chart

π divides by the third component, so this is the z = 1 chart (GnomonicFrame) and it exists only on the forward hemisphere. It is degenerate at 90° from the optical axis and cannot represent a ray with Z ≤ 0 at all. That is correct for a PINHOLE camera and wrong for a fisheye or omnidirectional one, whose rays routinely go past 90°.

It is legitimate where it is used — VisualGeometryToolkitEstimation's Pinhole PnP LM — and must not become the general path. The camera-general route is the ray-domain pixel residual (constraint_with_jacobians(model, pose, X, u) in cameras/camera_models.jl), which projects through the camera's own lens and therefore never assumes the chart exists. Calibrated two-view geometry takes the same view: it acts on unit bearings (unit rays, StaticVector{3}), never on this chart.

source
VisualGeometryToolkitCore.relabel — Method
relabel(Src::Type{<:Frame{N}}, Dst::Type{<:Frame{N}}, m::EuclideanMap{N}) -> EuclideanMap

Return m with its frame tags replaced by Src/Dst (map consumes Src-coordinates, produces Dst-coordinates). Zero-cost: the rotation and translation are reused bit-for-bit; only the type changes.

This is a DELIBERATE escape hatch: it overrides whatever frames m carried, with no check. Use it at trust boundaries (dataset loaders, solver outputs) where the caller — not the type system — knows the frames. Attaching a frame of the wrong dimension (e.g. a Frame{2} to an N = 3 map) is a type error (MethodError).

See also unlabel, Frame.

source
VisualGeometryToolkitCore.skew — Method
skew(v::AbstractVector) -> SMatrix{3,3,Float64,9}

Skew-symmetric (cross-product) matrix [v]_× such that [v]_× w = v × w for all w ∈ R³. Used in essential-matrix construction E = [t]_× R and in pose-perturbation Jacobians (see constraint_with_jacobians).

GTSAM spells this skewSymmetric; Sophus spells it SO3::hat (with vee for the inverse).

Errors if length(v) != 3.

Example

t = SVector{3,Float64}(1.0, 2.0, 3.0)
S = skew(t)        # 3×3 SMatrix
S * SVector(0.0, 1.0, 0.0) ≈ cross(t, SVector(0.0, 1.0, 0.0))   # true
source
VisualGeometryToolkitCore.strip_point — Function
strip_point(u::Units, p) -> p′
attach_point(u::Units, p) -> p′

A point's lengths as plain numbers in the unit u, and the inverse: a static vector of lengths (an Uncertain one with its covariance in the unit's square). A point that is plain already (already has units) is an ArgumentError. The moves a carrier of points makes in its own ustrip(u, x) and with_unit(x, u).

source
VisualGeometryToolkitCore.translation — Method
translation(E::EuclideanMap) -> StaticVector

The translation t of the rigid map x ↦ R·x + t: lengths in metres (FlexUnits base-form quantities) for a physical map, plain numbers for an inner-loop one (has_units).

source
VisualGeometryToolkitCore.translation_angle — Method
translation_angle(t1, t2) -> Float64

Angular distance (radians) between two translation directions — plain vectors, or lengths (compared in metres).

Since translation is only recoverable up to scale from epipolar geometry, this compares directions, scale-free: angle_between(t₁, t₂), atan(‖t₁ × t₂‖, t₁ · t₂), accurate at every angle. 0 when either vector is (numerically) zero.

source
VisualGeometryToolkitCore.unlabel — Method
unlabel(m::EuclideanMap{N}) -> EuclideanMap{..., UnlabeledFrame{N}, UnlabeledFrame{N}}

Erase the frame tags of m (both endpoints become UnlabeledFrame{N}). Zero-cost, deliberate escape hatch — the result composes with anything under the wildcard rule. Inverse direction of relabel.

source
VisualGeometryToolkitCore.with_unit — Method
with_unit(E::EuclideanMap, u::Units) -> EuclideanMap

The physical map of a plain E: its translation read as being in the length unit u and stored in metres, the rotation (of any parameterization) and the frames kept exactly. The inverse of ustrip(u, E); an ArgumentError on a map that already has units.

source
VisualGeometryToolkitCore.SimilarityMap — Type
SimilarityMap{N,T,S,R<:Rotation{N,T},V<:StaticVector{N},Src<:Frame{N},Dst<:Frame{N}} <: Transformation

The similarity transformation x ↦ s·R·x + t of N-space: a positive scale s, a rotation R (a Rotations.jl Rotation{N}) and a translation t.

S = SimilarityMap(2.0, RotMatrix{2}(π / 6), SVector(1.0, -3.0))
S(circle)                  # a Circle: centre mapped, radius · 2
S ∘ inv(S)                 # a SimilarityMap (the identity, up to rounding)
SimilarityMap(E)           # an EuclideanMap, s = 1 (also a rotation LinearMap, a Translation, a uniform scaling)

Accessors scale, rotation, translation. ∘ with a similarity, an EuclideanMap or a Translation (either order) is a SimilarityMap, and so is inv, both computed from the parts. It acts on primitives and feature frames in its class (the class table of the Transforms guide); to_homogeneous is [s·R t; 0 1].

Plain numbers only. No consumer holds a physical similarity, so a quantity translation or scale (or a unitful EuclideanMap converted or composed with one) is refused with an ArgumentError naming the fix: strip it in metres first, ustrip(m, t) or ustrip(m, E). check = true checks the rotation and that s is positive and finite.

The trailing Src/Dst parameters are coordinate-Frame tags, as for EuclideanMap: constructors that state none give UnlabeledFrame{N}, inv swaps them, ∘ checks a labeled junction (MethodError on a mismatch), relabel/unlabel are the escape hatches.

source
VisualGeometryToolkitCore.SimilarityMap — Method
SimilarityMap(E::EuclideanMap) -> SimilarityMap

The rigid map as a similarity of scale 1, its frame tags kept. A physical (unitful) map is refused: strip it first, SimilarityMap(ustrip(m, E)).

source
VisualGeometryToolkitCore.SimilarityMap — Method
SimilarityMap(s::Real, R::Rotation{N}, t::AbstractVector{<:Real}; check = true)

The similarity x ↦ s·R·x + t. A RotMatrix, QuatRotation or Angle2d is stored as given, any other Rotation as a RotMatrix; an integer scale or translation becomes its float type.

source
VisualGeometryToolkitCore.RotationMap — Type
RotationMap{N, T, R <: Rotation{N, T}, Src <: Frame{N}, Dst <: Frame{N}} <: AbstractMatrixMap

A rotation about the origin, x ↦ R·x, in N dimensions, carrying the frame tags an EuclideanMap carries: it consumes Src-coordinates and produces Dst-coordinates. Its one field is the rotation R, so it has no translation at all, rather than a zero one: it poses a camera that turns about its centre (OrientedCamera), and it holds no length, so it is the same map plain and physical (has_units is false).

  • RotationMap(R::Rotation{N}; check = true), unlabeled. A RotMatrix, a QuatRotation or an Angle2d is stored as given, any other Rotation as a RotMatrix of its own eltype, by EuclideanMap's rule. RotationMap(R::StaticMatrix{N, N}; check = true) takes a raw matrix as a RotMatrix. check verifies R·Rᵀ = I, det R = 1.
  • relabel(Src, Dst, M) and unlabel(M) set and erase the tags.
  • M(x) is R·x, for a vector or a point, plain or physical.
  • inv(M) is the inverse rotation with the tags swapped.
  • A ∘ B checks the junction (B's Dst meets A's Src, the unlabeled frame a wildcard). Two rotations compose to a RotationMap. A rotation and an EuclideanMap, in either order, compose to an EuclideanMap: R_A·(R_B·x + t_B) = (R_A R_B)·x + R_A t_B and R_A R_B·x + t_A.
  • rotation(M) is R. transform_deriv(M, x) is R. to_homogeneous(M) is the (N + 1) × (N + 1) block [R 0; 0 1]. convert_eltype(T, M) rebuilds the rotation in T and keeps the tags.
  • == compares the rotations as matrices and the tags. isapprox compares the rotations' matrices, and the tags must be equal.

It is isbits, so it is a device value as it is. A persisted rotation is a Hamilton quaternion, as for poses (the stack's persisted-geometry convention); no kind writes a RotationMap yet.

using Rotations: RotY
Rk = relabel(CameraFrame, WorldFrame, RotationMap(RotY(0.3)))   # a frame's orientation: camera → world
camera = PosedCamera(model, Rk)            # an OrientedCamera; stores the extrinsics inv(Rk)
pose(camera) == Rk
source
VisualGeometryToolkitCore.HomographyMap — Type
HomographyMap{N,T,M<:StaticMatrix{N,N,T},Src<:Frame,Dst<:Frame} <: Transformation

Projective transformation with cached inverse for efficient covariant (points) and contravariant (lines, conics) transformations.

A HomographyMap is ALWAYS projective in meaning, whatever its matrix: a circle or an ellipse maps to a Conic. The sub-projective classes are their own maps, acting in their class (the class table of the Transforms guide, "Maps Carry the Transform Class").

Type Parameters

  • N: Matrix dimension (3 for 2D projective, 4 for 3D projective)
  • T: Element type
  • M: Matrix type — a HomographyMat (a lifted homography matrix) or a plain SMatrix (every other lift and every mixed composition)
  • Src/Dst: coordinate-Frame tags of the point FRAMES the map connects — Frame{N-1}, since the N×N matrix acts on (N-1)-dimensional frames (a 3×3 homography is 2D↔2D projective: Frame{2} on both sides). Same design as EuclideanMap: the map consumes Src-frame coordinates and produces Dst-frame coordinates (Drake monogram H_AB ≙ Src = B, Dst = A), every frame-free constructor defaults both tags to UnlabeledFrame{N-1}, inv swaps the tags, and ∘ checks a labeled junction (identical wildcard truth table — see Base.:∘ below).

HomographyMat stays frame-FREE algebra; frames are declared at the Mat → Map lift:

H = HomographyMap{TargetPlaneFrame, PixelFrame{:colmap}}(H_mat)   # labeled lift
H = HomographyMap(H_mat)                                    # unlabeled lift

relabel/unlabel are the same explicit escape hatches as for EuclideanMap.

Transformation Rules (Hartley & Zisserman conventions)

  • Points (covariant): p' = H·p
  • Lines (contravariant): l' = H^{-T}·l
  • Conics (contravariant): Q' = H^{-T}·Q·H^{-1}

Dimension Convention

  • N×N matrix operates on (N-1)-dimensional Euclidean space
  • Point{N-1} → Euclidean input, lifted via to_affine
  • Point{N} → Homogeneous input, used directly

Examples

# Lift a homography matrix, or any structured map (its analytic inverse is stored)
h = HomographyMap(HomographyMat(camera))
h = HomographyMap(EuclideanMap(RotMatrix{2}(θ), t))
h = HomographyMap(AffineMap(A, b))

# Transform points (Euclidean → homogeneous output)
p′ = h(Point2(100, 200))  # → Point3 (homogeneous)

# Transform points (homogeneous → homogeneous output)
p′ = h(Point3(100, 200, 1))  # → Point3

# Transform conics: always a Conic (classify it, or apply the structured map itself)
h(circle)   # Conic
h(ellipse)  # Conic

# Transform lines
l′ = h(Line2D(1.0, -1.0, 0.0))  # → Line2D

# Composition
h_composed = h1 ∘ h2

# Inverse is O(1) - just swaps stored matrices
h_inv = inv(h)
source
VisualGeometryToolkitCore.HomographyMap — Method
(h::HomographyMap{3})(c::Circle) -> Conic

Transform a circle through a homography map: a Conic, whatever the map's matrix (a HomographyMap is projective in meaning). An EuclideanMap, a SimilarityMap, a translation, rotation or uniform scaling keeps a circle a Circle, and an affine map gives a HomEllipseMat: apply them directly.

The returned conic could be:

  • An ellipse (most common for camera projections of visible circles)
  • A hyperbola (circle partially behind camera)
  • A parabola (circle tangent to plane at infinity)
  • A degenerate conic (point or lines)

Use classify_conic() to determine the conic type, then construct the appropriate geometric primitive:

H = HomographyMap(HomographyMat(camera))
circle = Circle(Point2(100.0, 100.0), 50.0)
Q′ = H(circle)  # a Conic under a camera homography

trait = classify_conic(Q′)
if trait isa EllipseTrait
    ellipse = Ellipse(Q′)  # Construct geometric primitive
elseif trait isa HyperbolaTrait
    # Handle hyperbola case
end

Example: Filter to valid ellipses

H = HomographyMap(HomographyMat(camera))
conic_matrices = H.(circles)
ellipses = [Ellipse(Q) for Q in conic_matrices if classify_conic(Q) isa EllipseTrait]
source
VisualGeometryToolkitCore.HomographyMap — Method
(h::HomographyMap{3})(e::Ellipse) -> Conic

Transform an ellipse through a homography map: a Conic, which could be any conic type (an affine map applied directly gives a HomEllipseMat). Use classify_conic() to determine the type and construct appropriately.

Example

H = HomographyMap(HomographyMat(camera))
Q′ = H(ellipse)
if classify_conic(Q′) isa EllipseTrait
    ellipse′ = Ellipse(Q′)
end
source
VisualGeometryToolkitCore.HomographyMap — Method
HomographyMap(a) -> HomographyMap{3, T, SMatrix{3, 3, T, 9}}

Any structured planar map — a Translation, LinearMap or AffineMap, an EuclideanMap{2}, a SimilarityMap{2} — lifted to a homography: the forward matrix to_homogeneous(a), the stored inverse to_homogeneous(inv(a)) (the map's own analytic inverse, never a 3×3 inversion). An EuclideanMap's or SimilarityMap's frame tags ride along. The lift is projective in meaning: it maps a circle to a Conic.

source
VisualGeometryToolkitCore.HomographyMap — Method
(h::HomographyMap{N})(hp::Hyperplane{M}) -> Hyperplane{M}

Transform a hyperplane through a homography map.

Hyperplanes transform contravariantly (like lines): π' = H^{-T}·π

Example

H = HomographyMap(AffineMap(A, b))
line = Line2D(1.0, -1.0, 0.0)  # x - y = 0
line′ = H(line)

plane = Plane3D(Point3(0,0,0), Vec3(0,0,1))  # z = 0
# For 4×4 homography map:
# plane′ = H(plane)
source
VisualGeometryToolkitCore.transfer_ellipse — Method
transfer_ellipse(T::Transformation, shape::Union{Circle, Ellipse}) -> Ellipse | nothing

The ellipse-PRESERVING transfer of a circle/ellipse through any 2-D map of the warping paradigm. A homography carries the conic exactly (the method above); every other map is replaced by its local affinity at the shape's centre — c′ = T(c), Σ′ = J·Σ·Jᵀ with J = transform_deriv(T, c) — the same linearisation a detection's shape gets when it is carried across an undistortion. A ComposedTransformation transfers stage by stage, so the exact stages stay exact and only the non-projective ones linearise: for a lens camera, distortion_map(model) ∘ HomographyMap(pinhole camera) pushes the conic through the plane homography exactly and applies the lens's affinity at the transferred centre (see transfer_ellipse(::PosedCamera, _)). nothing where the map is not defined or not finite at the centre (a lens fold, the line at infinity).

source
VisualGeometryToolkitCore.transfer_ellipse — Method
transfer_ellipse(H::HomographyMap, shape::Union{Circle, Ellipse}) -> Ellipse | nothing

Push a circle/ellipse through the homography and classify: the transferred conic when it IS an ellipse, nothing when the map carries it across the line at infinity (hyperbola/parabola/degenerate). Downstream code spelled Q = H(region(...)); is_ellipse(Q) || continue; Ellipse(Q) at six call sites — this is that pattern, once.

source

The Homography Matrix

VisualGeometryToolkitCore.HomographyMat — Type
HomographyMat{T} <: StaticMatrix{3, 3, T}

A planar projective homography as a dense 3×3 matrix — algebra (a solver's, a DLT's output), frame-free. The geometric object is its HomographyMap, which carries the inverse and the frame tags. The product and the inverse of homographies are homographies.

source

Coordinate Conversions

VisualGeometryToolkitCore.to_affine — Method
to_affine(v::StaticVector{N,T}) -> StaticVector{N+1,T}

Convert euclidean coordinates to affine coordinates by appending 1 (setting w=1). Embeds points in the affine hyperplane of projective space. vcat preserves the input wrapper (Point2 → Point3, Vec2 → Vec3, SVector{2} → SVector{3}).

For a non-static AbstractVector fallback, returns SVector{N+1,T}.

source
VisualGeometryToolkitCore.to_euclidean — Method
to_euclidean(v::StaticVector{3}, w_ref::Real) -> Union{SVector{2}, Nothing}

Chirality-guarded perspective division: returns nothing unless the homogeneous coordinate is finite, nonzero, and on the SAME SIDE as w_ref (the w a reference point on the visible side produced — see homogeneous_w). Points beyond a plane's horizon flip w's sign and would otherwise wrap into view from behind the camera; the throwing one-arg form cannot express that, so callers hand-rolled this guard (three copies in BlobBoards, one of which forgot it — the 2026-09 audit's training-mask bug).

source
VisualGeometryToolkitCore.to_euclidean — Method
to_euclidean(v::AbstractVector{T}) -> SVector{N-1,T}

Convert homogeneous coordinates to euclidean by perspective division. Removes the last coordinate after dividing by it.

source
VisualGeometryToolkitCore.affine_part — Method
affine_part(M::StaticMatrix{3,3}) -> AffineMap

Extract the 2×2 linear part and 2D translation from a 3×3 homogeneous matrix. The bottom row is not read; for a camera calibration matrix K it is the pixel affine of the imaging chain, affine_part(K) ∘ lens.

Works with any 3×3 static matrix: a CameraCalibrationMat, a plain SMatrix{3,3} (to_homogeneous of an affine map), etc.

Example

K = CameraCalibrationMat(800.0px, [320.0px, 240.0px])
am = affine_part(K)  # AffineMap([800 0; 0 800], [320, 240])
source
VisualGeometryToolkitCore.to_homogeneous — Method
to_homogeneous(map) -> SMatrix{N+1, N+1, T}
to_homogeneous(A::StaticMatrix{N,N}, t::StaticVector{N}) -> SMatrix{N+1, N+1, T}

The homogeneous matrix [A t; 0 1] of an affine-family map of N-space, as a plain SMatrix — for algebra and interop. Every map has one: a Translation, a LinearMap (a rotation, a UniformScaling — 2-D — a Diagonal, any matrix), an AffineMap, an EuclideanMap, a SimilarityMap, a RotationMap, a composition a ∘ b (the product), and a HomographyMap (its forward matrix). Exact: the map's entries copied, the last row the constant 0 … 0 1.

The matrix carries no class: the map's TYPE does. Geometric operations take the map (S(circle), a ∘ b, inv(a)) and convert inside, once.

Example

M = @SMatrix [2.0 0.0; 0.0 1.5]  # scaling
t = SVector(1.0, 2.0)            # translation
H = to_homogeneous(M, t)         # 3×3 homogeneous matrix
source

Uncertainty Propagation

VisualGeometryToolkitCore.propagate — Method
propagate(J, Σ)

Propagate covariance Σ through the linear map with Jacobian J:

propagate(J, Σ) = J * Σ * Jᵀ

Specializations:

  • Σ = I (UniformScaling{Bool}): returns J * Jᵀ
  • Σ = λI (UniformScaling): returns λ * (J * Jᵀ)
  • Σ = diag(...) (Diagonal): returns J * Σ * Jᵀ via column scaling
  • General matrix: full sandwich product J * Σ * Jᵀ
  • Static J with a dense Σ (Matrix, Symmetric, Diagonal{Vector}): Σ is converted once to its static size, so the result is static like J

One of three covariance verbs: propagate goes THROUGH A MAP (its Jacobian, or a framed map carrying its frames), transform_covariance BETWEEN CHARTS of one model, implicit_covariance of an ESTIMATE.

GTSAM equivalent: a function's OptionalJacobianH, with the caller forming H·Σ·Hᵀ.

source
VisualGeometryToolkitCore.whitening_factor — Method
whitening_factor(G_x::SMatrix{dg,nx}, Σ_x = I) -> SMatrix{dg,dg}

The inverse Cholesky factor L⁻¹ of the propagated constraint covariance Σ_g = G_x·Σ_x·G_xᵀ = L·Lᵀ: whitened rows are L⁻¹·g and L⁻¹·G_θ, the factor frozen once (at a seed, or at the previous iteration's model of an iteratively reweighted fit) where whiten re-derives it per call — the Gauss–Newton currency of a constraint whose LM row holds its whitening fixed. A scalar constraint's gradient (an SVector) gives the number 1/√(G_x·Σ_x·G_xᵀ). A degenerate Σ_g (its relative rank verdict) is a ZERO factor — its rows drop out of a solve; NaN propagates.

source
VisualGeometryToolkitCore.propagate — Method
propagate(m::SimilarityMap{N}, Σ::CovMat{N})  -> CovMat{N, ..., Dst}
propagate(m::SimilarityMap{N}, Λ::PrecMat{N}) -> PrecMat{N, ..., Dst}
propagate(m::SimilarityMap{N}, u::Uncertain)  -> Uncertain

As for an EuclideanMap (the same junction check, the same two laws), with the constant Jacobian J = s·R: Σ ↦ s²·RΣRᵀ, Λ ↦ RΛRᵀ / s².

source
VisualGeometryToolkitCore.propagate — Method
propagate(m::EuclideanMap{N}, Σ::CovMat{N})  -> CovMat{N, ..., Dst}
propagate(m::EuclideanMap{N}, Λ::PrecMat{N}) -> PrecMat{N, ..., Dst}

Checked, frame-labeled propagation of a statistical form through a framed rigid map. The map's action on points is affine with constant Jacobian J = R, so no evaluation point is needed. The form's frame must meet the map's Src under the standing wildcard truth table (a labeled mismatch is a MethodError; UnlabeledFrame on either side is unchecked), and the result carries the map's Dst as-is.

The law dispatches on the REPRESENTATION — this is the point of the types:

  • covariance, a (2,0) tensor, pushes forward: Σ ↦ J Σ Jᵀ
  • precision, a (0,2) bilinear form, pulls back: Λ ↦ J⁻ᵀ Λ J⁻¹

(For a rigid map the two conjugations coincide numerically because R is orthogonal; the code follows the honest law per type so the invariant propagate(m, inv(Σ)) == inv(propagate(m, Σ)) holds by construction, not by coincidence.)

The raw-Jacobian layer propagate(J, Σ) is unchanged and unlabeled — use it when no framed map is in hand.

source
VisualGeometryToolkitCore.propagate — Method
propagate(h::HomographyMap{3}, u::Uncertain{<:StaticVector{2}}) -> Uncertain

Transport an uncertain 2D point through a framed collineation. The projective point action x ↦ π(H·x̃) is nonlinear, so the Jacobian is evaluated at the point — which is why the HomographyMap overloads take the observation (Uncertain), not a bare statistical form. Frame checking and labeling follow the framed-map contract (frame meets Src under the wildcard truth table; result carries Dst), and the law dispatches on the representation: covariance pushes forward Σ ↦ JΣJᵀ, precision pulls back Λ ↦ J⁻ᵀΛJ⁻¹ with the 2×2 point Jacobian J.

source
VisualGeometryToolkitCore.propagate — Method
propagate(m::EuclideanMap{N}, u::Uncertain) -> Uncertain

Transport an observation — geometry AND uncertainty — through a framed rigid map in one checked call: the value moves by the map's point action m(x), the statistical form by its own law (see the two-argument methods above, which also perform the frame check). The result's uncertainty carries the map's Dst frame.

source

Sampson Distance and Whitening

The Sampson residual is geometric (a model and its observation, no noise); whiten divides it by the observation's covariance along the normal.

VisualGeometryToolkitCore.constraint — Method
constraint(M, x...; kw...) -> g

The constraint value g(M; x...) alone — the <f> of the constraint_with_jacobians pair (conventions/jacobians.md): its first return, so a caller that keeps only g names the verb rather than discarding two Jacobians. A model may add a method that skips work the Jacobians need (the bearing TransferForm's g needs no backprojection Jacobians).

g is ALGEBRAIC: it scales with the model's homogeneous scale and with the observations' units (a fundamental matrix scaled by 10⁶ scales g by 10⁶), so it is not a distance and a threshold on it means nothing in pixels. The geometric distance is sampson_distance; the noise-whitened one mahalanobis_distance.

source
VisualGeometryToolkitCore.constraint_keywords — Method
constraint_keywords(M, xs) -> NamedTuple

The attribute columns of an observation set that the constraint of M takes as keywords, by keyword name, read per element by the batched Sampson functions: (; M₁ = xs.M₁, M₂ = xs.M₂) — the back-projection Jacobians — for the bearing forms of an EssentialMat and a TransferForm (xs of 3-vectors), none otherwise. A missing column is an ArgumentError naming it.

source
VisualGeometryToolkitCore.constraint_rounding_scale — Method
constraint_rounding_scale(M, x...; kw...) -> NamedTuple{(:gradient, :value)} | nothing

The rounding scales of the constraint g(M; x...) at an observation — the one input the Sampson degeneracy rule needs beyond (g, G_x): gradient, below which the observation gradient ‖G_x‖ is rounding (DEGENERATE), and value, below which |gᵢ| is rounding (the constraint VANISHES; a scalar, or one per component). Relative to the model's and the points' own magnitudes, so a model's homogeneous scale never decides. sampson_residual_with_normal consumes it; the default, nothing, makes only an exactly zero gradient degenerate and an exactly zero g vanishing. The epipolar models return their entries' rounding bounds (|F|·|s|, |d|ᵀ·|F|·|s|), a homography its rows' term-by-term bound, a conic, an ellipse, a circle and a radial camera theirs. The default is RIGHT, not missing, where a gradient cannot vanish to rounding: a Line2D's is its unit normal, and a TransferForm's blows up (its transfer goes to infinity) rather than vanishing, its rank verdict being the relative Cholesky test. A RANSAC problem answers it per point, constraint_rounding_scale(problem, model, i) (default nothing).

source
VisualGeometryToolkitCore.mahalanobis_distance — Method
mahalanobis_distance(M, x...; kw...) -> T

The first-order Mahalanobis distance of the observations x under the model M: the norm of whiten(sampson_residual_with_normal(M, x...)..., Σ), Σ the observations' own covariance per the observation-noise rule — a plain point unit isotropic (so a plain correspondence's Mahalanobis distance IS its sampson_distance), an Uncertain one its covariance or precision, an exact one zero (not every observation of a constraint: ArgumentError). In noise units: d² ~ χ²(dg). A correspondence may be given as a Pair; keywords are forwarded to the constraint.

GTSAM's noiseModel::Gaussian::mahalanobisDistance is the same (not squared); its squaredMahalanobisDistance is the square.

source
VisualGeometryToolkitCore.observation_gradient_jacobian — Function
observation_gradient_jacobian(M, x...; kw...) -> SMatrix{dg·nx, nθ}

The derivative of a constraint's observation gradient G_x (constraint_with_jacobians's second return, dg × nx) along the model coordinates of its G_θ: ∂vec(G_x)/∂θ (column-major vec; nx × nθ for a scalar constraint), the mixed second derivative ∂²g/∂x∂θ. It is what the Sampson residual's model Jacobian adds to G_θ: the normalizer L (LLᵀ = G_x·G_xᵀ) moves with the model (sampson_residual_with_jacobian). A model implements it beside its constraint — FundMat and EssentialMat on image-plane points and EssentialMat on bearings (with M₁, M₂), an EpipolarForm (its matrix's), HomographyMat, a TransferForm, Circle, RadialCamera, Conic, Ellipse. GTSAM and Ceres have no counterpart: they differentiate an explicit residual, not an implicit constraint's gradient; it is the term the FNS-type exact Sampson Jacobians (Chojnacki et al.) carry.

source
VisualGeometryToolkitCore.sampson_distance — Method
sampson_distance(M, x...; kw...) -> T

The Sampson distance of the observations x under the model M: the norm of its sampson_residual, the first-order geometric distance to the model's variety in the observations' own units — for the epipolar constraint u₂ᵀ F u₁ = 0 the familiar |u₂ᵀ F u₁| / sqrt((Fu₁)₁² + (Fu₁)₂² + (Fᵀu₂)₁² + (Fᵀu₂)₂²), for an ellipse the distance to its boundary to first order. Any model with a constraint_with_jacobians has one: FundMat, EssentialMat (pixel plane, or bearings with M₁, M₂), HomographyMat, an EpipolarForm, a TransferForm, a Conic, an Ellipse, a Circle, a Line2D. A correspondence may be given as a Pair.

Geometric only: an observation's covariance does not enter. The noise-whitened (Mahalanobis) distance is mahalanobis_distance. Degenerate observations follow sampson_residual_with_normal's rule (0 or Inf).

NOT SQUARED: this is ‖δ‖, the length of the first-order correction. Hartley & Zisserman's "Sampson error" and OpenCV's cv::sampsonDistance are its square, ‖δ‖² = gᵀ(G_x G_xᵀ)⁻¹g.

source
VisualGeometryToolkitCore.sampson_distances! — Method
sampson_distances!(R::AbstractMatrix, models, xs) -> R

The length(xs) × length(models) matrix of sampson_distances, model j in column j, cast to R's element type at storage: a pool's residual matrix (Float32). models is iterated once and has a length; a model's constraint_keywords columns (M₁, M₂ for an essential matrix on bearings) are read from xs. The distance of each element through the one path — one KernelAbstractions kernel per model column on the backend that owns R, the CPU's included; a buffer of another size is a DimensionMismatch.

source
VisualGeometryToolkitCore.sampson_jacobian_rows! — Method
sampson_jacobian_rows!(J, M, directions, xs) -> J

The model rows of sampson_residuals!: element i's EXACT derivative along the chart directions (an nθ × N matrix, retract_jacobian, or the models dM = (∂M/∂θ₁, …) of retract_directions) — sampson_residual_with_jacobian's — in rows dg(i−1)+1 … dg·i. GEOMETRIC and UNWEIGHTED, as its residuals: neither a noise nor a sqrt_weights column is read. The whitened rows are whitened_jacobian_rows!, the √w-weighted ones weighted_whitened_jacobian_rows!. A degenerate element has zero rows. One body: one KernelAbstractions kernel on the backend that owns J, the CPU's included.

source
VisualGeometryToolkitCore.sampson_residual_with_jacobian — Method
sampson_residual_with_jacobian(M, x...; directions = nothing, kw...)
    -> (s_N::SVector{dg}, J::SMatrix{dg, N})

The Sampson residual (sampson_residual) and its EXACT derivative along the model: with g, G_x, G_θ from constraint_with_jacobians and ∂G_x/∂θ from observation_gradient_jacobian, s_N = L⁻¹g (LLᵀ = G_x·G_xᵀ) differentiates as

∂s_N/∂θ_q = L⁻¹·∂g/∂θ_q − Φ(L⁻¹·∂P/∂θ_q·L⁻ᵀ)·s_N,   ∂P/∂θ_q = 2 sym(∂G_x/∂θ_q·G_xᵀ),

Φ the lower triangle with its diagonal halved (the Cholesky factor's derivative) — for a scalar constraint (G_θ − (g/‖G_x‖²)·(∂G_x/∂θ)ᵀG_x)/‖G_x‖. The normalizer moves with the model and its derivative is carried, so a Gauss–Newton or LM step descends exactly the residual that is scored (a whitening frozen at an anchor descends a different function away from it). Carried along the chart directions (nθ × N, e.g. retract_jacobian(ch, a), or the models dM = (∂M/∂θ₁, …) of retract_directions; nothing keeps the model's own coordinates, N = nθ). A degenerate observation (the rule of sampson_residual_with_normal) has ZERO rows: a least-squares row cannot carry ±Inf. Keywords are forwarded to the constraint (M₁, M₂ for an EssentialMat on bearings). Element-type generic and device-safe; sampson_jacobian_rows! runs it over an observation set.

Ceres and GTSAM differentiate an explicit residual (CostFunction::Evaluate, a factor's unwhitenedError Jacobians); this is the Jacobian of the Sampson residual of an implicit constraint, as Chojnacki et al.'s FNS differentiate the Sampson cost, which neither library forms.

source
VisualGeometryToolkitCore.sampson_residual_with_normal — Method
sampson_residual_with_normal(M, x...; kw...) -> (s_N::SVector{dg}, N::SMatrix{nx, dg})

The Sampson residual of the observations x under the model M and the orthonormal basis of the constraint's normal space it is expressed in: with g, G_x from constraint_with_jacobians(M, x...),

s_N = L⁻¹g,   N = G_xᵀL⁻ᵀ,   LLᵀ = G_x·G_xᵀ,

the first-order displacement onto the model's variety (‖s_N‖ is the sampson_distance), and N the directions it is measured along (NᵀN = I; one column for a scalar constraint, dg for a vector one). Always an SVector{dg}, dg the constraint's component count (1 for F, E, a conic, an ellipse, a circle or a line; 2 for a homography). The noise layer, whiten(s_N, N, Σ), takes both.

GEOMETRIC: a covariance attached to an observation (Uncertain, exact) is ignored here; the observation's point is used. A correspondence may be given as a Pair. Keywords are forwarded to the constraint (an EssentialMat on bearings takes its backprojection Jacobians M₁, M₂).

DEGENERATE (the observation gradient vanishes to rounding, at the model's constraint_rounding_scale — both points at the epipoles, a point a homography sends to the line at infinity, an ellipse's centre): each component is 0 when the constraint vanishes with it and ±Inf (the sign of g) when it does not, and N is zero. NaN propagates.

source
VisualGeometryToolkitCore.sampson_residuals! — Method
sampson_residuals!(r, M, xs) -> r

The Sampson residuals of the model M over the observation set xs (a PointSet or vector of points, a Correspondences or vector of Pairs): element i's sampson_residual in rows dg(i−1)+1 … dg·i of r. GEOMETRIC: a noise column is not read. UNWEIGHTED, like whitened_residuals!: a sqrt_weights column is not read either; the √w-weighted least-squares rows are weighted_whitened_residuals! (which, on a set without noise columns, are these rows times √wᵢ). The constraint's keyword columns (constraint_keywords: M₁, M₂ on bearings) are read per element. A degenerate element is ZERO (the row an LM can take; its distance is ±Inf under the rule). One body: one KernelAbstractions kernel on the backend that owns r, the CPU's included. Reductions compose: the residual sum of squares is sum(abs2, r). The noise- whitened rows are whitened_residuals!; the model rows sampson_jacobian_rows!.

source
VisualGeometryToolkitCore.whiten — Method
whiten(s_N, N, Σ) -> SVector{dg}

The noise layer: the Sampson residual s_N in the normal basis N (sampson_residual_with_normal) whitened by the observation covariance Σ (over the constraint's observations, nx × nx),

r = L_N⁻¹s_N,   L_N L_Nᵀ = NᵀΣN,

the first-order Mahalanobis residual — equal to the constraint whitened by its propagated covariance, L⁻¹g with LLᵀ = G_x·Σ·G_xᵀ. Σ = I returns s_N unchanged; r² is χ²(dg) under Gaussian noise of covariance Σ.

DEGENERATE: NᵀΣN losing rank to rounding relative to its diagonal (a observation exact along the normal) leaves the residual's zero components 0 and its others ±Inf; a degenerate Sampson residual (zero N) is returned unchanged. NaN propagates.

The least-squares rows of the whitened residual, exact (the factor differentiated with the model), are whitened_jacobian_rows!'s. GTSAM's noiseModel::Gaussian::whiten(v) is the same operation on an explicit residual (R·v, RᵀR = Σ⁻¹); here the residual is the Sampson residual of an implicit constraint and the noise is the observations' along its normal, so Σ enters through NᵀΣN.

source
VisualGeometryToolkitCore.whitened_residuals! — Method
whitened_residuals!(r, M, xs) -> r

sampson_residuals! whitened by the observations' noise columns (Σ of a point set, Σ₁/Σ₂ of a correspondence set — conventions/ observation-noise.md): element i's constraint L⁻¹gᵢ, LLᵀ = G_x·Σᵢ·G_xᵀ, in σ units (‖rᵢ‖² ~ χ²(dg); an exact source with a plain target is the transfer error in the target's units). UNWEIGHTED: a sqrt_weights column is not read — the same currency as the problem-level whitened_residuals!(r, problem, model); the √w-weighted rows a least-squares solver takes are weighted_whitened_residuals!. Without noise columns it is sampson_residuals! (both unweighted). GTSAM's NoiseModelFactor::whitenedError is this per factor (and unwhitenedError the constraint); a degenerate element is zero.

source

Observation Noise

A plain point is unit isotropic noise, an Uncertain point carries its covariance, an exact point has none (the Conventions page, "Observation noise").

VisualGeometryToolkitCore.exact — Method
exact(p::StaticVector) -> Uncertain

The observation p known EXACTLY: p with a ZERO covariance, the third case of the observation-noise rule (a plain point is unit isotropic noise, an Uncertain carries its own covariance). For data with no observation error in its frame — a calibration board's points in metres, a world point. Board → image correspondences built as exact.(board) .=> pixels score the transfer error in the image's pixels, whatever the board's units. A physical p (lengths) stays in metres, and its zero covariance is in m².

A correspondence needs one noisy endpoint: an estimator refuses a correspondence whose two endpoints are both exact (nothing to whiten).

The zero covariance is an ordinary CovMat — no second representation. It is read once, where an estimator's constructor turns endpoints into its covariance columns (noise_columns); the per-point matrices cost 4 numbers each at that boundary and nothing in the inner loop, where the columns are what is read.

source
VisualGeometryToolkitCore.noise_columns — Method
noise_columns(sources, targets) -> NamedTuple

The covariance columns of a correspondence set whose endpoints may be Uncertain (or exact): (; Σ₁, Σ₂), each a vector of static 2×2 covariances in the endpoints' own frame, with a side whose endpoints are all plain LEFT OUT (an absent column is unit isotropic noise) — so a plain set gives (;) and its estimator runs exactly as without. Merge them into the geometry-only set with withattrs(cs; noise_columns(s, t)...); the endpoints themselves are geom of these.

Throws ArgumentError for a correspondence whose two endpoints are both exact, or a covariance that is not finite.

source
VisualGeometryToolkitCore.pair_covariance — Method
pair_covariance(Σ₁, Σ₂)
pair_covariance(cs, i)

The 4×4 observation covariance of a correspondence over (x₁, y₁, x₂, y₂): the block diagonal of its endpoints' 2×2 covariances, I (the UniformScaling) when both are unit isotropic — the shape whiten and whitening_factor take as Σ_x. With a correspondence set cs, the endpoints' covariances are its Σ₁/Σ₂ columns (noise_columns), an absent column unit isotropic; a plain set returns I for every i.

source

Lie-Group Helpers and Rotation Fitting

VisualGeometryToolkitCore.SO3 — Type
SO3

The rotation group, as a type tag — the group a Lie-group helper is being asked about, in the slot a type argument takes (left_jacobian(SO3, ω)). Carries no data. Public, not exported.

source
VisualGeometryToolkitCore.exp_se3 — Method
exp_se3(ω::SVector{3}, ρ::SVector{3}) -> EuclideanMap

SE(3) exponential of the twist (ω, ρ): (exp_so3(ω), left_jacobian(SO3, ω)·ρ) as an EuclideanMap (constructed with check = false; the Rodrigues result is orthonormal to floating-point precision by construction).

GTSAM's Pose3::Expmap (log_se3 is Pose3::Logmap), Sophus' SE3::exp, Manifolds.jl's exp_lie. Note the twist ORDER differs between libraries — ours is (ω, ρ), rotation first.

Generic over Real: the two parts are promoted to one float type, so ForwardDiff Duals flow through it (a pose retraction exp_se3(ω, ρ) ∘ T differentiated by an NLLS solver), as through exp_so3 and left_jacobian.

source
VisualGeometryToolkitCore.exp_so3 — Method
exp_so3(ω::SVector{3,T}) -> SMatrix{3,3,T,9}

Rodrigues exponential map so(3) → SO(3): R = I + a·[ω]× + b·[ω]ײ with the second-order Taylor branch below the per-type small-angle threshold.

Plain SMatrix arithmetic — safe inside Zygote-tracked code (unlike RotMatrix(RotationVec(ω...)), which reverse-mode AD cannot penetrate) and generic over Real, so ForwardDiff Duals flow through it too. See also skew, left_jacobian, exp_se3.

GTSAM spells this Rot3::Expmap (and log_so3 is Rot3::Logmap), Sophus SO3::exp / SO3::log. Manifolds.jl's name is exp_lie(G, X) / log_lie: its exp(M, p, X) is the exponential at a BASE POINT, while this is the Lie-group exponential at the identity — which is why the group is not a first argument here.

source
VisualGeometryToolkitCore.left_jacobian — Method
left_jacobian(SO3, ω::SVector{3,T}) -> SMatrix{3,3,T,9}

Left Jacobian of SO(3): J = I + a·[ω]× + b·[ω]ײ with a = (1−cosθ)/θ², b = (θ−sinθ)/θ³. Maps a rotation-vector perturbation to the translation part of the SE(3) exponential: t = J(ω)·ρ. Not provided by Rotations.jl. Below θ = 1/4, where the closed forms cancel, a and b are their six-term Taylor series (within 4 eps of the exact value); above it the closed forms.

This is Sophus' SO3::leftJacobian. GTSAM's Rot3::ExpmapDerivative is its TRANSPOSE, the RIGHT Jacobian Jr(ω) = Jl(ω)ᵀ = Jl(−ω), because GTSAM retracts by right multiplication — do not equate the two. Measured here (2026-09-22): the left perturbation identity exp(ω+δ) = exp(Jl(ω)·δ)·exp(ω) holds to 5.8e-13 while the right-multiplied one is off by 1.8e-6, and Jl(ω) = Jl(−ω)ᵀ to 0.0.

source
VisualGeometryToolkitCore.log_se3 — Method
log_se3(E::EuclideanMap) -> (ω, ρ)

SE(3) logarithm of a rigid transform: ω = log_so3(R), ρ = left_jacobian(SO3, ω) \ t. Inverse of exp_se3. Element type follows the map's rotation eltype. NOT AD-safe under Zygote (see log_so3).

source
VisualGeometryToolkitCore.log_so3 — Method
log_so3(R::StaticMatrix{3,3,T}) -> SVector{3,T}

Rotation vector (axis · angle, angle ∈ [0, π]) of R — the SO(3) logarithm: the unit quaternion (w, v) of R by Shepperd's method (the largest of w, x, y, z computed from the diagonal, so every branch, the near-π one included, divides by a number ≥ ½), sign fixed to w ≥ 0, then 2·atan(‖v‖, w)·v/‖v‖ — with its Taylor series (2/w)(1 − ‖v‖²/(3w²))·v below the per-type small-angle threshold, so the derivative at the identity is exact. Generic over Real: ForwardDiff Duals flow through (within 2.5 ulp of Rotations' RotationVec, measured 2026-09-22).

source
VisualGeometryToolkitCore.reproject_so3 — Method
reproject_so3(R::AbstractMatrix) -> SMatrix{3, 3}

The rotation a drifted rotation matrix R stands for, by its unit quaternion (Rotations' QuatRotation of it, back to a matrix), in R's element type: the ONE re-anchoring policy of every chart with an SO(3) part (recenter of the rigid and essential charts), and the one a caller applies to a rotation that walked through many retractions or was read from a file before building an EuclideanMap — whose rotation check refuses a matrix past √eps off orthonormal. On an orthonormal R it changes it by rounding only. Retractions do not re-project; only re-anchoring does.

source
VisualGeometryToolkitCore.chordal_mean — Method
chordal_mean(Rs) -> RotMatrix{3, T}

Chordal L2 mean of a set of rotations: the rotation minimizing Σᵢ ‖R − Rᵢ‖²_F, i.e. Rotations.nearest_rotation of the elementwise sum. Suited to averaging rotations within a moderate spread (the chordal mean and the geodesic mean agree to second order); it is NOT a robust estimator.

Throws ArgumentError on an empty collection.

source
VisualGeometryToolkitCore.kabsch — Method
kabsch(cs) -> RotMatrix{3, T}

Kabsch/Wahba: the rotation minimizing Σᵢ ‖bᵢ − R·aᵢ‖² over the vector pairs cs (aᵢ => bᵢ, StaticVector{3} each side; any sources/targets container, a Correspondences preferred): one closed-form step, Rotations.nearest_rotation of the cross-covariance Σ bᵢaᵢᵀ (an SVD with the determinant-sign correction). A weights attribute column (withattrs(cs; weights = w), data-arguments.md) weighs the pairs, Σᵢ wᵢ‖bᵢ − R·aᵢ‖² and the cross-covariance Σ wᵢbᵢaᵢᵀ (Wahba's problem): fixed, finite, non-negative weights, one per pair (an ArgumentError otherwise); a zero weight drops its pair. The vectors are used as given, NOT centred: centre them at the call site when aligning point clouds rather than directions.

Physical pairs (lengths on both sides) are stripped in metres at entry; the rotation is unit-free, so it comes back the same either way.

Throws ArgumentError on fewer than 2 pairs, or fewer than 2 of positive weight (a single pair leaves a one-parameter family of solutions).

source
VisualGeometryToolkitCore.mirror — Method
mirror(pose, plane_point; normal = SVector(0., 0., 1.)) -> EuclideanMap

The second branch of the planar two-fold pose ambiguity: for a PLANAR target (world-plane normal normal, any point plane_point on it — conventionally the target center), the pose whose plane normal is reflected about the viewing ray to plane_point. Under weak perspective this mirror pose predicts image positions that agree with pose to second order, so the two tie on reprojection consensus; the rotation between them is twice the viewing tilt. Total: at a frontal view the reflection is (numerically) the identity and the returned pose coincides with pose — gate on rotation_angle(RotMatrix{3}(mirror.R' * pose.R)) if the degenerate case must be skipped.

Translation is carried over unchanged (same center ray and apparent size ⇒ same depth to first order); the result is a first-order INIT for the second basin, to be refined before any comparison.

source
VisualGeometryToolkitCore.spectral_rotation_averaging — Method
spectral_rotation_averaging(relative) -> Dict{K, RotMatrix{3, Float64}}

Spectral chordal rotation averaging (Singer 2011; Arie-Nachimson et al. 2012): the absolute rotations of a set of frames from their relative rotations, relative an AbstractDict keyed by frame pairs (i, j) whose value R_ij takes frame i's directions into frame j's (f_j = R_ij·f_i; with d = R_k·f_k a direction in the common frame, R_i = R_j·R_ij). The 3n × 3n symmetric block matrix with blocks w_ij·R_ij at (j, i) and its transpose at (i, j), identity diagonal blocks, normalized by the frames' weighted degrees (D^{-1/2} A D^{-1/2}), has the stacked R_kᵀ in its three leading eigenvectors; each 3 × 3 block of them is projected onto SO(3) (Rotations.nearest_rotation, the eigenvectors' sign chosen so most blocks are proper), and the result is gauged so the first of the sorted frames is the identity (the common frame is that frame's). Symmetric in the frames, exact on consistent relative rotations, one eigendecomposition — the seed of a robust rotation averaging, not a robust estimator. A pair carries its non-negative weight as Attributed(R_ij, (; weight)), read with weight_of; a bare R_ij weighs 1. A zero-weight pair contributes nothing. The pairs of positive weight must connect every frame: the caller checks it on its graph (VisualGeometryToolkitEstimation's fit_rotation_averaging refuses a split one through its ViewGraph); on a split graph the leading eigenspace is degenerate and the rotations returned are meaningless (a chain with one zero-weight bridge came back 180° off).

source

Index