Epipolar Forms API

The slot-labeled epipolar forms: which form a model is in, and how forms compose; see the Epipolar Forms guide. The solvers that estimate them are in the Solvers API.

VisualGeometryToolkitCore.EpipolarForm — Type
EpipolarForm{S1<:Frame{2}, S2<:Frame{2}, M<:AbstractFundMat}

Slot-labeled epipolar form: a bilinear form with two contravariant point arguments, wrapping a frame-free FundMat or EssentialMat.

Slot convention (pinned here; every other method defers to this): slot 1 takes points of frame S1 (view 1), slot 2 takes points of frame S2 (view 2), and the wrapped matrix M satisfies

x₂ᵀ M x₁ = 0

for corresponding x₁ ∈ S1, x₂ ∈ S2 (homogeneous). Slot 1 is the RIGHT argument of the matrix, matching the u₂ᵀ F u₁ convention used throughout the solver layer. In slot-monogram terms the form is M_{S2,S1} — first index left slot, second index right slot.

The frame tags label the slots of a form, not the endpoints of a map: an EpipolarForm is not a Transformation, does not compose with ∘, and has no inv (deliberately — see the note below). Its point ↦ line action is epipolar_map, and a frame change is composition on that map: EpipolarForm(H₂ ∘ epipolar_map(f) ∘ inv(H₁)) (see ∘ on EpipolarMap).

Construction is the labeled Mat → form lift; unlabeled lifts are allowed for gradual adoption (both slots UnlabeledFrame{2}):

form = EpipolarForm{ImageFrame{:a}, ImageFrame{:b}}(F)   # labeled lift
form = EpipolarForm(F)                                 # unlabeled lift

Evaluating the form applies the slot convention operationally: form(x₁, x₂) returns the scalar x̃₂ᵀ M x̃₁ (zero for an exact correspondence). The wrapper does NOT enforce the rank-2 (or E-manifold) constraint — that discipline belongs to the wrapped Mat type and the solvers that produce it.

Why there is no inv

The epipolar correlation is rank 2: its kernel is the epipole e₁ (M e₁ = 0), every point on one epipolar line maps to the SAME line in the other view, and the image is not all lines but the 1-parameter pencil through the other epipole e₂. A 2-parameter point domain collapsing onto a 1-parameter family of lines has no inverse — inv(::EpipolarForm) is deliberately left undefined (a MethodError), and transpose (swap the slots) is the only involution the form supports. Use epipoles to make the degeneracy explicit.

See also epipolar_map, EpipolarMap, epipoles, sampson_distance, ImageFrame.

source
VisualGeometryToolkitCore.EpipolarForm — Method
(f::EpipolarForm)(x₁, x₂) -> scalar

Evaluate the bilinear form on a point per slot: x̃₂ᵀ M x̃₁ with x̃ = [x; 1]. Zero (up to noise) iff (x₁, x₂) satisfy the epipolar constraint. x₁ feeds slot 1 (frame S1), x₂ slot 2 (frame S2) — the convention pinned at EpipolarForm.

source
VisualGeometryToolkitCore.EpipolarMap — Type
EpipolarMap{S1, S2, M}

Typed callable returned by epipolar_map: the rank-2 correlation of an EpipolarForm, taking a point in frame S1 to its epipolar line in frame S2; EpipolarMap(F) / EpipolarMap{S1, S2}(F) lift a Mat directly. Not a Transformation — it has no inverse (see the rank-2 note at EpipolarForm) — but it composes with collineations on either side (∘ below), which is how it changes frame.

source
VisualGeometryToolkitCore.HomographyMap — Method
HomographyMap(f::TransferForm{Src, Dst}) -> HomographyMap{Src, Dst}
TransferForm(h::HomographyMap{3, T, M, Src, Dst}) -> TransferForm{Src, Dst}

A transfer form's tags are endpoints, so its homography IS a framed HomographyMap with the same two frames, and its frame change is that map's composition (frame-checked by the map's own junction rule): with D₁, D₂ carrying the current endpoints into the new ones,

TransferForm(D₂ ∘ HomographyMap(f) ∘ inv(D₁))   has   H′ = D₂ · H · D₁⁻¹.

An estimator that solves in a model frame holds OBSERVATION maps Cᵢ carrying data-frame points into the model frame (x̃ᵢ = Cᵢ xᵢ; for calibrated homography Cᵢ = Kᵢ⁻¹), so moving the solved model back to the data frame is inv(C₂) ∘ HomographyMap(f) ∘ C₁. inv on a HomographyMap is an O(1) swap of its two stored matrices.

A residual is EQUIVARIANT, not invariant: g = π(H x̃₁) − x₂ is a displacement in the destination image, so a frame change moves it by that frame's local Jacobian — exactly (not to first order) for an affine D₂, whose Jacobian is constant. What IS frame-independent is the ZERO SET (an exact correspondence stays exact in every frame). Measured over 2000 random draws with κ(H), κ(D₁), κ(D₂) ≤ 100 (pinned, seed and all, in test/transfer_composition_tests.jl), as a relative error against max(|value|, 1):

quantityrelation under the compositionmeasured max
transferred point π(H x̃₁)commutes: π(H′ b̃₁) = D₂(π(H ã₁))8.2e-14
algebraic c = x̃₂ × H x̃₁c′ = (λ₁λ₂ det D₂)·D₂⁻ᵀ c3.5e-14
transfer residual g = π(H x̃₁) − x₂, affine dstg′ = A₂ g (A₂ = D₂'s linear part)3.2e-13

(The same grid puts the WRONG direction — the observation maps composed uninverted — no closer than 4.5e-2 on the first row.)

Public, not exported (as TransferForm is).

source
VisualGeometryToolkitCore.TransferForm — Type
TransferForm{Src, Dst}(H::HomographyMat)
TransferForm(H::HomographyMat)

The TRANSFER form of a homography constraint: the residual is measured in the image, g = π(H·ũ₁) − u₂, as against the ALGEBRAIC (cross-multiplied) form that constraint_with_jacobians(H, u₁, u₂) returns and the DLT minimizes.

The two tags are ENDPOINTS, not slots (see the note above): Src is u₁'s frame, Dst is the frame of u₂ and of the residual. The bare constructor lifts to UnlabeledFrame{2} endpoints, the gradual-typing default used by the Maps and by EpipolarForm.

Public, not exported.

source
VisualGeometryToolkitCore.compose_jacobian — Method
compose_jacobian(outer, m, inner) -> SMatrix{9, 9}

The derivative of the composition outer ∘ m ∘ inner in the model's matrix, ∂vec(M′)/∂vec(M) (column-major vec), for m an EpipolarMap or a HomographyMap{3} and outer/inner collineations (a HomographyMap{3} or a CoordinateTransformations affine). The composition is linear in M, so this is the constant Kronecker factor of its law (_conjugation_jacobian, the one ∂vec(A·M·B)/∂vec(M)), B = inner's matrix and A = outer's:

  • correlation, M′ = A⁻ᵀ·M·B: kron(Bᵀ, A⁻ᵀ);
  • collineation, H′ = A·H·B: kron(Bᵀ, A).

What a constraint evaluated on the composed model needs to pull its model gradient back into the model's own coordinates: ∂g/∂vec(M) = Jᵀ·∂g/∂vec(M′) for a scalar constraint's 9-vector, G_θ·J for a 2×9. Frame-checked like ∘; in the promoted element type of the model and the maps.

source
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(f::EpipolarForm, x₁, x₂) -> (g, G_x, G_θ)

The epipolar constraint of the wrapped Mat, with the slot convention of EpipolarForm — x₁ feeds slot 1 (frame S1), x₂ slot 2 (frame S2). So a form has everything a constraint gives: sampson_distance, sampson_residual, mahalanobis_distance.

The delegation is CHECKED where a frame is available: an Uncertain observation whose CovMat/PrecMat carries a frame label must meet the slot it feeds (standing wildcard truth table — a labeled mismatch is a MethodError; plain points and unlabeled matrices are the wildcard), before the Sampson layer strips the covariance.

source
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(f::TransferForm, u₁, u₂) -> (g, G_x, G_θ)

The transfer constraint of the wrapped homography and its Jacobians: g = π(H·ũ₁) − u₂, the transfer error in the destination image, with G_x = [Jπ·H₁₂ −I₂] (2×4, over (x₁, y₁, x₂, y₂)) and G_θ = Jπ·(ũ₁ᵀ ⊗ I₃) (2×9, ∂g/∂vec(H) column-major), Jπ the perspective derivative at H·ũ₁ and H₁₂ the first two columns of H.

The ALGEBRAIC counterpart is constraint_with_jacobians(H, u₁, u₂) on the bare Mat — same verb, same return shape, different residual. Which one you get is decided by whether the model is wrapped in a TransferForm, not by a second function name. Generic in the element type; device-safe. Image points only: u₁, u₂ are 2-vectors, and a 3-vector (a bearing) is a MethodError.

source
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(f::TransferForm, f₁, f₂; M₁, M₂) -> (g, G_x, G_θ)

The transfer constraint of the wrapped homography on UNIT BEARINGS — the infinite homography of a calibrated camera (under a pure rotation H = R): g = P·(H f₁/‖H f₁‖), the transferred ray in the tangent plane of f₂ (P the 2×3 orthonormal basis of s2_tangent_basis(f₂)), zero when f₂ ∥ H f₁ — the constraint f₂ × H f₁ = 0 at codimension 2. Its Jacobians:

  • G_x = [D·H·M₁ −P·M₂] (2×4) in PIXEL coordinates, M₁/M₂ the 3×2 back-projection Jacobians of backproject_with_jacobian and D = P·(I − ĥĥᵀ)/‖H f₁‖, the derivative of g along the joint observation (both images' pixels), so that at Σ = I (assumed by the rows; detection covariances are a planned extension) the Sampson denominator G_x G_xᵀ = P(H̄M₁M₁ᵀH̄ᵀ + M₂M₂ᵀ)Pᵀ (H̄ the normalized transfer, R for a rotation) is the Sampson covariance in f₂'s tangent plane. The f₂ block is the first-order derivative at ĥ = f₂ (Sampson's linearization at the observation), exact there.
  • G_θ = D·(f₁ᵀ ⊗ I₃) (2×9, ∂g/∂vec(H) column-major) — the pixel form's _transfer_params_jacobian with D in the place of Jπ.

The pixel sibling constraint_with_jacobians(f, u₁, u₂) is the same constraint in the z = 1 chart. An antipodal pair (ĥ·f₂ < 0) also gives a small g; a scorer must refuse it separately (the hemisphere of the transferred ray).

source
VisualGeometryToolkitCore.epipolar_map — Method
epipolar_map(f::EpipolarForm{S1, S2}) -> EpipolarMap{S1, S2}

The honest epipolar correlation: a callable taking a point x₁ in slot 1's frame S1 and returning its epipolar LINE in slot 2's frame S2,

ℓ₂ = M x̃₁

as a Line2D — VGC's 2D Hyperplane, i.e. the covector of the incidence constraint ℓ₂ᵀ x̃₂ = 0. The returned line is itself a one-slot form (a linear form on S2-frame points), which is why a correlation lands on Line2D and not on a point type: point ↦ line, not point ↦ point.

epipolar_map(transpose(f)) goes the other way (S2 points to S1 lines, via Mᵀ). There is no inverse map — the correlation is rank 2 (see EpipolarForm); the two degenerate directions are exposed by epipoles.

Note

Since 0.8 this name refers ONLY to the form action. The former epipolar_map(source::PosedCamera, target::PosedCamera, extrinsics) — a unit-depth point-TRANSFER closure, not an epipolar correlation — is renamed depth_transfer.

source
VisualGeometryToolkitCore.epipoles — Method
epipoles(f::EpipolarForm{S1, S2}) -> (e₁, e₂)

The two kernels of the rank-2 form, labeled by their frames: e₁ is the epipole in slot 1's frame S1 (right kernel, M e₁ ≈ 0) and e₂ the epipole in slot 2's frame S2 (left kernel, Mᵀ e₂ ≈ 0), returned as a NamedTuple (e₁ = ..., e₂ = ...) of homogeneous SVector{3} (unit norm, from the SVD null vectors — homogeneous because an epipole can lie at infinity; dehomogenize with to_euclidean when e[3] ≠ 0).

These are the degenerate directions that make the correlation one-way: epipolar_map sends e₁ to the zero "line", and every returned line goes through e₂ (the pencil property). Computed by SVD of the wrapped matrix; for a noisy or rank-3 matrix the returned vectors are the nearest-kernel directions.

source
Base.transpose — Method
transpose(f::EpipolarForm{S1, S2}) -> EpipolarForm{S2, S1}

Swap the slots of the form: wraps Mᵀ, so the constraint reads x₁ᵀ Mᵀ x₂ = 0 — the same incidence relation with the argument order exchanged. An involution (transpose(transpose(f)) recovers f), and the ONLY inversion-like operation the form supports — the rank-2 correlation has no inv (see EpipolarForm). The wrapped Mat type is preserved: the transpose of a fundamental matrix is fundamental, and Eᵀ remains on the essential manifold.

adjoint (f') is the same operation (real coefficients).

source

Index