Epipolar Forms API
The slot-labeled epipolar forms: which form a model is in, and how forms compose; see the Epipolar Forms guide. The solvers that estimate them are in the Solvers API.
VisualGeometryToolkitCore.EpipolarForm — Type
EpipolarForm{S1<:Frame{2}, S2<:Frame{2}, M<:AbstractFundMat}Slot-labeled epipolar form: a bilinear form with two contravariant point arguments, wrapping a frame-free FundMat or EssentialMat.
Slot convention (pinned here; every other method defers to this): slot 1 takes points of frame S1 (view 1), slot 2 takes points of frame S2 (view 2), and the wrapped matrix M satisfies
x₂ᵀ M x₁ = 0for corresponding x₁ ∈ S1, x₂ ∈ S2 (homogeneous). Slot 1 is the RIGHT argument of the matrix, matching the u₂ᵀ F u₁ convention used throughout the solver layer. In slot-monogram terms the form is M_{S2,S1} — first index left slot, second index right slot.
The frame tags label the slots of a form, not the endpoints of a map: an EpipolarForm is not a Transformation, does not compose with ∘, and has no inv (deliberately — see the note below). Its point ↦ line action is epipolar_map, and a frame change is composition on that map: EpipolarForm(H₂ ∘ epipolar_map(f) ∘ inv(H₁)) (see ∘ on EpipolarMap).
Construction is the labeled Mat → form lift; unlabeled lifts are allowed for gradual adoption (both slots UnlabeledFrame{2}):
form = EpipolarForm{ImageFrame{:a}, ImageFrame{:b}}(F) # labeled lift
form = EpipolarForm(F) # unlabeled liftEvaluating the form applies the slot convention operationally: form(x₁, x₂) returns the scalar x̃₂ᵀ M x̃₁ (zero for an exact correspondence). The wrapper does NOT enforce the rank-2 (or E-manifold) constraint — that discipline belongs to the wrapped Mat type and the solvers that produce it.
Why there is no inv
The epipolar correlation is rank 2: its kernel is the epipole e₁ (M e₁ = 0), every point on one epipolar line maps to the SAME line in the other view, and the image is not all lines but the 1-parameter pencil through the other epipole e₂. A 2-parameter point domain collapsing onto a 1-parameter family of lines has no inverse — inv(::EpipolarForm) is deliberately left undefined (a MethodError), and transpose (swap the slots) is the only involution the form supports. Use epipoles to make the degeneracy explicit.
See also epipolar_map, EpipolarMap, epipoles, sampson_distance, ImageFrame.
VisualGeometryToolkitCore.EpipolarForm — Method
(f::EpipolarForm)(x₁, x₂) -> scalarEvaluate the bilinear form on a point per slot: x̃₂ᵀ M x̃₁ with x̃ = [x; 1]. Zero (up to noise) iff (x₁, x₂) satisfy the epipolar constraint. x₁ feeds slot 1 (frame S1), x₂ slot 2 (frame S2) — the convention pinned at EpipolarForm.
VisualGeometryToolkitCore.EpipolarMap — Type
EpipolarMap{S1, S2, M}Typed callable returned by epipolar_map: the rank-2 correlation of an EpipolarForm, taking a point in frame S1 to its epipolar line in frame S2; EpipolarMap(F) / EpipolarMap{S1, S2}(F) lift a Mat directly. Not a Transformation — it has no inverse (see the rank-2 note at EpipolarForm) — but it composes with collineations on either side (∘ below), which is how it changes frame.
VisualGeometryToolkitCore.HomographyMap — Method
HomographyMap(f::TransferForm{Src, Dst}) -> HomographyMap{Src, Dst}
TransferForm(h::HomographyMap{3, T, M, Src, Dst}) -> TransferForm{Src, Dst}A transfer form's tags are endpoints, so its homography IS a framed HomographyMap with the same two frames, and its frame change is that map's composition (frame-checked by the map's own junction rule): with D₁, D₂ carrying the current endpoints into the new ones,
TransferForm(D₂ ∘ HomographyMap(f) ∘ inv(D₁)) has H′ = D₂ · H · D₁⁻¹.An estimator that solves in a model frame holds OBSERVATION maps Cᵢ carrying data-frame points into the model frame (x̃ᵢ = Cᵢ xᵢ; for calibrated homography Cᵢ = Kᵢ⁻¹), so moving the solved model back to the data frame is inv(C₂) ∘ HomographyMap(f) ∘ C₁. inv on a HomographyMap is an O(1) swap of its two stored matrices.
A residual is EQUIVARIANT, not invariant: g = π(H x̃₁) − x₂ is a displacement in the destination image, so a frame change moves it by that frame's local Jacobian — exactly (not to first order) for an affine D₂, whose Jacobian is constant. What IS frame-independent is the ZERO SET (an exact correspondence stays exact in every frame). Measured over 2000 random draws with κ(H), κ(D₁), κ(D₂) ≤ 100 (pinned, seed and all, in test/transfer_composition_tests.jl), as a relative error against max(|value|, 1):
| quantity | relation under the composition | measured max |
|---|---|---|
transferred point π(H x̃₁) | commutes: π(H′ b̃₁) = D₂(π(H ã₁)) | 8.2e-14 |
algebraic c = x̃₂ × H x̃₁ | c′ = (λ₁λ₂ det D₂)·D₂⁻ᵀ c | 3.5e-14 |
transfer residual g = π(H x̃₁) − x₂, affine dst | g′ = A₂ g (A₂ = D₂'s linear part) | 3.2e-13 |
(The same grid puts the WRONG direction — the observation maps composed uninverted — no closer than 4.5e-2 on the first row.)
Public, not exported (as TransferForm is).
VisualGeometryToolkitCore.TransferForm — Type
TransferForm{Src, Dst}(H::HomographyMat)
TransferForm(H::HomographyMat)The TRANSFER form of a homography constraint: the residual is measured in the image, g = π(H·ũ₁) − u₂, as against the ALGEBRAIC (cross-multiplied) form that constraint_with_jacobians(H, u₁, u₂) returns and the DLT minimizes.
The two tags are ENDPOINTS, not slots (see the note above): Src is u₁'s frame, Dst is the frame of u₂ and of the residual. The bare constructor lifts to UnlabeledFrame{2} endpoints, the gradual-typing default used by the Maps and by EpipolarForm.
Public, not exported.
VisualGeometryToolkitCore.compose_jacobian — Method
compose_jacobian(outer, m, inner) -> SMatrix{9, 9}The derivative of the composition outer ∘ m ∘ inner in the model's matrix, ∂vec(M′)/∂vec(M) (column-major vec), for m an EpipolarMap or a HomographyMap{3} and outer/inner collineations (a HomographyMap{3} or a CoordinateTransformations affine). The composition is linear in M, so this is the constant Kronecker factor of its law (_conjugation_jacobian, the one ∂vec(A·M·B)/∂vec(M)), B = inner's matrix and A = outer's:
- correlation,
M′ = A⁻ᵀ·M·B:kron(Bᵀ, A⁻ᵀ); - collineation,
H′ = A·H·B:kron(Bᵀ, A).
What a constraint evaluated on the composed model needs to pull its model gradient back into the model's own coordinates: ∂g/∂vec(M) = Jᵀ·∂g/∂vec(M′) for a scalar constraint's 9-vector, G_θ·J for a 2×9. Frame-checked like ∘; in the promoted element type of the model and the maps.
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(f::EpipolarForm, x₁, x₂) -> (g, G_x, G_θ)The epipolar constraint of the wrapped Mat, with the slot convention of EpipolarForm — x₁ feeds slot 1 (frame S1), x₂ slot 2 (frame S2). So a form has everything a constraint gives: sampson_distance, sampson_residual, mahalanobis_distance.
The delegation is CHECKED where a frame is available: an Uncertain observation whose CovMat/PrecMat carries a frame label must meet the slot it feeds (standing wildcard truth table — a labeled mismatch is a MethodError; plain points and unlabeled matrices are the wildcard), before the Sampson layer strips the covariance.
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(f::TransferForm, u₁, u₂) -> (g, G_x, G_θ)The transfer constraint of the wrapped homography and its Jacobians: g = π(H·ũ₁) − u₂, the transfer error in the destination image, with G_x = [Jπ·H₁₂ −I₂] (2×4, over (x₁, y₁, x₂, y₂)) and G_θ = Jπ·(ũ₁ᵀ ⊗ I₃) (2×9, ∂g/∂vec(H) column-major), Jπ the perspective derivative at H·ũ₁ and H₁₂ the first two columns of H.
The ALGEBRAIC counterpart is constraint_with_jacobians(H, u₁, u₂) on the bare Mat — same verb, same return shape, different residual. Which one you get is decided by whether the model is wrapped in a TransferForm, not by a second function name. Generic in the element type; device-safe. Image points only: u₁, u₂ are 2-vectors, and a 3-vector (a bearing) is a MethodError.
VisualGeometryToolkitCore.constraint_with_jacobians — Method
constraint_with_jacobians(f::TransferForm, f₁, f₂; M₁, M₂) -> (g, G_x, G_θ)The transfer constraint of the wrapped homography on UNIT BEARINGS — the infinite homography of a calibrated camera (under a pure rotation H = R): g = P·(H f₁/‖H f₁‖), the transferred ray in the tangent plane of f₂ (P the 2×3 orthonormal basis of s2_tangent_basis(f₂)), zero when f₂ ∥ H f₁ — the constraint f₂ × H f₁ = 0 at codimension 2. Its Jacobians:
G_x = [D·H·M₁ −P·M₂](2×4) in PIXEL coordinates,M₁/M₂the 3×2 back-projection Jacobians ofbackproject_with_jacobianandD = P·(I − ĥĥᵀ)/‖H f₁‖, the derivative ofgalong the joint observation (both images' pixels), so that atΣ = I(assumed by the rows; detection covariances are a planned extension) the Sampson denominatorG_x G_xᵀ = P(H̄M₁M₁ᵀH̄ᵀ + M₂M₂ᵀ)Pᵀ(H̄the normalized transfer,Rfor a rotation) is the Sampson covariance inf₂'s tangent plane. Thef₂block is the first-order derivative atĥ = f₂(Sampson's linearization at the observation), exact there.G_θ = D·(f₁ᵀ ⊗ I₃)(2×9, ∂g/∂vec(H) column-major) — the pixel form's_transfer_params_jacobianwithDin the place ofJπ.
The pixel sibling constraint_with_jacobians(f, u₁, u₂) is the same constraint in the z = 1 chart. An antipodal pair (ĥ·f₂ < 0) also gives a small g; a scorer must refuse it separately (the hemisphere of the transferred ray).
VisualGeometryToolkitCore.epipolar_map — Method
epipolar_map(f::EpipolarForm{S1, S2}) -> EpipolarMap{S1, S2}The honest epipolar correlation: a callable taking a point x₁ in slot 1's frame S1 and returning its epipolar LINE in slot 2's frame S2,
ℓ₂ = M x̃₁as a Line2D — VGC's 2D Hyperplane, i.e. the covector of the incidence constraint ℓ₂ᵀ x̃₂ = 0. The returned line is itself a one-slot form (a linear form on S2-frame points), which is why a correlation lands on Line2D and not on a point type: point ↦ line, not point ↦ point.
epipolar_map(transpose(f)) goes the other way (S2 points to S1 lines, via Mᵀ). There is no inverse map — the correlation is rank 2 (see EpipolarForm); the two degenerate directions are exposed by epipoles.
Since 0.8 this name refers ONLY to the form action. The former epipolar_map(source::PosedCamera, target::PosedCamera, extrinsics) — a unit-depth point-TRANSFER closure, not an epipolar correlation — is renamed depth_transfer.
VisualGeometryToolkitCore.epipoles — Method
epipoles(f::EpipolarForm{S1, S2}) -> (e₁, e₂)The two kernels of the rank-2 form, labeled by their frames: e₁ is the epipole in slot 1's frame S1 (right kernel, M e₁ ≈ 0) and e₂ the epipole in slot 2's frame S2 (left kernel, Mᵀ e₂ ≈ 0), returned as a NamedTuple (e₁ = ..., e₂ = ...) of homogeneous SVector{3} (unit norm, from the SVD null vectors — homogeneous because an epipole can lie at infinity; dehomogenize with to_euclidean when e[3] ≠ 0).
These are the degenerate directions that make the correlation one-way: epipolar_map sends e₁ to the zero "line", and every returned line goes through e₂ (the pencil property). Computed by SVD of the wrapped matrix; for a noisy or rank-3 matrix the returned vectors are the nearest-kernel directions.
Base.transpose — Method
transpose(f::EpipolarForm{S1, S2}) -> EpipolarForm{S2, S1}Swap the slots of the form: wraps Mᵀ, so the constraint reads x₁ᵀ Mᵀ x₂ = 0 — the same incidence relation with the argument order exchanged. An involution (transpose(transpose(f)) recovers f), and the ONLY inversion-like operation the form supports — the rank-2 correlation has no inv (see EpipolarForm). The wrapped Mat type is preserved: the transpose of a fundamental matrix is fundamental, and Eᵀ remains on the essential manifold.
adjoint (f') is the same operation (real coefficients).
Index
VisualGeometryToolkitCore.EpipolarFormVisualGeometryToolkitCore.EpipolarFormVisualGeometryToolkitCore.EpipolarMapVisualGeometryToolkitCore.HomographyMapVisualGeometryToolkitCore.TransferFormBase.transposeVisualGeometryToolkitCore.compose_jacobianVisualGeometryToolkitCore.constraint_with_jacobiansVisualGeometryToolkitCore.constraint_with_jacobiansVisualGeometryToolkitCore.constraint_with_jacobiansVisualGeometryToolkitCore.epipolar_mapVisualGeometryToolkitCore.epipoles