Imaging Chain and Targets API
The framed imaging chain (frame-to-frame stages, ray maps, frame transitions) and calibration targets. See the The Framed Imaging Chain and Ray Maps and Frame Transitions guides; rigs, stereo and captures are on the Rigs, Stereo and Captures API page.
The Framed Imaging Chain
VisualGeometryToolkitCore.Cameras.CameraCalibrationMat — Method
CameraCalibrationMat(m::FramedAffine) -> CameraCalibrationMatThe intrinsics whose calibration stage is m, K = [A t; 0 0 1] — the inverse of calibration_stage. A K solved in the NormalizedImageFrame comes back in pixels as CameraCalibrationMat(inv(norm) ∘ calibration_stage(DistortedFrame{id}, NormalizedImageFrame{id}, LogicalIntrinsics(K_norm))) (framed affines fuse under ∘, the junction checked). The lens coefficients need no conversion: they live in K⁻¹·pixel, which the normalization leaves where it was. Checked as any K is (positive focals, upper-triangular): a rotated stage is an ArgumentError.
VisualGeometryToolkitCore.Cameras.FrameTransition — Type
FrameTransition{Src <: RetinalFrame, Dst <: RetinalFrame}The framed transition map between two retinal frames of the SAME view, {Src → Dst} — a pure radial reparameterization of the viewing sphere, computed once through the sphere as the conjugation
section(Dst) ∘ lift(Src)so every pair among the five frames (GnomonicFrame, EquidistantFrame, StereographicFrame, EquisolidFrame, OrthographicFrame) is served by ONE implementation, with no per-pair closed forms to keep consistent. Azimuth is preserved; the radial profile map is profile_Dst(profile_Src⁻¹(r)).
Mint with frame_transition; inv is the reverse pair (an honest inverse — on the sphere the frame projections are invertible sections). As a FramedStage it composes junction-checked with the rest of the framed surface, and being isbits it composes into a per-point map that pointwise! runs over a batch.
Domain: the intersection of the two frames' honest domains, enforced — points outside Src's frame radius (equisolid r > 2, orthographic r > 1) and rays outside Dst's domain (gnomonic Z ≤ 0, orthographic Z < 0, the stereographic/equidistant antipode) map to NaN at the edge rather than aliasing.
VisualGeometryToolkitCore.Cameras.FramedAffine — Type
FramedAffine{Src <: Frame{2}, Dst <: Frame{2}, T}Framed 2D affine stage x ↦ A·x + t — the calibration stage of the imaging chain, {DistortedFrame{id} → ImageFrame{id}} (or {ray frame → ImageFrame{id}} under the no-distortion frame collapse; the identity-K logical case is just A = I, t = 0). Same arithmetic as the AffineMap in the legacy imaging_transform; the frames are the addition. inv swaps the frames around the inverse affine. Mint with calibration_stage.
VisualGeometryToolkitCore.Cameras.FramedAffine — Method
FramedAffine(ImageFrame{id}, NormalizedImageFrame{id}, sz::Size2) -> FramedAffine
FramedAffine(PixelFrame{O}, NormalizedImageFrame{id}, sz::Size2) -> FramedAffineThe normalization map of an image of size sz (pixels, plain or px): x ↦ (x − c) / s, c the image centre and s = hypot(width, height) / 2 the half-diagonal — DiagonalNormalization, framed — so the image corners land at unit radius. The storage frame of the benchmark pools, where one noise level and one threshold must mean the same across resolutions; estimation normalizes by HartleyNormalization. The centre is written in the source's convention: (width/2, height/2) for an ImageFrame (always read :colmap), moved by EuclideanMap(PixelFrame{:colmap}, PixelFrame{O}) for a PixelFrame{O}, so every convention lands on the same normalized point. inv de-normalizes. The view ids must agree (a MethodError otherwise).
VisualGeometryToolkitCore.Cameras.FramedChain — Type
FramedChain{Src <: Frame, Dst <: Frame, T <: Tuple} <: FramedStage{Src, Dst}
FramedChain(stages::FramedStage...)Composite of framed stages, applied FIRST-TO-LAST (stages[1] innermost): a FramedChain consumes src_frame(stages[1]) coordinates and produces dst_frame(stages[end]) coordinates, and is itself a FramedStage — so chains compose with stages and with each other under the same checked ∘, flattening (no nested chains from composition).
The vararg constructor checks EVERY inner junction under the standing wildcard truth table and throws an ArgumentError naming the first mismatched junction; ∘ between stages performs the same check as a MethodError (the house composition idiom). The callable unrolls at compile time. inv inverts and reverses the stages (frames swap) — and is therefore a MethodError for chains containing a ray projection, whose fiber collapse has no inverse.
VisualGeometryToolkitCore.Cameras.FramedDistort — Type
FramedDistort{Src <: Frame{2}, Dst <: Frame{2}, L <: AbstractLensModel}Framed forward-distortion stage {ray frame → DistortedFrame} wrapping a lens model. The math dispatches on the Src frame KIND:
Src <: GnomonicFrame: the wrapped lens callable itself — the lens contract already IS a map on the gnomonic frame.Src <: EquidistantFrame: the model's radial map in θ (KannalaBrandt's polynomial; COLMAP's thin-prism fisheye's additive terms evaluated on equidistant-frame coordinates, COLMAP's convention).
Mint with distort_stage (which tags the honest frame via distortion_frame); inv yields the matching FramedUndistort with the frames swapped. Direct type-parameter construction is an unchecked expert surface, like relabel.
VisualGeometryToolkitCore.Cameras.FramedEquidistantProjection — Type
The framed EQUIDISTANT-AZIMUTHAL ray-frame projection θ·(cos ψ, sin ψ) onto EquidistantFrame, defined past 90° too: an alias of FramedProjection.
VisualGeometryToolkitCore.Cameras.FramedEquisolidProjection — Type
The framed EQUISOLID (equal-area) ray-frame projection 2·sin(θ/2)·(cos ψ, sin ψ) onto EquisolidFrame, the whole sphere: an alias of FramedProjection.
VisualGeometryToolkitCore.Cameras.FramedLensProjection — Type
FramedLensProjection{Src <: Frame{3}, Dst <: Frame{2}, L <: AbstractLensModel}
FramedLensProjection(lens; id::Symbol = :_)The fused MODEL-CARRYING ray projection of a lens whose projection has no fixed frame (a DoubleSphere, Unified or ExtendedUnified projection): {ViewFrame{id} → DistortedFrame{id}} in ONE stage, project(lens, X) on ray representatives — the lens's whole valid field of view, Z ≤ 0 rays included, NaN outside it. imaging_chain composes it with the framed K-affine:
ViewFrame{id} ─ FramedLensProjection(lens) → DistortedFrame{id}
─ calibration_stage(K) → ImageFrame{id}Like the other ray projections it is a fiber collapse — no inv (a MethodError by design); the closed-form inverse is backproject.
VisualGeometryToolkitCore.Cameras.FramedOrthographicProjection — Type
The framed ORTHOGRAPHIC (sine-law) ray-frame projection sin θ·(cos ψ, sin ψ) onto OrthographicFrame, NaN past the fold at 90°: an alias of FramedProjection.
VisualGeometryToolkitCore.Cameras.FramedPerspective — Type
The framed GNOMONIC ray-frame projection (X, Y, Z) ↦ (X/Z, Y/Z) onto GnomonicFrame: an alias of FramedProjection.
VisualGeometryToolkitCore.Cameras.FramedProjection — Type
FramedProjection{Src <: Frame{3}, Dst <: Frame{2}} <: FramedStage{Src, Dst}The framed ray-frame projection (3D → 2D): the view's unit rays onto the retinal frame Dst, the section of the ray sphere that Dst's family defines (_sphere_section, one definition per family, shared with the lens contract in lens_rays.jl). The stage wears the frames; the math is the lens contract's. One type serves the five geometry-defined families, and the five names below are aliases with the Dst family fixed:
| alias | Dst family | image of a ray at polar angle θ, azimuth ψ |
|---|---|---|
FramedPerspective | GnomonicFrame | tan θ·(cos ψ, sin ψ) — (X/Z, Y/Z); NaN for Z ≤ 0 |
FramedEquidistantProjection | EquidistantFrame | θ·(cos ψ, sin ψ), θ ∈ [0, π) |
FramedStereographicProjection | StereographicFrame | 2·tan(θ/2)·(cos ψ, sin ψ); NaN at the antipode |
FramedEquisolidProjection | EquisolidFrame | 2·sin(θ/2)·(cos ψ, sin ψ), the whole sphere |
FramedOrthographicProjection | OrthographicFrame | sin θ·(cos ψ, sin ψ); NaN for Z < 0 (the frame folds) |
Each alias has the keyword constructor Alias(; id::Symbol = :_), which mints FramedProjection{ViewFrame{id}, Family{id}}() (anonymous view :_ by default — see DistortedFrame); Alias{Src, Dst}() and x isa Alias work as they would on a type of its own. The azimuth is ψ = 0 at the on-axis poles (X = Y = 0). A ray outside a frame's honest domain is answered NaN, never aliased onto an in-frame point.
No inv — deliberate
A projection collapses each ray (a 1-parameter fiber) to a point: a 3D → 2D fiber collapse has no inverse, so inv is absent by design (a MethodError, pinned by test; the explicit throw is needed because CoordinateTransformations defines a generic inv(::Transformation) fallback, an ErrorException, which would misreport the ABSENCE as a missing definition). The right-inverses — the SECTIONS of the projection, each selecting one point per fiber via a depth — are unproject (pixel + depth → 3D point) and backproject (pixel → unit ray, the norm-one section); use those to go back.
Serves the lenses by distortion_frame(lens): the perspective-plane lenses use the gnomonic alias, the omnidirectional ones the others.
VisualGeometryToolkitCore.Cameras.FramedStage — Type
FramedStage{Src <: Frame, Dst <: Frame} <: TransformationAbstract supertype of the framed imaging-chain stages: a map consuming Src-frame coordinates and producing Dst-frame coordinates (same tag semantics as EuclideanMap/HomographyMap). Concrete stages: FramedPerspective, FramedEquidistantProjection, FramedStereographicProjection, FramedEquisolidProjection, FramedOrthographicProjection (3D → 2D ray projections), FramedLensProjection (the fused model-carrying 3D → 2D projection of a lens with a parametric projection), FramedDistort, FramedUndistort, FramedAffine, FrameTransition (2D → 2D), the sphere self-maps RayMap/InverseRayMap (3D → 3D), and the composite FramedChain.
∘ between framed stages is junction-checked under the standing wildcard truth table (see Base.:∘ at EuclideanMap) and produces a flat FramedChain; a labeled junction mismatch is a MethodError. Composing with a foreign (unframed) Transformation falls through to CoordinateTransformations' lazy composition — gradual, as always.
VisualGeometryToolkitCore.Cameras.FramedStereographicProjection — Type
The framed STEREOGRAPHIC ray-frame projection 2·tan(θ/2)·(cos ψ, sin ψ) onto StereographicFrame, the conformal fisheye: an alias of FramedProjection.
VisualGeometryToolkitCore.Cameras.FramedUndistort — Type
FramedUndistort{Src <: Frame{2}, Dst <: Frame{2}, L <: AbstractLensModel}Framed inverse-distortion stage {DistortedFrame → ray frame} — the frames-swapped inverse of FramedDistort, minted by undistort_stage or inv. Math dispatches on the Dst frame kind: undistort_normalized(lens, ·) toward a GnomonicFrame; the closed-form/Newton θ-inverse toward an EquidistantFrame.
Feeding stages the wrong way around — e.g. composing an undistort stage directly after another undistort stage, or after the ray projection (which lands on the ray frame, not the distorted frame) — is a junction mismatch and throws a MethodError: distort-vs-undistort DIRECTION is what the frame tags check, view identity or not.
VisualGeometryToolkitCore.Cameras.InverseRayMap — Type
InverseRayMap{F <: Frame{3}, C <: RetinalFrame, L <: AbstractLensModel}The inverse sphere self-map D⁻¹: S² → S² — the undistort conjugation lift ∘ undistort ∘ section, minted by inv of a RayMap (and inverting back). Same frame typing, ray semantics, unit-norm output, and frame-domain behavior as RayMap.
VisualGeometryToolkitCore.Cameras.RayMap — Type
RayMap{F <: Frame{3}, C <: RetinalFrame, L <: AbstractLensModel}The lens's SPHERE SELF-MAP D: S² → S² as a framed stage {ViewFrame{id} → ViewFrame{id}}: the lens's forward distortion free of any image frame, computed as the conjugation
D = lift ∘ distort ∘ sectionwhere section is the projection onto the lens's defining frame (distortion_frame) restricted to the unit sphere and lift is its inverse. C records the defining frame the conjugation runs through; D itself needs no 2D frame tag — its input and output are rays of the view's 3D frame.
Acts on RAYS: any 3D representative may be supplied (the input norm is irrelevant — both frame projections are scale-invariant on ray representatives), and the output is always the UNIT bearing vector of the re-oriented ray.
Restricted to the sphere the frame projections are sections — invertible, unlike the fiber-collapsing full projections FramedPerspective / FramedEquidistantProjection — which is why RayMap has an honest inv (the InverseRayMap undistort conjugation) while the projection stages deliberately have none.
Domain: the conjugation is defined exactly where the defining frame is — the forward hemisphere Z > 0 for gnomonic-frame lenses, θ ∈ [0, π) (all but the exact backward ray) for equidistant-frame lenses. Rays outside the frame's domain map to NaN.
Mint with ray_map; direct type-parameter construction is an unchecked expert surface, like relabel.
VisualGeometryToolkitCore.Cameras.calibration_stage — Method
calibration_stage(intrinsics::AbstractIntrinsics; id::Symbol = :_) -> FramedAffine
calibration_stage(Src::Type{<:Frame{2}}, Dst::Type{<:Frame{2}}, intrinsics)Mint the framed calibration stage of intrinsics: the affine action of K (A = K[1:2, 1:2], t = K[1:2, 3] — bit-identical to the legacy imaging_transform's AffineMap), labeled {DistortedFrame{id} → ImageFrame{id}}. The 3-argument form states the frames explicitly — used by imaging_chain for the no-distortion collapse, where K consumes the ray frame directly.
VisualGeometryToolkitCore.Cameras.distort_stage — Method
distort_stage(lens::AbstractLensModel; id::Symbol = :_) -> FramedDistortMint the framed forward-distortion stage of lens, {distortion_frame(lens){id} → DistortedFrame{id}}, with the honest frame tag read from distortion_frame. For COLMAP's thin-prism fisheye the stage's math is its additive bundle, read in the equidistant frame (the equidistant half is the stage-1 ray projection, not distortion). Errors for lenses with no distortion stage: identity-in-frame models (Pinhole, a pure Lens(Equidistant()) — the chain collapses instead, see DistortedFrame) and scoped-out models (a parametric projection, which has no frame split).
VisualGeometryToolkitCore.Cameras.distortion_frame — Method
distortion_frame(lens::AbstractLensModel) -> Type{<:RetinalFrame} | nothingThe ray frame a lens model's distortion is DEFINED in — the Src frame kind of its framed distortion stage, and the frame the stage-1 ray projection of imaging_chain must land on. For a Lens it is its projection's frame (projection_frame): GnomonicFrame for the perspective projection, EquidistantFrame for the equidistant one (Kannala–Brandt's θ_d = θ·(1 + k₁θ² + …) is a radial factor there), and so on. A LiftLens is read on the gnomonic frame. nothing for a parametric projection, whose frame its coefficients define — ModelDefined in the FrameDependence vocabulary, which no Symbol tag can carry — so there is no frame + distortion SPLIT; imaging_chain serves such a lens through the fused model-carrying FramedLensProjection onto DistortedFrame{id}.
Relation to the Bool capability traits: has_distortion(lens) == false means the distortion is the identity in this frame, and the chain collapses the ray frame onto DistortedFrame (no distortion stage) — see the collapse rule there. All these frames assume a single center of projection.
VisualGeometryToolkitCore.Cameras.frame_transition — Method
frame_transition(C1::Type{<:RetinalFrame{id}}, C2::Type{<:RetinalFrame{id}}) -> FrameTransitionMint the framed transition map {C1 → C2} between two retinal frames of view id (the id must MATCH — a transition between frames of different views is a MethodError, the labeled-junction rule):
t = frame_transition(EquidistantFrame{:_}, GnomonicFrame{:_})
t(Vec2(θ, 0.0)) # == Vec2(tan(θ), 0.0)
inv(t) # the reverse transition {Gnomonic → Equidistant}This makes a first-class named object of what production code usually hides in an inline expression: OpenCV's cv::fisheye::undistortPoints ends with the line scale = tan(θ)/θ_d — that line IS frame_transition(EquidistantFrame{id}, GnomonicFrame{id}), the equidistant → gnomonic frame change applied after the polynomial θ-inverse, unnamed and unchecked in the C++. Here the same reparameterization is a framed stage with checked junctions and honest domain edges (see FrameTransition).
VisualGeometryToolkitCore.Cameras.imaging_chain — Method
imaging_chain(intrinsics, lens; id::Symbol = :_) -> FramedChain
imaging_chain(model::CameraModel; id::Symbol = :_)Mint the fully-typed framed imaging composite {ViewFrame{id} → ImageFrame{id}} — the frame-checked counterpart of imaging_transform's frame-free affine_part(K) ∘ lens. Every inner junction is checked at construction:
ViewFrame{id} ─ ray-frame projection → distortion_frame(lens){id}
─ distort_stage(lens) → DistortedFrame{id}
─ calibration_stage(K) → ImageFrame{id}Stage 1 is FramedPerspective for perspective-plane lenses and FramedEquidistantProjection for equidistant-frame (omnidirectional) lenses — see distortion_frame. When the lens is the identity in its defining frame (Pinhole; a pure Lens(Equidistant())), the distortion stage is omitted and K consumes the ray frame directly (frame collapse — see DistortedFrame). A lens with a parametric projection is served by one fused FramedLensProjection onto DistortedFrame{id}.
The id keyword threads the view identity; the default is the reserved anonymous view :_ (junctions then check by frame KIND — the distort-vs-undistort direction check needs no view identity). Rig-derived chains label from the rig's minted ids:
a, b = ids(rig)
chain₁ = imaging_chain(model(source(rig)); id = a) # {ViewFrame{a} → ImageFrame{a}}Numerically the chain matches imaging_transform: bit-identical for perspective-plane lenses (identical operations in identical order); equivalent to floating-point round-off for equidistant-frame lenses, whose framed route works in the model's defining frame instead of hopping through the gnomonic plane (atan/tan cancellation — the legacy callable's route).
VisualGeometryToolkitCore.Cameras.ray_map — Method
ray_map(lens::AbstractLensModel; id::Symbol = :_) -> RayMapMint the lens's sphere self-map D: S² → S² as the framed stage {ViewFrame{id} → ViewFrame{id}} — what a central lens actually does, stated with no image frame: it RE-ORIENTS rays. The familiar 2D distortion polynomial is the conjugation section ∘ D ∘ section⁻¹ in the lens's defining frame (distortion_frame); ray_map returns D itself.
Input is any 3D ray representative (norm irrelevant); output is the unit bearing vector of the re-oriented ray. inv yields the undistort conjugation (InverseRayMap) — honest, because on the sphere the frame projections are invertible sections. Rays outside the defining frame's domain (gnomonic: Z ≤ 0; equidistant: the exact backward ray) map to NaN — see RayMap.
For lenses whose distortion is the identity in their defining frame (Pinhole, a pure Lens(Equidistant())), D is the identity map — still framed, still returning the unit representative. For COLMAP's thin-prism fisheye the conjugation runs through its equidistant frame, where its distortion is the additive bundle. A lens with a parametric projection has no frame split and is scoped out (ArgumentError), exactly like distort_stage.
D exists only for CENTRAL lenses (single center of projection, which every VisualGeometryToolkitCore lens model satisfies): a finite or thick pupil has no sphere self-map, because its rays do not share a center — that regime is ray tracing, downstream.
VisualGeometryToolkitCore.Cameras.similarity_scale — Method
similarity_scale(m::FramedAffine) -> Real
similarity_scale(a::AffineMap) -> RealThe isotropic scale σ of a (framed) similarity x ↦ σ R x + t — a Hartley normalization's side included: lengths in Src are σ times as long in Dst, so a pixel threshold τ is σ τ after the map (and τ / σ back through inv(m)). For the NormalizedImageFrame map σ = 1/s, the inverse half-diagonal. Throws an ArgumentError unless m is a similarity (AᵀA = σ² I to rounding): a sheared or anisotropic map has no single scale.
VisualGeometryToolkitCore.Cameras.undistort_stage — Method
undistort_stage(lens::AbstractLensModel; id::Symbol = :_) -> FramedUndistortMint the framed inverse-distortion stage {DistortedFrame{id} → distortion_frame(lens){id}} — inv of distort_stage. The bare undistort_normalized surface is unchanged; this is its framed, direction-checked wrapper.
Targets
A calibration target is a MODEL, the analogue of a camera model: AbstractTarget promises points(target) in its own TargetFrame{id}; PointTarget holds explicit (nominal or estimated) points, PlanarTarget a design — its nominal points, or the reference they are resolved from — and a TargetCalibration taking the design to the physical target. A target whose geometry an estimator refines implements the calibratable-target protocol (parameter_vector, with_parameters, minimal_manifold, regularizer, and CoordinateTransformations' transform_deriv_params).
A standard layout is a constructor, an AbstractTargetLayout: Kalibr's AprilGrid is AprilGrid(; tag_rows, tag_cols, tag_size, tag_spacing, apriltag_family, first_id) (lengths, the spacing the gap between two tags), PlanarTarget(grid; calibration, board, canvas, placement) its tag corners in Kalibr's point order and frame, and point_index(grid, tag, corner) the point of a detected corner. Schöps et al.'s tagged star pattern is SchoepsPattern(; squares_x, squares_y, square_length, apriltags, star_segments, apriltag_family), and PlanarTarget(pattern; calibration, board, canvas, placement) its star features in Schöps' feature order (feature id k is point k + 1). The target records its layout, layout(t), so that VisualGeometryToolkitIO writes it as one (its tag in TARGET_LAYOUT_TAGS).
Target calibration
What a target calibration estimates about the physical target, as a camera calibration estimates intrinsics and a lens: a CoordinateTransformations Transformation of the design, its parameter Jacobian CT's transform_deriv_params and its point Jacobian CT's transform_deriv. The kinds:
| kind | map | parameters | Jacobian | regularizer | free directions |
|---|---|---|---|---|---|
Rigid | p ↦ p | none | 2 × 0 | none | 0 |
Skew (N = 2) | p ↦ U p, U unit upper-triangular | u12 (U's strictly-upper entries, row by row) | 2 × 1 | none | 1 |
Aspect (N = 2) | p ↦ diag(e^{a}, e^{−a}) p | a1 | 2 × 1 | none | 1 |
Scale | p ↦ e^{k} p | log_scale | 2 × 1 | none | 1 |
a planar EuclideanMap | p ↦ R(ϑ) p + t | angle, tx, ty | 2 × 3 | none | 3 |
ThinPlateSpline | p ↦ p + (f_x(p), f_y(p)), no affine term | fx1…fxK, fy1…fyK | 2 × 2K | bending energy, 2K rows | 2K − 6 |
HeightField | p ↦ (p, h(p)), no affine term | h1…hK | 3 × K | bending energy, K rows | K − 3 |
a composition (CT's ∘) | the members, the inner first | the members', tag-prefixed, the outer member's first | CT's chain rule, hcat(p1, m1 * p2) | stacked, the outer member's first | the members' sum |
The affine print error is its Iwasawa factors. An affine map of the design p ↦ A p + b factors as A = s·R·D·U (the KAN decomposition): a rotation, a scale, a det-1 diagonal (the aspect) and a unit upper-triangular matrix (the skew). The calibration composes them in that canonical order, EuclideanMap ∘ Scale ∘ Aspect ∘ Skew, and holds only the ones its data observes. With the target's pose estimated jointly, the rotation, the translation and the scale are the pose's — a target and the same target turned, shifted or scaled are the same images — so the print error is Aspect ∘ Skew, two parameters and no gauges; a measured length (a ruler) adds Scale; a known pose (a fixture, motion capture) the rigid member too. The counts agree: an affine map of R^N has N² + N parameters, the pose N(N−1)/2 + N and the scale one, which leaves (N−1) + N(N−1)/2. Aspect, Skew and Scale are N-generic; a planar target takes N = 2.
The thin-plate kinds are pure bending. An in-plane spline's displacements and a height field's heights have no affine term in their interpolant: the spline through each nodal value vector is Σ wᵢ U(·, pᵢ) alone, its affine coefficients a (tps_coefficients) zero — intrinsic to the spline, whatever the node distribution. The constructor and with_parameters project what they are given onto that subspace, v ↦ v − M a(v) with M = [1 u v] at the nodes, keeping the bending (values whose affine term is within 1e-12 of them are kept bit for bit, so RadiometricCamera's board heights, the bending part of its spline at its chart nodes, are kept unchanged). The condition is defined on the nodes the spline is built on: the radial basis U(r) = r² log r is not scale-invariant, and scaling the nodes moves a constant (a piston) between the affine and the bending parts — build the thin-plate kinds on a dimensionless chart of the board. An affine displacement belongs to the affine factors and a plane of heights to the pose; a spline built from either is the identity. So the canonical print error with a jointly estimated pose is
HeightField(nodes, h) ∘ ThinPlateSpline(nodes, fx, fy) ∘ Aspect(a) ∘ Skew(u)and no kind has a gauge: every direction a composition's parameters may move in is observable, and the joint pose/calibration Jacobian has full column rank. The stored parameters stay the nodal values (2K, K), with 2K − 6 (K − 3) free directions; a covariance over them has that rank.
A PlanarTarget takes one kind, or kinds in the canonical order HeightField ∘ ThinPlateSpline ∘ EuclideanMap ∘ Scale ∘ Aspect ∘ Skew, any of them left out, and refuses any other composition with an ArgumentError; it stores a composition right-associated, whatever association ∘ built. A composition's parameters, names, bounds, Jacobian columns and regularizer rows are in CoordinateTransformations' order, the outer member's first. The 0.4 AffineMap kind (tag "affine") is retired.
The thin-plate kinds are splines over K fixed nodes in the target's plane (the numerics' ThinPlateSplineSystem), their parameters the nodal values; a height is tps_evaluate of its spline, bit for bit. Lengths (a rigid member's translation, nodes, displacements, heights) are in the design's form, plain or physical, and move with ustrip/with_unit; Scale, Aspect and Skew are dimensionless. A calibration may be Uncertain(c, CovMat(Σ)), Σ over its parameters, as a pose may be.
A calibration on the board's own chart. "No affine term" is a property of the nodes a thin-plate kind is built on (r² log r is not scale-invariant: rescaling the nodes moves a piston into the affine part), so a field computed on a dimensionless chart of the board — RadiometricCamera's page [0, 1]² — stays on that chart, and the target places the chart with a placement, Core's SimilarityMap{3}S₃ of scale side, the identity rotation and the translation (origin, 0) (the design point p = origin + side·c): PlanarTarget(…; calibration, placement) applies its calibration as the composition S₃ ∘ lift ∘ C ∘ S₂⁻¹, a height scaled by side. The calibration is plain chart numbers whatever the points' form; the placement is fixed data, plain numbers in the points' unit (metres for a physical target, rescaled by ustrip/with_unit), and adds no parameters. A page whose chart runs down the design's y is placed at its top-left corner on the reflected chart nodes (u, −v), which keep the bending pure.
nodes = [SVector(n[1], -n[2]) for n in grid_nodes(5; lo = 0.0, hi = 1.0)] # the page's chart, rows down
t = PlanarTarget(:page; calibration = HeightField(nodes, heights ./ side), # heights in chart units
placement = SimilarityMap(side, one(RotMatrix{3, Float64}), SVector(x0, ytop, 0.0)), board, canvas)Planarity is in the type.is_planar is true for Rigid, the affine factors and ThinPlateSpline, false once a HeightField is present; points(t) is then SVector{2}s on z = 0 or SVector{3}s, and the result type is inferred. A method that assumes the plane takes t::PlanarTarget{<:PlanarCalibration} and refuses a height field by dispatch.
The manifold.TargetManifold(t) — minimal_manifold(t) — is linear: the identity basis for the affine factors, the thin-plate kinds' free directions (an orthonormal null-space basis of the affine-coefficient rows, from a pivoted Householder QR: reproducible) for the splines. There is nothing to free or fix: what the data observes is chosen by the composition's members.
t = PlanarTarget(pts; calibration = HeightField(nodes) ∘ ThinPlateSpline(nodes) ∘ Aspect(0.0) ∘ Skew(0.0))
ch = minimal_manifold(t) # (K − 3) + (2K − 6) + 2 directions, all observable
PlanarTarget(pts; calibration = Scale(0.0) ∘ Aspect(0.0) ∘ Skew(0.0)) # a ruler: the scale is observableVisualGeometryToolkitCore.Cameras.PlanarCalibration — Type
PlanarCalibrationThe planar calibration kinds — Rigid, ThinPlateSpline, the affine factors and their compositions — as one type, for a method that assumes the design stays on z = 0: f(t::PlanarTarget{<:PlanarCalibration}) refuses a target with a HeightField by dispatch (a MethodError).
VisualGeometryToolkitCore.Cameras.TARGET_CALIBRATION_ORDER — Constant
TARGET_CALIBRATION_ORDERThe canonical order of a target calibration's composition, the outer map first, as the kinds of TARGET_CALIBRATION_TAGS:
(HeightField, ThinPlateSpline, EuclideanMap, Scale, Aspect, Skew)A PlanarTarget holds one kind, or two to six of them composed in this order, any left out, each once (Rigid composes with nothing). calibration_tag.(TARGET_CALIBRATION_ORDER) are the tags a composition's record lists its members by, in that order.
VisualGeometryToolkitCore.Cameras.TARGET_CALIBRATION_TAGS — Constant
TARGET_CALIBRATION_TAGSThe TagRegistry of the target calibration kinds — the one map from a kind's tag to its type, read by a composition's parameter names (:aspect_a1) and by VisualGeometryToolkitIO's target record ("type": "aspect"): "rigid", "euclidean" (Core's EuclideanMap, the family's suffix Map stripped), "scale", "aspect", "skew", "thin_plate_spline", "height_field", snake_case of the type's name. A composition has no tag: its record is written by its structure.
RETIRED: "affine", CoordinateTransformations' AffineMap as one kind with six parameters (A row by row, then b) and four gauges. Its print error is now the composition of the gauge-free factors, Aspect ∘ Skew (Scale ∘ with a measured length, EuclideanMap ∘ Scale ∘ with a known pose); the registry names its outer member, "aspect", as its successor.
VisualGeometryToolkitCore.Cameras.TargetCalibration — Type
TargetCalibrationWhat a target calibration estimates about a planar target: a CoordinateTransformations Transformationc(p) of its design points to the physical target — one of Rigid, the affine factors (a planar EuclideanMap, Scale, Aspect, Skew), ThinPlateSpline, HeightField, or a composition of them with CT's ∘. The union of those, the one type every calibration-typed method takes. The rigid member's parameters are (:angle, :tx, :ty), and it holds its rotation as a Rotations.jl Angle2d, the angle itself (a member built from another Rotation{2} is converted once, where a target stores it), so its parameters round-trip bit for bit.
A PlanarTarget holds one kind, or kinds composed in the CANONICAL ORDER, any of them left out:
HeightField ∘ ThinPlateSpline ∘ EuclideanMap ∘ Scale ∘ Aspect ∘ SkewThe print error with a jointly estimated pose is HeightField ∘ ThinPlateSpline ∘ Aspect ∘ Skew — the affine part as the gauge-free factors (see Aspect), the spline and the height field pure bending (no affine term in the interpolant, see ThinPlateSpline); a measured length adds Scale, a known pose the rigid member. No kind has a gauge, so every parameter of HeightField ∘ ThinPlateSpline ∘ Aspect ∘ Skew is observable with the pose estimated jointly; a rigid member and a Scale duplicate that pose's rotation, translation and scale, so the joint-pose manifold (TargetManifold, minimal_manifold(target; known)) refuses them unless known states the length (:length, Scale) or the pose (:pose, both) — drop them, or absorb them into the pose, for a joint refinement. (VisualGeometryToolkitIO's migration of a retired "affine" record writes all four factors EuclideanMap ∘ Scale ∘ Aspect ∘ Skew: the same map, held exactly.) Any other composition — a Rigid member, members out of order or repeated — is refused with an ArgumentError. A target stores a composition right-associated (a ∘ (b ∘ c)) whatever association ∘ built it in: the members, their order and so the parameters are the same.
The protocol: parameter_vector, parameter_names, parameter_bounds, with_parameters(c, θ), CoordinateTransformations' transform_deriv_params(c, x) (∂c(x)/∂θ) and transform_deriv(c, x) (∂c(x)/∂x), regularizer(c), length_mask. A composition's are stacked in CT's order, the outer member's first; its names carry each member's tag, :height_field_h1, …, :aspect_a1, :skew_u12. Jacobians are plain numbers; for a physical calibration the point and the length parameters are in metres.
VisualGeometryToolkitCore.Cameras.Aspect — Type
Aspect(a::StaticVector{M}) <: CoordinateTransformations.Transformation N = M + 1
Aspect(a::Real) N = 2
Aspect{N}() the identityThe aspect of an affine print error, x ↦ D·x with D = diag(e^{d₁}, …, e^{d_N}) and Σ dᵢ = 0: a det-1 diagonal, which stretches one axis against the others and changes no area. Its parameters are the N − 1 free log-stretches aᵢ = dᵢ (i < N), named (:a1, …, :a(N-1)), and d_N = −Σ aᵢ: for N = 2, D = diag(e^{a}, e^{−a}) — a = 0.001 stretches x by 0.1 % and shrinks y by as much. det D = 1 exactly in exact arithmetic (Σ dᵢ = 0 holds exactly in floating point too; the product of the e^{dᵢ} is 1 to rounding).
Dimensionless (it carries lengths through, so it sits with plain and physical designs alike): no length parameters, no gauges, no regularizer, unbounded. transform_deriv(c, x) = D; transform_deriv_params(c, x) is N × (N − 1), column j is e_j D_jj x_j − e_N D_NN x_N. inv(c) negates a. A kind of the Iwasawa factorization of an affine print error (see Skew, the factor it follows in Aspect ∘ Skew).
VisualGeometryToolkitCore.Cameras.HeightField — Type
HeightField(nodes, heights; project = true) <: CoordinateTransformations.Transformation
HeightField(nodes) flatAn out-of-plane deformation, p ↦ (p, h(p)): one thin-plate spline (numerics/thinplatespline.jl) over K fixed nodes in the target's plane through the nodal heights, both in the design's length form (plain, or lengths). The height at p is tps_evaluate of the spline, nothing else. Parameters (:h1, …, :hK), unbounded; transform_deriv_params is 3 × K (zero in the plane, the spline's nodal_weights through the pure-bending projection in z: the derivative of with_parameters); regularizer the bending-energy rows (K rows).
A target with a height field is not planar (is_planar): its points are 3-vectors. It composes after an in-plane kind or chain, HeightField ∘ ThinPlateSpline ∘ Aspect ∘ Skew (the canonical print error), HeightField ∘ Aspect ∘ Skew, …, its nodes then in that chain's output (the calibrated plane, not the design).
Its heights are pure bending: no affine term in the interpolant, a = 0 (the field's a, of tps_coefficients), and the constructor and with_parameters PROJECT the heights they are given onto that subspace, h ↦ h − M a(h) with M = [1 u v] at the nodes — the bending kept, the plane dropped (heights whose affine term is below 1e-12 of them are kept bit for bit, as RadiometricCamera's board heights, the bending part of its spline, are). A plane of heights — a piston and two tilts — is the target pose's, so a height field built from one is flat.
"No affine term" is defined ON THE NODES THE SPLINE IS BUILT ON: the radial basis U(r) = r² log r is not scale-invariant (U(s r) = s² U(r) + s² log(s) r², and Σ wᵢ |p − pᵢ|² is a constant under the spline's side conditions), so scaling the nodes moves a constant — a piston — between the affine and the bending parts (translating, rotating or reflecting them does not). Build a height field on a dimensionless chart of the board, not on nodes in physical units, to keep its heights where they were computed: the board in units of ONE length, its width ([0, 1] × [0, H/W], RadiometricCamera's page), a uniform scaling. Never scale x and y apart (a non-square board onto [0, 1]²): a thin-plate spline is the same surface only under a similarity of its domain, so per-axis scaling makes its bending depend on the board's aspect. Place that chart on the target with a placement, a SimilarityMap{3} (PlanarTarget(…; calibration, placement)): the field stays plain, its heights in chart units, and the placement scales them to the target's. A physical field's condition holds in metres, its stored form; a unit move (ustrip(u, c), with_unit) converts the values and does not re-project them (the geometry stays where it is), so in another unit the interpolant may carry the piston that unit's scale adds, and a with_parameters there projects in that unit's numbers. project = false keeps the heights as given, the form a reader restores a field in after checking it was pure bending in the unit it was written in (VisualGeometryToolkitIO). The K stored heights have K − 3 free directions, the TargetManifold's tangent, and no gauges; a covariance over them has rank K − 3.
VisualGeometryToolkitCore.Cameras.HeightField — Method
HeightField(nodes, points .=> heights; smoothing = 0) -> HeightFieldThe HeightField over nodes that fits the scattered heights at the plane points (point => height pairs, Core's correspondence form) in the least-squares sense, the PLANE REMOVED: the heights are modelled as the pure-bending field plus a plane, zₖ ≈ h(pₖ) + a₁ + a_u uₖ + a_v vₖ, and the plane (the pose's piston and two tilts) is fitted and dropped, so the result is pure bending (is_pure_bending) and a planar input gives a flat field. A one-step algorithm (one pivoted-QR solve, `), the scattered-point counterpart ofHeightField(nodes, heights), which interpolates nodal values. The unknowns are the field'sK − 3free directions (its target manifold's null-space basis of the interpolant's affine rows) and the three plane coefficients; the rows are the spline's [nodalweights](@ref) at each point.smoothing = λ ≥ 0adds the bending energyλ hᵀQh([bendingenergy](@ref)) asKresidual rows (a smoothing thin-plate spline); withλ = 0the fit needs at leastK` points in general position.
nodes, points and heights share one form: plain numbers, or lengths (the field is then physical, as HeightField(nodes, heights) is).
VisualGeometryToolkitCore.Cameras.HeightField — Method
HeightField(system::ThinPlateSplineSystem, heights; project = true)The same field on a solver already built on its nodes — plain numbers, the nodes in metres for heights that are lengths (their form decides the field's): what a caller that has checked is_pure_bending on that system builds, without forming it again.
VisualGeometryToolkitCore.Cameras.Rigid — Type
Rigid() <: CoordinateTransformations.TransformationThe design, exactly: the identity, no parameters, no regularizer, no lengths (so it sits with plain and physical designs alike).
VisualGeometryToolkitCore.Cameras.Scale — Type
Scale(log_scale::Real = 0.0) <: CoordinateTransformations.TransformationA uniform scale, x ↦ e^{k}·x with k the log-scale, any dimension: one parameter, named (:log_scale,); transform_deriv(c, x) = e^{k} I, transform_deriv_params(c, x) = e^{k} x (one column). Dimensionless, no gauges, no regularizer, unbounded.
The scale of an affine print error is the target's pose's when the pose is estimated jointly (the pose's translation scales with it), so it belongs in a calibration only when a LENGTH is measured — a ruler, a known pose: Scale ∘ Aspect ∘ Skew, or EuclideanMap ∘ Scale ∘ Aspect ∘ Skew.
VisualGeometryToolkitCore.Cameras.Skew — Type
Skew(u::StaticVector{L}) <: CoordinateTransformations.Transformation L = N(N − 1)/2
Skew(u::Real) N = 2
Skew{N}() the identityThe skew of an affine print error, x ↦ U·x with U unit upper-triangular: ones on the diagonal, zeros below it, and its L = N(N − 1)/2 strictly-upper entries the parameters, ROW BY ROW — (U₁₂, U₁₃, …, U₁N, U₂₃, …, U₂N, …), named (:u12, :u13, …, :u23, …); a record persists them in that order. For N = 2, x ↦ (x₁ + u·x₂, x₂): a shear of x along y. det U = 1 exactly.
Dimensionless: no length parameters, no gauges, no regularizer, unbounded. transform_deriv(c, x) = U; transform_deriv_params(c, x) is N × L, the column of U_ij being x_j e_i. inv(c) is the skew of U⁻¹ (−u for N = 2). The inner factor of the canonical print error Aspect ∘ Skew.
VisualGeometryToolkitCore.Cameras.ThinPlateSpline — Type
ThinPlateSpline(nodes, fx, fy; project = true) <: CoordinateTransformations.Transformation
ThinPlateSpline(nodes) zero displacementAn in-plane residual, p ↦ p + (f_x(p), f_y(p)): two thin-plate splines (numerics/thinplatespline.jl) over the same K fixed nodes in the target's plane, through the nodal displacements fx and fy. Nodes and displacements are in the design's length form (plain, or lengths). Parameters (:fx1, …, :fxK, :fy1, …, :fyK), unbounded; transform_deriv_params is 2 × 2K (the splines' nodal_weights through the pure-bending projection: the derivative of with_parameters); regularizer the two splines' bending-energy rows (2K rows; a quadratic block, see its docstring).
It is a pure BENDING residual: no affine term in the interpolant. Each of the two splines, through fx and through fy, has zero affine coefficients (a₁ = a_u = a_v = 0 of tps_coefficients: six conditions), and the constructor and with_parameters PROJECT the displacements they are given onto that subspace, v ↦ v − M a(v) with M = [1 u v] at the nodes — which keeps the bending and drops the affine term (displacements whose affine term is below 1e-12 of them are kept bit for bit). A displacement that is an affine map of the design is not the spline's to carry: four of its six directions are the target pose's, and the other two (and the rest, with a measured length or a known pose) belong to the affine factors composed after it — the canonical print error is HeightField ∘ ThinPlateSpline ∘ Aspect ∘ Skew. A spline built from an affine displacement is the identity. As for a HeightField, "no affine term" is defined on the nodes the spline is built on (scaling them moves a constant between the affine and bending parts): build it on a dimensionless chart. project = false keeps the displacements as given — for a reader restoring values that were pure bending in the unit they were written in (a unit move does not re-project, see HeightField); the caller has checked that condition. The 2K stored displacements have 2K − 6 free directions, the TargetManifold's tangent, and no gauges; a covariance over them has rank 2K − 6. transform_deriv_params is the derivative along that subspace (the nodal weights; the projection is the identity there).
VisualGeometryToolkitCore.Cameras.ThinPlateSpline — Method
ThinPlateSpline(system::ThinPlateSplineSystem, fx, fy; project = true)The same spline on a solver already built on its nodes — plain numbers, the nodes in metres for displacements that are lengths (their form decides the spline's): what a caller that has checked is_pure_bending on that system builds, without forming it again.
VisualGeometryToolkitCore.Cameras.calibration_tag — Function
calibration_tag(::Type{C}) -> StringThe tag of the calibration kind C: the prefix of its parameter names in a composition (:aspect_a1) and the type of its record — its TARGET_CALIBRATION_TAGS tag. A composition has none (an ArgumentError).
VisualGeometryToolkitCore.Cameras.is_affine — Function
is_affine(::Type{C}) -> Bool
is_affine(c::TargetCalibration) -> BoolWhether a calibration's IN-PLANE part — where it takes a design point in the target's plane, a height field's lift aside — is an affine map of the design, p ↦ A p + b: a function of the TYPE, true for Rigid, the affine factors and their compositions, HeightField (it leaves the plane point where it is) and a height field after affine factors; false for a ThinPlateSpline and HeightField ∘ ThinPlateSpline. A planar target's surface reads its chart's affine off such a calibration (its image of the design origin and its point Jacobian there) and refuses any other.
VisualGeometryToolkitCore.Cameras.is_planar — Function
is_planar(::Type{C}) -> Bool
is_planar(c::TargetCalibration) / is_planar(t::PlanarTarget) -> BoolWhether a calibration keeps the design in its plane — a function of the TYPE, so it folds at compile time: true for Rigid, the affine factors and their compositions, and ThinPlateSpline; false for a HeightField and a composition with one. A planar target's points are SVector{2}s (on z = 0 of its frame), a non-planar one's SVector{3}s. An Uncertain calibration is planar when its kind is.
VisualGeometryToolkitCore.Cameras.is_pure_bending — Method
is_pure_bending(system::ThinPlateSplineSystem{K}, v::AbstractVector) -> BoolWhether the nodal values v (length K) of a thin-plate spline on system are PURE BENDING, the rule of ThinPlateSpline and HeightField: the affine term of their interpolant, M a(v) with M = [1 u v] at the nodes, is within 1e-12 of them (‖M a(v)‖ ≤ 1e-12 ‖v‖). Exactly the values a thin-plate kind's constructor keeps bit for bit; it projects any other. Defined on the nodes the system is built on (scaling them moves a piston between the affine and the bending parts).
VisualGeometryToolkitCore.Cameras.iwasawa_parameters — Method
iwasawa_parameters(A::StaticMatrix{2, 2}) -> SVector{4}
iwasawa_parameters_with_jacobian(A::StaticMatrix{2, 2}) -> (θ, J)The parameters of the Iwasawa factors of a planar linear map A with det A > 0, A = s·R·D·U: θ = (ϑ, log s, a₁, u₁₂) — the angle of R(ϑ) and the parameter vectors of Scale, Aspect and Skew — so that EuclideanMap(Angle2d(ϑ), b) ∘ Scale(θ[2]) ∘ Aspect(θ[3]) ∘ Skew(θ[4]) is the affine map p ↦ A·p + b, held exactly.
It is the QR factorization A = Q·T (T upper-triangular with a positive diagonal, so Q is a rotation when det A > 0), in closed form — Gram–Schmidt on A's columns: ϑ = atan(a₂₁, a₁₁), s = √(t₁₁ t₂₂) = √det A, a₁ = ½ log(t₁₁/t₂₂), u₁₂ = t₁₂/t₁₁. J is the 4 × 4 Jacobian of θ in A's entries ROW BY ROW, (a₁₁, a₁₂, a₂₁, a₂₂) (ForwardDiff's, in one evaluation). An ArgumentError for det A ≤ 0 (a reflection or a degenerate map, which has no such factorization).
VisualGeometryToolkitCore.Cameras.iwasawa_parameters_with_jacobian — Method
iwasawa_parameters_with_jacobian(A::StaticMatrix{2, 2}) -> (θ, J)iwasawa_parameters(A) and its 4 × 4 Jacobian J in A's entries row by row, (a₁₁, a₁₂, a₂₁, a₂₂); the same ArgumentError for det A ≤ 0.
VisualGeometryToolkitCore.Cameras.length_mask — Function
length_mask(::Type{C}) -> SVector{n, Bool}
length_mask(c::TargetCalibration) -> SVector{n, Bool}Which entries of parameter_vector(c) are lengths: true for an entry that scales with a unit change (stored in metres for a physical calibration), false for a dimensionless one — a function of the TYPE. Rigid: none; Scale, Aspect, Skew: none (every entry false); a planar EuclideanMap: (false, true, true) (the angle, then the translation); ThinPlateSpline and HeightField: every entry; a composition: its members', the outer member's first. A covariance over θ moves with a unit change by this mask; a record of the calibration states its lengths by it.
VisualGeometryToolkitCore.Cameras.members — Method
members(c) -> TupleThe members of a target calibration's composition, its leaves in CoordinateTransformations' order, the outer map first, whatever association ∘ built it in: members(H ∘ (A ∘ U)) == members((H ∘ A) ∘ U) == (H, A, U). A kind that is not a composition is its own one member, (c,).
VisualGeometryToolkitCore.Cameras.planar_kind — Method
planar_kind(K::Type, n_nodes::Integer = 0) -> TypeThe type a planar target holds for the calibration kind K (a TARGET_CALIBRATION_TAGS value): EuclideanMap{2}, Aspect{2} and the planar Skew (N = 2), ThinPlateSpline{n_nodes} and HeightField{n_nodes} over their nodes, Rigid and Scale as they are. Its length_mask is the kind's parameter layout, so length(length_mask(planar_kind(K, n))) is the number of parameters a record of the kind holds.
VisualGeometryToolkitCore.Cameras.regularizer — Function
regularizer(c | target) -> SVectorThe prior's residual rows r(θ) at the current parameters: ‖r‖² is the penalty a calibrating estimator adds (a thin-plate spline's bending energy, a shrinkage toward the design). Empty for Rigid and the affine factors (Aspect, Skew, Scale, an EuclideanMap).
The rows are a QUADRATIC BLOCK: only ‖r‖² (equivalently LᵀL) is defined. The thin-plate kinds' rows are L v with L an eigen-factor of the bending form, and an eigenbasis is not unique (a symmetric node grid has repeated eigenvalues; the basis and the row signs follow the LAPACK build). So never compare rows one by one (across machines or against stored values), and never weight them one by one (a robust per-row weight would depend on that basis): weight the block as a whole.
VisualGeometryToolkitCore.Cameras.with_parameters — Method
with_parameters(u::Uncertain{<:TargetCalibration}, θ) -> TargetCalibrationThe BARE kind rebuilt from θ: an estimate's covariance describes the parameters it was estimated at and does not survive a change of them, so the result is not Uncertain. (A PlanarTarget's with_parameters does the same to its calibration.)
VisualGeometryToolkitCore.Cameras.TARGET_LAYOUT_TAGS — Constant
TARGET_LAYOUT_TAGSThe TagRegistry of the target layouts (AbstractTargetLayout): "april_grid" (AprilGrid) and "schoeps_pattern" (SchoepsPattern), snake_case of the type's name — the type of VisualGeometryToolkitIO's target layout record. (Kalibr's own YAML spells its target type aprilgrid; that is the foreign file's word, read by its reader, not a tag.)
VisualGeometryToolkitCore.Cameras.AbstractTarget — Type
AbstractTargetA calibration target as a model: points(target) returns its points in its own frame TargetFrame{id} — 2-vectors on the plane z = 0 for a planar target, 3-vectors for a solid one or a planar design lifted off its plane by a HeightField — plain or Uncertain. See the header of targets.jl for the calibratable-target protocol.
Placement is a trait, is_posed, as for cameras: a target type such as PlanarTarget or PointTarget is unposed, is its own model, and refuses pose and extrinsics with an ArgumentError; a PosedTarget is posed. The calibratable-target protocol and TargetManifold are a target MODEL's, as the lens chart is a camera model's: on a PosedTarget they are an ArgumentError naming model(pt).
VisualGeometryToolkitCore.Cameras.AbstractTargetLayout — Type
AbstractTargetLayoutA parametric design of a planar target — AprilGrid or SchoepsPattern: the few numbers its nominal points follow from, and whatever else the printed design states (its tag family and ids). PlanarTarget(layout; …) is the target of its points, and it records the layout (layout(t)) so that a writer can store the design by its layout rather than by its points.
VisualGeometryToolkitCore.Cameras.AprilGrid — Type
AprilGrid(; tag_rows, tag_cols, tag_size, tag_spacing, apriltag_family = "tag36h11", first_id = 0)The layout of an AprilGrid, Kalibr's grid of AprilTags (the aprilgrid target of Kalibr's target YAML: tagRows, tagCols, tagSize, tagSpacing): tag_rows × tag_cols square tags of side tag_size, tag_spacing apart — the gap between two neighbouring tags, a length like the side (Kalibr's tagSpacing is the ratio tag_spacing / tag_size). Both are lengths (90mm, 27mm) or both plain numbers. The tags are of the AprilTag family apriltag_family (Kalibr's grids are tag36h11), with ids from first_id (≥ 0) on. Tag i (0-based: the family id less the grid's first id) sits in row i ÷ tag_cols, column i % tag_cols.
PlanarTarget(grid; calibration = Rigid(), board = nothing, canvas = nothing, placement = nothing) is the target of its tag corners (recording the grid, layout) in Kalibr's point order and frame: the 2tag_rows × 2tag_cols corners row by row, point r·2tag_cols + c (0-based, r and c 0-based) at
x = (c ÷ 2)(tag_size + tag_spacing) + (c % 2)·tag_size
y = (r ÷ 2)(tag_size + tag_spacing) + (r % 2)·tag_size— tag 0's first corner at the origin, x along the columns and y along the rows: the design chart (TargetFrame), x right and y up as the print is seen from the front. point_index(grid, tag, corner) is the point of a tag's corner. The points are in the layout's form (lengths, stored in metres, or plain numbers); a board or canvas is in that form too, and a page whose lower-left corner is not tag 0's first corner says so by its origin (Canvas(size, origin)).
VisualGeometryToolkitCore.Cameras.PlanarTarget — Type
PlanarTarget(nominal; calibration = Rigid(), board = nothing, canvas = nothing, placement = nothing) a design given by its points
PlanarTarget(design::Symbol; nominal = nothing, calibration = Rigid(), board = nothing, canvas = nothing, placement = nothing) a content-addressed designA planar target: a DESIGN — its nominal plane points, and/or the reference design they are resolved from (a BlobBoards board uid) — and a TargetCalibrationC taking the design to the physical target, points(t) = C.(nominal). An unresolved target (a reference, no nominal points yet) is a pose-only board; resolve supplies its points, and its points is an error until then. VGC never parses a design file: the design's package resolves the reference.
The calibration may be Uncertain — Uncertain(c, CovMat(Σ)), Σ over parameter_vector(c) — as a PosedTarget's pose may be; the type parameter C is then the calibration kind it wraps. calibration(t) returns it as stored, design(t) the design and reference(t) the reference (or nothing).
The nominal points are plain numbers (stored as Float64) or lengths (stored in metres), all one form (has_units); a calibration with lengths (a rigid member's translation, a thin-plate spline's nodes and displacements, a height field's nodes and heights) is in the same form, or the constructor raises FlexUnits' DimensionError. ustrip(u, t) and with_unit(t, u) move the points and the calibration together. An unresolved target holds no points: it is neither form, and both move only its calibration.
board is the board the design is printed on, a Board (its size and its corner in the design chart), or nothing — the target's analogue of a camera's Sensor: Rect(board) is the printed domain in the design chart, and surface bounds the target's primitive by it. The board is in the points' form — lengths for physical points, plain numbers for plain ones (FlexUnits' DimensionError otherwise) — and the points lie inside Rect(board) to within a rounding margin of 16 eps of its coordinates, the margin surface grows the board by (an ArgumentError otherwise). Being in the design chart, the board is the same rectangle under any calibration. board(t) reads it; ustrip and with_unit move it with the points.
canvas is the printed page the board lies on, a Canvas (its size, its corner in the design chart and optionally the printed image, a CanvasTexture), or nothing: the record of what was printed, which the points, the calibration and surface never read. It is in the points' form like the board, and the board lies on it to within the same rounding margin (an ArgumentError otherwise). canvas(t) reads it; ustrip and with_unit move it with the points.
placement is a SimilarityMap{3}, or nothing: where the calibration's dimensionless chart lies in the target's frame (scale the chart's side, the identity rotation, the translation (origin, 0); see "The placement" in targets.jl). With one, the calibration acts in that chart — a HeightField or ThinPlateSpline built on the chart's nodes (RadiometricCamera's page [0, 1]²), its values in chart units — and is plain numbers whatever the points' form (an ArgumentError for a calibration with units); the placement is plain numbers in the points' unit (metres for a physical target): points(t) = (S₃ ∘ lift ∘ C ∘ S₂⁻¹).(nominal), a height scaled by the placement's side. The calibration's parameters, manifold and regularizer are its own: the placement is fixed data and adds none. A unit move moves the placement and leaves the calibration as it is. placement(t) reads it.
A target built from a parametric design, PlanarTarget(layout; calibration, board, canvas, placement) (an AbstractTargetLayout: AprilGrid, SchoepsPattern), holds the layout's points as its nominal points and records the layout, layout(t); every other construction records none.
VisualGeometryToolkitCore.Cameras.PointTarget — Type
PointTarget(points)A target given by its points, explicitly, in its own frame: 2-vectors (a plane's) or 3-vectors, all of one dimension, plain or Uncertain (an estimated constellation with its covariances). A motion-capture body's marker constellation, or a board whose calibration was resolved to measured points.
A physical target's points are lengths, stored in metres, and an Uncertain physical point's covariance is in m²: values are held in base units, and the covariance is stored as given, never rescaled — supply it in m², not in the squared unit the point was written in. Plain points are stored as Float64. All are one form or the other (has_units); ustrip(u, t) and with_unit(t, u) move between them.
VisualGeometryToolkitCore.Cameras.SchoepsPattern — Type
SchoepsPattern(; squares_x, squares_y, square_length, apriltags, star_segments = 16, apriltag_family = "tag36h11")The layout of a tagged star pattern of Schöps et al.'s cameracalibration (their `createcalibrationpattern.py; the pattern YAML'ssquaresx,squaresy,squarelengthinmeters,numstarsegmentsandapriltags): asquaresx × squaresygrid of squares of sidesquarelength— a length (8mm) or a plain number — with a star feature ofstarsegmentssegments (even, ≥ 4) at every inner corner, and AprilTags of the familyapriltagfamily, each coveringwidth × heightsquares from square(x, y)(the YAML'stagx,tag_y,width,height: 0-based squares, x to the right and y DOWN as the pattern is printed) and carrying the family idid(the YAML'sindex).apriltagsis a collection of(; x, y, width, height)or(; x, y, width, height, id)— without ids the tags are numbered 0, 1, … in order — stored as(; x, y, width, height, id)`; each tag lies on the pattern, and the ids are distinct and ≥ 0.
PlanarTarget(pattern; calibration = Rigid(), board = nothing, canvas = nothing, placement = nothing) is the target (recording the pattern, layout) of its star features in Schöps' feature order (FeatureDetectorTaggedPattern::GetCorners): the inner corners (gx, gy), gx ∈ 0:squares_x − 2, gy ∈ 0:squares_y − 2, row by row (gy outer), the corners next to a tag left out (IsValidFeatureCoord: x − 1 ≤ gx ≤ x − 1 + width and y − 1 ≤ gy ≤ y − 1 + height for some tag), so for a YAML's first pattern (pattern index 0) feature id k (0-based, Schöps' and a dataset.bin's) is point k + 1. GetCorners numbers the features of a YAML with several patterns across them, in order: pattern i's feature id k is point k + 1 − (n₀ + … + nᵢ₋₁) of its own target, nⱼ the number of points of pattern j's target. Each point is at
square_length · (gx + 1, squares_y − 1 − gy)— the design chart (TargetFrame): the pattern's lower-left corner at the origin, x right and y up as the print is seen from the front. Schöps' own geometry (KnownGeometry: cell_length_in_meters · (gx, gy, 0), y down) is the same points under a half turn about x and a shift of one square, a rigid motion the pose absorbs. The points are in the layout's form (lengths, stored in metres, or plain numbers); a board or canvas is in that form too, and a page whose lower-left corner is not the pattern's says so by its origin (Canvas(size, origin)).
VisualGeometryToolkitCore.Cameras.TargetManifold — Type
TargetManifold(target; known = ()) <: ManifoldA calibratable target's manifold anchored at target: minimal_manifold(target; known). No calibration kind has a gauge (the affine factors leave the pose's directions out of the composition, the thin-plate kinds hold no affine or plane part), so the manifold is every direction the calibration's parameters may move in, and with the target's pose estimated jointly every one of them is observable.
That holds for the print error a jointly estimated pose leaves, HeightField ∘ ThinPlateSpline ∘ Aspect ∘ Skew. A rigid member (EuclideanMap) duplicates the pose's rotation and translation, and a Scale the pose's scale (its translation's length): with the pose estimated jointly they are the pose's gauges, and the manifold REFUSES them (ArgumentError) unless the caller states what fixes them — known lists :length (a measured length, a ruler: Scale is observable) or :pose (a known pose, a fixture or motion capture: EuclideanMap and Scale both are). Otherwise drop them from the calibration, or absorb them into the pose, before a joint refinement. (A vgio_migrate of a retired "affine" record keeps all four factors EuclideanMap ∘ Scale ∘ Aspect ∘ Skew, the same map; that record takes known = (:pose,) or that reduction.)
The chart is linear: retract(ch, v) = with_parameters(target, θ₀ + B v), B an n × m matrix with orthonormal columns, so tangent_size(ch) = m and retract_jacobian(ch) = B. For the affine factors B is the identity and the coordinates are the parameters' offsets, named by parameter_names. For a ThinPlateSpline (a HeightField) B spans the 2K − 6 (K − 3) pure-bending directions, the null space of the interpolant's affine-coefficient rows — the complement of an orthonormal basis of those rows' span, from a Householder QR with column pivoting (LAPACK, so one reproducible basis per node set and parameter layout); a composition's the complement of its members' constraints together. action_jacobian(ch, x) = transform_deriv_params(c, x) · B at the design point x; local_coordinates(ch, t) = Bᵀ (θ(t) − θ₀). restrict(ch, names) keeps only the parameters names free, the others held.
NO GAUGE WITH THE POSE, AND AN OBLIQUE COMPLEMENT. The tangent has no direction a change of the jointly estimated pose reproduces: the pose's nodal fields (a piston and two tilts of a height field, an in-plane translation and rotation of a thin-plate spline) are affine, and the thin-plate kinds' free directions are pure bending, their interpolant's affine part zero. That complement of the pose's directions is OBLIQUE, not orthogonal: the basis is orthogonal to the interpolant's affine ROWS (the pose's fields in the metric K⁻¹ of the radial basis), not to the plane fields themselves, so a bending direction correlates with a piston at a set of sampled points while no combination of them is one. The deformation's and the pose's spans intersect trivially — the stacked point Jacobian has full rank (test/mappedsurfacetests.jl pins its condition number and the smallest principal angle between the spans per composition) — which is the gauge-freedom statement; a Euclidean orthogonality in point space is not a property any parametrization can have. The spans can come close: with a 4 × 4 node grid reaching past the detections (nodes at ±0.6, samples to ±0.5 × ±0.4), the nearest bending direction is a nearly constant field of nodal heights — a bowl whose curvature lies outside the data, almost a piston on it — at about 5° (0.095 rad) from the pose's span, which inflates the variance along that one mode about 110×; nodes covered by detections (0.36 rad with the grid inside the samples) and the bending regularizer control it. A Scale and an in-plane EuclideanMap ARE pose directions, hence refused without known.
THE BASIS IS FIXED AT THE ANCHOR, and recenter keeps it: the constrained directions are constants of the nodes, so the slice is the same wherever it is anchored, and the manifold's type does not change.
B is an SMatrix while n ≤ 18 and a Matrix{Float64} above: a static basis compiles in time that grows steeply with its size (the first call of the manifold's verbs, cold: n = 18, 0.6 s static against 0.8 s as a Matrix; n = 32, 2.3 s against 0.7 s; a 5 × 5 in-plane spline, n = 50, 9.9 s against 0.7 s; 7 × 7, n = 98, 100 s against 2.6 s). In the large case retract's step, action_jacobian and local_coordinates are a Vector or a Matrix, and tangent_size is a run-time value. Zero-dimensional for a Rigid target.
VisualGeometryToolkitCore.Cameras.board — Method
board(t::PlanarTarget) -> Union{Nothing, Board}The board the target's design is printed on (Board), or nothing when the target does not carry one. Rect(board(t)) is its domain in the design chart, plain metres.
VisualGeometryToolkitCore.Cameras.calibration — Method
calibration(t::PlanarTarget) -> TargetCalibration or UncertainThe target's calibration as stored: a TargetCalibration, or an Uncertain one carrying its covariance (geom unwraps it). VisualGeometryToolkitIO's record of who calibrated a target (the tool, the date, the RMS) is a different thing with another name, the attribute Attributed(t, (; calibration_record = …)).
VisualGeometryToolkitCore.Cameras.canvas — Method
canvas(t::PlanarTarget) -> Union{Nothing, Canvas}The page the target's board is printed on (Canvas, with its printed image when it carries one), or nothing when the target does not carry one.
VisualGeometryToolkitCore.Cameras.design — Method
design(t::PlanarTarget) -> Vector{SVector{2}} or SymbolThe target's design: its nominal plane points when it is resolved (is_resolved), the reference they are resolved from (a Symbol) when it is not. The physical target is calibration(t) applied to it.
VisualGeometryToolkitCore.Cameras.is_resolved — Method
is_resolved(t) -> BoolWhether a target carries its points (a PlanarTarget holding only a design reference does not).
VisualGeometryToolkitCore.Cameras.layout — Method
layout(t::PlanarTarget) -> Union{Nothing, AbstractTargetLayout}The parametric design the target's nominal points were built from — an AprilGrid or a SchoepsPattern, given to PlanarTarget(layout; …) — or nothing for a target given by its points or by a reference. A unit move moves it with the points and a new calibration (with_parameters) keeps it; resolve, which supplies other points, drops it.
VisualGeometryToolkitCore.Cameras.placement — Method
placement(t::PlanarTarget) -> Union{Nothing, SimilarityMap{3}}Where the target's calibration chart lies in its design chart (a SimilarityMap{3}: the chart's side, the identity rotation, the translation (origin, 0), plain numbers in the target's form), or nothing when the calibration acts on the design chart itself.
VisualGeometryToolkitCore.Cameras.point_index — Method
point_index(grid::AprilGrid, tag, corner) -> IntThe 1-based index, in PlanarTarget(grid)'s points, of corner corner of tag tag (both 0-based: Kalibr's tag id less the grid's first id, and its corner order 0 (min column, min row), 1 (max column, min row), 2 (max column, max row), 3 (min column, max row)) — Kalibr's GridCalibrationTargetAprilgrid point of a detected corner.
VisualGeometryToolkitCore.Cameras.reference — Method
reference(t::PlanarTarget) -> Union{Nothing, Symbol}The content-addressed design the target's points are resolved from (a BlobBoards board uid), or nothing for a design given by its points alone.
VisualGeometryToolkitCore.Cameras.resolve — Method
resolve(t::PlanarTarget, nominal) -> PlanarTargetThe target with its design's nominal points supplied — what the design's owner returns for the reference reference(t).
The surface a ray meets
surface(pt) gives a posed planar target's geometric primitive, in the pose's destination frame (metres when physical), with the DESIGN chart as its chart — an affine calibration (the factors, Aspect ∘ Skew, …) is the primitive's in-plane affine, so a hit's chart coordinates are the nominal ones it shows — bounded by the target's Board (board = … on PlanarTarget, Rect(board) in the design chart) or else by its nominal points: a Quad for a Rigid or affine calibration, a MappedSurface for any other — a HeightField, an in-plane ThinPlateSpline, either after the affine factors — the surface of the composed chart map pose ∘ S₃ ∘ lift ∘ C ∘ S₂⁻¹, built by MappedSurface(pose, calibration, placement, rect) (a device caller builds it from adapted parts). The choice is by type, so a rigid target's ray caster never compiles the root.
On the device.HeightField, ThinPlateSpline and their ThinPlateSplineSystem take Adapt.adapt (their matrices become device arrays) and convert_eltype (a Float32 form); tps_pure_bending and tps_coefficients give a field's pure-bending values and spline coefficients from nodal values in a kernel (with_parameters on the adapted kind), the same functions the host constructors call. HeightField(nodes, points .=> heights; smoothing) fits a pure-bending field to scattered heights, the plane removed (a one-step algorithm).
No gauge freedom. The thin-plate kinds' pure bending is the OBLIQUE complement of the pose's directions (values orthogonal to the interpolant's affine rows, not to the plane fields), so no deformation direction is a pose change: the stacked Jacobian of the posed points in the deformation and the pose has full rank (test/mappedsurfacetests.jl pins its condition number and the smallest principal angle between the two spans for each composition). The spans can still come close: with nodes reaching past the detections the nearest bending mode is a nearly constant bowl, almost a piston on the data, about 5° from the pose's span (a variance inflation of about 110× along that mode); a node layout covered by detections and the bending regularizer control it. See The Surface a Ray Meets, where surface is documented with the primitives.
Index
VisualGeometryToolkitCore.Cameras.TARGET_CALIBRATION_ORDERVisualGeometryToolkitCore.Cameras.TARGET_CALIBRATION_TAGSVisualGeometryToolkitCore.Cameras.TARGET_LAYOUT_TAGSVisualGeometryToolkitCore.Cameras.AbstractTargetVisualGeometryToolkitCore.Cameras.AbstractTargetLayoutVisualGeometryToolkitCore.Cameras.AprilGridVisualGeometryToolkitCore.Cameras.AspectVisualGeometryToolkitCore.Cameras.CameraCalibrationMatVisualGeometryToolkitCore.Cameras.FrameTransitionVisualGeometryToolkitCore.Cameras.FramedAffineVisualGeometryToolkitCore.Cameras.FramedAffineVisualGeometryToolkitCore.Cameras.FramedChainVisualGeometryToolkitCore.Cameras.FramedDistortVisualGeometryToolkitCore.Cameras.FramedEquidistantProjectionVisualGeometryToolkitCore.Cameras.FramedEquisolidProjectionVisualGeometryToolkitCore.Cameras.FramedLensProjectionVisualGeometryToolkitCore.Cameras.FramedOrthographicProjectionVisualGeometryToolkitCore.Cameras.FramedPerspectiveVisualGeometryToolkitCore.Cameras.FramedProjectionVisualGeometryToolkitCore.Cameras.FramedStageVisualGeometryToolkitCore.Cameras.FramedStereographicProjectionVisualGeometryToolkitCore.Cameras.FramedUndistortVisualGeometryToolkitCore.Cameras.HeightFieldVisualGeometryToolkitCore.Cameras.HeightFieldVisualGeometryToolkitCore.Cameras.HeightFieldVisualGeometryToolkitCore.Cameras.InverseRayMapVisualGeometryToolkitCore.Cameras.PlanarCalibrationVisualGeometryToolkitCore.Cameras.PlanarTargetVisualGeometryToolkitCore.Cameras.PointTargetVisualGeometryToolkitCore.Cameras.RayMapVisualGeometryToolkitCore.Cameras.RigidVisualGeometryToolkitCore.Cameras.ScaleVisualGeometryToolkitCore.Cameras.SchoepsPatternVisualGeometryToolkitCore.Cameras.SkewVisualGeometryToolkitCore.Cameras.TargetCalibrationVisualGeometryToolkitCore.Cameras.TargetManifoldVisualGeometryToolkitCore.Cameras.ThinPlateSplineVisualGeometryToolkitCore.Cameras.ThinPlateSplineVisualGeometryToolkitCore.Cameras.boardVisualGeometryToolkitCore.Cameras.calibrationVisualGeometryToolkitCore.Cameras.calibration_stageVisualGeometryToolkitCore.Cameras.calibration_tagVisualGeometryToolkitCore.Cameras.canvasVisualGeometryToolkitCore.Cameras.designVisualGeometryToolkitCore.Cameras.distort_stageVisualGeometryToolkitCore.Cameras.distortion_frameVisualGeometryToolkitCore.Cameras.frame_transitionVisualGeometryToolkitCore.Cameras.imaging_chainVisualGeometryToolkitCore.Cameras.is_affineVisualGeometryToolkitCore.Cameras.is_planarVisualGeometryToolkitCore.Cameras.is_pure_bendingVisualGeometryToolkitCore.Cameras.is_resolvedVisualGeometryToolkitCore.Cameras.iwasawa_parametersVisualGeometryToolkitCore.Cameras.iwasawa_parameters_with_jacobianVisualGeometryToolkitCore.Cameras.layoutVisualGeometryToolkitCore.Cameras.length_maskVisualGeometryToolkitCore.Cameras.membersVisualGeometryToolkitCore.Cameras.placementVisualGeometryToolkitCore.Cameras.planar_kindVisualGeometryToolkitCore.Cameras.point_indexVisualGeometryToolkitCore.Cameras.ray_mapVisualGeometryToolkitCore.Cameras.referenceVisualGeometryToolkitCore.Cameras.regularizerVisualGeometryToolkitCore.Cameras.resolveVisualGeometryToolkitCore.Cameras.similarity_scaleVisualGeometryToolkitCore.Cameras.undistort_stageVisualGeometryToolkitCore.Cameras.with_parameters