Lenses API
The lens families — two lens kinds, the projections, the terms and their combinators — the lens contract on rays with everything derived from it, the coefficient chart and record, and the batch surfaces. convert_eltype rebuilds a lens, a LogicalIntrinsics, a CameraModel or a grid model in another float type (Float32 for a GPU launch); it is the conversion the batched and fused kernels apply. See the Lens Families and Back-projection and Undistortion guides.
Projection and Lens Models
VisualGeometryToolkitCore.Cameras.AbstractDistortionTerm — Type
AbstractDistortionTerm <: CoordinateTransformations.TransformationA distortion term in a projection's frame: a MAP x ↦ f(x) (Radial, Tilt, FieldOfView) or a DELTA x ↦ Δ(x) added to the point (Tangential, ThinPrism) — see Additive. A term is a CoordinateTransformations Transformation, callable on a frame point (t(x), a delta summed with the point, as a lone Additive); terms applied in sequence are its ∘, D₂ ∘ D₁ (D₁ first).
VisualGeometryToolkitCore.Cameras.AbstractLensModel — Type
AbstractLensModelBase type for all lens models.
A lens model determines how 3D rays through the optical center map to points on the sensor, including both the projection geometry and any aberrations.
Interface — the lens contract is on rays
A lens is a map between RAYS and NORMALIZED image points, and the pair project(lens, X) (a ray → its image point) and backproject(lens, xd) (an image point → its unit ray) is the whole contract (geometry/cameras/lens_rays.jl): NaN where the lens does not image, generic in the number type, allocation-free. There are two kinds of lens: a forward Lens — a projection onto a frame followed by a distortion in that frame — and a backward LiftLens, whose image point lifts to its ray in closed form (lenses.jl). Everything else is derived from the pair, for every lens: transform_deriv(lens, X), transform_deriv_params(lens, X), project_with_jacobians, the radial profile (projected_radius / view_angle), and the perspective views distort_normalized / undistort_normalized with the functor lens(x).
A lens is a CoordinateTransformations.Transformation, so its derivative is that package's transform_deriv(lens, x) (ForwardDiff through the perspective view), and its back-projection is a map too: inv(lens) is an InverseLens whose call is undistort_normalized and whose transform_deriv is ForwardDiff through it. Compositions inherit both — transform_deriv(K ∘ inv(lens) ∘ K⁻¹, x) is the chain rule. The other generic defaults are imaging_transform(intrinsics, lens) and the capability trait has_distortion(lens). Every lens model here is a central projection (single center of projection); finite-aperture models are downstream.
Subtyping CoordinateTransformations.Transformation makes every imaging chain a ComposedTransformation regardless of lens (the Pinhole and distorted paths used to produce DIFFERENT composition types, breaking structural pattern-matches like pnp_pixel_whitening).
VisualGeometryToolkitCore.Cameras.AbstractProjection — Type
AbstractProjectionA ray → frame-point projection: the first stage of a forward Lens. The five fixed projections (Perspective, Equidistant, Stereographic, Equisolid, Orthographic) each land on a retinal frame (projection_frame); the parametric ones (DoubleSphere, Unified, ExtendedUnified) carry coefficients and land on a frame their coefficients define.
VisualGeometryToolkitCore.Cameras.Additive — Type
Additive(m, Δ₁, …, Δₙ)
Additive(Δ₁, …, Δₙ)Terms evaluated at the SAME point and summed: m(x) + (Δ₁(x) + … + Δₙ(x)), the deltas left to right; without a leading map, x + (Δ₁(x) + …). OpenCV's projectPoints and COLMAP's Distortion() are of this form.
VisualGeometryToolkitCore.Cameras.Division — Type
Division(k₁, …, k_Q)The division model (Fitzgibbon 2001): x_u = x_d / (1 + k₁ρ² + … + k_Qρ²Q), the lift (x_d, 1 + k₁ρ² + …): LiftLens(Radial((), (k₁, …))).
VisualGeometryToolkitCore.Cameras.DoubleSphere — Type
DoubleSphere(xi, alpha) <: AbstractProjectionUsenko, Demmel & Cremers' double-sphere projection (3DV 2018): with d₁ = ‖X‖, z_s = ξ·d₁ + X₃ and d₂ = √(X₁² + X₂² + z_s²), x = X₁₂ / (α·d₂ + (1 − α)·z_s). ξ = α = 0 is the perspective projection. It images the rays out to the fold (or pole) of its radial profile, in closed form — not the paper's conservative field-of-view bound — and the closed-form inverse has the same domain. The coefficients' domain is ξ > −1, α ∈ [0, 1] (parameter_bounds), which LensManifold keeps.
VisualGeometryToolkitCore.Cameras.Equidistant — Type
Equidistant projection r = θ, onto the EquidistantFrame; θ < 180°.
VisualGeometryToolkitCore.Cameras.Equisolid — Type
Equisolid-angle projection r = 2 sin(θ/2), onto the EquisolidFrame; θ ≤ 180°.
VisualGeometryToolkitCore.Cameras.Even — Type
The power basis of a Radial factor: Even — ρ², ρ⁴, … (the default); Full — ρ², ρ³, ρ⁴, ….
VisualGeometryToolkitCore.Cameras.ExtendedUnified — Type
ExtendedUnified(alpha, beta) <: AbstractProjectionThe enhanced unified camera model of Khomutenko et al. (2016) (Kalibr's eucm): with d = √(β(X₁² + X₂²) + X₃²), x = X₁₂ / (α·d + (1 − α)·X₃).
VisualGeometryToolkitCore.Cameras.FieldOfView — Type
FieldOfView(omega)The field-of-view model of Devernay and Faugeras (2001) (COLMAP FOV, Kalibr fov): x ↦ x·atan(2r·tan(ω/2)) / (r·ω).
VisualGeometryToolkitCore.Cameras.Full — Type
The power basis of a Radial factor: Even — ρ², ρ⁴, … (the default); Full — ρ², ρ³, ρ⁴, ….
VisualGeometryToolkitCore.Cameras.KannalaBrandt — Type
KannalaBrandt(k₁, …, k_N)Kannala & Brandt's fisheye (2006) in OpenCV's/COLMAP's form, θ_d = θ·(1 + k₁θ² + … + k_Nθ²ᴺ): Lens(Equidistant(), Radial((k₁, …, k_N))).
VisualGeometryToolkitCore.Cameras.Lens — Type
Lens(projection[, distortion]) <: AbstractLensModelA forward lens: the ray is projected onto projection's frame and the frame point is distorted by distortion (nothing, a term, an Additive or terms in sequence, CoordinateTransformations' D₂ ∘ D₁ — each stage a map term or an Additive). Projection is closed form; back-projection inverts the distortion (backproject).
A composed distortion is stored right-associated, whatever association it was built in: Lens(p, D₃ ∘ D₂ ∘ D₁) (Base folds ∘ to the left) and Lens(p, D₃ ∘ (D₂ ∘ D₁)) are one value, with distortion the latter, the form lens_from_record builds.
VisualGeometryToolkitCore.Cameras.LiftLens — Type
LiftLens(radial::Radial) <: AbstractLensModelA backward lens: the image point x_d lifts to the ray (x_d·N(ρ), D(ρ)), ρ = |x_d| — closed-form back-projection; projection solves the image radius as a root on the ray. D crossing zero is the 90° circle and D < 0 the field behind it, so the lens images past 90°.
VisualGeometryToolkitCore.Cameras.Orthographic — Type
Orthographic projection r = sin θ, onto the OrthographicFrame; θ ≤ 90°.
VisualGeometryToolkitCore.Cameras.Perspective — Type
Perspective projection x = X₁₂/X₃ (r = tan θ), onto the GnomonicFrame; θ < 90°.
VisualGeometryToolkitCore.Cameras.Pinhole — Type
The ideal perspective lens, Lens(Perspective()).
VisualGeometryToolkitCore.Cameras.Radial — Type
Radial{B, P, Q, T}(k::NTuple{P}, d::NTuple{Q})
Radial(k, d = ()) # B = Even
Radial{Full}(k, d)The radial factor F(ρ) = N(ρ)/D(ρ) with N = 1 + Σ kᵢ φᵢ(ρ) and D = 1 + Σ dⱼ φⱼ(ρ), where the basis B sets φᵢ: Even — ρ²ⁱ; Full — ρⁱ⁺¹ (no linear term: the factor is smooth on the axis).
In a Lens it is the map x ↦ F(|x|)·x in the projection's frame — OpenCV's rational radial factor (k₁…k₃ / k₄…k₆) in the perspective frame, Kannala–Brandt's θ-polynomial in the equidistant frame. In a LiftLens it is the undistortion of the image point, x_u = F(ρ_d)·x_d, represented as the ray (x_d·N, D) — the division model, BabelCalib's rational model, Scaramuzza's polynomial.
Coefficient names: k1…kP then d1…dQ.
The constructor also stores where the factor's imaged branch ends, in the basis variable u (ρ² for Even): u_fold, the first positive root of the profile's derivative (the fold), and u_end, the smaller of the fold and the pole (the first positive root of D); Inf when there is none. They are not parameters, and a per-point branch test is one comparison against them; every other reader of the branch's end (monotone_radius, view_angle_limit, the inverses' brackets) reads them too.
The ends follow from the coefficients in the precision they were built in. A WIDENING rebuild — convert_eltype to a wider type, the Duals of a parameter Jacobian and their primal — casts them instead of sweeping again, so a Float32 factor seen at Float64 keeps its own branch (and primal_value of a Float64 differentiation of a Float32 lens is _precision(Float64, lens)); such a factor can differ from one constructed at Float64 from the same coefficients by the narrow precision's rounding of the end. A narrowing conversion recomputes them. An integer factor is not a supported stored form: the outer constructors and every rebuild float their coefficients, and only a direct call of the inner Radial{B, P, Q, <:Integer} reaches one; it constructs (the ends stored rounded down, typemax for Inf) but is internal only — the lens contract and the branch readers (_branch_end) take float factors.
VisualGeometryToolkitCore.Cameras.Scaramuzza — Type
Scaramuzza(a₂, …, a_{N+1})Scaramuzza's omnidirectional model (Scaramuzza, Martinelli & Siegwart 2006; MATLAB's fisheyeIntrinsics), normalized: the lift (x_d, 1 + a₂ρ² + a₃ρ³ + …) — LiftLens(Radial{Full}((), (a₂, …))). The coefficients are named d1, d2, … for a₂, a₃, ….
VisualGeometryToolkitCore.Cameras.Stereographic — Type
Stereographic projection r = 2 tan(θ/2), onto the StereographicFrame; θ < 180°.
VisualGeometryToolkitCore.Cameras.Tangential — Type
Tangential(p1, p2)Brown's decentering delta in OpenCV's convention: Δ = (2p₁uv + p₂(r² + 2u²), 2p₂uv + p₁(r² + 2v²)).
VisualGeometryToolkitCore.Cameras.ThinPrism — Type
ThinPrism{N, T}(sx::NTuple{N}, sy::NTuple{N})The thin-prism delta Δ = (Σ sxᵢ r²ⁱ, Σ syᵢ r²ⁱ): OpenCV's (s₁, s₂; s₃, s₄) is N = 2, COLMAP's (sx₁; sy₁) is N = 1.
VisualGeometryToolkitCore.Cameras.Tilt — Type
Tilt(tx, ty)OpenCV's tilted-sensor map (CALIB_TILTED_MODEL, τx, τy): the projective map of computeTiltProjectionMatrix(τx, τy) applied to (x, 1).
VisualGeometryToolkitCore.Cameras.Unified — Type
Unified(xi) <: AbstractProjectionThe unified camera model (Geyer & Daniilidis; Mei & Rives 2007; Kalibr's omni, OpenCV's omnidir): x = X₁₂ / (X₃ + ξ·‖X‖).
VisualGeometryToolkitCore.Cameras.isdelta — Method
isdelta(term) -> BoolIs the term a delta (added to the point by Additive) rather than a map?
VisualGeometryToolkitCore.Cameras.projection_frame — Method
projection_frame(projection) -> Type{<:RetinalFrame} | nothingThe retinal frame a fixed projection lands on; nothing for a parametric projection, whose frame its coefficients define.
VisualGeometryToolkitCore.Cameras.radial — Method
radial(lens) -> Radial | nothingThe lens's radially symmetric factor: a lone Radial distortion, the leading Radial of an Additive, the lift of a LiftLens; nothing otherwise. Its folds and poles bound what the lens images (monotone_radius, view_angle_limit), and it is what a regression onto a radially symmetric family fits.
VisualGeometryToolkitCore.Cameras.InverseLens — Type
InverseLens{L} <: CoordinateTransformations.TransformationThe back-projection of a lens as a first-class map: inv(lens) returns one, inv(inv(lens)) === lens, its call is undistort_normalized(lens, xd), and its transform_deriv is ForwardDiff through that map (lensrays.jl). Composes like any transformation, so the pixel-domain undistortion of a camera is `inv(distortionmap(model))with the chain rule for its derivative. APinhole` is its own inverse.
VisualGeometryToolkitCore.Cameras.has_distortion — Method
has_distortion(lens) -> BoolDoes the lens bend rays at all? (false ⇒ undistort_normalized is the identity.) Relation to the framed imaging chain (0.10): has_distortion(lens) == false means the distortion is the identity in the lens's defining frame, so imaging_chain COLLAPSES the ray frame onto DistortedFrame — the chain has no distortion stage and K consumes the ray frame directly (see the collapse rule at DistortedFrame and distortion_frame). Every frame here assumes a single center of projection.
VisualGeometryToolkitCore.Cameras.imaging_transform — Method
imaging_transform(intrinsics, lens) -> ComposedTransformation
imaging_transform(model::CameraModel)The composed camera-space → pixel-space map as a value: affine_part(K) ∘ lens, the lens applied to the ray (its functor is project on 3-vectors). It is project(model, X) for every lens and every ray — past 90° for a lens that images there, NaN where the lens does not (for a Pinhole, every Z ≤ 0). Its derivative is the chain rule through transform_deriv(lens, X). The fully-typed, junction-checked counterpart is imaging_chain, which mints the framed composite {ViewFrame{id} → ImageFrame{id}} through the frame chain documented at DistortedFrame.
VisualGeometryToolkitCore.Cameras.monotone_radius — Method
monotone_radius(lens) -> RealThe distorted radius up to which the lens's perspective view is a monotone bijection — the bound a whole-extent perspective undistortion clips to (undistorted_rect, valid_rect, division_bounded). Beyond it the view folds or diverges and project answers NaN. The profile's first fold or pole, whichever comes first, mapped to the distorted radius; both are found exactly, as first positive polynomial roots. Inf for a lens whose profile never folds. A point EXACTLY at this radius can be NaN (it rounds onto the closed end of the branch); a caller that needs a finite value steps inside, as the rect builders' margin does.
- the radial factor (
radial) of a forward lens: the fold or pole of its profiler ↦ r·F(r²)in the projection's frame, capped at the frame's edge (a ray cannot leave the sphere); decentering terms do not enter, as inview_angle_limit. - a
LiftLens: the first pole ofD(the 90° circle) or fold of the lift —1/√|k|forDivision(k). - a composed distortion (CoordinateTransformations'
∘of stages): its FIRST radial stage (aRadial, or anAdditiveled by one), wherever it sits — the stageprojecttests — read as above at that stage's output; the stages around it do not enter (for a stage behind a tilt, the frame's edge caps the stage's input). - a
SplineLens: the largest normalized radius aboutx = 0(its gauge's principal point, the pixel imaging ẑ) whose disc lies inside the lens's support — its backward area and the lattice cells whose 4 × 4 window holds present nodes only, where the grid is defined (and, for a calibrated grid, one-to-one);0whenx = 0is outside it.
VisualGeometryToolkitCore.Cameras.radial_part — Method
radial_part(lens) -> AbstractLensModelThe radially symmetric part of lens: its projection with only its radial factor (radial) — the decentering terms dropped; a lift is its own radial part. What view_angle_limit bounds and what a regression onto a radially symmetric family can represent.
VisualGeometryToolkitCore.Cameras.view_angle_limit — Method
view_angle_limit(lens; cap) -> θ_maxThe largest view angle the lens's radial profile honestly represents: where the profile stops being finite and strictly increasing (a fold, a pole, or the frame's own edge). Closed forms where the lens has one — the fold or pole of a forward lens's radial factor read through its frame, never past the frame's edge (π/2 − 10⁻³ for the perspective frame); a lift's fold; the double sphere's FOV bound — and a monotonicity scan of projected_radius otherwise. A composed distortion (∘) with a radial stage is bounded by that stage, tested at its own input as project tests it: the angle where project first turns NaN, bisected to rounding. When a stage precedes the radial one (a tilt) that angle depends on the azimuth, and the limit is its minimum over azimuths, so every ray within it is imaged; a lens whose bound is radially symmetric reads it at azimuth 0. cap bounds the answer: the generic scan runs to it (default π − 10⁻³); the closed forms clamp to it (default none). A ray projected EXACTLY at the limit can be NaN (a lift's root there is √eps-conditioned); a caller that needs a finite value steps inside, as the valid domain's disc radius and the spline lattice's sampler do.
VisualGeometryToolkitCore.Cameras.AbstractPanoramaProjection — Type
AbstractPanoramaProjection <: CoordinateTransformations.TransformationA projection of directions onto a (longitude, vertical) rectangle, the cylindrical family: Equirectangular, Cylindrical, CylindricalEqualArea, Mercator. The call p(X) is project(p, X) (its transform_deriv(p, X) the 2 × 3 Jacobian), and inv(p) is its Backprojection (inv(p)(q) == backproject(p, q), inv(inv(p)) === p). All are kernel-lawful (singletons, no allocation, no throw), so a chain built from them runs in image_warp! and pointwise! on any backend.
VisualGeometryToolkitCore.Cameras.Backprojection — Type
Backprojection(m) <: CoordinateTransformations.TransformationThe back-projection of m as a map: its call is backproject(m, q) and transform_deriv(b, q) its 3 × 2 Jacobian (a CameraModel's or a NoncentralSplineModel's own, last(backproject_with_jacobian(m, q)); any other model's — a panorama projection, a BoundedCamera — by ForwardDiff through its backproject). m is a panorama projection (inv(p) builds it, inv(Backprojection(p)) === p) or a camera model, a pixel → ray map: Backprojection(model) of a CameraModel(...; lens = Lens(Perspective())) is a rectilinear virtual camera, of Lens(Equidistant()) or Lens(Stereographic()) a fisheye one, and LinearMap(R) ∘ Backprojection(model) aims it. Kernel-lawful when m's backproject is.
VisualGeometryToolkitCore.Cameras.Cylindrical — Type
Cylindrical()The central cylindrical projection of directions onto the unit cylinder about the up axis, in the camera frame's axes (x right, y down, z forward, up = −y): the longitude about the axis, and the height h = tan φ at which the ray through the origin meets the cylinder of radius 1 (φ the latitude). A cylindrical panorama raster is an affine map of (λ, h): its columns are the longitude, its rows the height, so verticals stay vertical and straight.
project(Cylindrical(), X)— the directionX(any 3-vector off the up axis; its length does not enter) toSVector(λ, h): the longitudeλ = atan(X₁, X₃) ∈ (−π, π](0 forward, +π/2 right) andh = −X₂ / √(X₁² + X₃²)(+ up). NaN for a zero, non-finite or polar (X₁ = X₃ = 0) vector: the poles have no height.backproject(Cylindrical(), p)—p = (λ, h)to the unit direction(sin λ, −h, cos λ) / √(1 + h²), defined for every finitep(periodic inλ).
Its longitude is Equirectangular's, and h = tan φ of its latitude. An AbstractPanoramaProjection.
VisualGeometryToolkitCore.Cameras.CylindricalEqualArea — Type
CylindricalEqualArea()Lambert's cylindrical equal-area projection of directions, in the camera frame's axes (x right, y down, z forward, up = −y): the longitude and v = sin φ of the latitude — the height at which the ray's parallel meets the unit cylinder when projected ALONG the planes of constant height (orthogonally onto the axis' cylinder), so equal areas of the raster are equal solid angles (dΩ = cos φ dφ dλ = dv dλ): a uniform raster's pixels all subtend the same solid angle.
project(CylindricalEqualArea(), X)—X(any nonzero 3-vector; its length does not enter) toSVector(λ, v),λ = atan(X₁, X₃)andv = −X₂ / ‖X‖ ∈ [−1, 1]. NaN for a zero or non-finite vector; the poles arev = ±1withatan's longitude.backproject(CylindricalEqualArea(), p)—p = (λ, v)to the unit direction(√(1 − v²) sin λ, −v, √(1 − v²) cos λ); NaN for|v| > 1.
VisualGeometryToolkitCore.Cameras.Equirectangular — Type
Equirectangular()The equirectangular (latitude/longitude) projection of directions, in the camera frame's axes (x right, y down, z forward, up = −y).
project(Equirectangular(), X)— the directionX(any nonzero 3-vector; its length does not enter) toSVector(λ, φ)in radians: the longitudeλ = atan(X₁, X₃) ∈ (−π, π](0 forward, +π/2 right) and the latitudeφ = atan(−X₂, √(X₁² + X₃²)) ∈ [−π/2, π/2](+π/2 up). NaN for a zero or non-finite vector. At the poles the longitude isatan's value of the remaining components (0 for an exact pole).backproject(Equirectangular(), p)—p = (λ, φ)to the unit direction(cos φ sin λ, −sin φ, cos φ cos λ), defined for everyp(periodic inλ).
A panorama raster is an affine map of (λ, φ) to pixels, and a rotation R with d_world = R·d puts the projection in another frame: project(Equirectangular(), R' * d_world). An AbstractPanoramaProjection.
VisualGeometryToolkitCore.Cameras.Mercator — Type
Mercator()The Mercator projection of directions, in the camera frame's axes (x right, y down, z forward, up = −y): the longitude and v = asinh(tan φ) of the latitude — the conformal member of the cylindrical family (angles are kept locally; the scale grows as 1/cos φ).
project(Mercator(), X)—X(any 3-vector off the up axis; its length does not enter) toSVector(λ, v),λ = atan(X₁, X₃)andv = asinh(−X₂ / √(X₁² + X₃²)). NaN for a zero, non-finite or polar vector.backproject(Mercator(), p)—p = (λ, v)to the unit direction(sin λ / cosh v, −tanh v, cos λ / cosh v), defined for every finitep.
VisualGeometryToolkitCore.Cameras.RegionBeyondSupport — Type
RegionBeyondSupport(who, region, support) <: ExceptionWhat check_region_support throws: who (the fit that refused) was asked to work over region, which reaches beyond support, the source camera's calibrated support (its valid_domain) — a fit there would read the model where it is extrapolated.
VisualGeometryToolkitCore.Cameras.check_region_support — Method
check_region_support(camera, region; who::AbstractString) -> region
check_region_support(support, region; who::AbstractString) -> regionTHE guard of every fit over a camera's region (optical_axis, approximate_camera, refine_camera, …): region back when it lies inside the source's calibrated support — valid_domain(camera), or a support region given directly (a source camera's, when the camera fitted from is a re-gridded copy whose own domain no longer records it) — and a RegionBeyondSupport naming who, the region and the support otherwise. Never a warning: a fit over an extrapolated model is refused. Containment is region ⊆ support, Core's one region containment over a Rect, an Ellipse, a PolygonRegion (a decentred lens's domain), their ∩ and the WholePlane.
VisualGeometryToolkitCore.Cameras.farthest_radius — Method
farthest_radius(region, center) -> Float64
farthest_radius(region, center, direction) -> Float64How far region reaches from the point center, in the region's units (pixels for an imaged_region).
farthest_radius(region, center)is the largest distance fromcenterto a point of the region. ARecthas the exact value, its farthest corner. Any other region uses the second form on 720 azimuths, refines the largest by Brent's method between its neighbours (grid_brent_minimum, Core's one 1-D minimizer) and returns the largest value it evaluated.farthest_radius(region, center, direction)is the largesttwithcenter + t·directionin the region, bisected to rounding, in units of‖direction‖. A direction through a linear map,A·d, gives the radius in the coordinates thatAmaps to pixels.
The region is any bounded 2-D region that answers p ∈ region: a Rect, an Ellipse, a Circle, a PolygonRegion, or their ∩. It must contain center and be star-shaped about it, which every intersection of convex parts containing center is (and a polygon valid domain about its distortion centre). A Rect's farthest point is a corner from any center, inside or not (an image's farthest corner from a distortion centre: division_bounded).
The sensor clips the result. An imaged_region is the sensor's Rect intersected with the lens coverage, so an image circle larger than the sensor, or cut by it, reaches only as far as the sensor does.
The whole plane has no farthest point, so it is an ArgumentError (from its Rect2). So is a center outside any other region.
r = imaged_region(camera)
farthest_radius(r, principal_point) # a radial profile's domain: ρ ≤ 1 on every imaged pixelVisualGeometryToolkitCore.Cameras.image_circle — Method
image_circle(camera, sensor::Sensor, diameter::Quantity) -> Ellipse{Float64}
image_circle(camera::BoundedCamera, diameter::Quantity) -> Ellipse{Float64}The image circle of a lens: the region of the sensor it lights, from a lens datasheet's image-circle diameter, a length, carried into pixels by the sensor's pitch — semi-axes diameter / (2 · pitch.width) and diameter / (2 · pitch.height) px about the principal point, axis-aligned, in the camera's pixel frame (the frame of its calibration matrix K; Rect(extent) is the COLMAP-convention extent of the same frame). The datasheet circle is physical, on the sensor plane, and the pitch defines the pixel axes, so K's skew and focal lengths do not enter. A DimensionError when diameter is not a length. A BoundedCamera bounded by its physical Sensor takes the two-argument form; one bounded by a bare extent is an ArgumentError (it has no pitch).
The image circle is never predicted from the lens model: where the model's formula is valid is valid_domain, a different region. A measured circle (an Ellipse segmented from images) serves wherever this one does. A SplineLens camera's circle is centred on its gauge principal point, the pixel imaging ẑ. Where the camera images is imaged_region(camera).
cam = BoundedCamera(CameraModel(K; lens = KannalaBrandt(-0.135)), sensor("Sony", "IMX250"))
q = image_circle(cam, 6.0mm) # from the datasheet, through the 3.45 μm pitch
Rect(extent(cam)) ∈ q # is the whole sensor lit?VisualGeometryToolkitCore.Cameras.imaged_region — Method
imaged_region(camera::BoundedCamera) -> regionThe region of the sensor the camera images, in pixels in its pixel frame: the sensor Rect(extent(camera)) ∩ its lens_coverage (where light reaches the sensor: the lens's image circle, measured or from a datasheet's image_circle, or the whole sensor) ∩ the valid_domain of its model (where the lens formula holds, or where a grid model is defined). The one place the three are combined, so a consumer asks the camera and never carries an image circle of its own. A lens coverage larger than the sensor, or cut by it, is clipped to the sensor.
A lazy RegionIntersection, or a Rect (the sensor's, in its eltype) when the whole sensor is covered and the model has no limit (∩ returns it). The pixel mask of any residual is u ∈ imaged_region(camera); the free parameters of a local model are free_nodes(model, imaged_region(camera)).
VisualGeometryToolkitCore.Cameras.pixel_centre — Method
pixel_centre([T = Float64,] I::CartesianIndex{2}) -> SVector{2, T}The centre of the pixel at the array index I = (row, col) (Julia's 1-based indices, img[row, col]) in the COLMAP frame, (col − ½, row − ½), in T: the one conversion from an image array's index to a pixel point. Allocation- and branch-free, so a device functor over CartesianIndices calls it at the kernel's element type.
VisualGeometryToolkitCore.Cameras.pixel_centres — Method
pixel_centres(region, us, vs) -> Vector{SVector{2, Float64}}The COLMAP centres (u − ½, v − ½) of the pixels (u, v), u ∈ us, v ∈ vs (Julia's 1-based pixel indices, as pixel_mask indexes them), that lie in region, v outer and u inner: the pixels of a sub-lattice (a stride, an offset) without the full image's mask. pixel_mask(region, W, H)[v, u] is whether pixel (u, v)'s centre is among pixel_centres(region, 1:W, 1:H).
VisualGeometryToolkitCore.Cameras.pixel_mask — Method
pixel_mask(region, frame_map, width, height) -> BitMatrixThe pixels of a width × height image whose centres, mapped by frame_map into the frame region is stated in, lie in it: mask[v, u] is frame_map((u − ½, v − ½)) ∈ region. For an image on another pixel lattice than the region's — a Bayer plane or channel against a region on the sensor (pixel_mask(imaged_region(camera), bayer_plane_map(L, c), w, h)), a window of a mosaic (raw_frame_map(m)) — the membership is decided in the region's own frame, at each pixel's mapped centre, never by mapping the region. Without a map it is pixel_mask(region, width, height) (the identity).
VisualGeometryToolkitCore.Cameras.pixel_mask — Method
pixel_mask(region, width, height) -> BitMatrixThe pixels of a width × height image whose CENTRES lie in region (any 2-D region answering p ∈ region: an imaged_region, a Rect, an Ellipse, their ∩, the WholePlane), as a height × widthBitMatrix in Julia's array layout: mask[v, u] is pixel (u, v), whose centre is the COLMAP point (u − ½, v − ½) — the conversion pixel_centres shares, made in one place.
region = imaged_region(camera) # the sensor ∩ its lens coverage ∩ where its lens model is valid
mask = pixel_mask(region, width, height) # the pixels the camera imagesVisualGeometryToolkitCore.Cameras.valid_domain — Method
valid_domain(camera) -> Union{Ellipse{Float64}, PolygonRegion, WholePlane}Where the camera's lens MODEL is valid, in pixels in the camera's own pixel frame (the frame of its calibration matrix K): the image of the cone of view angles θ ≤ θ_max =view_angle_limit(lens), on which the lens's radial profile is finite and strictly increasing, so the model is radially injective. The conic K⁻ᵀ · diag(1, 1, −ρ²) · K⁻¹ with ρ the lens's image radius at θ_max (projected_radius for a radially symmetric lens): an Ellipse centred at the principal point, a circle (equal axes) only when K is isotropic without skew and the pitch square.
This is a property of the formula, not where light falls on the sensor: a fitted lens's fold can lie well outside the lit disc, or inside the extent where the sensor is dark. Where light falls is the lens's image_circle(camera, sensor, diameter) from its datasheet, or a measured circle; a bounded camera holds either as its lens_coverage.
For a lens that is not radially symmetric — a composed distortion (∘) with a tilt stage, tangential or thin-prism terms (Brown–Conrady, OpenCV), or a lens type without the RadialSymmetry trait — the fold radius depends on the azimuth, and the domain is a PolygonRegion: the star polygon about the distortion centre (the image of the optical axis) through each azimuth's OWN fold. On each of 720 ray azimuths the image radius |project(lens, ray(θ, ψ))| is followed out in θ to where it first stops being finite and strictly increasing (refined past the 1024-sample scan), and that point goes through K. The polygon never claims past a fold: each vertex is pulled in by a relative 10⁻⁹ and by a bound on how far the fold curve can bulge inward of the chord between neighbours. Beside a jump of the fold radius between two samples (more than 10 % between neighbours: an azimuth range that does not fold beside one that does) a vertex is capped at the nearer neighbour's fold, so no chord across the jump claims past it, and none comes nearer the centre than the disc of the smallest fold (inside every fold). A discontinuity narrower than one sample's spacing (0.5°) that both neighbours miss cannot be seen. Its chords lie within R·(1 − cos(π/720)) ≈ R·10⁻⁵ of a smooth fold curve (0.012 px at a fold radius of 1300 px). Until Core 0.10.9 such a lens got the disc at the SMALLEST fold over the azimuths, which excluded the pixels between it and their own azimuth's fold.
Cost: building the polygon is about 10 ms (720 azimuth folds; the disc of a radially symmetric lens about 13 µs), and membership in it about 0.1–0.5 µs a point. Build it ONCE per model and hold it — a fit's support, an imaged_region, a bundle's camera variable — never per point or per residual.
The WholePlane when the lens has no limit of its own: view_angle_limit reports only the edge it stops at by construction — the perspective or orthographic chart's edge, or the sphere's π − 10⁻³ (an undistorted equidistant lens, a lens whose profile stays monotone, a lift that never folds). That edge is a cap, not the lens: a conic there would measure the cap constant. A lens WITH a limit — a fold, a pole, the double sphere's bound, a chain's bounded stage — always gets its conic, even one larger than the extent. The rule is generic over AbstractLensModel: any lens with a view_angle_limit (every lens has one, by the generic scan) and a project gets an answer. A PosedCamera or BoundedCamera forwards to its model.
Related radii, different quantities: horizon_radius is fx times monotone_radius, the radial factor's perspective-monotone bound (decentering ignored, fx only, Inf without a fold) — equal to this ellipse's radius for an isotropic, skew-free K and a radially symmetric folding lens; the imaging domain (where backproject turns NaN) may be wider than a whole-plane answer admits, e.g. r ≤ π·f for an undistorted equidistant lens.
cam = BoundedCamera(CameraModel(K; lens = KannalaBrandt(-0.135)), sensor("Sony", "IMX250"))
v = valid_domain(cam) # where the lens formula is injective, about the principal point
Rect(extent(cam)) ∈ v # is the model valid over the whole sensor?VisualGeometryToolkitCore.Cameras.BoundaryRule — Type
BoundaryRule(m, region; order = 2)The boundary rule of the lattice model m (a LocalSupport model whose nodes are the CartesianIndices of a 2-D lattice) under the imaged region: its interior nodes (node_mask(m, region), the parameters) and, per dependent node, the linear combination of interior nodes its CORRECTION is, extrapolated outward ring by ring (Chebyshev rings around the interior) with a zero order-th difference across the border:
order = 2(the default): the natural boundary conditionf'' = 0(Δ²c = 0), the outermost coefficient of BSplineKit's natural recombination (natural_extrapolation,c₀ = 2 c₁ − c₂) — a linear correction is extended exactly;order = 3: the curvature-preserving Δ³c = 0,c₀ = 3 c₁ − 3 c₂ + c₃, which carries the slope and the curvature across the border — a quadratic correction is extended exactly.
Per dependent node: along each lattice axis on which the order nodes inward are already defined, averaged over those axes; with none, the same along the diagonals; with none of those either, the mean of the defined neighbours (constant extrapolation). The coefficients of each dependent node sum to one. extend_controls applies the rule; boundary_rule_jacobian is its Jacobian. An ArgumentError when the region meets no node's support or order is not 2 or 3.
VisualGeometryToolkitCore.Cameras.BoundaryRule — Method
BoundaryRule(interior::AbstractMatrix{Bool}; order = 2)The boundary rule of a lattice whose interior nodes are interior (true), the rest dependent (BoundaryRule(m, region; order) with the mask given).
VisualGeometryToolkitCore.Cameras.SupportTrait — Type
SupportTrait
GlobalSupport()
LocalSupport()How a model's parameters are supported on the sensor. GlobalSupport (the default for every type): every parameter influences the whole image, so every parameter is free whatever the camera images. LocalSupport: the model is a set of nodes, each influencing a region of the sensor — it implements nodes(m) and support(m, i), and opts in with support_trait(::Type{<:MyModel}) = LocalSupport().
VisualGeometryToolkitCore.Cameras.boundary_rule_jacobian — Method
boundary_rule_jacobian(rule::BoundaryRule) -> Matrix{Float64}The Jacobian of extend_controls with respect to the interior controls, per scalar component: row = a node of the lattice (linear index), column = an interior node in linear-index order (findall(rule.interior)). An interior node's row is its unit vector, a dependent node's its coefficients; every row sums to one.
VisualGeometryToolkitCore.Cameras.cubic_bspline_footprint — Method
cubic_bspline_footprint(center, spacing) -> Rect2{Float64}The footprint of one node of a uniform cubic B-spline lattice: the closed rectangle center ± 2·spacing (two cells on each side, the 4×4 cells the node's basis function is nonzero on), in the lattice's pixel frame. center is the node's position and spacing the lattice's cell size, each a pair (x, y). The one footprint every uniform bicubic lattice model reads for its support.
VisualGeometryToolkitCore.Cameras.directions — Method
directions(m) -> AbstractMatrix{<:SVector{3}}The direction controls of a SplineLens camera (its backward spline's) or of a NoncentralSplineModel, [gy, gx]: kept as given, not normalized.
VisualGeometryToolkitCore.Cameras.extend_controls — Method
extend_controls(rule::BoundaryRule, controls, reference) -> matrixThe full lattice of controls under rule: the interior nodes of controls as given, each dependent node its reference control plus the extrapolated correction, reference[j] + Σ cₖ (controls[k] − reference[k]) over its interior terms. controls and reference are matrices of the lattice's size (the dependent entries of controls are not read) of any control type with +, - and scalar * (3-vectors, scalars, duals). With controls == reference the result is reference.
VisualGeometryToolkitCore.Cameras.free_nodes — Method
free_nodes(m, region) -> indicesThe parameters of m free under the imaged region (an imaged_region, a Rect, an Ellipse, …): for a LocalSupport model the nodes whose support meets the region, [i for i in nodes(m) if intersects(region, support(m, i))], less any node the model does not hold (a spline model's absent nodes, present_nodes); for a GlobalSupport model every parameter, Colon() — an index that selects them all (θ[free_nodes(m, region)]), the same answer node_mask gives. A consumer that needs to iterate nodes asks a LocalSupport model (support_trait(m) isa LocalSupport).
VisualGeometryToolkitCore.Cameras.grid_size — Method
grid_size(m) -> (gx, gy)The size of the control grid of a SplineLens camera or a NoncentralSplineModel, (columns, rows) — the grid_size keyword's order, the reverse of size(directions(m)). A window reports its block's.
VisualGeometryToolkitCore.Cameras.lattice_spacing — Method
lattice_spacing(m) -> (hx, hy)The cell size of a SplineLens camera's or a NoncentralSplineModel's lattice in pixels: the calibrated pixel span over its cells, (w / (gx − 3), h / (gy − 3)).
VisualGeometryToolkitCore.Cameras.node_mask — Method
node_mask(m, region) -> BitArrayfree_nodes as a mask in the shape of nodes(m) (true = free), for assembling a Levenberg–Marquardt system. For a GlobalSupport model Colon(), as free_nodes answers: an index meaning "every parameter", so θ[node_mask(m, region)] reads the same for every model.
VisualGeometryToolkitCore.Cameras.node_position — Method
node_position(m, i) -> Union{Point2, Nothing}Where node i of a model sits on the sensor, in pixels (for placing nodes); nothing when the model does not say. A contract generic with that default.
VisualGeometryToolkitCore.Cameras.nodes — Function
nodes(m) -> indicesThe node ids of a LocalSupport model — CartesianIndices of a lattice, 1:N for scattered nodes — in the shape node_mask returns. A contract generic: a node model implements it.
VisualGeometryToolkitCore.Cameras.present_nodes — Method
present_nodes(m) -> BitMatrixThe nodes a SplineLens camera or a NoncentralSplineModel holds, [gy, gx] like its controls: true = present. Every node for a model built without a present mask.
VisualGeometryToolkitCore.Cameras.support — Function
support(m, i) -> regionThe region of the sensor node i of a LocalSupport model influences, in pixels in the camera's pixel frame: a Rect for a node of a uniform B-spline lattice (cubic_bspline_footprint), WholePlane() for a global basis function. A contract generic: a node model implements it.
VisualGeometryToolkitCore.Cameras.support_trait — Method
support_trait(m) -> SupportTrait
support_trait(::Type{M}) -> SupportTraitWhether m's parameters have global or local support on the sensor (SupportTrait). GlobalSupport() unless the type opts in; the instance form forwards to the type form.
VisualGeometryToolkitCore.Cameras.window_offset — Method
window_offset(w, m) -> (row, col)The 0-based (row, col) of the first control of the window w in the control grid of m, the camera or model it was cut from (w = window(m, rows, cols): rows = row + 1 : …, cols = col + 1 : …), so a control directions(w)[i, j] is directions(m)[row + i, col + j]. Read off the two lattices' areas; an ArgumentError when w does not lie on m's lattice.
The Lens Contract, Coefficients and Batches
VisualGeometryToolkitCore.Cameras.distort_normalized — Method
distort_normalized(lens, x::StaticVector{2}) -> Vec2The image point of the ray through the perspective point x: project(lens, (x₁, x₂, 1)). The functor lens(x) is this call.
VisualGeometryToolkitCore.Cameras.project! — Method
project!(u, m, X; exec = OnBackend(get_backend(u))) -> u
backproject!(X, lens, u; exec = OnBackend(get_backend(X))) -> X
distort_normalized!(xd, lens, x; exec = OnBackend(get_backend(xd))) -> xd
undistort_normalized!(x, lens, xd; exec = OnBackend(get_backend(x))) -> xproject / backproject / distort_normalized / undistort_normalized of every point of a batch, into u / X / … — arrays of static vectors on any backend. m is a lens or anything else project takes (a camera). A point the lens does not image comes back NaN. One kernel on the output's backend, the CPU's included (pointwise!); exec launches it on another backend value.
VisualGeometryToolkitCore.Cameras.project_with_jacobians — Method
project_with_jacobians(lens, X::StaticVector{3}) -> (u, ∂u/∂X, ∂u/∂p)The image point and both Jacobians — transform_deriv(lens, X) and transform_deriv_params(lens, X) — in ONE ForwardDiff evaluation over (X, p): the entry point for a bundle's or a PnP solver's per-observation residual.
This is GTSAM's project(point, Dpose, Dpoint, Dcal), with the Jacobians RETURNED rather than written into optional output arguments; the Ceres analogue is what a CostFunction::Evaluate fills.
VisualGeometryToolkitCore.Cameras.projected_radius — Method
projected_radius(lens, θ) -> ρ_dA lens's radial projection function, read off the contract: the image radius of the ray at view angle θ (the classical r(θ) — equidistant r = θ, stereographic 2·tan(θ/2), Kannala–Brandt's θ-polynomial, …), i.e. project at azimuth 0, so it holds past 90° wherever the lens does. NaN where the lens does not image; θ ≥ π — the exact backward ray, a circle rather than a point — never is. Its inverse is view_angle.
VisualGeometryToolkitCore.Cameras.radial_ratio — Method
radial_ratio(r::Radial, u) -> N(u)/D(u)The radial factor of r at its basis variable u (ρ² for the Even basis): for a LiftLens, the undistortion x_u = x_d · radial_ratio(radial(lens), ρ_d²).
VisualGeometryToolkitCore.Cameras.undistort_normalized — Method
undistort_normalized(lens, xd::StaticVector{2}) -> Vec2The perspective point of the ray behind the image point xd — the view of backproject(lens, xd) on the plane Z = 1. NaN where the lens does not image xd, and where its ray has Z ≤ 0 (no perspective point exists).
VisualGeometryToolkitCore.Cameras.view_angle — Method
view_angle(lens, ρ_d) -> θThe view angle of the ray behind the image point at radius ρ_d — the inverse of projected_radius, read off backproject at azimuth 0. NaN where the lens does not image.
VisualGeometryToolkitCore.backproject — Method
backproject(lens::AbstractLensModel, xd::StaticVector{2}) -> Vec3The default backproject of a lens that defines only project: Newton's method on project(lens, X(m)) = x_d over the equidistant chart m = θ·(cos φ, sin φ) of the ray X(m) = (sin θ·m̂, cos θ) — so view angles past 90° are reached — started at the pinhole ray's chart point atan(|x_d|)·x̂_d, each step capped at 0.3 rad, a fixed 30 trips (kernel-safe: no throw, no allocation), the 2 × 2 Jacobian by forward-mode differentiation of project. The unit ray, or NaN unless the iterate converged (the residual within the solve tolerance of its precision), the forward map keeps orientation there (det J > 0) and θ < π. Generic in the number type: Float32 and ForwardDiff Duals.
Every lens of this package has its own inverse (closed form, or a solver that follows the imaged branch), which takes precedence; this one is for a downstream lens type, and agrees with each of them in its domain (tested to 95% of each lens's view_angle_limit; within a few percent of a fold Newton converges too slowly and the answer is NaN).
VisualGeometryToolkitCore.backproject — Method
backproject(lens, xd::StaticVector{2}) -> Vec3The UNIT ray behind the normalized image point xd — the other half of the lens contract, with project(lens, X). NaN where the lens does not image the point (outside its frame's domain, past a fold, a non-converged inverse), never a throw.
Where the in-plane distortion folds, a point has several preimages, and the answer is the one on the imaged branch: det J > 0 there and every Radial factor before its fold, reached from the axis. Two consequences:
projecttests only the radial factors, so a ray past a fold of the deltas of anAdditiveor of a composition'sTiltprojects to a finite point whosebackprojectis the imaged branch's preimage (a different ray), or NaN — never that ray.- The acceptance test is pointwise. Where the map overlaps itself INSIDE the axis-connected component, a point has several preimages on the branch and
backprojectreturns the first the solver reaches — rare (0.05 % of the interior points of deliberately strong OpenCV chains), but an undistortion map built frombackprojectmay jump between preimages from one pixel to the next there. Whether a lens is injective over a sensor is a property of the LENS, for an optional check over the sensor rectangle, not something tested per pixel.
VisualGeometryToolkitCore.backproject — Method
backproject(lens::LiftLens, xd) -> Vec3The lift (x_d·N, D), normalized: D = 0 is the 90° ray and D < 0 a ray behind it; only the Euclidean point x_d·N/D has a pole there, not the direction. Past the lift's first fold (where its view angle first stops increasing) the point is not an image of the lens and the ray is NaN — found exactly, so a point past a second fold is NaN too: the same branch project solves on.
VisualGeometryToolkitCore.project — Method
project(lens, X::StaticVector{3}) -> Vec2The NORMALIZED image point of the ray through X (any positive scale) — half of the lens contract, with backproject(lens, xd). NaN where the lens does not image the ray, never a throw; allocation-free and generic in the number type (Float32 kernels, ForwardDiff Duals).
A Lens projects the ray onto its projection's frame and distorts the frame point there; no perspective hop is involved unless the projection IS the perspective one, whose domain genuinely ends at 90°. A LiftLens solves the image radius as a root on the ray.
A ray past the fold (or pole) of a radial factor is NaN: every Radial factor is tested at its own input — a lone one, an Additive's head, each radial stage of a composed distortion (∘). The other folds are NOT tested forward, since that would cost a Jacobian per projected point: where the deltas of an Additive (tangential, thin prism) or a composition's Tilt fold the map (det J ≤ 0 inside the radial factor's disk), project returns a finite point, and backproject of that point is not the ray — it is the imaged branch's preimage, a different ray, or NaN where the branch does not reach the point. So the round trip backproject(lens, project(lens, X)) ≈ X / ‖X‖ holds on the imaged branch (the region connected to the axis where det J > 0 and every radial factor is before its fold), not past such a fold.
VisualGeometryToolkitCore.project — Method
project(lens::LiftLens, X) -> Vec2The image point of a ray under the lift, solved ON THE RAY: with (r, z) = (sin θ, cos θ) the image radius ρ is the root of h(ρ) = ρ·N·z − r·D on the lift's monotone branch. NaN past the fold, where the lens images nothing.
VisualGeometryToolkitCore.Cameras.LensManifold — Type
LensManifold(lens) <: ManifoldThe lens's ParameterManifold: parameter_vector's coordinates as offsets from the anchor lens, each kept inside its parameter_bounds by the chart that table gives (the double sphere's α ∈ [0, 1] through its logit, its ξ > −1 through a log). minimal_manifold(lens) is this manifold; restrict it to a target's free coefficients by mask or by name (restrict(ch, (:k1, :k2))).
VisualGeometryToolkitCore.Cameras.ParameterManifold — Type
ParameterManifold(x) <: ManifoldThe manifold of a model x with the parameter protocol — a lens, a LogicalIntrinsics, a parametric CameraModel (intrinsics then lens) — anchored at x: parameter_vector's coordinates, each moved by a chart that keeps it inside its parameter_bounds, the one statement of the model's domain:
| bound (open ends) | chart coordinate v of the value t (anchor t₀) |
|---|---|
| none finite | t = t₀ + v |
a | t = a + (t₀ − a)·exp(v) |
b | t = b − (b − t₀)·exp(−v) |
a and b | t = a + (b − a)·logistic(logit((t₀ − a)/(b − a)) + v) |
and a CLOSED endpoint — a value the model takes, such as α = 0 or ω = 0 (parameter_bounds lists them) — is a mirror: the open ends' chart (the plain t₀ + v when it has none) reflected at it, so a value past it folds back inside (with two closed ends, a fold of period 2(b − a)) and the slope keeps its magnitude with its sign flipped per reflection — never the flat residual of a clamp, at which a fixed-chart LM that overshot the end would stall. In range the value is the plain chart's, bit for bit. So no step leaves the domain and every fitter (VisualGeometryToolkitEstimation's fit_camera, a calibration bundle) keeps the model valid without knowing why, and an anchor at a closed endpoint moves with slope 1, not from the far end of a logit. An anchor on (or past) an OPEN end is moved inside it by a margin of 10⁻⁶ of the interval (of max(1, |a|) for a one-sided bound); an anchor past a closed end is clamped onto it; a mirrored coordinate (ω, whose sign is not part of the map) is anchored at |ω|. ω = 0 is a stationary point of the field-of-view model itself — its map is even in ω, so ∂project/∂ω = 0 there whatever the chart — so a fit of ω starts at a small ω ≠ 0. A zero chart offset is the anchor's value bit for bit, so a coordinate restrict holds fixed never drifts across recenters. restrict it to the free coefficients by mask or by name (restrict(ch, (:k1, :k2)); a camera's restrict(ch, (:f, :cx, :cy, :k1)) fixes its skew and aspect). minimal_manifold(lens) and minimal_manifold(camera) are this manifold. action_jacobian(ch, x) is the model's parameter Jacobian of its action on x through the chart: the lens's transform_deriv_params(lens, x), a camera's ∂project(camera, X)/∂θ (ForwardDiff).
LensManifold is its name for a lens.
A solver's box
ParameterManifold(x; box) adds an optimizer's search box — the solver option parameter_bounds leaves out (a prior range such as a focal length between 0.1 and 50, not a model fact): a NamedTuple of (lo, hi) intervals keyed by parameter_names, a name not listed unboxed. The box is intersected with the domain and acts at the step: recenter clamps the stepped free parameters into it (a coordinate restrict holds fixed is never clamped), so an NLLSsolver variable on the manifold (or a restriction of it) never leaves it; retract is the chart's alone. The anchor is taken as given. parameter_box reads the intersected box.
A coordinate's scale
ParameterManifold(x; scale) scales coordinates — a NamedTuple of positive factors keyed by parameter_names, a name not listed 1: the coordinate v of a parameter with scale s moves its chart by s·v (t = t₀ + s·v unbounded, t = a + (t₀ − a)·exp(s·v) bounded below, …), so retract_jacobian carries the factor and local_coordinates divides by it. The manifold is the same set; only the units of its coordinates change. An optimizer that damps every coordinate alike (λ·I) needs them comparable: a pixel camera's principal point (pixels, with scale = (; cx = W, cy = H) one coordinate per image width) next to its log-charted focal length and its unit-free lens coefficients. coordinate_scale reads it.
VisualGeometryToolkitCore.Cameras.convert_eltype — Function
convert_eltype(::Type{T}, x) where {T <: Real}x rebuilt with every stored number in T: a lens (its coefficients), a LogicalIntrinsics (K, with K⁻¹ recomputed in T), a CameraModel (both), a BoundedCamera (its camera, the bound kept), a plain EuclideanMap or RotationMap (rotation and translation, representation and frames kept), a RadialHomography (its H, coefficients and centre; T <: AbstractFloat there), a SplineLens or a NoncentralSplineModel (the splines' controls; their areas stay as stored; T <: AbstractFloat there). Use it to move a camera to Float32 before a GPU launch, or back to Float64, or to a ForwardDiff Dual for a Jacobian by forward differentiation (a Dual is a Real, not an AbstractFloat).
A lens's and a grid model's method is also what the batched and fused kernels apply once per launch, so per-element math never promotes (Metal has no Float64): a downstream lens type that adds a method here is converted by them too.
Public, not exported; a downstream camera or lens type adds its own method.
VisualGeometryToolkitCore.Cameras.coordinate_scale — Method
coordinate_scale(ch::ParameterManifold) -> SVector{N, T} or nothingThe coordinate scales of a ParameterManifold built with scale, entry by entry over parameter_vector (1 for a name the scale does not list), in the float type of the scales given (Float32 scales keep a retraction of Float32 coordinates in Float32); nothing for a manifold without one (every scale 1).
VisualGeometryToolkitCore.Cameras.free_parameter_names — Method
free_parameter_names(model, fixed) -> NTuple{N, Symbol}
free_parameter_names(names::Tuple{Vararg{Symbol}}, fixed) -> NTuple{N, Symbol}The parameters a calibration moves: parameter_names(model) (or names), in that order, less :skew and less the fixed ones. A fixed entry is a parameter name (:f, :cx, :k2, :d1) or a GROUP, a coefficient family's stem standing for all of its numbered coefficients (:k for k1…, :d for d1…, :p for the tangential terms, :sx/:sy for the thin prism's). :skew is the gauge, held always, so naming it is accepted and changes nothing (fixed = parameter_names(model) holds everything). An entry that names nothing of the model, or a parameter named twice (directly or through a group), is an ArgumentError.
LensSpec's fixed and VisualGeometryToolkitEstimation's refine_camera_and_poses resolve their fixed here.
VisualGeometryToolkitCore.Cameras.lens_from_record — Method
lens_from_record([T = Float64,] d::AbstractDict) -> AbstractLensModelThe lens of a lens_record, its coefficients in T. Unknown keys, missing or extra coefficients, and a second distortion are refused, naming the offending key.
VisualGeometryToolkitCore.Cameras.lens_label — Method
lens_label(lens) -> StringA lens's display name — a report's label, a run's name — by the tag rule: a named family is snake_case of its name ("pinhole", "kannala_brandt", "division", "scaramuzza"); any other lens is its lens_record's leaf tags in the record's order, joined by _ (a lift is lift_ and its term): Lens(DoubleSphere(ξ, α)) is "double_sphere", an OpenCV-14 lens "perspective_tilt_radial_tangential_thin_prism". A label, never read back: lens_from_record reads the record.
VisualGeometryToolkitCore.Cameras.lens_record — Method
lens_record(lens) -> Dict{String, Any}The lens as a record: {"equidistant": [], "radial": [[k₁, …], []]}, {"perspective": [], "add": [{"radial": …}, {"tangential": [p₁, p₂]}]}, {"lift": {"radial": [[], [k₁]]}} — keys are the leaves' tags, arities are array lengths, nothing is named or padded. lens_from_record inverts it exactly: written with shortest round-trip floats, a Float64 lens reads back bitwise identical.
VisualGeometryToolkitCore.Cameras.parameter_bounds — Function
parameter_bounds(x) -> (lo::SVector, hi::SVector)The interval each coefficient of parameter_vector may take, entry by entry — the model's DOMAIN, where it is defined: (-Inf, Inf) unless the model says otherwise. Any interval (a, b), a < b, either end infinite: the double sphere's ξ > −1 and α ∈ [0, 1], the unified model's ξ > −1, the extended unified model's α ∈ [0, 1] and β ≥ 0, the field-of-view term's ω ∈ [0, π), a LogicalIntrinsics's f > 0 and aspect > 0; a CameraModel's are its intrinsics' then its lens's.
An endpoint is open unless the model is defined there, as written above: the closed ones are the double sphere's α = 0 and α = 1, the extended unified model's α = 0 (the pinhole), α = 1 and β = 0, and the field-of-view term's ω = 0 (the identity). ω and −ω are one map, so a FieldOfView(ω) with ω < 0 is the lens FieldOfView(|ω|), which is how ParameterManifold anchors it.
This is the only place a bound is stated: ParameterManifold (a lens's LensManifold) reads it, so no step leaves the domain. An optimizer's box — a prior range a solver searches, such as a focal length between 0.1 and 50 — is not a model fact and is not stated here: it is the solver's option, keyed by parameter_names.
VisualGeometryToolkitCore.Cameras.parameter_box — Method
parameter_box(ch::ParameterManifold) -> (lo::SVector{N, Float64}, hi::SVector{N, Float64}) or nothingThe search box of a ParameterManifold built with box, entry by entry over parameter_vector and intersected with the model's parameter_bounds (an entry the box does not name is its domain); nothing for a manifold without one.
VisualGeometryToolkitCore.Cameras.parameter_names — Function
parameter_names(x) -> NTuple{N, Symbol}The names of parameter_vector, in its order — what a configuration refers to when it frees or fixes coefficients by name. A lens's are generated from its leaves: k1…, d1… (Radial), p1, p2, sx1…, sy1…, tx, ty, omega, xi, alpha, beta. A LogicalIntrinsics's are (:f, :aspect, :skew, :cx, :cy).
VisualGeometryToolkitCore.Cameras.parameter_offsets — Method
parameter_offsets(ch::ParameterManifold, x) -> SVectorThe chart coordinates of the model x on the parameter manifold ch of a model of the same structure: its parameters' offsets from the manifold's model (through the bounded parameters' charts, in the manifold's scale). What local_coordinates(ch, x) is for every model with the parameter protocol (parameter_vector, with_parameters); a model of another package defines local_coordinates(ch::ParameterManifold, x::ItsModel) = parameter_offsets(ch, x). A model of another structure is an ArgumentError.
VisualGeometryToolkitCore.Cameras.parameter_vector — Function
parameter_vector(x) -> SVectorThe coefficients of a model x — a lens, a calibratable target or its calibration, or a LogicalIntrinsics — in the order of parameter_names. Inverse of with_parameters.
A lens's are the projection's, then the distortion's terms depth-first, left to right — for terms in sequence, D₂ ∘ D₁, CoordinateTransformations' order, the outer D₂'s first — each term in its own documented order (Radial: k…, d…; Tangential: p1, p2; ThinPrism: sx…, sy…; …); empty for a lens without coefficients. A PlanarTarget's are its calibration's (empty for Rigid). A LogicalIntrinsics's are f, aspect, skew, cx, cy of K = [f s cx; 0 f·aspect cy; 0 0 1], absolute and in pixels.
VisualGeometryToolkitCore.Cameras.parameters — Method
parameters(x) -> NamedTupleThe model's coefficients by name: parameters(KannalaBrandt(-0.04, 0.01)).k2.
VisualGeometryToolkitCore.Cameras.with_parameters — Method
with_parameters(x, θ::AbstractVector) -> xThe model rebuilt from the coefficient vector θ (parameter_vector' inverse): the same structure and arities, the number type promoted to θ's.
VisualGeometryToolkitCore.Cameras.CameraModelSpec — Type
CameraModelSpec(lens::LensSpec; intrinsics::Type{<:AbstractIntrinsics} = LogicalIntrinsics)
CameraModelSpec(:spline; grid_size = nothing, calib_area = nothing)
CameraModelSpec(table::AbstractDict)What a calibration is for: an intrinsics kind (the type parameter I: metric LogicalIntrinsics, or the projective ProjectiveIntrinsics of a partial calibration) and a LensSpec or a SplineLensSpec (the field model). The spline form is always metric.
toml_value(spec) is the table Dict("intrinsics" => "metric" | "projective", "lens" => …, …) (a LensSpec table plus intrinsics) or Dict("type" => "spline", "grid_size" => […], "calib_area" => […]) with the unset keys left out; CameraModelSpec(table) reads it back, and intrinsics defaults to "metric" when absent (a spline table may also name it, as "metric" only). == compares the specification.
A parameterized intrinsics (LogicalIntrinsics{Float64}) names its kind: it is normalized to the wrapper (LogicalIntrinsics), so the specification round trips equal.
Throws an ArgumentError for an intrinsics other than metric or projective (and toml_value for an intrinsics kind with neither name), for a spline table whose intrinsics is not metric, for an unknown type or key (the grid's type before Core 0.9 included: it names "spline"), and for whatever LensSpec or SplineLensSpec refuses.
VisualGeometryToolkitCore.Cameras.LensSpec — Type
LensSpec(family::Symbol; fixed = (), structure...)
LensSpec(record::AbstractDict)
LensSpec(lens::AbstractLensModel)A lens model specified by family name and term counts, not by a record of zeros: the lens a calibrator starts from (see lens(::LensSpec)) and the coefficients it moves (free_names).
The families and their structure keywords:
:division,:kannala_brandt:coefficients(an integer ≥ 1), thed1…dnofDivision, thek1…knofKannalaBrandt;:brown_conrady:radial(≥ 1) andtangential(aBool, defaultfalse), thek1…kn,p1, p2of OpenCV's perspective model;:rational:numerator(≥ 0) anddenominator(≥ 1), OpenCV's rational radial factork1…k(numerator)overd1…d(denominator);:perspective,:equidistant,:stereographic,:equisolid,:orthographic,:double_sphere,:unified,:extended_unified: a projection with no distortion, no keywords.
The reference values of a family are those of the member that equals its base projection: all-zero coefficients for the polynomial families, and for the parametric projections the parameters that reduce them to perspective (DoubleSphere(0, 0), Unified(0), ExtendedUnified(0, 1)).
fixed lists the coefficients a calibrator HOLDS (the stack's one parameter-holding keyword), each by parameter_names of the lens or by a group (:k for k1…; free_parameter_names); the default () moves all of them, and free_names is the complement. free is refused, naming fixed.
The record form, LensSpec(Dict("record" => lens_record(lens))), covers every composition the table does not name, and is the only form with numbers: a lens with non-zero starting or held coefficients. LensSpec(lens) is the inverse of lens(spec): the named form only when the lens's structure has a name AND its coefficients are the reference values (compared whatever their element type, so a Float32 lens at the reference values is named; a -0.0 is not the reference 0.0), else the record form, so no value is ever dropped.
toml_value(spec) is Dict("lens" => "division", "coefficients" => 3, …) or Dict("lens" => Dict("record" => …)), plus "fixed" when not empty; LensSpec(::AbstractDict) reads it back (the keys of the table the spec sits in). == compares the specification, not the lens.
Throws an ArgumentError for an unknown family (listing the known ones), a keyword the family does not take or lacks, a count out of range, a fixed name that is not a parameter name of the lens, a retired free, and a record that is not a lens record.
VisualGeometryToolkitCore.Cameras.SplineLensSpec — Type
SplineLensSpec(; grid_size = nothing, calib_area = nothing)The lattice of a SplineLens camera (the central grid camera: one ray per lattice node, no lens family): grid_size, the (columns, rows) of the control grid, and calib_area, the (x0, y0, x1, y1) pixel rectangle the grid covers; nothing leaves either to the calibrator.
Throws an ArgumentError for a grid_size that is not two positive integers or a calib_area that is not four integers.
VisualGeometryToolkitCore.Cameras.free_names — Method
free_names(spec::LensSpec) -> NTuple{N, Symbol}The coefficients a calibrator moves, by parameter_names of lens(spec), in that order: every one but spec's fixed (free_parameter_names: an entry names a coefficient or a group, :k for k1…).
VisualGeometryToolkitCore.Cameras.intrinsics_kind — Method
intrinsics_kind(spec::CameraModelSpec) -> Type{<:AbstractIntrinsics}The intrinsics type spec calibrates: LogicalIntrinsics for a metric calibration.
Index
VisualGeometryToolkitCore.Cameras.AbstractDistortionTermVisualGeometryToolkitCore.Cameras.AbstractLensModelVisualGeometryToolkitCore.Cameras.AbstractPanoramaProjectionVisualGeometryToolkitCore.Cameras.AbstractProjectionVisualGeometryToolkitCore.Cameras.AdditiveVisualGeometryToolkitCore.Cameras.BackprojectionVisualGeometryToolkitCore.Cameras.BoundaryRuleVisualGeometryToolkitCore.Cameras.BoundaryRuleVisualGeometryToolkitCore.Cameras.CameraModelSpecVisualGeometryToolkitCore.Cameras.CylindricalVisualGeometryToolkitCore.Cameras.CylindricalEqualAreaVisualGeometryToolkitCore.Cameras.DivisionVisualGeometryToolkitCore.Cameras.DoubleSphereVisualGeometryToolkitCore.Cameras.EquidistantVisualGeometryToolkitCore.Cameras.EquirectangularVisualGeometryToolkitCore.Cameras.EquisolidVisualGeometryToolkitCore.Cameras.EvenVisualGeometryToolkitCore.Cameras.ExtendedUnifiedVisualGeometryToolkitCore.Cameras.FieldOfViewVisualGeometryToolkitCore.Cameras.FullVisualGeometryToolkitCore.Cameras.InverseLensVisualGeometryToolkitCore.Cameras.KannalaBrandtVisualGeometryToolkitCore.Cameras.LensVisualGeometryToolkitCore.Cameras.LensManifoldVisualGeometryToolkitCore.Cameras.LensSpecVisualGeometryToolkitCore.Cameras.LiftLensVisualGeometryToolkitCore.Cameras.MercatorVisualGeometryToolkitCore.Cameras.OrthographicVisualGeometryToolkitCore.Cameras.ParameterManifoldVisualGeometryToolkitCore.Cameras.PerspectiveVisualGeometryToolkitCore.Cameras.PinholeVisualGeometryToolkitCore.Cameras.RadialVisualGeometryToolkitCore.Cameras.RegionBeyondSupportVisualGeometryToolkitCore.Cameras.ScaramuzzaVisualGeometryToolkitCore.Cameras.SplineLensSpecVisualGeometryToolkitCore.Cameras.StereographicVisualGeometryToolkitCore.Cameras.SupportTraitVisualGeometryToolkitCore.Cameras.TangentialVisualGeometryToolkitCore.Cameras.ThinPrismVisualGeometryToolkitCore.Cameras.TiltVisualGeometryToolkitCore.Cameras.UnifiedVisualGeometryToolkitCore.Cameras.backproject!VisualGeometryToolkitCore.Cameras.boundary_rule_jacobianVisualGeometryToolkitCore.Cameras.check_region_supportVisualGeometryToolkitCore.Cameras.convert_eltypeVisualGeometryToolkitCore.Cameras.coordinate_scaleVisualGeometryToolkitCore.Cameras.cubic_bspline_footprintVisualGeometryToolkitCore.Cameras.directionsVisualGeometryToolkitCore.Cameras.distort_normalizedVisualGeometryToolkitCore.Cameras.distort_normalized!VisualGeometryToolkitCore.Cameras.extend_controlsVisualGeometryToolkitCore.Cameras.farthest_radiusVisualGeometryToolkitCore.Cameras.free_namesVisualGeometryToolkitCore.Cameras.free_nodesVisualGeometryToolkitCore.Cameras.free_parameter_namesVisualGeometryToolkitCore.Cameras.grid_sizeVisualGeometryToolkitCore.Cameras.has_distortionVisualGeometryToolkitCore.Cameras.image_circleVisualGeometryToolkitCore.Cameras.imaged_regionVisualGeometryToolkitCore.Cameras.imaging_transformVisualGeometryToolkitCore.Cameras.intrinsics_kindVisualGeometryToolkitCore.Cameras.isdeltaVisualGeometryToolkitCore.Cameras.lattice_spacingVisualGeometryToolkitCore.Cameras.lens_from_recordVisualGeometryToolkitCore.Cameras.lens_labelVisualGeometryToolkitCore.Cameras.lens_recordVisualGeometryToolkitCore.Cameras.monotone_radiusVisualGeometryToolkitCore.Cameras.node_maskVisualGeometryToolkitCore.Cameras.node_positionVisualGeometryToolkitCore.Cameras.nodesVisualGeometryToolkitCore.Cameras.parameter_boundsVisualGeometryToolkitCore.Cameras.parameter_boxVisualGeometryToolkitCore.Cameras.parameter_namesVisualGeometryToolkitCore.Cameras.parameter_offsetsVisualGeometryToolkitCore.Cameras.parameter_vectorVisualGeometryToolkitCore.Cameras.parametersVisualGeometryToolkitCore.Cameras.pixel_centreVisualGeometryToolkitCore.Cameras.pixel_centresVisualGeometryToolkitCore.Cameras.pixel_maskVisualGeometryToolkitCore.Cameras.pixel_maskVisualGeometryToolkitCore.Cameras.present_nodesVisualGeometryToolkitCore.Cameras.project!VisualGeometryToolkitCore.Cameras.project_with_jacobiansVisualGeometryToolkitCore.Cameras.projected_radiusVisualGeometryToolkitCore.Cameras.projection_frameVisualGeometryToolkitCore.Cameras.radialVisualGeometryToolkitCore.Cameras.radial_partVisualGeometryToolkitCore.Cameras.radial_ratioVisualGeometryToolkitCore.Cameras.supportVisualGeometryToolkitCore.Cameras.support_traitVisualGeometryToolkitCore.Cameras.undistort_normalizedVisualGeometryToolkitCore.Cameras.undistort_normalized!VisualGeometryToolkitCore.Cameras.valid_domainVisualGeometryToolkitCore.Cameras.view_angleVisualGeometryToolkitCore.Cameras.view_angle_limitVisualGeometryToolkitCore.Cameras.window_offsetVisualGeometryToolkitCore.Cameras.with_parametersVisualGeometryToolkitCore.backprojectVisualGeometryToolkitCore.backprojectVisualGeometryToolkitCore.backprojectVisualGeometryToolkitCore.projectVisualGeometryToolkitCore.project