Frames API
Coordinate-frame tags: the open Frame{N} hierarchy carried by EuclideanMap/HomographyMap (Src/Dst), by CovMat/PrecMat (the trailing F), and by the framed imaging-chain and batched stages. See the Coordinate Frames guide.
Frame Tags and Image Frames
VisualGeometryToolkitCore.BodyFrame — Type
BodyFrame{id} <: Frame{3}The 3D frame of a SPECIFIC rigid body, tagged by an instance identity: id::Symbol names which body — the ViewFrame instance-identity idiom applied to things that are tracked or mounted as a unit (a motion-capture rigid body, a camera rig, a board rig carrying several targets). A body is one frame, and a trajectory is its pose over time: EuclideanMap{..., Src = BodyFrame{id}, Dst = WorldFrame} per instant, never a new frame per instant.
What is mounted on a body is posed in it: a target on a board rig is EuclideanMap{..., Src = TargetFrame{m}, Dst = BodyFrame{:board_rig}}, a camera on a camera rig EuclideanMap{..., Src = BodyFrame{:camera_rig}, Dst = ViewFrame{id}} (extrinsics: rig coordinates into the camera's), so a hand-eye chain is junction-checked by dispatch.
See also TargetFrame, ViewFrame, WorldFrame, Frame.
VisualGeometryToolkitCore.CameraFrame — Type
CameraFrame <: Frame{3}The camera-centered coordinate frame (origin at the optical center, +Z along the optical axis in the CV convention). Source frame of a camera POSE, destination frame of camera EXTRINSICS (X_CW ≙ EuclideanMap{..., Src = WorldFrame, Dst = CameraFrame}).
VisualGeometryToolkitCore.Frame — Type
Frame{N}Abstract supertype for coordinate-frame tags of dimension N.
A frame tag names a coordinate frame (3D, or a 2D image frame) at the type level; EuclideanMap carries a source and destination frame as trailing type parameters so that composition and inversion are frame-checked by dispatch. In Drake monogram notation, X_AB (pose of B in A, maps B-coordinates to A-coordinates) corresponds to EuclideanMap{..., Src = B, Dst = A}.
The hierarchy is open: downstream packages declare their own frames (struct LidarFrame <: Frame{3} end) and participate in frame-checked composition with zero changes to VisualGeometryToolkitCore.
See also UnlabeledFrame, WorldFrame, CameraFrame, TargetPlaneFrame, PixelFrame, unlabel, relabel.
VisualGeometryToolkitCore.TargetFrame — Type
TargetFrame{id} <: Frame{3}The 3D body frame of a SPECIFIC calibration target (board) instance, tagged by an instance identity: id::Symbol names which target — the ViewFrame instance-identity idiom applied to the target side of a calibration. Target-local geometry lives in this frame (for planar boards, the pattern plane is z = 0); the target's POSE in a common frame F is EuclideanMap{..., Src = TargetFrame{id}, Dst = F} (target-local coordinates into F).
For content-addressed targets (e.g. BlobBoards board uids) the symbol is the content hash, so the frame tag doubles as the geometry reference. Minted from a target rig's keys by target_frames.
See also ViewFrame, TargetPlaneFrame (the 2D frame of the board plane), Frame.
VisualGeometryToolkitCore.UnlabeledFrame — Type
UnlabeledFrame{N} <: Frame{N}The gradual-typing default frame: "no frame recorded".
Every EuclideanMap constructor that does not state frames produces Src = Dst = UnlabeledFrame{N}, so pre-frame code compiles and behaves exactly as before. UnlabeledFrame is the composition wildcard: it composes with anything, the junction is unchecked, and known endpoints are kept as-is (the truth table at Base.:∘ for EuclideanMap; the rule is _junction_ok, below). This anti-virality rule means frame labels never force themselves onto code that does not opt in.
VisualGeometryToolkitCore.ViewFrame — Type
ViewFrame{id} <: Frame{3}The 3D camera-centered frame of a SPECIFIC view, tagged by an instance identity: id::Symbol names which camera (ViewFrame{:left} ≠ ViewFrame{:right}). This extends the PixelFrame{:colmap} idiom from convention-identity to instance-identity — the symbol parameter distinguishes coexisting instances of the same kind of frame, so that a labeled junction between two different cameras' frames is a MethodError instead of a silent CameraFrame collision.
Axis conventions are those of CameraFrame (origin at the optical center, +Z along the optical axis); the tag adds WHICH camera, not a new convention. In Drake monogram terms a two-view relative pose is X_{V_b V_a} ≙ EuclideanMap{..., Src = ViewFrame{:a}, Dst = ViewFrame{:b}}.
Gradual as always: nothing forces view identities — UnlabeledFrame (and the singleton CameraFrame) remain the defaults everywhere. StereoRig mints per-view identities on request (see view_frames).
See also ImageFrame (the 2D image frame of a view), CameraFrame, Frame.
VisualGeometryToolkitCore.WorldFrame — Type
WorldFrame <: Frame{3}The world (scene) coordinate frame. Destination frame of a camera POSE (X_WC ≙ EuclideanMap{..., Src = CameraFrame, Dst = WorldFrame}), source frame of camera EXTRINSICS.
VisualGeometryToolkitCore.DistortedFrame — Type
DistortedFrame{id} <: Frame{2}The post-lens, pre-calibration 2D frame of view id: the frame a lens model's forward distortion lands on, one framed K-affine short of pixels. Completes the imaging-frame vocabulary:
| frame | meaning |
|---|---|
GnomonicFrame{id} | gnomonic ray frame (calibrated, undistorted): x̂ = π(K⁻¹ũ) |
EquidistantFrame{id} | equidistant ray frame (radius θ): the omni models' home |
StereographicFrame{id} | stereographic ray frame (radius 2·tan(θ/2), conformal) |
EquisolidFrame{id} | equisolid ray frame (radius 2·sin(θ/2), equal-area) |
OrthographicFrame{id} | orthographic ray frame (radius sin θ; folds at 90°) |
DistortedFrame{id} | post-lens, pre-K: where forward distortion lands |
ImageFrame{id} | canonical pixel frame of the view (:colmap convention) |
PixelFrame{Origin} | pixel ORIGIN conventions — orthogonal to view identity |
The ray-frame rows are all frames on the same viewing sphere S² (the RetinalFrame family): the gnomonic frame (perspective plane) is ONE frame on S², degenerate at 90° and blind to Z ≤ 0 rays; the equidistant and stereographic frames cover θ ∈ [0, π), the equisolid frame the whole sphere, and the orthographic frame only the closed forward hemisphere (it folds at the equator). "Perspective precedes lens" is therefore a property of perspective-plane MODELS, not a law of the imaging chain — each lens model's distortion is labeled with the frame it is actually defined in (distortion_frame), and the typed chain of imaging_chain reads
ViewFrame{id} ─ ray-frame projection → Gnomonic/EquidistantFrame{id}
─ lens distortion → DistortedFrame{id}
─ framed K-affine → ImageFrame{id}with every junction frame-checked. When a lens's distortion is the identity in its defining frame (has_distortion == false, e.g. Pinhole; also the pure Lens(Equidistant()) on its equidistant frame), the distortion stage is omitted and the ray frame meets K directly — the two frames COLLAPSE rather than being separated by a manufactured identity stage.
Anonymous view id. The id axis is optional: the reserved symbol :_ is the designated ANONYMOUS view identity, and every stage mint defaults to it. Frame KIND then still does the junction checking — a lone CameraModel with no view identity gets GnomonicFrame{:_} → DistortedFrame{:_} stages whose distort-vs-undistort direction is checked by kind — while rig-derived chains mint real ids (imaging_chain(model; id = ids(rig)[1])). The wildcard rules for UnlabeledFrame are unchanged; :_ is a real label, not a wildcard.
See also GnomonicFrame, EquidistantFrame, ImageFrame, Frame.
VisualGeometryToolkitCore.EquidistantFrame — Type
EquidistantFrame{id} <: RetinalFrame{id}The equidistant-azimuthal 2D ray frame of view id: the frame on the viewing sphere S² whose point for a ray at polar angle θ (from the optical axis) and azimuth ψ is θ·(cos ψ, sin ψ) — radial coordinate equal to the ANGLE, azimuth preserved. Well-defined for θ ∈ [0, π) (every ray except the exact backward direction), so it represents fields of view past 90° where the gnomonic GnomonicFrame is degenerate.
"Equidistant" (radial coordinate r = θ) names this frame unambiguously within the fisheye family, whose classical frames are all functions of the polar angle — equidistant r = θ, stereographic r = 2·tan(θ/2), equisolid r = 2·sin(θ/2), orthographic r = sin θ — so "the angle frame" would not single one out. The other radial profiles slot under RetinalFrame beside this one (0.13): StereographicFrame, EquisolidFrame, OrthographicFrame.
This is the DEFINING frame of the omnidirectional distortion models: KannalaBrandt is the radial polynomial θ_d = θ·(1 + k₁θ² + …) on this frame, the pure Lens(Equidistant()) is the IDENTITY on it (the "distortion" of a pure equidistant fisheye is entirely the frame choice), and COLMAP's THIN_PRISM_FISHEYE applies its additive terms to this frame's coordinates. The framed projection onto this frame is FramedEquidistantProjection (imaging_chain stage 1 for equidistant-frame lenses); see the frame table at DistortedFrame for how it slots into the imaging chain.
Not to be confused with Manifold: that machinery parameterizes MODEL manifolds (essential matrices, homographies) for estimation; EquidistantFrame is a coordinate frame on the viewing sphere of one camera.
See also GnomonicFrame, DistortedFrame, ImageFrame, Frame.
VisualGeometryToolkitCore.EquisolidFrame — Type
EquisolidFrame{id} <: RetinalFrame{id}The equisolid-angle 2D ray frame of view id: the frame on the viewing sphere S² whose point for a ray at polar angle θ and azimuth ψ is 2·sin(θ/2)·(cos ψ, sin ψ) — the Lambert azimuthal equal-area projection. Well-defined on the WHOLE sphere θ ∈ [0, π] (the only retinal frame that reaches the antipode), with bounded radius r ∈ [0, 2]; equal areas on the sphere map to equal areas in the frame, which is why equisolid ("equal-area fisheye") lenses are the standard for sky photometry and illuminance observation. The antipode θ = π maps to radius 2 under the pole azimuth convention ψ = 0.
The defining frame of the pure equisolid fisheye projection: Lens(Equisolid()) is the IDENTITY on it, and the framed projection onto this frame is FramedEquisolidProjection (imaging_chain stage 1). Transitions to the other retinal frames are minted by frame_transition; see the family table at RetinalFrame and the chain table at DistortedFrame.
See also StereographicFrame, OrthographicFrame, EquidistantFrame, GnomonicFrame, Frame.
VisualGeometryToolkitCore.FrameDependence — Type
FrameDependenceHoly trait classifying a 2D frame TAG by what DEFINES the frame it names — the retina's geometry alone, or a model's continuous parameters:
GeometryDefined(): the frame is fixed by geometry and conventions — everyRetinalFramekind (gnomonic / equidistant / stereographic / equisolid / orthographic),ImageFrame,NormalizedImageFrame(fixed by the image size),PixelFrame{Origin},TargetPlaneFrame.ModelDefined(): the frame is pinned down by a model's continuous parameters —DistortedFrame(which frame it IS depends on the view's lens coefficients).
The trait encodes a COMMENSURABILITY rule: points in GeometryDefined frames of the same kind and view are always commensurable — the tag alone determines the frame, so a model-free frame_transition between two such frames exists and the sphere sections/lifts behind it are defined. Points in ModelDefined frames are commensurable only under the one-lens-per-view invariant that makes DistortedFrame{id} honest, and no model-free transition into or out of them exists — the model-carrying maps (distort_stage/undistort_stage, or a fused model-carrying projection stage) are the only bridges.
Deliberate boundary: the trait classifies the KIND of a frame tag; it cannot and does not verify parameter equality between two ModelDefined uses — parameter values are a runtime question, exactly as rank lives on the manifold rather than in the matrix type. Query with frame_dependence.
VisualGeometryToolkitCore.GeometryDefined — Type
Frame fixed by the retina's geometry/conventions alone. See FrameDependence.
VisualGeometryToolkitCore.GnomonicFrame — Type
GnomonicFrame{id} <: RetinalFrame{id}The calibrated (normalized) 2D image frame of view id: the frame whose points are gnomonic coordinates x̂ = π(K⁻¹ũ) — pixel coordinates with the intrinsics removed, i.e. the perspective projection of the view's unit-focal image plane. EssentialMat-backed epipolar forms are honest on this frame (x̂₂ᵀ E x̂₁ = 0); ImageFrame remains the PIXEL frame (canonical :colmap convention) and is never overloaded for gnomonic coordinates.
Formally, this is the GNOMONIC chart of the view's ray sphere S² — the central projection of directions onto the unit-focal plane. Like every gnomonic chart it covers only the forward hemisphere: it is degenerate at 90° from the optical axis and cannot represent rays with Z ≤ 0. It is the defining frame of the perspective projection's distortions (OpenCV's radial-tangential-thin-prism models); omnidirectional models define their distortion on the EquidistantFrame instead — see the frame table at DistortedFrame. The framed projection onto this frame is FramedPerspective (imaging_chain stage 1 for perspective-plane lenses).
The calibrated → pixel collineation of view id is the labeled lift of the view's intrinsics, HomographyMap{GnomonicFrame{id}, ImageFrame{id}}(K) — the slot maps whose composition with the correlation computes F = K₂⁻ᵀ E K₁⁻¹ with the frame labels verifying that each K met the slot it was meant for. Minted by essential_form(rig) / EpipolarForm(rig) on a StereoRig.
See also EquidistantFrame, DistortedFrame, ImageFrame, ViewFrame, Frame.
VisualGeometryToolkitCore.ImageFrame — Type
ImageFrame{id} <: Frame{2}The 2D image frame of view id — the per-view pixel frame used to label the argument slots of epipolar forms (see EpipolarForm) and the endpoints of inter-view 2D maps.
Deliberate division of labor with PixelFrame: PixelFrame{Origin} tags a pixel-origin CONVENTION (:colmap/:julia/:opencv) — a property of how coordinates are written; ImageFrame{id} tags WHICH VIEW an image point belongs to — a property of what the coordinates refer to. The two are orthogonal, and ImageFrame does not encode an origin: convert origin conventions FIRST (via the EuclideanMap(PixelFrame{A}, PixelFrame{B}) frame maps, landing on the canonical :colmap convention), then label the points' frame with their view. An ImageFrame{id} point is therefore always read in the canonical :colmap convention.
See also ViewFrame (the 3D frame of the same view), GnomonicFrame (the calibrated frame of the same view), image_frames, Frame.
VisualGeometryToolkitCore.ModelDefined — Type
Frame pinned down by a model's continuous parameters. See FrameDependence.
VisualGeometryToolkitCore.NormalizedImageFrame — Type
NormalizedImageFrame{id} <: Frame{2}The diagonal-normalized 2D image frame of view id: ImageFrame coordinates moved to the image centre and divided by the half-diagonal s = hypot(width, height) / 2, so the image corners sit at unit radius. Pixel-scale constants are sensor-independent here: a focal length is the dimensionless f / s (1 for a 90° diagonal field of view) and a pixel threshold τ becomes τ / s. Mint the map with the factory FramedAffine(ImageFrame{id}, NormalizedImageFrame{id}, sz) (or from a PixelFrame{Origin} source); its inv de-normalizes.
Not to be confused with GnomonicFrame, the CALIBRATED frame K⁻¹·pixel: this frame needs no intrinsics, only the image size.
VisualGeometryToolkitCore.OrthographicFrame — Type
OrthographicFrame{id} <: RetinalFrame{id}The orthographic 2D ray frame of view id: the frame on the viewing sphere S² whose point for a ray at polar angle θ and azimuth ψ is sin θ·(cos ψ, sin ψ) — parallel projection of the sphere straight down onto the equatorial plane, the image formed by an orthographic ("sine law") fisheye lens.
Honest domain: θ ∈ [0, π/2] ONLY — the frame FOLDS at the equator.r = sin θ increases to 1 at θ = π/2 and then DECREASES again: past 90° every radius value is taken twice, so the projection is non-injective on any domain crossing the equator and the frame is restricted to the closed forward hemisphere. This is the poster child for the honest-domain discipline of the retinal frames — a ray with Z < 0 is not "a large radius" in this frame, it is OUTSIDE the frame, and the section/lift machinery throws a DomainError rather than silently aliasing it onto a forward ray (the lift always selects the forward sheet cos θ = +√(1 − r²)). Radius is bounded, r ∈ [0, 1].
The defining frame of the pure orthographic fisheye projection: Lens(Orthographic()) is the IDENTITY on it, and the framed projection onto this frame is FramedOrthographicProjection (imaging_chain stage 1). Transitions to the other retinal frames are minted by frame_transition; see the family table at RetinalFrame and the chain table at DistortedFrame.
See also StereographicFrame, EquisolidFrame, EquidistantFrame, GnomonicFrame, Frame.
VisualGeometryToolkitCore.PixelFrame — Type
PixelFrame{Origin} <: Frame{2}2D pixel frame of an image, tagged by its origin convention: Origin::Symbol names the pixel-origin placement. The registered origins:
| frame | first-pixel center | corner offset |
|---|---|---|
PixelFrame{:opencv} | (0, 0) | (-0.5, -0.5) |
PixelFrame{:colmap} | (0.5, 0.5) | (0, 0) |
PixelFrame{:julia} | (1, 1) | (0.5, 0.5) |
(x right / y down throughout; :julia is additionally read row-major (row, col) at the array boundary — that permutation is image_warp's RCSwap, not part of the frame.)
Origin conversion is a labeled rigid map between two pixel frames (identity rotation, constant translation), built by the constructor-as-factory on the frame TYPES:
T = EuclideanMap(PixelFrame{:colmap}, PixelFrame{:julia})
T isa EuclideanMap{2, …, PixelFrame{:colmap}, PixelFrame{:julia}} # true
T.t == Vec2(0.5, 0.5) # true
pt_julia = T(pt_colmap)The map composes (frame-checked) with other labeled 2D maps and fuses in image_warp chains like any EuclideanMap{2}. This is the ONLY origin conversion API: the 0.6-era ImageOrigin value surface (COLMAP_ORIGIN, JULIA_ORIGIN, Translation(::ImageOrigin, ...), Translation(::Symbol, ::Symbol)) is removed in 0.7.
VisualGeometryToolkitCore.RetinalFrame — Type
RetinalFrame{id} <: Frame{2}Abstract supertype of the canonical 2D ray frames of view id — the coordinate systems on the view's SPHERICAL RETINA, the sphere S² of rays through the center of projection (the manifold-level object of the Geyer–Daniilidis unified imaging model). Every central camera has a spherical retina; what differs between camera models is the FRAME its distortion is expressed in, so frames — not retinas — are the type currency of the imaging chain. The "planar retina" of the perspective camera is not a different manifold but the gnomonic frame on this one (GnomonicFrame); the equidistant frame (EquidistantFrame) covers the fields of view the gnomonic frame cannot.
Subtypes share the azimuth and differ in the radial profile — the five classical azimuthal frames of the fisheye-projection family, each a function of the polar angle θ with its own honest domain:
| frame | radial profile | domain |
|---|---|---|
GnomonicFrame | r = tan θ | θ ∈ [0, π/2) |
EquidistantFrame | r = θ | θ ∈ [0, π) |
StereographicFrame | r = 2·tan(θ/2) | θ ∈ [0, π) |
EquisolidFrame | r = 2·sin(θ/2) | θ ∈ [0, π] |
OrthographicFrame | r = sin θ | θ ∈ [0, π/2] ONLY |
(the orthographic frame FOLDS at the equator — see its docstring). The transition maps between them are pure radial reparameterizations, minted as first-class framed stages by frame_transition. A point of the retina written in no 2D frame needs no 2D tag — it is a unit 3-vector in ViewFrame{id}.
imaging_chain stage 1 (the ray-frame projection family) maps ViewFrame{id} → RetinalFrame{id}; distortion_frame(lens) names which subtype a lens model's distortion is defined in. DistortedFrame is deliberately NOT a subtype: it is defined by the lens map's image, not as a canonical frame of the retina.
See also GnomonicFrame, EquidistantFrame, StereographicFrame, EquisolidFrame, OrthographicFrame, DistortedFrame, Frame.
VisualGeometryToolkitCore.StereographicFrame — Type
StereographicFrame{id} <: RetinalFrame{id}The stereographic 2D ray frame of view id: the frame on the viewing sphere S² whose point for a ray at polar angle θ (from the optical axis) and azimuth ψ is 2·tan(θ/2)·(cos ψ, sin ψ) — the classical stereographic projection of the sphere from its backward pole onto the tangent plane at the forward pole. Well-defined for θ ∈ [0, π) (every ray except the exact antipode), and CONFORMAL — the only angle-preserving member of the fisheye-projection family, which is why stereographic ("fisheye of least visual distortion") lenses are prized in panorama work. Radius is unbounded as θ → π.
The defining frame of the pure stereographic fisheye projection: Lens(Stereographic()) is the IDENTITY on it, and the framed projection onto this frame is FramedStereographicProjection (imaging_chain stage 1). Transitions to the other retinal frames are minted by frame_transition; see the family table at RetinalFrame and the chain table at DistortedFrame.
See also EquidistantFrame, EquisolidFrame, OrthographicFrame, GnomonicFrame, Frame.
VisualGeometryToolkitCore.TargetPlaneFrame — Type
TargetPlaneFrame{id} <: Frame{2}2D frame of a calibration target's plane, the target named by its id (a Symbol, as TargetFrame{id} names its 3-D frame): target-local (x, y) on the plane z = 0 of TargetFrame{id}. A file names it "target:<id>" (VisualGeometryToolkitIO: a blur on a print, a target homography's source). Equal coordinates in two targets' planes are different points: the frames differ by their id.
The bare TargetPlaneFrame (no id) is the plane of a target the value does not name — the source of a pinhole camera's plane-induced homography, HomographyMap(camera::PosedPinholeCamera), Src = TargetPlaneFrame, Dst = PixelFrame{:colmap} (monogram H_PB).
VisualGeometryToolkitCore.frame_dependence — Function
frame_dependence(C::Type{<:Frame{2}}) -> FrameDependenceWhat defines the frame named by tag C: GeometryDefined for the RetinalFrame family, ImageFrame, NormalizedImageFrame, PixelFrame, and TargetPlaneFrame; ModelDefined for DistortedFrame. Downstream packages minting their own frame tags add a method; there is no default — an unclassified frame tag is a MethodError, not a silent guess. See FrameDependence for the commensurability rule the trait encodes.
Index
VisualGeometryToolkitCore.BodyFrameVisualGeometryToolkitCore.CameraFrameVisualGeometryToolkitCore.DistortedFrameVisualGeometryToolkitCore.EquidistantFrameVisualGeometryToolkitCore.EquisolidFrameVisualGeometryToolkitCore.FrameVisualGeometryToolkitCore.FrameDependenceVisualGeometryToolkitCore.GeometryDefinedVisualGeometryToolkitCore.GnomonicFrameVisualGeometryToolkitCore.ImageFrameVisualGeometryToolkitCore.ModelDefinedVisualGeometryToolkitCore.NormalizedImageFrameVisualGeometryToolkitCore.OrthographicFrameVisualGeometryToolkitCore.PixelFrameVisualGeometryToolkitCore.RetinalFrameVisualGeometryToolkitCore.StereographicFrameVisualGeometryToolkitCore.TargetFrameVisualGeometryToolkitCore.TargetPlaneFrameVisualGeometryToolkitCore.UnlabeledFrameVisualGeometryToolkitCore.ViewFrameVisualGeometryToolkitCore.WorldFrameVisualGeometryToolkitCore.frame_dependence