Rigs, Stereo and Captures API
Rigs of cameras and targets, stereo rectification, and the per-frame values of a capture. Split from the Imaging Chain and Targets API page, which holds the imaging chain and the targets. See the Stereo Rigs and Uncertainty guide.
Rigs
A Rig is members with fixed poses in one frame, each a (model, pose) pair: a camera (its model and rig-relative extrinsics) or a PosedTarget. Sensors only, targets only, or both (a camera rig carrying its marker constellation).
VisualGeometryToolkitCore.Cameras.PosedTarget — Type
PosedTarget(target::AbstractTarget, pose)A target as a Rig member, the target-side counterpart of a PosedCamera and, like it, of its kind's hierarchy: an AbstractTarget whose is_posed is true. It holds the TARGET — its geometry, an unposed AbstractTarget, independent of where it is mounted, so the same target can sit in several rigs — and its POSE, the rigid map taking target-local coordinates (the pattern plane z = 0 for a planar target, the points a HeightField lifts off it as they are) into the rig frame, stored unlabeled (the rig mints TargetFrame{id} → F on access). The pose may be Uncertain (an estimated pose, its 6×6 covariance in the RigidExpManifold6 chart, rotation first); the rig hands it back uncertain, its covariance unchanged by the labels.
Accessors: model(m) is the target (as model(::PosedCamera) is a camera's model, so code over rig members calls model(member) for either kind), pose(m) the pose, extrinsics(m) its inverse (an ArgumentError for an uncertain pose, whose covariance does not invert by relabeling), and points(m) the target's own points. The target must be unposed: posing a PosedTarget is an ArgumentError, as posing a posed camera is.
A physical pose (a translation in metres) holds a physical target (points in metres; for an unresolved planar target, a calibration or board with lengths in metres), a plain pose a plain one: a unitful pose on plain points, or the reverse, is FlexUnits' DimensionError at construction, never a promotion. ustrip(u, m) and with_unit(m, u) move both together.
VisualGeometryToolkitCore.Cameras.PosedTarget — Method
PosedTarget(target::AbstractTarget)
PosedTarget(board::Board[, pose])The target posed at the origin of its rig frame, its frame the rig's (the usual choice of a calibration's world, "the target's frame is the world's"): the identity pose, physical when the target is (EuclideanMap(I, [0m, 0m, 0m])), plain otherwise. A Board alone is a target too, a planar target with no pattern points, the board it is printed on: posed by pose, at the origin without one.
VisualGeometryToolkitCore.Cameras.Rig — Type
Rig{F, M}A rigid assembly: members with FIXED poses in a common frame F, each a (model, pose) pair. Three storages, one interface:
M <: Tuple{Vararg{AbstractCamera}}— a physical multi-sensor rig. Sensori's extrinsics is the labeled mapF → ViewFrame{idᵢ}, stored inside the camera (a camera IS its model and its pose); the ids, the reference frame and every relative transform are read off those labels — there is no cached inverse, no id tuple, no second copy to drift. The tuple makes the sensor count static:StereoRigIS the 2-tuple case, dispatchable with no runtime length check. Many views over time are NOT a rig — they are frames of one (see the header comment) — which is why aVectorof hundreds of cameras is not accepted.M <: AbstractDict{Symbol, <:PosedTarget}— a rig of targets: each aPosedTargetkeyed by identity, its pose stored unlabeled and mintedTargetFrame{id} → Fon access.M <: NamedTuple{(:sensors, :targets)}— both: a sensor tuple and a target dictionary posed in one frame (a camera rig carrying its motion-capture marker constellation, onBodyFrame{:camera_rig}). Built byRig{F}(sensors, targets);sensor_rigandtarget_rigare its two halves as rigs of their own. Ids are distinct across the two.
Type parameters
F <: Frame{3}: the frame every member is posed in. THE FRAME IS THE GAUGE: whenFis a member's own frame —ViewFrame{id₁}for a stereo rig built by theStereoRigconstructors,TargetFrame{uid}for a calibration target gauged on boarduid— that member's pose is the identity, checked at construction. A rig posed in an external frame (a body frame, the world, a tracker's room) names that frame instead and has no gauge member.M: the storage, above.
Accessors
ids, pose(rig, id) / extrinsics(rig, id) (label-minting or label-preserving per storage; sensors also by position), rig[id], length, haskey; view_frames / image_frames on sensor rigs, target_frames and points(rig, id) — the target's points in the rig frame, the WORLD POINTS of the assembly — on target rigs; the stereo surface (source, destination, extrinsics(rig), pose(rig)) on the 2-sensor alias.
VisualGeometryToolkitCore.Cameras.ids — Method
ids(rig::Rig) -> NTuple{N, Symbol} | Vector{Symbol}The member identities: a sensor rig's view-identity symbols, read off the frame parameters of each stored sensor's labeled extrinsics (F → ViewFrame{id}) — an accessor, not a field, resolved at compile time from the rig's type; a target rig's keys, sorted for determinism; a rig of both, the sensors' then the targets'.
VisualGeometryToolkitCore.Cameras.image_frames — Method
image_frames(rig::Rig) -> (ImageFrame{id1}, ImageFrame{id2}, …)The labeled 2D per-view image-frame TAGS of a sensor rig's sensors — ImageFrame{id} types minted from ids(rig), the slot labels for epipolar forms over this rig. See ImageFrame for the convention-vs-view division of labor.
VisualGeometryToolkitCore.Cameras.reference_frame — Method
reference_frame(rig::Rig{F}) -> FThe rig's frame type — the frame every member is posed in (F → ViewFrame{idᵢ} for sensors, TargetFrame{id} → F for targets). Unexported; resolved from the type.
VisualGeometryToolkitCore.Cameras.sensor_rig — Method
sensor_rig(rig::Rig) -> Rig{F, <:Tuple}The sensor half of a rig carrying both sensors and targets, as a rig of its own in the same frame (the identity on a sensors-only rig). The target half is target_rig.
VisualGeometryToolkitCore.Cameras.target_frames — Method
target_frames(rig::Rig) -> Vector{DataType}The labeled 3D per-target frame TAGS of a target rig — TargetFrame{id} types minted from ids(rig).
VisualGeometryToolkitCore.Cameras.target_rig — Method
target_rig(rig::Rig) -> Rig{F, <:AbstractDict}The target half of a rig carrying both sensors and targets, as a rig of its own in the same frame (the identity on a targets-only rig). The sensor half is sensor_rig.
VisualGeometryToolkitCore.Cameras.view_frames — Method
view_frames(rig::Rig) -> (ViewFrame{id1}, ViewFrame{id2}, …)The labeled 3D per-view frame TAGS of a sensor rig's sensors — ViewFrame{id} types minted from ids(rig), ready to be attached at a lift (e.g. relabel(ViewFrame{:left}, WorldFrame, pose)). See ViewFrame for the instance-identity design.
VisualGeometryToolkitCore.Cameras.with_model — Method
with_model(camera, model) -> PosedCamera/BoundedCamera
with_model(rig::Rig, i => model) -> Rig
with_model(rig::Rig, model) -> Rig (single-sensor convenience)Rebuild with sensor i's camera MODEL replaced — after a fit_camera regression or a generic-model init — keeping everything geometric: the labeled rig-relative extrinsics, the ids, any BoundedCamera extent. The rig is reconstructed through its validating constructor (never a raw field update, which would bypass the frame invariants); a lens/K change is metric-neutral to the rigid transforms, so the swap is geometrically exact.
NOTE: swapping the model does NOT re-estimate poses. Poses optimized under the old model keep their values; for an APPROXIMATING model (the fit_camera use case) that is the intent — the approximation error is the fit report's — while for a refinement the new model is the START of another bundle run, not a substitute for one.
VisualGeometryToolkitCore.extrinsics — Method
extrinsics(rig::Rig, i::Integer | id::Symbol) -> EuclideanMapSensor i's (or view id's) stored rig-relative extrinsics, label-preserving: the map F → ViewFrame{idᵢ} (NOT the WorldFrame → CameraFrame relabel that extrinsics(::PosedCamera) applies — a rig sensor's frame is the rig's, and the labels are kept). For a target rig, F → TargetFrame{id} — common-frame coordinates into target-local coordinates, inv(pose(rig, id)) on demand.
VisualGeometryToolkitCore.pose — Method
pose(rig::Rig, i::Integer | id::Symbol) -> EuclideanMapA member's pose in the rig frame. For a sensor, inv(extrinsics(rig, i)) computed on demand (ViewFrame{idᵢ} → F — the typed algebra swaps the labels); for a target, the stored map minted with its labels, TargetFrame{id} → F (target-local coordinates into the rig frame).
VisualGeometryToolkitCore.Cameras.StereoRig — Type
StereoRig{F, U, V} = Rig{F, Tuple{U, V}}Stereo camera pair — the 2-sensor specialization of Rig: a stereo rig IS a rig whose camera container is a 2-tuple, so the pair's length is in the type and the stereo surface dispatches on it with no runtime check.
The rig has no source, destination or extrinsics fields (none since 0.16): it stores its two cameras rig-relative (reference frame F = ViewFrame{id₁}; camera 1's stored extrinsics is the identity, camera 2's is the labeled relative transform ViewFrame{id₁} → ViewFrame{id₂}), and the three are method-form accessors:
source(rig)/destination(rig): the two cameras (rig-relative — the constructors re-express world-posed inputs).extrinsics(rig): the relative rigid transform (monogramX_{V₂V₁}), DERIVED from the stored per-camera maps asextrinsics(rig, 2) ∘ pose(rig, 1)— it consumes source-view (ViewFrame{id₁}) coordinates and produces destination-view (ViewFrame{id₂}) coordinates, i.e.X₂ = R·X₁ + t.pose(rig):inv(extrinsics(rig))(the typed algebra swaps the frames,X_{V₁V₂}).ids(rig): the identity symbols, read off the stored labels.
The labels ARE the single source of truth — the twin-typing and id-duplication policy violations of 0.6–0.8 remain unrepresentable, and the 0.9-era separately-stored relative transform is gone too (one authority: the per-camera rig-relative maps).
Constructors
StereoRig(left_camera, right_camera) # Computes relative transform
StereoRig(source, destination, extrinsics) # Explicit transform, CHECKED against
# the cameras (check = false to skip)
StereoRig(left, right; ids = (:left, :right)) # Mint per-view identitiesThe ids keyword mints the view identities at construction (label-at- source: the constructor re-expresses the cameras in the rig frame ViewFrame{id₁} and labels the relative transform ViewFrame{id₁} → ViewFrame{id₂}, overriding whatever frames the inputs carried — the rig's ids are authoritative for its own transforms). The labeled per-view tags are exposed by view_frames (3D ViewFrame{id}) and image_frames (2D ImageFrame{id}); essential_form / EpipolarForm(rig) mint the labeled epipolar forms. Purely additive: nothing in the rig forces identities onto callers who ignore them.
The 3-arg constructor validates the given transform against extrinsics(destination) ∘ pose(source) computed from the GIVEN cameras (check = true by default, atol tolerance on the rotation angle and the relative translation deviation). Supply check = false for synthetic rigs whose cameras are placeholders and do not encode the relative geometry. check = false also admits UNPOSED members (a BoundedCamera with no pose): the rig poses them, so only the given relative transform matters. The 2-argument form computes the transform from the cameras and so requires them posed.
VisualGeometryToolkitCore.Cameras.StereoRig — Method
StereoRig(K1, K2, R_rel, t_rel) -> StereoRigConstruct a stereo rig from two intrinsics matrices plus the relative pose (R_rel, t_rel) from camera 1 to camera 2 (extrinsic convention: X_cam2 = R_rel · X_cam1 + t_rel). Camera 1 is placed at the world origin with identity rotation; camera 2 gets the given relative pose.
Convenience constructor for benchmarks where ground-truth is expressed as two calibration matrices plus a per-pair relative pose (e.g. HEB's K_<img> and pose_<pair> keys).
VisualGeometryToolkitCore.Cameras.depth_transfer — Method
depth_transfer(source::PosedCamera, destination::PosedCamera, extrinsics)
depth_transfer(rig::StereoRig)Depth-ASSUMED point transfer from the source camera to the destination camera: returns a closure mapping a source pixel to a destination pixel by backprojecting to a ray, fixing unit depth in the source camera frame, transforming through extrinsics, and projecting into the destination image.
WARNING: this is a point-to-point transfer under an arbitrary depth assumption — it is NOT the epipolar correlation. Without depth, a pixel backprojects to a ray, not a point; the honest depth-free two-view object is the point-to-LINE map of an EpipolarForm (see epipolar_map, which since 0.8 names only that action). For real transfer with known depth, use transfer with a depth-attributed point; for correspondence search, use triangulation with matched features.
Renamed from epipolar_map in 0.8 — the old name promised a correlation and delivered a transfer.
Arguments
source::PosedCamera: Source cameradestination::PosedCamera: destination cameraextrinsics::EuclideanMap: Relative transform from source to destination
Returns
- Function mapping pixel coordinates from source to destination camera
VisualGeometryToolkitCore.Cameras.destination — Method
destination(rig::StereoRig) -> AbstractCameraThe destination (right, view 2) camera — stored rig-relative: its extrinsics is the labeled relative transform ViewFrame{id₁} → ViewFrame{id₂}.
VisualGeometryToolkitCore.Cameras.essential_form — Method
essential_form(rig::StereoRig)
-> EpipolarForm{GnomonicFrame{id₁}, GnomonicFrame{id₂}, EssentialMat}The rig's CALIBRATED epipolar form: E = [t]×R from the labeled relative transform (x̂₂ᵀ E x̂₁ = 0 for gnomonic coordinates x̂ = π(K⁻¹ũ)), with the slots labeled by the per-view GnomonicFrame tags — NOT ImageFrame, which is documented as the canonical-:colmap PIXEL frame and is never overloaded for gnomonic coordinates. Slot 1 is the source view, matching the pinned x₂ᵀMx₁ convention of EpipolarForm.
Composing its epipolar_map with the calibrated → pixel lifts HomographyMap{GnomonicFrame{idᵢ}, ImageFrame{idᵢ}}(Kᵢ) turns this into the pixel-frame fundamental form — which is exactly what EpipolarForm(rig) does.
VisualGeometryToolkitCore.Cameras.source — Method
source(rig::StereoRig) -> AbstractCameraThe source (left, view 1) camera — stored rig-relative: its extrinsics is the identity map on the rig frame ViewFrame{id₁}.
VisualGeometryToolkitCore.Cameras.validate_gt_inliers — Method
validate_gt_inliers(rig::StereoRig, correspondences, depth_src, depth_tgt;
rel_tol = 0.05,
metric_tol::Quantity,
gross_outlier_pixel_frac = 0.01,
bidirectional_depth = true) -> BitVectorValidate correspondences against GT rig + per-view depth maps, returning a BitVector of inliers. The threshold is a physical metric length at the surface (e.g. 5mm, 50mm); see _gt_inliers_metric for the math.
Pixel-based dispatch was removed in favor of metric-only to keep the GT truth process apples-to-apples across datasets at different resolutions/focal lengths.
VisualGeometryToolkitCore.Cameras.validate_gt_metric — Method
validate_gt_metric(rig, correspondences, depth_src, depth_tgt;
rel_tol = 0.05, gross_outlier_pixel_frac = 0.01,
bidirectional_depth = true)
-> (residuals::Vector{Vec3{Float64}}, valid::BitVector)Geometric pre-validation in metric space, without any pixel-residual threshold. A correspondence is valid when it satisfies, in order:
- a gross-outlier gate: its GT-pose Sampson distance is at most
gross_outlier_pixel_frac× the destination image diagonal (pose only, no depth); depth_srcis finite atp_Aanddepth_tgtis finite atp_B;- the forward warp of
p_Alands wheredepth_tgtis finite, and the predicted and observed destination depths agree withinrel_tol(relative); - with
bidirectional_depth, warpingp_Bback at its observed depth agrees withdepth_srcthe same way.
The residual is 3D, in meters: forward, in the destination camera frame,
residuals[i] = unproject(tgt, p̂_B, d̂_B) − unproject(tgt, p_B, d_obs)and, with bidirectional_depth, the longer of that and the backward residual (in the source camera frame), so ‖residuals[i]‖ is the worse-direction metric disagreement at the surface. Used by calibrate_metric_tol to estimate the noise floor without applying a tolerance gate.
VisualGeometryToolkitCore.extrinsics — Method
extrinsics(rig::StereoRig) -> EuclideanMapThe source→destination relative rigid transform (monogram X_{V₂V₁}), DERIVED on demand from the stored per-camera rig-relative maps as extrinsics(rig, 2) ∘ pose(rig, 1) — labeled ViewFrame{id₁} → ViewFrame{id₂}, i.e. X₂ = R·X₁ + t. With the stereo convention (reference = view 1, camera 1's stored map the identity) this is numerically exactly camera 2's stored extrinsics.
VisualGeometryToolkitCore.pose — Method
pose(rig::StereoRig) -> EuclideanMapThe destination-to-source relative transform, inv(extrinsics(rig)) computed on demand (monogram X_{V₁V₂}; the typed algebra swaps the frame labels to ViewFrame{id₂} → ViewFrame{id₁}).
VisualGeometryToolkitCore.transfer — Method
transfer(src_model::CameraModel, extrinsics::EuclideanMap,
tgt_model::CameraModel, p::Attributed{<:StaticVector{2},<:Real})
-> Attributed{Point2, T}Primitive form: transfer a depth-attributed pixel from one view to another through an explicit (src_model, extrinsics, tgt_model) triple. Composes unproject(src_model, ·, ·), extrinsics(·), and project(tgt_model, ·), binding the resulting destination-frame depth as the attribute on the returned point.
Multi-view analog of unproject / project (see also depth_transfer, which is the unit-depth-assumed special case).
VisualGeometryToolkitCore.transfer — Method
transfer(rig::StereoRig, p::Attributed{<:StaticVector{2},<:Real})
-> Attributed{Point2, T}Convenience wrapper: transfer through rig in the source→destination direction. For the reverse direction, call the four-arg primitive form directly with destination(rig).model, pose(rig), source(rig).model.
VisualGeometryToolkitCore.Cameras.rectify — Method
rectify(rig::StereoRig; K = nothing) -> (; rig, R, K)
rectify([T = Float64], rig::StereoRig, images::Tuple; K = nothing, out_size = nothing, scale = 1,
device = default_device()) -> (; rig, R, K, images::NTuple{2, FramedImage})Epipolar rectification of a stereo rig.
Geometry form: computes the rectifying rotations R = (Q₁, Q₂) (RotMatrix3s, original view-i orientation → the common rectified orientation, Q₂ = Q₁·R_relᵀ), the shared pinhole calibration K (SMatrix{3,3}; default: mean focal of the two views, mean principal point, zero skew; a SplineLens camera contributes its gauge K, the pinhole at the pixel imaging ẑ), and the RECTIFIED rig — a StereoRig of two pinhole cameras with the same ids whose relative transform is the pure baseline translation. Rectified frames are NEW frames; the returned rig's ViewFrame{id}s label those, not the input views.
Image form: additionally resamples the two source images into the shared rectified pixel frame — for each output pixel, K⁻¹ → rotate the ray by Qᵢᵀ → project through view i's OWN lens model → sample: distortion and rectification in ONE fused warp per view, any AbstractLensModel. Returns two FramedImages over the same window Rect(0, 0, W, H) of the rectified pixel frame (out_size = (W, H), default: the elementwise max of the input sizes; scale downsamples like the rect form of undistort). After rectification, corresponding points share a row and disparity = fx·B/z.
KernelAbstractions end to end, on device (default_device() by default: an accelerator when one is functional, else KA.CPU()): each image is uploaded there once, the fused warp (one bilinear kernel; the isbits chain evaluated per output pixel on the device, no remap grid) runs there, and each canvas comes back in its image's array kind (an Array gives an Array, a device array stays on its device with no copy). rectify(Float32, rig, images) keeps the warp math single-precision for GPU throughput (the type leads, as in Base's zeros(Float32, n)).
The construction assumes a predominantly HORIZONTAL baseline (the new x-axis is sign-fixed toward +x); a vertical rig wants the transposed convention. Throws for a near-zero baseline.
Captures
The per-frame values of a capture, keyed by frame index: a body's Trajectory, a detector's TargetPoses, the TargetObservations — per frame and target the observations index => point of the target's points in a pixel or 3D frame (pixels, or 3D markers), with its FrameStamps — and convert_frame, the one way to change a pixel frame. The camera's settings per frame are a CaptureSettings: each frame's ExposureSettings and nominal LensSettings, with the DeviceIdentity of the body and the lens beside them (a camera model holds no names) and is_constant, whether every frame was exposed alike.
VisualGeometryToolkitCore.Cameras.PLANAR_MAP_TAGS — Constant
PLANAR_MAP_TAGSThe tags of the planar maps a record embeds by their parameters (the parametric record {"type": <tag>, "parameters": [θ…]}): "homography_map" → HomographyMap. A TagRegistry; the target calibrations, which are also planar maps, keep their own family (TARGET_CALIBRATION_TAGS) — a homography is an observation of a placement, not a calibration of a target.
VisualGeometryToolkitCore.Cameras.CaptureSettings — Type
CaptureSettings(frames; body = DeviceIdentity(), lens = DeviceIdentity())The camera settings of a capture, per frame index: cs[t] is frame t's FrameSettings, (; exposure, lens_settings) (a frame's value is anything with those two properties). The capture's hardware rides along as cs.body and cs.lens, DeviceIdentitys. The frame index is the join key every per-frame value of one capture shares (Trajectory, TargetObservations); a frame without settings is absent. Otherwise a Dict: keys, pairs, iteration. is_constant says whether every frame was exposed alike.
VisualGeometryToolkitCore.Cameras.DeviceIdentity — Type
DeviceIdentity(make, model, serial)
DeviceIdentity(; make = nothing, model = nothing, serial = nothing)What a file says a piece of hardware — a camera body, a lens — is: its maker, model and serial number, each a String or nothing where unknown (an empty string is an ArgumentError: state nothing). A CLAIM, not a fact checked here: EXIF names an adapted lens by whatever the adapter reports, so a calibration pipeline may prefer an identity stated in its config. Held beside the values it identifies (CaptureSettings, a stored camera's calibration record), never inside a camera or lens model.
VisualGeometryToolkitCore.Cameras.ExposureSettings — Type
ExposureSettings(exposure_time, f_number, iso)The exposure a frame was taken with: exposure_time a time quantity (0.004s, with s = parse_unit("s"); stored in seconds, FlexUnits' base form), f_number the aperture's f-number and iso the ISO speed, plain numbers stored as Float64. Each is positive and finite (ArgumentError); a plain exposure time is a DimensionError (its unit is not stated), a quantity of another dimension a ConversionError.
VisualGeometryToolkitCore.Cameras.FrameSettings — Type
FrameSettings = @NamedTuple{exposure::ExposureSettings, lens_settings::LensSettings}One frame's camera settings in a CaptureSettings: its ExposureSettings and LensSettings.
VisualGeometryToolkitCore.Cameras.FrameStamps — Type
FrameStamps(ts; images = Dict(), sensors = Dict(), times = Dict(), time_origin = nothing)The per-frame stamps of a capture value whose frame indices are ts: per index, optionally, images[t] (the image taken then), sensors[t] (what measured it — one name for every frame, or a map) and times[t] (seconds since time_origin, an ISO 8601 date-time, stated iff any time is). Each map names only indices of ts (ArgumentError). Read as fields: s.images, s.sensors, s.times, s.time_origin. Held by TargetObservations, which forwards the four as its own properties.
VisualGeometryToolkitCore.Cameras.LensSettings — Type
LensSettings(focal_length, focus_distance = nothing)The lens settings a frame was taken with: focal_length, the NOMINAL focal length the lens is marked with or reports (a length; a zoom's setting) — not a calibrated focal length, which is the camera model's — and focus_distance, the distance the lens was focused at (a length), or nothing where it is not known. Both are positive and finite (ArgumentError): a lens reporting "infinity", or a marker value, is nothing here, never a distance. Lengths are stored in metres.
VisualGeometryToolkitCore.Cameras.TargetHomographies — Type
TargetHomographies{F <: PixelFrame}(frames; images, sensors, times, time_origin, targets = String[])Per frame index, the homography of every target seen, each a HomographyMap taking the target's plane (its chart coordinates, millimetres) to pixels of the pixel frame F, part of the TYPE as in TargetObservations — the projective counterpart of TargetPoses: th[t] is frame t's target id → homography (stored unlabeled, plain or Uncertain, as TargetPoses stores its poses), homography(th, t, id) the labeled map TargetPlaneFrame{id} → F (the target's plane frame, by its id, as TargetFrame{id} is its 3-D frame; the id is the entry's key). A given map is unlabeled, or labeled exactly so; any other labels are an ArgumentError. An H is RadialHomography's H: it acts BEFORE the lens of the camera sensors[t] names, about that camera's centre (radial_homography(camera, H) pairs them). An Uncertain homography's covariance is 8×8, in the HomographyManifold8 chart about its canonical matrix (parameter_vector's: unit Frobenius norm, H[3,3] ≥ 0). targets names the target definitions the ids resolve in by content identity (VisualGeometryToolkitIO's content_id: a rig's or a single target's). images[t] names the image, sensors[t] the camera (one name for every frame, or a map), times[t] seconds since time_origin. The pixel frame is always stated: there is no form without F. Throws ArgumentError for a stamp (images, sensors, times) naming a frame index with no value, times without a time_origin, or a targets naming a definition twice. Otherwise a Dict: keys, pairs, iteration.
VisualGeometryToolkitCore.Cameras.TargetObservations — Type
TargetObservations{F <: Frame}(frames; images, sensors, times, time_origin, targets = String[])
TargetObservations{F <: Frame}(frames; stamps::FrameStamps, targets = String[])Per frame index, per target, the observations of the target's points (detections associated with the target's design points): td[t] is frame t's target id (Symbol) → correspondence set, and td[t][id] a Correspondencesindex => observation — index 1-based into points(target) (unique within the set), observation the measured point — so the pairing with the design is points(target)[i] => x. Only detected points are held (sparse, as COLMAP's and Kalibr's detections are); a target not detected in a frame is absent. Built from frames = Dict(t => Dict(id => cs)), each cs any vector of index => point pairs (correspondences(indices, points)), with optional per-detection attribute columns (withattrs(cs; weights = w), a bundle's weights), per the data-arguments convention.
The dimension is the frame's: F <: PixelFrame holds pixels in that pixel frame (plain SVector{2, Float64} or UncertainPixel, covariances in px²; convert_frame moves them), a named F <: Frame{3} 3D points in it (WorldFrame for a tracker's room; UnlabeledFrame is refused) — plain Float64, or lengths in metres, an Uncertain one's covariance in m², stored as supplied (has_units; ustrip(u, x) and with_unit(x, u) move every point). One form per container: every set holds the same point type and the same attribute columns (ArgumentError), all physical or all plain (DimensionError).
POINTS only: a detection of a non-point feature (an edge, a conic) is another type. MEASURED only, never computed by posing a design through a pose (that is derived, a function, and not stored). A motion-capture capture's target is the tracked body's marker constellation, never a board mounted on the body. Memory holds metres; a file holds its declared length_unit (VisualGeometryToolkitIO's observations kind).
The per-frame stamps are a FrameStamps (td.stamps) — given as the four keywords or as one stamps value, not both (ArgumentError) — read as td.images, td.sensors, td.times and td.time_origin; targets names the target definitions the indices resolve in by content identity (VisualGeometryToolkitIO's content_id: a rig's of targets, or a single unposed target's, whose detections are named by its design reference), unique. ==, isequal and hash agree: the frame type, the frame keys, every set with its attribute columns, the stamps and the targets — equal numbers in different frames are different detections. Otherwise a Dict: keys, pairs, iteration.
A 2D bundle's per-point weight is a weights column. The 0.9 containers of 2D and 3D target points, and their element aliases, are folded into this one container (0.10): a file of the old kinds is read by VisualGeometryToolkitIO's version steps into it.
VisualGeometryToolkitCore.Cameras.TargetPoses — Type
TargetPoses{F}(frames; images = Dict(), sensors = Dict(), times = Dict(), time_origin = nothing)Per frame index, the pose of every target seen, each mapping TargetFrame{id} → F (F the measuring camera's frame, CameraFrame for a detector's output): tp[t] is frame t's target id → pose (stored unlabeled, plain or Uncertain), pose(tp, t, id) the labeled pose. images[t] names the image, sensors[t] the camera (one name for every frame, or a map), times[t] seconds since time_origin. What a target IS — its points — belongs to its definition (a target Rig), not to every pose of it. Otherwise a Dict: keys, pairs, iteration.
VisualGeometryToolkitCore.Cameras.Trajectory — Type
Trajectory{S, D}(poses; images = Dict(), times = Dict(), time_origin = nothing, recovered = Set{Int}())
Trajectory(S, D, poses; kwargs...)The pose of frame S in frame D (x_D = R x_S + t) at each capture instant, keyed by frame index: tr[t] is a labeled EuclideanMap{…, S, D} — or an Uncertain one, its covariance kept. Both frames are named (a trajectory is the pose of ONE body, BodyFrame{:rig}, a view or a target, in one frame, WorldFrame): an unlabeled pose is labeled S → D, a labeled one must already be. Optional, per index: images[t], times[t] (seconds since time_origin), and recovered, the indices whose pose was substituted under a stated method rather than measured. Otherwise a Dict: keys, pairs, iteration; tr.poses is the underlying Dict.
VisualGeometryToolkitCore.Cameras.UncertainPixel — Type
UncertainPixel = Uncertain{SVector{2, Float64}, CovMat{2, Float64, SMatrix{2, 2, Float64, 4}, UnlabeledFrame{2}}}A measured pixel with its 2×2 covariance (px²), in the full form. A TargetObservations set keeps the storage its points' covariances hold (isotropic, diagonal or full; a set of mixed storage in the coarser one), so a detector's isotropic noise stays one number per point; a container typed with this alias converts a structured covariance to the full form on the way in.
VisualGeometryToolkitCore.Cameras.UncertainPoint3 — Type
UncertainPoint3 = Uncertain{SVector{3, Float64}, CovMat{3, Float64, SMatrix{3, 3, Float64, 9}, UnlabeledFrame{3}}}A measured plain 3D point with its 3×3 covariance (in the point's units squared), the full form. A physical one holds metres, its covariance in m² — stored as given, never rescaled from the unit the point was written in.
VisualGeometryToolkitCore.Cameras.convert_frame — Function
convert_frame(G::Type{<:PixelFrame}, x) -> x in GThe ONE way to change the pixel frame of frame-typed pixel data (a pixel TargetObservations; a package adds a method for its own frame-typed values): every pixel is moved by EuclideanMap(F, G) and the result is typed G, so a frame is never relabeled without its numbers moving with it. A pixel-frame change is a translation, so a covariance does not move.
VisualGeometryToolkitCore.Cameras.convert_frame — Method
convert_frame(G::Type{<:PixelFrame}, camera::CameraModel{<:ProjectiveIntrinsics}) -> CameraModelThe projective camera with its principal point moved from the package's COLMAP pixel frame to G by EuclideanMap(PixelFrame{:colmap}, G), so that project of the result is project of the camera in G's pixels. The lens is unchanged: it acts on centred coordinates, which a translation of the pixel origin does not touch. The result is a CameraModel in G's convention; the camera itself holds no frame tag, so converting it again is the caller's bookkeeping. The element type is preserved.
VisualGeometryToolkitCore.Cameras.is_constant — Method
is_constant(cs::CaptureSettings) -> BoolWhether every frame of cs has the same exposure time, f-number and ISO (true for none or one frame). The lens settings are not compared: a fixed exposure under a refocused lens is still one exposure. With it, a per-frame brightness gain fitted across the capture is the light changing, not the camera.
VisualGeometryToolkitCore.Cameras.parameter_names — Method
parameter_names(::HomographyMap{3}) -> NTuple{9, Symbol}
parameter_vector(h::HomographyMap{3}) -> SVector{9}
with_parameters(h::HomographyMap{3}, θ) -> HomographyMap{3}The parameter protocol of a plane homography: its nine entries h11 … h33, row-major, of its CANONICAL matrix — unit Frobenius norm and H[3,3] ≥ 0 (sign_normalize); a matrix already canonical (within 4 ulp) is returned as held, so a round trip is bit for bit. Nine numbers, not eight: a homography has 8 degrees of freedom but no global 8-number chart (fixing h33 = 1 fails at h33 = 0), so the normalization is declared instead. A negligible or non-finite matrix is an ArgumentError. with_parameters rebuilds the map (its frames kept) from nine row-major entries as given; a singular one is an ArgumentError. No entry is a length (length_mask). Its tag is PLANAR_MAP_TAGS' "homography_map".
VisualGeometryToolkitCore.pose — Method
pose(tp::TargetPoses{F}, t, id) -> EuclideanMap{…, TargetFrame{id}, F} | UncertainTarget id's pose in frame t, minted with its labels; an Uncertain one keeps its covariance.
Index
VisualGeometryToolkitCore.Cameras.PLANAR_MAP_TAGSVisualGeometryToolkitCore.Cameras.CaptureSettingsVisualGeometryToolkitCore.Cameras.DeviceIdentityVisualGeometryToolkitCore.Cameras.ExposureSettingsVisualGeometryToolkitCore.Cameras.FrameSettingsVisualGeometryToolkitCore.Cameras.FrameStampsVisualGeometryToolkitCore.Cameras.LensSettingsVisualGeometryToolkitCore.Cameras.PosedTargetVisualGeometryToolkitCore.Cameras.PosedTargetVisualGeometryToolkitCore.Cameras.RigVisualGeometryToolkitCore.Cameras.StereoRigVisualGeometryToolkitCore.Cameras.StereoRigVisualGeometryToolkitCore.Cameras.TargetHomographiesVisualGeometryToolkitCore.Cameras.TargetObservationsVisualGeometryToolkitCore.Cameras.TargetPosesVisualGeometryToolkitCore.Cameras.TrajectoryVisualGeometryToolkitCore.Cameras.UncertainPixelVisualGeometryToolkitCore.Cameras.UncertainPoint3VisualGeometryToolkitCore.Cameras.convert_frameVisualGeometryToolkitCore.Cameras.convert_frameVisualGeometryToolkitCore.Cameras.depth_transferVisualGeometryToolkitCore.Cameras.destinationVisualGeometryToolkitCore.Cameras.essential_formVisualGeometryToolkitCore.Cameras.idsVisualGeometryToolkitCore.Cameras.image_framesVisualGeometryToolkitCore.Cameras.is_constantVisualGeometryToolkitCore.Cameras.parameter_namesVisualGeometryToolkitCore.Cameras.rectifyVisualGeometryToolkitCore.Cameras.reference_frameVisualGeometryToolkitCore.Cameras.sensor_rigVisualGeometryToolkitCore.Cameras.sourceVisualGeometryToolkitCore.Cameras.target_framesVisualGeometryToolkitCore.Cameras.target_rigVisualGeometryToolkitCore.Cameras.validate_gt_inliersVisualGeometryToolkitCore.Cameras.validate_gt_metricVisualGeometryToolkitCore.Cameras.view_framesVisualGeometryToolkitCore.Cameras.with_modelVisualGeometryToolkitCore.extrinsicsVisualGeometryToolkitCore.extrinsicsVisualGeometryToolkitCore.poseVisualGeometryToolkitCore.poseVisualGeometryToolkitCore.poseVisualGeometryToolkitCore.transferVisualGeometryToolkitCore.transfer