Core API

The package docstring, units and quantities, utilities, the benchmark pool record, and the visualization stubs. The core composition and uncertainty types are on the Core Types API page, synthetic data and the Monte Carlo calibration on the Synthetic Data API page.

The Package

VisualGeometryToolkitCore.VisualGeometryToolkitCore — Module
VisualGeometryToolkitCore

Geometry, camera models and the robust-estimation core of the visual-geometry stack. VisualGeometryToolkitFeatures, VisualGeometryToolkitIO, VisualGeometryToolkitEstimation and BlobBoards build on it.

Submodules

  • Primitives: conics and feature frames (Ellipse, Circle, Disc, OrientedDisc, EllipticFrame, AffineFrame), hyperplanes (Line2D, Plane3D), 3-D conics (Ellipse3D, Circle3D) and Paraboloid3D.
  • Cameras: sensors (CMOS_SENSORS), intrinsics, lens models (Lens, LiftLens, Pinhole, KannalaBrandt, Division, …), PosedCamera, CameraModel, Rig, projection, calibration fits and P3P.
  • Synthetic: ground-truth generators and metrics for benchmarking estimators.

Key types in the parent module

  • Transform maps carry the class: EuclideanMap ⊂ SimilarityMap ⊂ CoordinateTransformations' affines ⊂ HomographyMap; the algebraic matrices are HomographyMat, FundMat, EssentialMat, CameraCalibrationMat (five parameters) and the symmetric conics.
  • Frame-tagged maps: EuclideanMap, SimilarityMap and HomographyMap carry Src/Dst tags from the open Frame hierarchy (ViewFrame, PixelFrame, GnomonicFrame, …); composition and inv are frame-checked.
  • Minimal solvers (homography_4pt, p3p, the 5-point essential and the fundamental-matrix solvers) and the manifolds their models are refined on.
  • The robust-loss family (AbstractLossShape, deploy), the block-coordinate-descent/LO interface shared with VisualGeometryToolkitEstimation, and FitResult, the one fit return type.
  • Numerics: the B-spline kernels, the thin-plate spline (ThinPlateSplineSystem, tps_coefficients, tps_evaluate, tps_gradient), public, not exported.
  • Units: FlexUnits as shipped, plus the registered px, pd, dpi and pt (mm, px, dpi, Size2).
  • The sensor image: RawMosaic (a decoded Bayer RAW, an AbstractMatrix of its photosites that knows its colour phase and frame), its channel-first photosite view (a method of ImageCore's channelview), and LinearRaw, its exact linear values on the 2×2 cells, with the exact maps onto the sensor (raw_frame_map, bayer_plane_map). Decoding a file is VisualGeometryToolkitFeatures' (LibRaw).

Plotting loads as a package extension with any Makie backend.

See ARCHITECTURE.md (module graph, include order, extension points) and LOSSES.md (the robust-loss family) at the repository root.

source

Units and Quantities

VisualGeometryToolkitCore.mm — Constant
mm

The millimetre, 1//1000 m exactly: 142mm is 71//500 m. A length is stored in metres, so read one back in millimetres with ustrip(mm, q).

source
VisualGeometryToolkitCore.px — Constant
px

The pixel, a dimensionless count (FlexUnits' dimensions are the seven SI ones, so px cannot be told from a radian or a ratio). Pixel COORDINATES keep their frame through PixelFrame tags; px is for sizes and densities. A length becomes pixels through a density: q * 300dpi.

source
VisualGeometryToolkitCore.BaseQuantity — Type
BaseQuantity

FlexUnits' concrete stored form of a base quantity, computed once as typeof(ubase(1.0mm)) (today Quantity{Float64, Dimensions{FixRat32}}): the type a struct field holds, so a container of mixed dimensions is concrete. It is NOT a stack quantity type — public signatures take any FlexUnits Quantity and convert with base_quantity — and it is named here only so the stack spells FlexUnits' stored form once instead of restating its parameters (see conventions/units.md).

source
VisualGeometryToolkitCore.base_quantity — Method
base_quantity(q::Quantity) -> Quantity

q in FlexUnits' base form with a floating-point value: float(ubase(q)). The ONE conversion a constructor applies to a quantity it stores. A quantity built from an integer (142mm, 3px) holds an exact Rational (71//500 m); its stored form is the correctly rounded Float64, 0.142 m, the decimal-nearest value. A float value keeps its type (Float64, Float32, a ForwardDiff Dual), so base_quantity(1.0mm) is ubase(1.0mm). A Rational never reaches a struct field or a kernel.

source
VisualGeometryToolkitCore.has_units — Method
has_units(x) -> Bool

Whether x holds FlexUnits quantities — a physical value (a pose's translation, a target's points, a rig's mounts, in metres) — rather than plain numbers, the inner loop's form. Defined for numbers, arrays, and every carrier of a physical length (EuclideanMap, Uncertain values, targets, mounts, rigs, cameras, trajectories, target poses and 3D target points).

source
VisualGeometryToolkitCore.parse_unit — Method
parse_unit(s::AbstractString; compound = false) -> Units

The unit a serialized name s spells, resolved by the stack's registry (StackUnits) and its generated uparse — guarded, since uparse evaluates its parse. By default s must be ONE registered unit symbol ("mm", "inch", "px", "dpi", …); compound = true also admits a product, quotient or power of them ("mm/px", "px/mm", "mm^2"), checked on the parsed expression before anything is evaluated. Anything else is an ArgumentError, never a default.

source
VisualGeometryToolkitCore.strip_metres — Method
strip_metres(x) -> plain numbers

A length, or a vector of lengths, as plain metres: the one strip at a public boundary that takes physical values and computes on plain numbers (a pose's translation, a 3D point, a threshold given beside physical data). It refuses plain numbers where a length is meant — a unit the caller did not state — with FlexUnits' DimensionError, as the arithmetic would raise; a quantity of another dimension is FlexUnits' ConversionError. A static vector stays static. Values are stored in metres, so the strip is exact; a quantity built from an integer (2m, exactly rational) comes out as its correctly rounded float.

source
VisualGeometryToolkitCore.strip_or_bare — Method
strip_or_bare(u::Units, x)

x as plain numbers in unit u: a quantity, or a struct of quantities with a ustrip(u, ·) method (Size2, the feature frames), stripped in u; plain numbers returned as they are. A stripped quantity is a float (float of an exact Rational from 3px). For the producers that emit either (the ASIFT frames are plain Float64, the detectors' are px). A quantity of another dimension is FlexUnits' ConversionError.

source
VisualGeometryToolkitCore.strip_pixels — Method
strip_pixels(x) -> Real

x as a plain number of pixels: a plain number as it is, a px quantity stripped (a float); a quantity of another dimension (a length) is a DimensionError. What a constructor applies to a pixel-sized argument (a centre, a radius on the sensor).

source
VisualGeometryToolkitCore.unit_string — Method
unit_string(u::Units) -> String

The serialized name of a registered unit: FlexUnits' own printing, string(u), which for a registry unit is its symbol. A unit the registry has no symbol for (mm/px, 1000mm) is an ArgumentError: its printing is no name parse_unit reads back. The inverse of parse_unit.

source
VisualGeometryToolkitCore.with_unit — Function
with_unit(x, u::Units)

The inverse of ustrip(u, x) on the stack's carriers: x's plain lengths read as being in the unit u and stored as FlexUnits base-form quantities (metres) — an Uncertain pose's covariance rescaled with them. An ArgumentError when x already has units (has_units). A plain EuclideanMap takes its unit this way too, its rotation kept exactly (construction, EuclideanMap(rotation(E), translation(E) .* u), re-stores a non-RotMatrix rotation as one).

source
VisualGeometryToolkitCore.Size2 — Type
Size2{T}

Width/height container with (optionally unit-carrying) elements for image and canvas sizes.

  • width: columns (x-axis)
  • height: rows (y-axis)

Constructors:

  • Size2(; width, height) keyword constructor
  • Size2(matrix) from a matrix's (width, height)

Examples:

Size2(width=800, height=600)
Size2(width=210mm, height=297mm)
Size2(width=800px, height=600px)
source
VisualGeometryToolkitCore.StackUnits — Module
StackUnits

The stack's unit registry: FlexUnits' RegistryTools over Units{Dimensions{FixRat32}, AffineTransform{Rational{Int}}}, every scale an exact rational in base units. It holds the units the stack reads and writes and nothing else:

unitscale (base units)
m, kg, s, A, K, cd, mol1the SI base units
km, cm, mm, μm (um)1000, 1//100, 1//1000, 1//1000000m
inch (in), ft, pt127//5000, 381//1250, 127//360000m (the international inch; pt = inch/72)
px, pd1, dimensionlessa pixel count; pd the board JSONs' print dot
dpi5000//127px/inch, of dimension 1/m

Its generated uparse, qparse and @ud_str are VisualGeometryToolkitCore's; a name from outside is read with parse_unit, never uparse.

source

Utilities

VisualGeometryToolkitCore.RetiredKeyword — Type
RetiredKeyword()
refuse_retired_keyword(value, old, new) -> nothing

The stack's one refusal of a retired keyword, never an alias: a function keeps the retired keyword old with the default RetiredKeyword() and calls refuse_retired_keyword(old_value, :old, successor) first, which throws ArgumentError("the keywordoldis retired: givenew") for any value a caller supplies and returns nothing for the default. new names what replaces it (a keyword, or the argument that now carries it). LensSpec refuses free this way (it takes fixed), every find_*, ransac and locally_optimize refuses local_optimization (it takes local_optimization_strategy), every refine_* and fit_* the strategy spellings (the owner's ruling, 2026-11-11); a downstream package refuses its own retired keywords through the same pair.

source
VisualGeometryToolkitCore.filter_kwargs — Method
filter_kwargs(T::Type; kwargs...) -> (matched::NamedTuple, dropped::NamedTuple)

Split keyword arguments into those that match fieldnames of type T and those that don't.

This is useful for forwarding a subset of kwargs to a struct constructor while forwarding the remainder to another function.

Arguments

  • T::Type: Type whose fieldnames determine which kwargs are "matched"
  • kwargs...: Keyword arguments to filter

Returns

A tuple of two NamedTuples:

  • matched: kwargs whose names are fields of T
  • dropped: kwargs whose names are not fields of T

Examples

Base.@kwdef struct Config
    width::Int = 100
    height::Int = 100
end

matched, dropped = filter_kwargs(Config; width=200, extra=true)
# matched = (width = 200,)
# dropped = (extra = true,)

config = Config(; matched...)
source
VisualGeometryToolkitCore.require_extension — Method
require_extension(missing_error, parent::Module, extension::Symbol) -> nothing
require_extension(missing_error, f, args::Tuple, parent::Module, extension::Symbol)

The stack's one check for a package extension a call needs. missing_error is a zero-argument function returning the exception to throw when the extension is not loaded: the caller's own message, naming what to install and load (using X), built only on that path.

  • require_extension(missing_error, parent, extension): nothing when parent's extension extension is loaded (Base.get_extension(parent, extension)), else throw(missing_error()). A function calls it first, before any work that would end in the extension.
  • require_extension(missing_error, f, args, parent, extension): the fallback method of a function f whose methods live in the extension, called with the arguments args it received. Without the extension it throws missing_error(). With the extension loaded the arguments fit none of its methods, so it throws the MethodError(f, args) the call would have raised, at the caller's world age (Base.tls_world_age()), so that a call compiled before the extension loaded keeps Julia's "method too new" hint.
_fit(args...; kwargs...) = require_extension(_fit, args, @__MODULE__, :MyPackageEstimationExt) do
    ArgumentError("_fit needs `using VisualGeometryToolkitEstimation` (its package extension is not loaded)")
end
source
VisualGeometryToolkitCore.saturation_bounds — Method
saturation_bounds(::Type{T}) -> (lo, hi)
saturation_bounds(::Type{<:Colorant}) -> (lo, hi)
saturation_bounds(img::AbstractMatrix) -> (lo, hi)

The open interval of pixel values that are NOT clipped by the sensor or the encoding, as Float64s: (1e-6, 1 - 1e-6) for float images, one code inside typemin/typemax for integer ones, one eps inside for fixed-point ones. A pixel outside it carries no intensity information (it is saturated) and must not enter a photometric fit. Colorants defer to their element type; an image to its eltype.

source
VisualGeometryToolkitCore.sha256_hex — Method
sha256_hex(s::AbstractString) -> String
sha256_hex(bytes::AbstractVector{UInt8}) -> String
sha256_hex(io::IO) -> String

Lower-case hex SHA-256 — the one hash behind every content identity in the stack (sha256_hex(canonical_json(tree)), canonical_json) and every file digest (a pinned model's, a data artifact's). A string is hashed as its bytes (never opened as a path); a stream is read to its end. A file's digest is the stream form, open(sha256_hex, path).

source
VisualGeometryToolkitCore.stable_seed — Method
stable_seed(seed, keys...) -> UInt64

The seed derivation of every random stream in the stack: the first eight bytes (big-endian) of the SHA-256 of seed and keys printed as text, separated by the unit separator \x1f. A fixed algorithm, unlike Base.hash (which changes across Julia versions), and keyed by NAME, unlike seed + i: a scene's draw depends on that scene's name only, so adding, removing or reordering other scenes leaves it unchanged, and no arithmetic coincidence gives two key tuples one stream. Keys are told apart by their printed text (string): 2, "2" and Int32(2) are one key, and a key whose text holds \x1f can collide with two keys; integers and plain names are always distinct. Seed a StableRNGs StableRNG with it (StableRNG(stable_seed(seed, "scene", i))), whose stream is fixed across Julia versions too.

julia> using VisualGeometryToolkitCore: stable_seed

julia> stable_seed(7, "budget", 2, 1)
0xf5154d7e8f0e4d1c

julia> stable_seed(1000, 1, 3) == stable_seed(0, 2, 3)
false
source
VisualGeometryToolkitCore.validate_dir — Method
validate_dir(dir; create=false) -> String

Validate a directory path and optionally create it.

Arguments

  • dir: Directory path (String or AbstractString)
  • create::Bool=false: If true, create the directory (and parents) if it doesn't exist

Returns

The validated directory path as a String.

Throws

  • ArgumentError if dir is not a directory and create=false
  • ArgumentError if dir exists but is not a directory (e.g., it's a file)

Examples

# Validate existing directory
path = validate_dir("/tmp")

# Create directory if needed
path = validate_dir("/tmp/mydir"; create=true)
source
VisualGeometryToolkitCore.canonical_json — Method
canonical_json(io::IO, tree)
canonical_json(tree) -> String

The RFC 8785 (JCS) serialization of a lowered JSON tree (see the file header): members sorted by their names' UTF-16 code units, no whitespace, minimal string escapes, every number in ECMAScript's form of its double (1.0 is 1, 1e-5 is 0.00001, 1.0e21 is 1e+21). Throws on a non-finite number, an Int64 past ±2^53 (not exactly a double) and any value outside the lowered vocabulary — a tree that cannot be encoded must fail loudly, never hash to something. Core.sha256_hex(canonical_json(tree)) is a content identity.

source
GeometryBasics.Rect — Method
Rect(s::Size2{T}) where T

Create a Rect from a Size2 with origin at (0,0) and widths filled from the size dimensions.

source
FlexUnits.ustrip — Method
ustrip(u, r::HyperRectangle)

Remove units from HyperRectangle, expressing magnitudes in unit u.

source
VisualGeometryToolkitCore.canonical_basis — Method
canonical_basis(::Type{SVector{N,T}}) where {N,T}

Generate all N canonical basis vectors for ℝⁿ as a tuple.

Returns a tuple containing all canonical basis vectors e₁, e₂, ..., eₙ where eᵢ has 1 in the i-th position and 0 elsewhere. This is type-stable and compile-time optimized using StaticArrays.

Arguments

  • SVector{N,T}: Type of vectors to generate (N dimensions, element type T)

Returns

An N-tuple of SVector{N,T} containing all canonical basis vectors

Examples

# 3D basis
e1, e2, e3 = canonical_basis(SVector{3,Float64})
# e1 = [1.0, 0.0, 0.0]
# e2 = [0.0, 1.0, 0.0]
# e3 = [0.0, 0.0, 1.0]

# 2D basis with Float32
ex, ey = canonical_basis(SVector{2,Float32})

# Use in transformation
R = RotMatrix(...)  # 3×3 rotation matrix
e1, e2, e3 = canonical_basis(SVector{3,Float64})
x_axis = R * e1  # Transform basis vectors
y_axis = R * e2
z_axis = R * e3

See also: VisualGeometryToolkitCore.basis_vector (unexported)

source
VisualGeometryToolkitCore.centroid — Method
centroid(r::HyperRectangle)

Get the centroid (geometric center) of a HyperRectangle (Rect/Rect2).

Note: This returns the midpoint, not the origin. For the corner/origin, use r.origin or GeometryBasics.origin(r).

Examples

rect = Rect2(0.5, 0.5, 1920.0, 1200.0)
c = centroid(rect)  # [960.5, 600.5]
source
VisualGeometryToolkitCore.default_device — Method
default_device() -> KA.Backend

The device every KernelAbstractions-generic entry point in the package family uses when the caller supplies none: an accelerator when one is loaded AND functional, else KA.CPU(). The accelerator's backend value is built with the compiler options its extension chooses (the CUDA extension's always_inline = true); on_device treats it as the same memory as the backend of an array on that device.

Memoized: resolution asks the vendor package whether it is functional, which is not free and cannot change within a session. _DEVICE_RESOLVER[] overrides it (including back to KA.CPU()); reset_default_device! clears the memo so the next call resolves again.

Supplying device = explicitly always wins — this only decides what "unspecified" means, and it is deliberately NOT the cheapest device (see the file header).

source
VisualGeometryToolkitCore.device_free_bytes — Method
device_free_bytes(dev) -> Int

Free memory on the KernelAbstractions backend dev, in bytes: the host's for KA.CPU(), the accelerator extension's query for its backend (CUDA's free_memory), else typemax(Int) with a one-time warning (a caller sizing chunks from it then does not chunk).

source
VisualGeometryToolkitCore.device_names — Method
device_names() -> Vector{String}

The names of the accelerator devices this process sees (the extension's query: CUDA's devices when CUDA is functional); none without an accelerator extension.

source
VisualGeometryToolkitCore.device_resident_threads — Method
device_resident_threads(kernel, args...; ndrange) -> Int

How many threads of kernel can be resident on its device at once: the work its device runs in one wave. kernel is a KernelAbstractions kernel instance — OnBackend(dev)(f!) or f!(dev[, workgroupsize]), the one the caller launches as kernel(args...; ndrange) — and args, ndrange that launch's: the count is of THAT kernel compiled for those arguments, at the workgroup size the launch takes.

  • A GPU (the accelerator extension's backend; CUDA): register- and shared-memory-aware. The compiled kernel's occupancy (CUDA's active_blocks at the launch's block size, the size the backend picks for a dynamic workgroup) times the block size times the multiprocessors. Never the device's raw maximum of threads per multiprocessor: a kernel that uses 255 registers a thread fits about an eighth of it.
  • The host (KA.CPU()): Threads.nthreads(), the threads its kernels run on.
  • Another backend: an ArgumentError (no query; a caller must not guess a capacity).
source
VisualGeometryToolkitCore.device_zeros — Method
device_zeros(T, device, dims...) -> array

A zero array of element type T and size dims on the KernelAbstractions backend device (an Array on KA.CPU()), as ka_upload puts one there: the stack's one spelling of it (KernelAbstractions' own zeros is not public).

source
VisualGeometryToolkitCore.free_device_buffers! — Method
free_device_buffers!(x::AbstractArray, xs::AbstractArray...) -> nothing

Release each device array's memory NOW (GPUArrays' unsafe_free!), rather than when the collector finalizes its few-byte host wrapper — which, with no host memory pressure, it has no reason to do while the device fills. A host array is left alone. Only for buffers the caller allocated and never reads again: a freed array must not be used (GPUArrays throws on its next use). An array derived from another (a reshape) shares its memory, which is released when the last of them is freed. KernelAbstractions' unsafe_free! is not used: CUDA.jl does not extend it, so it freed nothing on the GPU.

source
VisualGeometryToolkitCore.ka_upload — Method
ka_upload(T, device, A) -> array

A copy of A with element type T on device: one Array{T} on the host (Array{T}(A), which also downloads a device array), else a KA.allocated array filled by one copyto! — from A's own memory when it is a dense array of T or a host wrapper of T that is a dense array's memory in order (a reinterpret, a reshape, a contiguous view: no host copy), else from T.(A) (a strided view, an adjoint, another eltype). An integer image (UInt8, UInt16, N0f8, N0f16: an integer or fixed-point eltype narrower than a float T) crosses as it is and is converted ON the device by one KernelAbstractions kernel, T(x) per element, on every backend (the CPU's included): bit for bit the host's T.(A), at sizeof(eltype(A)) bytes a pixel moved instead of sizeof(T).

source
VisualGeometryToolkitCore.on_device — Method
on_device([T = eltype(A)], device, A) -> array

A itself when it already lives on device with element type T, else one ka_upload(T, device, A). Also the way back: a result handed back in the caller's array kind is on_device(get_backend(input), result).

source
VisualGeometryToolkitCore.reclaim_device! — Method
reclaim_device!(dev) -> nothing

Hand the memory dev's backend pool holds back to the device (CUDA.jl's reclaim), so an allocator outside that pool (ONNX Runtime's) can use it; nothing for the host or a backend without a pool query.

source
VisualGeometryToolkitCore.primal_float — Method
primal_float(T::Type) -> Type

The real type under every ForwardDiff Dual layer of T (Float64 for a Dual{Tag, Dual{Tag2, Float64, 1}, 2}, T itself for a non-Dual type): the precision a branch on primal values computes in. A type-level recursion, so it folds at compile time inside a kernel.

source
VisualGeometryToolkitCore.primal_value — Method
primal_value(x) -> Real

The plain number under every ForwardDiff Dual layer of x (x itself when it is no Dual): what a branch, a support region or a tolerance reads of a value that may carry derivatives. Its type is primal_float(typeof(x)).

source

Drawing into an array without Makie (masks, debug frames; public, not exported):

VisualGeometryToolkitCore.draw_ellipse! — Method
draw_ellipse!(img, center, S, cutoff, color; n = 180, w = 2)

Draw the cutoff·σ boundary of the elliptic frame (center, S) (image coordinates): boundary point = center + cutoff · S · [cos t, sin t], n samples around the ring, each stamped w pixels thick.

source
VisualGeometryToolkitCore.draw_projected_ellipse! — Method
draw_projected_ellipse!(img, H, center, S, cutoff, color; n = 180, w = 2)

Draw the cutoff·σ boundary of a WORLD-plane elliptic frame after projecting each boundary point through the world→image homography H (the map before the shape it maps, after the canvas it draws on). The projection of an ellipse is only approximately an ellipse under perspective, so points are mapped one by one rather than through a shape matrix; samples beyond the plane's horizon are skipped (chirality-guarded to_euclidean), never wrapped into view.

source

TOML Configuration

The config-of-record machinery shared by the package family's config types (public, not exported — import by name).

VisualGeometryToolkitCore.allowed_keys — Method
allowed_keys(::Type{T}, extra...) -> Vector{String}

The keys a [T] table may carry: the same field walk toml_table emits, plus extra (a type discriminator, say). Derived, never hand-listed, so a field is readable and writable together — emitting a key the reader then refuses is exactly the asymmetry this avoids.

source
VisualGeometryToolkitCore.check_keys — Method
check_keys(d::AbstractDict, allowed, section::AbstractString; retired = (), renamed = ())

Refuse unknown keys in table d with an ArgumentError naming the section, the offending keys and the allowed set. A typo'd knob that silently does nothing is the failure a config loader exists to prevent — a config that looks like it set max_trials but spelled it max_trial would otherwise run at the default and report success.

retired is a collection of exact (section, key) pairs: knobs removed because nothing read them, which recorded run sidecars (immutable provenance) legitimately still contain. Those load with a one-time warning; every other unknown key throws. Retirement is section-scoped on purpose — the same key in another table is a typo and must still fail.

renamed is a collection of (section, key) => new_key pairs: keys whose knob lives on under a new name. Each is refused with an ArgumentError naming its new key (there is no alias: the old spelling never loads), before the unknown keys are checked; section-scoped like retired.

source
VisualGeometryToolkitCore.convert_field — Method
convert_field(::Type{T}, v)
convert_field(::Type{T}, v, key)

A TOML value → the field type (Float64(v), Int(v), Bool(v), Symbol(v)): the one converter behind a typed present_kwargs spec.

The keyed form checks the value's shape and names key in every error (an ArgumentError): a table where a value belongs is refused; a Tuple field takes an array of exactly its length (any length for a Vararg tuple such as Tuple{Vararg{Float64}}), converted element by element; a Vector field an array, converted element-wise; any other array field (an SVector) an array, converted by its constructor as the unkeyed form does; any other value for a tuple or array field (a string would convert character by character) is refused; a value that does not convert (MethodError, InexactError, ArgumentError, DimensionMismatch) is reported as not a T. read_table's default converter is the keyed form.

source
VisualGeometryToolkitCore.parse_defaults — Method
parse_defaults(path::AbstractString) -> Dict{String, Any}

Parse a package's defaults file (its configs/default.toml) and record it as a precompilation dependency of the package being precompiled, so editing the file recompiles it. Call it at a module's top level with an absolute path: const _TOML_DEFAULTS = parse_defaults(default_config_path()).

source
VisualGeometryToolkitCore.present_kwargs — Method
present_kwargs(d::AbstractDict, spec::Pair{String}...; fallback = nothing, field_of = Symbol)
    -> Vector{Pair{Symbol, Any}}

Typed per-key overrides. For each "key" => convert in spec, emit field_of("key") => convert(d["key"]) when d has the key, else the same from fallback when THAT has it, else nothing at all — so an absent key falls through to the struct's own default rather than to a parallel literal here. field_of maps a TOML spelling to a field name; the default Symbol keeps the key as the field. A loader of structs with Greek fields (:μ read from an ASCII "mu") supplies field_of =toml_field_name.

source
VisualGeometryToolkitCore.read_table — Method
read_table(::Type{T}, d::AbstractDict; skip = (), from_toml) -> Vector{Pair{Symbol, Any}}

The reader's half of toml_table: the keyword overrides of a [T] table d, one f => from_toml(T, Val(f), d[toml_key(T, f)]) for every field f of T whose key is PRESENT in d, in field order — the same fieldnames / toml_skip_fields / toml_key walk the writer emits, minus the fields in skip the caller reads itself. An absent key yields nothing, so the field falls through to the struct's own default: T(; read_table(T, d)...). from_toml(T, Val(f), v) converts one value (a package's per-field hook); the default is convert_field to the field's type. Unknown keys are check_keys' job, with allowed_keys.

source
VisualGeometryToolkitCore.required_key — Method
required_key(d, key, where) -> value

d[key], or an ArgumentError naming where (the file and the table) and the missing key, rather than a KeyError from deep inside a parsed record. The one reader of a key a record or a config must hold.

source
VisualGeometryToolkitCore.toml_default — Method
toml_default(defaults::AbstractDict, table::AbstractString, key::AbstractString)

Look a default up in a parsed defaults file. table is a dotted path ("ransac", "optimization_strategy.hq"; "" is the top level). Throws a KeyError naming the missing table or key BY DESIGN: a keyword default that is not in the file would be a silent second source of truth, which is what the mechanism exists to prevent.

source
VisualGeometryToolkitCore.toml_field — Method
toml_field(c, ::Val{F})

The TOML value of field F of c; default toml_value(getfield(c, F)). Override for the non-mechanical cases only — a nested config struct (toml_table(getfield(c, F))), a tagged union (type = ... discriminator), a sentinel (typemax(Int) written as "unbounded").

source
VisualGeometryToolkitCore.toml_field_name — Method
toml_field_name(key::AbstractString) -> Symbol

The field name a TOML key spells: the inverse of toml_key's Greek spelling ("mu" → :μ), Symbol(key) for every other key. A loader whose structs carry the Greek fields hands it to present_kwargs as field_of = toml_field_name; it is not that function's default, since a struct whose fields are themselves ASCII (nu, gamma, alpha) must not be renamed.

source
VisualGeometryToolkitCore.toml_key — Method
toml_key(::Type, f::Symbol) -> String
toml_key(c, f::Symbol) -> String

The TOML spelling of field f of this type; default String(f), except that a Greek field name is spelled ASCII (:μ → "mu", :α_min → "alpha_min", the table behind toml_field_name) so a config is typeable everywhere. Dispatched on the TYPE so the reader's allowed-key list and the writer share one mapping.

source
VisualGeometryToolkitCore.toml_skip_fields — Method
toml_skip_fields(::Type) -> Tuple{Vararg{Symbol}}

Fields the walker must not emit for this type: data supplied at load (a camera), a property derived from the inputs (a polarity), a scale interface that is not a config key. Default: none. Declared per type so the reason lives next to the type it applies to.

source
VisualGeometryToolkitCore.toml_value — Method
toml_value(x)

A leaf in TOML-representable form: numbers, booleans and strings are returned unchanged; symbols become strings; vectors, tuples and dicts recurse. Anything else is handed to the TOML writer as is (which refuses what it cannot represent).

source
VisualGeometryToolkitCore.write_toml — Method
write_toml(path, body::AbstractDict; header = "") -> path

Write body as sorted TOML, creating the directory, with header (one or more # comment lines, no trailing newline needed) first. The one writer behind every package's write_config.

source

Benchmark Pools

The pool record and the raw-pair interface (using VisualGeometryToolkitCore.Pools). The pool is the stack's one declared exception to "frame in = frame out": its data is stored in NormalizedImageFrame, and every accessor that crosses the boundary names it.

VisualGeometryToolkitCore.Pools — Module
Pools

The benchmark pool record and the raw-pair interface (using VisualGeometryToolkitCore.Pools):

The record holds no residuals: a pool's residual matrices are a separate file, bound to the record by a content key, read by the evaluation package.

source
VisualGeometryToolkitCore.Pools.PoolPair — Type
PoolPair{M}

One image pair of a scoring pool: the light record — everything about the pair except the per-model residual matrix, which is a separate file bound to the record by a content key.

The pool is the one place in the stack that stores data normalized: every point datum and model is in NormalizedImageFrame (image centre, half-diagonal unit), through the per-image maps the record carries. See "Coordinate frame" below.

Type Parameters

  • M: Model type, expressed in NormalizedImageFrame (FundMat{Float64} for the E and F pools, HomographyMat{Float64} for the H pools).

Fields

  • correspondences: Correspondences in NormalizedImageFrame — NOT pixels, stored as the columnar correspondences(sources, targets) of SVector{2,Float64} sides (any vector of 2-vector pairs is converted to it at construction). Each side is mapped pixel → normalized by its own map (source_map, destination_map). pixel_correspondences is the exact way back to pixels.
  • K1::SMatrix{3,3,Float64,9}, K2::SMatrix{3,3,Float64,9}: the pixel calibration of the source and target cameras (calibration_matrices).
  • R_gt::SMatrix{3,3,Float64,9}, t_gt::SVector{3,Float64}: the GT relative motion, source → target (X₂ = R_gt·X₁ + t_gt; gt_pose).
  • models::Vector{M}: Precomputed minimal-sample models, expressed in NormalizedImageFrame (e.g. fundamentals conjugated by the per-image maps).
  • sample_indices::Matrix{Int}: (m, num_models) — 1-based sample indices per model.
  • errors_r::Vector{Float64}: (num_models,) — rotation error [deg] per model vs GT.
  • errors_t::Vector{Float64}: (num_models,) — translation error [deg] per model vs GT.
  • gt_inliers::BitVector: (n,) — GT inlier mask (from bundle adjustment), one entry per correspondence (checked at construction).
  • gt_variance::Float64: σ̂²p = RSS/ν of the residuals under the oracle pose (from the GT R, t) on the SfM-track detections — NOT a DLT refit. `ν = n·dg` (oracle: no fitted DOF). NaN if too few tracks.
  • source_map::SMatrix{3,3,Float64,9}, destination_map::SMatrix{3,3,Float64,9}: the per-image maps T (pixel → NormalizedImageFrame) the data is stored in — the stored truth of the pool's frame, and the two sides of its pool_normalization (source_map(N), destination_map(N)); framed, they are pool_frame_maps.
  • snn::Union{Nothing,Vector{Float32}}: (n,) — the per-correspondence second-nearest-neighbour ratio the pool's hypotheses were sampled by, where the generator's is known; nothing otherwise.

Coordinate frame

All point data (correspondences, models, and the separate residuals) lives in NormalizedImageFrame: each image is normalized by its own T (source_map/destination_map), isotropically — translate to the image centre and scale BOTH axes by 1/s, s = ½√(W²+H²) HALF the image diagonal: (ũ,ṽ) = (u−W/2, v−H/2)/s, which puts the image corners at unit radius (DiagonalNormalization(size) per side: pool_normalization(size1, size2), what the pool generator writes). Every width on pool data — the outlier half-width, the loss width, the IG scale prior — is in this half-diagonal unit. The stored maps are the truth of the frame: every conversion goes through them and none is reconstructed from the image size.

Isotropy is the key property. Feature noise is isotropic in pixels (Σ_px = σ²·I); a single scale maps it to Σ_pool = (σ²/s²)·I, still isotropic, so the scalar σ², the single global outlier half-width, the IG scale prior, and the Sampson distance all stay correctly specified. (A per-axis box normalization would make Σ_pool anisotropic and force a full 2×2 noise covariance through the scoring.) Normalizing by image size — not focal — also keeps the outlier half-width resolution/FOV-uniform: the K⁻¹ calibrated CHART — the z = 1 plane, VisualGeometryToolkitCore's GnomonicFrame — would not (extent ≈ tan(fov/2), so a single outlier half-width drifts ~4× with focal across PhotoTourism), which is why the pool frame is the scoring frame even for the calibrated (essential) problem.

Re-solving / local-optimization applications

Because the pool-frame maps T are stored, the record is self-contained for re-solving and LO: the metric frame is recovered EXACTLY (no reconstruction from image size or principal point). With T and the pixel K, the pool-frame calibration is Kc = T·K (pool_frame_calibrations), and:

  • pool frame → the calibrated z = 1 CHART (GnomonicFrame, not a unit ray): x_chart = inv(Kc) · x_pool, a unit BEARING being x_chart / ‖x_chart‖;
  • a pool-frame fundamental F → metric essential: EssentialMat(F, Kc₁, Kc₂);
  • scoring an essential E in the pool frame: FundMat(E, Kc₁, Kc₂) Sampson.

This is exact, so the global pool-frame outlier half-width needs no per-pair conversion. A pixel tolerance enters the pool frame only through pool_pixel_scale.

Construction

PoolPair(; correspondences, K1, K2, R_gt, t_gt, models, sample_indices, errors_r,
         errors_t, gt_inliers, gt_variance, source_map, destination_map, snn = nothing)
PoolPair(; correspondences, …, gt_variance, normalization::CorrespondenceNormalization, snn = nothing)
PoolPair{M}(; …)

The keyword form is the one to build a record with (every field named; M from models unless given): the maps come either as source_map and destination_map or as the normalization whose sides they are, never both. The positional forms below take the fields in declaration order.

PoolPair{M}(correspondences, K1, K2, R_gt, t_gt, models, sample_indices,
            errors_r, errors_t, gt_inliers, gt_variance,
            source_map, destination_map, snn)
PoolPair{M}(correspondences, K1, K2, R_gt, t_gt, models, sample_indices,
            errors_r, errors_t, gt_inliers, gt_variance,
            N::CorrespondenceNormalization, snn)

(M is inferred from models when omitted.) The maps, or the pool_normalization whose sides they are, are REQUIRED: a record without them is refused with an ArgumentError, never read as the identity — an identity default once silently read a map-less record, a pool from a generator that did not persist its maps, as if it were already in the pool frame. Synthetic data built directly in the pool frame supplies the identity EXPLICITLY. Each map must be affine (last row [0 0 1], else ArgumentError), and the columns must agree in length (DimensionMismatch): per point, gt_inliers and snn (when present) have one entry per correspondence; per model, sample_indices has one column and errors_r, errors_t one entry. Every pool on disk satisfies this (all are format v4, loaded through these checks). The record is built by the loader from plain data; it is never serialized as a struct.

source
VisualGeometryToolkitCore.Pools.best_model_index — Method
best_model_index(score, M::Integer) -> Int

The one pool argmax: the index of the best of models 1:M under score(j). The FIRST strictly best model wins; a non-finite score (NaN, ±Inf) never wins; 0 (never a valid index) means no model scores a finite value.

source
VisualGeometryToolkitCore.Pools.check_residual_matrix — Method
check_residual_matrix(pair::PoolPair, R::AbstractMatrix) -> nothing

The pool record's residual-matrix contract: R is num_points(pair) × num_models(pair), its rows the pair's correspondences and its columns the stored models. Any other shape is a DimensionMismatch, so a matrix of another pair is refused rather than read. Every scorer entry that reads a pair's residuals calls it.

source
VisualGeometryToolkitCore.Pools.gt_pose — Method
gt_pose(pair::PoolPair) -> (R_gt, t_gt)

The ground-truth relative pose (source → target, X₂ = R_gt·X₁ + t_gt) as (SMatrix{3,3,Float64,9}, SVector{3,Float64}). t_gt is the stored (unnormalized) translation; pose-error callers use it direction-only.

source
VisualGeometryToolkitCore.Pools.pixel_correspondences — Method
pixel_correspondences(pair::PoolPair) -> StructArray

The pair's correspondences back in PIXELS: each side through the inverse of its stored map (pool_frame_maps), so the result is in the pixel frame the pool's maps were built in (the generator's input frame). The exact inverse of the pool's normalization up to rounding; the way out of the pool frame for a consumer that works in pixels.

source
VisualGeometryToolkitCore.Pools.pixel_residuals — Method
pixel_residuals(pair::PoolPair) -> Matrix{Float32}

The image-plane Sampson residuals of the pair's stored models in the PIXEL frame, num_points × num_models. Pixels are the frame the SIMFIT++ reference works in, so its pixel bounds apply to them without any scalar approximation (the pool frame scales the two images independently, so pool_pixel_scale is a bound on the scale, not a conversion). One n×M scan of the light record: the correspondences come back to pixels through the stored maps (pixel_correspondences), each model by composition with the pool normalization's two sides, inv(D) ∘ M ∘ S (a pool F → T₂ᵀ F T₁, a pool H → T₂⁻¹ H T₁). A model type with neither law (a bare matrix) has no sampson_distance and is a MethodError.

It never reads the pool's stored residual matrix, so it does not depend on that matrix's residual kind. The stored residuals of an essential pool are TANGENT Sampson distances (the pool generator's residual_kind = :tangent_sampson), those of an F or H pool image-plane Sampson (:sampson); this exit is image-plane Sampson for every pool, the quantity SIMFIT++'s pixel bounds are stated in.

source
VisualGeometryToolkitCore.Pools.pool_frame_calibrations — Method
pool_frame_calibrations(pair::PoolPair) -> (Kc1, Kc2)

The POOL-FRAME calibration matrices Kc = T·K of the source and target cameras: the pixel calibrations of calibration_matrices moved into the pool frame by the stored maps source_map, destination_map through reframed_calibration (each map an isotropic scale and a translation; any other affine map is an ArgumentError), which map pool-frame coordinates ↔ the calibrated z = 1 chart (GnomonicFrame) exactly. What a problem built on the stored pool data takes as its K.

source
VisualGeometryToolkitCore.Pools.pool_frame_maps — Method
pool_frame_maps(pair::PoolPair) -> (S, D)

The stored per-image maps as frame-typed affine maps, S::FramedAffine{ImageFrame{:cam1}, NormalizedImageFrame{:cam1}} and D::FramedAffine{ImageFrame{:cam2}, NormalizedImageFrame{:cam2}}: pixels of each view in, the pool frame out; inv goes back. The frame lives on the maps in memory only — the stored maps are plain matrices.

source
VisualGeometryToolkitCore.Pools.pool_imc — Method
pool_imc(pair::PoolPair, j::Integer) -> Float64

The stored selection pose error [deg] of pool model j — the "imc" every pool evaluation reports for the model it selects — dispatched on the pool's model type:

  • a homography pool (HomographyMat): the rotation error errors_r[j] alone — a homography's translation is known only up to the plane's scale, so its stored errors_t is not part of the selection error;
  • every other pool (FundMat, EssentialMat): max(errors_r[j], errors_t[j]).

The one definition: the scorers' staged pairs bake it in, and so every staged grid.

source
VisualGeometryToolkitCore.Pools.pool_normalization — Method
pool_normalization(pair::PoolPair) -> CorrespondenceNormalization
pool_normalization(size1, size2) -> CorrespondenceNormalization
pool_normalization(raw::StereoRawPair) -> CorrespondenceNormalization

The pool's two-sided normalization (pixel → NormalizedImageFrame per image), with its inverses. Of a record: its stored maps, pair.source_map and pair.destination_map. Of two image sizes: the map a pool of such images is written in, DiagonalNormalization per side (image centre, half diagonal, corners at unit radius) — THE construction of the pool frame from an image size; a raw pair's is that of its two image sizes; a record built from it (PoolPair{M}(…, N, snn)) stores its sides. A stored pool model M comes back to pixels by composition with the two sides, inv(D) ∘ M ∘ S with S, D = source_map(N), destination_map(N), and goes the other way by D ∘ M ∘ inv(S). It returns a normalization, not a frame: *Frame names frame tags only. The same maps, frame-typed, are pool_frame_maps.

source
VisualGeometryToolkitCore.Pools.pool_pixel_scale — Method
pool_pixel_scale(pair::PoolPair) -> Float64

Pool units per PIXEL of the pair: the isotropic scale (similarity_scale) of its pixel → NormalizedImageFrame maps, the larger of the two views (a gate takes one tolerance; the two half-diagonals differ by a few percent, so the denser view's 0.1 px reads as ≈ 0.1 px in the other). THE entry of a pixel tolerance into the pool frame: τ_pool = pool_pixel_scale(pair) · τ_px.

source
VisualGeometryToolkitCore.Pools.rig — Method
rig(pair::PoolPair) -> StereoRig

The GT stereo rig of the pair, built on access from the stored pixel calibrations and GT pose (camera 1 at the origin, camera 2 at the GT pose; view ids :cam1, :cam2, the ids of pool_frame_maps). The record stores plain matrices, not the rig: a change to the rig type never makes a stored pool unreadable.

source
VisualGeometryToolkitCore.Pools.StereoRawPair — Type
StereoRawPair

Abstract supertype of a dataset reader's raw image pair: the input a pool pair is drawn from, in PIXELS. Read through its accessors, whose defaults read the field of the same name (a reader that stores a quantity differently overrides the accessor):

AccessorDefault fieldMeaning
pair_correspondencescorrespondencespixel correspondences x₁ => x₂
pair_snnsnnper-correspondence second-nearest-neighbour ratio (low = good)
pair_K1, pair_K2K1, K2pixel calibration matrices
pair_size1, pair_size2size1, size2image sizes (Size2)
pair_R_gt, pair_t_gtR_gt, t_gtGT relative motion, source → target
pair_gt_inliersgt_inliersGT inlier mask
pair_scene—the pair's source-scene label, nothing unless overridden

Its pool frame is pool_normalization(raw), the normalization of its two image sizes.

source

Spatial Queries

Allocation-free queries on SpatialIndexing.jl's R-tree (using VisualGeometryToolkitCore.SpatialQueries): the intersection query with early exit and the branch-and-bound nearest neighbour, with a reusable state for batches, and the in-place OMT bulk load. bbox gives a shape's query rectangle.

VisualGeometryToolkitCore.SpatialQueries — Module
SpatialQueries

Allocation-free queries on a SpatialIndexing.jl RTree (using VisualGeometryToolkitCore.SpatialQueries):

  • foreach_intersecting: call a function on every element whose bounding box intersects a query Rect, with early exit;
  • nearest_neighbor: the element nearest a point (branch-and-bound), within an optional squared radius;
  • nearest_neighbor! with a NearestNeighborState: the same query with its state supplied, so a batch of queries allocates nothing;
  • bulk_load!: SpatialIndexing's OMT bulk load, sorting the caller's elements in place.

The trees are SpatialIndexing's own (RTree, SpatialElem, Rect); build them with bulk_load!, or SpatialIndexing's insert! or load!. VisualGeometryToolkitCore's bbox gives a shape's Rect.

source
VisualGeometryToolkitCore.SpatialQueries.bulk_load! — Method
bulk_load!(tree::RTree{T,N,V}, elems::Vector{V}) -> tree

Bulk-load elems into the empty tree by SpatialIndexing's OMT method (Lee & Lee, "OMT: Overlap Minimizing Top-down Bulk Loading Algorithm for R-tree"), sorting elems in place: the caller hands the vector over and must not rely on its order afterwards. The tree is the one SpatialIndexing.load!(tree, elems) builds, node for node, without the copies load! sorts at every level.

tree = RTree{Float64, 2}(Int, Nothing)
bulk_load!(tree, [SpatialElem(SpatialIndexing.Rect(p, p), j, nothing) for (j, p) in enumerate(points)])
source
VisualGeometryToolkitCore.SpatialQueries.foreach_intersecting — Method
foreach_intersecting(f, tree::RTree{T,N}, rect::SpatialIndexing.Rect{T,N}) -> Bool

Call f(elem) on every element of tree whose bounding box intersects rect (closed boxes: touching counts). f returns true to stop the query early and false to continue. Returns true when f stopped it.

The elements are SpatialIndexing's (SpatialElem: SpatialIndexing.id(elem), SpatialIndexing.mbr(elem)), visited in tree order. Allocation-free when f is: unlike intersects_with, no iterator state is built per element.

hits = 0
foreach_intersecting(tree, SpatialIndexing.Rect((0.0, 0.0), (1.0, 1.0))) do elem
    hits += 1
    false                                   # keep going
end
source
VisualGeometryToolkitCore.SpatialQueries.nearest_neighbor — Method
nearest_neighbor(tree::RTree{T,N}, point::NTuple{N,<:Number}; max_sqdist = Inf) -> (id, sqdist) | nothing

The element of tree nearest point: its SpatialIndexing.id and its squared Euclidean distance, measured to the element's bounding box (for point elements, the point-to-point distance). nothing when no element lies within max_sqdist (inclusive) or the tree is empty. Ties keep the element visited last in tree order.

Depth-first branch-and-bound: a subtree whose bounding box is farther than the best distance so far (seeded by max_sqdist) is skipped, so a query touches only the neighbourhood of point. Allocates its one state; a batch of queries supplies one to nearest_neighbor! instead.

source

Visualization Stubs

Stubs implemented by the Makie/GLMakie package extension (load a Makie backend to activate them); see the Visualization guide.

VisualGeometryToolkitCore.imshow — Function

Display an image with correct orientation using a standard Makie Axis. Requires a Makie backend (implementation lives in VisualGeometryToolkitCoreMakieExt).

source
VisualGeometryToolkitCore.poly! — Function

Add a polygon/shape to an LScene. Not exported to avoid Makie conflicts. Requires a Makie backend (implementation lives in VisualGeometryToolkitCoreMakieExt).

source

Index