Getting Started

This guide covers installation and the concepts the rest of the documentation assumes.

Installation

Install VisualGeometryToolkitCore from the General registry (Julia 1.13+):

using Pkg
Pkg.add("VisualGeometryToolkitCore")

The Schur complement of the LM driver

The stack's Levenberg–Marquardt driver is NLLSsolver. Release 5.3.1 contains our pull requests #61–#65 (allocation-free copies, OptimizeCache, the trial-point reuse, a NaN trial rejected, a non-finite start stopped). One remains open, #60, the Schur-complement linear system for eliminable variables: without it everything is correct, but the relative-pose bundle solves the full system and runs about 2× slower. To use it now, pin our fork once in your environment:

using Pkg
Pkg.add(PackageSpec(url = "https://github.com/prittjam/NLLSsolver.jl", rev = "stack"))

stack is the latest upstream release plus the unmerged pull requests, rebased onto each new release. Verify with VisualGeometryToolkitCore.nlls_features().schur → true. For a run that feeds a figure, replace rev = "stack" by the commit sha so every machine resolves the same tree. When #60 is released, Pkg.free("NLLSsolver") returns the environment to the registry and this note is removed.

For visualization support, also install GLMakie:

Pkg.add("GLMakie")

Basic Usage

using VisualGeometryToolkitCore

With visualization:

using VisualGeometryToolkitCore
using GLMakie  # Activates the plotting extension

Quick Start

The README's first example, run by the docs build (a doctest):

julia> using VisualGeometryToolkitCore

julia> ellipse = Ellipse(Point2(0.0, 0.0), 3.0, 2.0, π/4);

julia> circle = Circle(Point2(1.0, 2.0), 1.5);

julia> line = Line2D(Point2(0.0, 0.0), Point2(1.0, 1.0));

julia> centroid(circle)
2-element GeometryBasics.Point{2, Float64} with indices SOneTo(2):
 1.0
 2.0

julia> signed_distance(line, Point2(0.0, 1.0))   # + on the normal's side
0.7071067811865475

julia> classify_conic(ellipse)
EllipseTrait()

julia> VisualGeometryToolkitCore.has_boundary(circle)   # public, not exported
VisualGeometryToolkitCore.HasBoundary()

Key Concepts

Physical Units

VisualGeometryToolkitCore uses FlexUnits.jl for physical quantities; a pixel (px) is a dimensionless count:

julia> using VisualGeometryToolkitCore

julia> focal = 35.0mm; pixel_size = 3.45μm;   # lengths

julia> x_coord = 100.0px;                       # pixels (logical)

julia> ustrip(mm, 2.5cm)                        # strip in a named unit
25.0

julia> ustrip(px, 1inch * 300dpi)               # a length times a density is pixels
300//1

julia> VisualGeometryToolkitCore.base_quantity(142mm)   # an integer literal is exact; stored as Float64
0.142 m

Values are stored in base units (metres): strip in a named unit, ustrip(mm, distance) == 50.0. The registry's scales are exact rationals, so 142mm is exactly 71/500 m until a constructor stores its Float64 form (base_quantity). ud"…" and uparse parse unit strings, and dimension(q) == dimension(mm) checks a dimension at an API boundary (a pixel is dimensionless, so dimension(px) is also an angle's or a ratio's).

Coordinate Conventions

Different vision libraries place the pixel origin differently:

framefirst-pixel center
PixelFrame{:opencv}(0, 0)
PixelFrame{:colmap}(0.5, 0.5)
PixelFrame{:julia}(1, 1)

VGC math is COLMAP convention throughout. Origin conversion is a labeled rigid map between pixel frames:

T = EuclideanMap(PixelFrame{:colmap}, PixelFrame{:julia})
pt_julia = T(pt_colmap)

See Coordinate Frames for the frame-tag system this is part of.

Coordinate Frames

Maps carry Src/Dst frame tags at the type level; composition and inversion are frame-checked, and unlabeled maps compose with anything (the gradual-typing wildcard). A camera pose and its extrinsics are the same EuclideanMap type under swapped labels — inv(::CameraPose)is a CameraExtrinsics. This system is opt-in and pervades the package; the Coordinate Frames guide is the place to start.

Module Structure

  • VisualGeometryToolkitCore.Cameras — camera models, sensors, projection, stereo rigs, the imaging chain, undistortion.
  • VisualGeometryToolkitCore.Primitives — geometric primitives.
  • Transforms, frames, epipolar forms, solvers, and robust-estimation primitives are defined in the top-level module.

Commonly-used names are re-exported from the main module:

# These are equivalent:
camera = VisualGeometryToolkitCore.Cameras.PosedCamera(model, pose)
camera = PosedCamera(model, pose)  # Re-exported

Feature detection lives in VisualGeometryToolkitFeatures.jl; the estimation engines live in VisualGeometryToolkitEstimation.jl (see Robust Estimation).

The Attributed{V,A} Pattern

Values that carry metadata use a single composition type: Attributed{V,A} wraps a value V with attributes A, with property access forwarded transparently. Uncertain{V,M} is the same type with a covariance or precision matrix as the attributes.

julia> using VisualGeometryToolkitCore

julia> p = Attributed(Point2(320.5, 240.5), 1.35);   # a depth-attributed pixel, as consumed by transfer(rig, p)

julia> p.value                                        # the Point2
2-element GeometryBasics.Point{2, Float64} with indices SOneTo(2):
 320.5
 240.5

julia> p.attributes                                   # the depth
1.35

julia> u = Uncertain(Point2(100.0, 200.0), CovMat{2}(σ = 0.5));   # an uncertain point, Σ = σ²·I

See Design: Attributed{V,A} for the rationale.

Next Steps