Batch API
Per-point maps over arrays, the B-spline kernels and the thin-plate spline. See the Batches guide; the lens surfaces (project!, …) are documented with the Lenses API.
Pointwise maps
VisualGeometryToolkitCore.OnBackend — Type
OnBackend(backend; workgroupsize = nothing)
(exec::OnBackend)(kernel[, ndrange]) -> KernelAbstractions.KernelThe stack's ONE kernel-launch policy. Called on a KernelAbstractions @kernel, it instantiates it on backend's launch value: one per backend type, so a kernel compiles once whichever value of that type the caller holds (Core's CUDA extension compiles every CUDA launch with always_inline = true, the default_device() it registers, and get_backend(::CuArray)'s CUDABackend() launches the same), with the workgroup workgroupsize names, per backend:
nothing: the backend's choice (on the CPU, KernelAbstractions' default: up to 1024 items as one workgroup on the calling task, a larger range in groups of 1024 items over the threads).- one work item (
1,(1, 1),(1, 1, 1)): one item per workgroup on the CPU, so a kernel whose work item is a long loop (a pool's (pair, model[, grid point]) item) runs its items on every thread where the default's 1024-item groups leave most threads idle; on a device, one warp (32 items) per workgroup: one item would leave 31 of 32 lanes idle, and the backend's own choice puts these small ranges of long items on few multiprocessors (RANSACScoringDoneRight's grids on a B200: the Student-t grid 30.6 ms at one item, 1.55 at the backend's choice, 1.35 at one warp; its mixture marginal 62.1, 90.4 and 17.7 ms). - any other size (a kernel's tile, a measured device size): that size, on every backend.
A static ndrange instantiates a statically sized kernel at workgroupsize as given (the CPU's allocation-free one-item launch). Every stack kernel is instantiated here.
As pointwise!'s exec, it is that one kernel's launch; without it the kernel runs on get_backend(out) with the defaults. A functor whose fields are each one of its type parameters or an isbits value reaches the kernel as its fields, rebuilt on the device: one large argument costs local memory there.
VisualGeometryToolkitCore.pointwise! — Method
pointwise!(f, out, xs...; exec = OnBackend(get_backend(out))) -> outout[i] = f(xs[1][i], xs[2][i], …) for every index, as ONE KernelAbstractions kernel on the backend that owns out (every backend, the CPU's included), or on the backend OnBackendexec names. out and xs share one shape (no broadcasting: a DimensionMismatch otherwise). f is any per-point function; for a GPU array it must be isbits and kernel-lawful (no allocation, no throw, no dynamic dispatch), as every lens and camera map is. On the CPU the launch has finished on return; on a device it is asynchronous, as a device broadcast is: KernelAbstractions.synchronize(backend) before out is read on the host.
VisualGeometryToolkitCore.pointwise_kernel — Method
pointwise_kernel(f, out, xs...; exec = OnBackend(get_backend(out))) -> (kernel, args, ndrange)The one launch pointwise!(f, out, xs...; exec) makes, unlaunched: the KernelAbstractions kernel instance, its arguments and its ndrange, so kernel(args...; ndrange) is that call. pointwise! is exactly this followed by the launch (none for an empty out). Its use is to ask about the launch without restating how it is built: device_resident_threads(pointwise_kernel(f, out, xs...)...) counts the threads of the very kernel pointwise! runs (the functor adapted to the backend and given to the kernel as its fields or whole, the kernel compiled for those arguments). The arguments are checked as pointwise! checks them (DimensionMismatch).
B-spline Kernels
The B-spline algebra on the CPU is BSplineKit's (knot vectors, evaluation and derivatives, Galerkin matrices, recombined bases). Core keeps only what a GPU kernel needs and BSplineKit cannot give it: the scalar Cox–de Boor basis bspline_basis(knots, Val(p), x) — an SVector of every basis function at x on a static knot vector, isbits and generic in the element type, so it runs in a kernel and under ForwardDiff (hoisted from RadiometricCamera, its arithmetic unchanged; equal to BSplineKit's evaluate_all on the same knots) — its broadcast over a device array, and the tensor-product surface/4D spline kernels built on it; the ChainRulesCore rrules attach via the package extension.
using BSplineKit: BSplineBasis, BSplineOrder, knots
k = SVector{11}(knots(BSplineBasis(BSplineOrder(4), range(-0.5, 0.5; length = 5)))) # 7 cubic control points on [-½, ½]
N = bspline_basis(k, Val(3), 0.12) # SVector{7}, sum(N) == 1VisualGeometryToolkitCore.bspline_4d_partial — Method
bspline_4d_partial(u1, u2, u3, u4, degrees, k1, k2, k3, k4, control_points, dim)∂S/∂udim of the 4D spline at the sample points, via the derivative surface (degree − 1 along dim, differenced CPs, trimmed knots). AD-transparent wrt `controlpoints`.
VisualGeometryToolkitCore.bspline_basis — Method
bspline_basis(knots::SVector{K}, Val(p), x) -> SVector{K − p − 1}
bspline_basis(knots::SVector{K}, p::Integer, x)Every degree-p B-spline basis function on knots at x, N_{1,p}(x), …, N_{K−p−1,p}(x) (Cox–de Boor, Piegl & Tiller 1997): 0/0 is 0, and the last non-empty knot span is closed on the right, so the values sum to one on the whole closed interval [knots[1], knots[K]]. x outside it is clamped onto it (constant extrapolation).
The basis (its knots and degree) leads, the point follows (signatures.md). The degree is a Val so the result's size is static and nothing allocates; the Integer form is the convenience. Scalar, isbits and generic in the element type (promote_type of x's and the knots'): it runs inside a KernelAbstractions kernel, in Float32, and under ForwardDiff. Why it is Core's and not BSplineKit's: a kernel needs an isbits basis on a static knot vector with a static result; on the CPU, BSplineKit's evaluate_all gives the same values (the tests check it). bspline_basis_iterative is its broadcast over an array.
VisualGeometryToolkitCore.bspline_basis_iterative — Method
bspline_basis_iterative(u, degree, knots) -> Vector{<:AbstractArray{Float32}}For each of the n_basis = length(knots) − degree − 1 basis functions, its values at every point of the device array u: bspline_basis broadcast over u (on u's backend) in Float32, split per basis function — the structure-of-arrays form the tensor-product kernels below consume. u outside the knots is clamped onto them, as by bspline_basis.
Kept because the tensor-product kernels below run on the device array's backend; validated against BSplineKit.BSplineBasis on the downstream demo's actual θ knot vector — see test/bspline_tests.jl.
VisualGeometryToolkitCore.bspline_surface_partial — Method
bspline_surface_partial(u, v, degree_u, degree_v, knots_u, knots_v, control_points, along)First partial derivative of a B-spline surface along parameter direction along (1 = u, 2 = v), via the derivative-surface trick: difference the control points, drop one knot at each end, evaluate at degree − 1 along the differentiated direction. Shared by the evaluate_bspline_surface rrule (package extension) and the downstream spline-lens Jacobians.
VisualGeometryToolkitCore.contract_basis_4d_weighted — Method
contract_basis_4d_weighted(u1, u2, u3, u4, degrees, k1, k2, k3, k4, w)Σ_s w_s · (4D basis product at sample s), returned in control-point shape — the single memory-critical primitive behind every 4D control-point gradient (the 4D spline rrule and the downstream boundary/render accumulations). Weights are folded to per-sample scalars FIRST, so only one (n1·n2, N)×(N, n3·n4) GEMM remains and the dense per-sample basis tensor is never materialized.
VisualGeometryToolkitCore.evaluate_bspline_4d — Method
evaluate_bspline_4d(u1, u2, u3, u4, degrees, k1, k2, k3, k4, control_points)
-> AbstractArray{Float32}4D tensor-product B-spline: S(u) = Σijkl N1i(u1)·N2j(u2)·N3k(u3)·N4_l(u4)·cp[i,j,k,l]. The four inputs must share one shape (callers materialize broadcasts first); the result has that shape. degrees is indexable per dim (NTuple or Vector). Differentiable wrt the inputs and the control points via the ChainRulesCore rrule (package extension).
VisualGeometryToolkitCore.evaluate_bspline_surface — Method
evaluate_bspline_surface(u, v, degree_u, degree_v, knots_u, knots_v, control_points)Tensor-product B-spline surface evaluation on the compute device (kept for the device: BSplineKit evaluates on the CPU; tested equal to its Spline evaluation on the same knots). Inputs u, v are broadcasted device arrays of equal shape; the surface is evaluated point-wise and an array of the same shape is returned. Differentiable wrt u, v, and the control points via the ChainRulesCore rrule (package extension). Periodicity in the second parameter direction is handled by the CALLER tying the wrap control-point columns — no knot wrapping happens here.
The bending energy of a B-spline
bending_gram is ∫∫ f_xx² + 2 f_xy² + f_yy² of a tensor-product B-spline as a quadratic form in its coefficients, from BSplineKit's Galerkin matrices; bending_rows factors it into a regularizer's rows and bending_stencils gives it cell by cell on a large uniform lattice (cubic_lattice_basis, Core's generic-model lattice). The natural boundary condition f'' = 0 of BSplineKit's recombined basis is natural_extrapolation, the rule BoundaryRule applies to a lattice's nodes outside an imaged region by default (order = 2); its order = 3 is the curvature-preserving Δ³c = 0.
VisualGeometryToolkitCore.bending_gram — Method
bending_gram(B) -> Matrix{Float64}
bending_gram(Bx, By) -> Matrix{Float64}The bending energy of a B-spline as a quadratic form in its coefficients: ∫ f''² of a curve on the BSplineKit basis B; ∫∫ f_xx² + 2 f_xy² + f_yy² of the tensor-product surface on Bx ⊗ By, vec(c)ᵀ G vec(c) with c laid [i, j] (the first index fastest). BSplineKit's Galerkin matrices of the basis's derivatives, over the bases' domains. Positive semidefinite, the affine functions its null space. A regularizer reads it as rows (bending_rows) or, on a large uniform lattice, cell by cell (bending_stencils, and over a mask of held nodes bending_rows(mask)); a lattice of vector-valued controls applies it per component.
VisualGeometryToolkitCore.bending_rows — Method
bending_rows(mask::AbstractMatrix{Bool}; spacing = (1, 1)) -> SparseMatrixCSC{Float64, Int}The bending energy's rows over the nodes of a uniform cubic lattice that mask holds (true: a node the rows may read), as a sparse operator on the lattice's column-major vector. These are the bending_stencils of the size(mask) lattice (cell size spacing) over the cells whose 16 nodes are all in mask: each cell's rows L in cell order, the first index fastest. It is the masked bending operator beside the masked differences of difference_rows(mask, order). A regularizer of a lattice that holds only the nodes an imaged region reaches reads it as a penalty operator (mean_positive_eigenvalue, row_stencils). A lattice of vector-valued controls applies it per component. A mask that holds no complete cell gives an operator with no rows.
VisualGeometryToolkitCore.bending_rows — Method
bending_rows(G) -> Matrix{Float64}
bending_rows(tps::ThinPlateSplineSystem) -> Matrix{Float64}The bending energy's rows, the stack's second penalty operator kind beside difference_rows: a factor L of a positive semidefinite bending form — a B-spline lattice's Gram matrix (bending_gram), or a thin-plate spline system's Q (bending_energy) — with ‖L c‖² = cᵀ G c to rounding, so √λ · L (c − c₀) are a regularizer's residual rows (the chart LM's row protocol) and L a penalty operator for mean_positive_eigenvalue and cgls. L = Λ₊^½ Vᵀ from G = V Λ Vᵀ, one row per positive eigenvalue (the null space — a system's affine values — and its rounding residue have no row; bending_factor is the same factor with a row for every node, the fixed shape a target calibration kind stores); only LᵀL is defined.
VisualGeometryToolkitCore.bending_stencils — Method
bending_stencils(nx, ny; spacing = (1, 1)) -> (; windows, L)The bending energy of an nx × ny uniform cubic lattice (cubic_lattice_basis per axis, cell size spacing) cell by cell, for a lattice too large for a dense form: E = Σₖ ‖L c[windows[k]]‖² over the (nx − 3)(ny − 3) cells, windows[k] the 16 linear indices of cell k's 4 × 4 window (the first index fastest) and L the factor of one cell's 16 × 16 form — bending_gram of a one-cell lattice, shared by every cell of a uniform lattice. A row involves one window only, so a solve that blocks the lattice keeps its rows local.
VisualGeometryToolkitCore.clamped_knots — Method
clamped_knots(n, p, lo, hi) -> SVector{n + p + 1, Float64}The clamped (open) uniform knot vector of n basis functions of degree p on [lo, hi]: p + 1 knots at each end and n − p − 1 interior knots spaced evenly, BSplineKit's knots(BSplineBasis(BSplineOrder(p + 1), range(lo, hi; length = n − p + 1))) as the static vector bspline_basis takes (a kernel's knots). The spline interpolates its first and last coefficient at lo and hi. Needs n > p.
VisualGeometryToolkitCore.cubic_lattice_basis — Method
cubic_lattice_basis(n; spacing = 1) -> BSplineKit.BSplineBasisThe uniform cubic B-spline lattice of n nodes (Core's generic-model lattice) as a BSplineKit basis: knots (−3:n) · spacing, so basis j (node j) is centred at (j − 2) · spacing and the spline's domain is the n − 3 cells [0, (n − 3) · spacing] where the basis sums to one.
VisualGeometryToolkitCore.difference_rows — Method
difference_rows(mask::AbstractMatrix{Bool}, order; axes = (1, 2), mixed = 0) -> SparseMatrixCSC{Float64, Int}
difference_rows(dims::Dims{2}, order; axes = (1, 2), mixed = 0)The forward differences Δᵒʳᵈᵉʳ (difference_stencil, any order ≥ 1: 1 a gradient, 2 the natural rule's, 3 the P-spline's) of a 2-D lattice of coefficients along the lattice axes (a subset of (1, 2), in that order), as a sparse operator on the lattice's column-major vector: one row per window of order + 1 consecutive nodes that lies wholly in mask (true: a node the rows may read), first the windows along the first axis ((i, j) … (i + order, j), j outer), then those along the second. dims is the unmasked lattice (trues(dims)): a full texel page (TW, TH) at order = 1 is its gradient, the horizontal differences first. A lattice of vector-valued controls applies it per component.
DᵀD is the P-spline difference penalty (Eilers & Marx 1996): at order = 3 it annihilates a quadratic along each axis, so only the variation beyond it is penalized. An N × 1 mask is a profile (rows along its one axis only). The isotropic pairing of the order-1 rows is gradient_pairs; a device solve takes the operator as a CSRMatrix.
A nonzero mixed (order 2 only) appends the mixed differences Δ₁Δ₂c = c(i, j) − c(i + 1, j) − c(i, j + 1) + c(i + 1, j + 1), weighted by mixed, one row per 2 × 2 window that lies wholly in mask (j outer), after the axis rows. The mixed rows read both axes, so they come with axes = (1, 2), or alone with axes = (); a one-axis operator takes none (an ArgumentError). At mixed = √2, ‖D c‖² is the discrete second-order bending energy Σ Δ₁₁² + 2Δ₁₂² + Δ₂₂² of a lattice of square cells, which annihilates exactly the affine functions of the node indices (the thin-plate discretization; on cells h₁ × h₂, weight the three blocks 1/h₁², 1/h₂² and √2/(h₁h₂), as extend_lattice's extrapolation does). The default mixed = 0 gives the axis rows alone.
VisualGeometryToolkitCore.difference_stencil — Method
difference_stencil(order) -> NTuple{order + 1, Float64}The coefficients of the forward difference Δᵒʳᵈᵉʳ of a lattice's coefficients, Δᵒc₀ = Σₖ sₖ cₖ₋₁ over order + 1 consecutive nodes: the binomial coefficients with alternating sign, (1, −3, 3, −1) for order = 3. The one source of a P-spline's difference penalty (VisualGeometryToolkitEstimation's fit_camera) and of BoundaryRule's curvature-preserving extrapolation (Δ³c = 0 solved for the outermost coefficient).
VisualGeometryToolkitCore.gradient_pairs — Method
gradient_pairs(mask::AbstractMatrix{Bool}) -> Matrix{NTuple{2, Int}}
gradient_pairs(dims::Dims{2})Each node's isotropic gradient pair among the rows of difference_rows(mask, 1): at node (i, j), (r₁, r₂) the rows of its forward differences along the first axis ((i, j) → (i + 1, j)) and along the second ((i, j) → (i, j + 1)), 0 where that difference is not a row (the lattice's last node on the axis, or a neighbour outside mask). A penalty on the gradient's magnitude ((D x)[r₁]² + (D x)[r₂]², an L0 or total-variation step) reads both entries of a pair together.
VisualGeometryToolkitCore.natural_extrapolation — Method
natural_extrapolation() -> (w₁, w₂)The natural boundary condition f'' = 0 at a border knot of a uniform cubic B-spline as the outermost coefficient's extrapolation from the two inward, c₀ = w₁ c₁ + w₂ c₂: the relation BSplineKit's RecombinedBSplineBasis(B, Natural()) imposes (its recombination matrix's span), (2, −1) — the rule BoundaryRule applies outward from an imaged region.
Penalized least squares
Every penalized solve of the stack is min ‖A x − b‖² + λ‖D (x − c)‖², one owner per piece. The operator D: difference_rows (orders 1–3 on a masked lattice or a texel page, along chosen axes; gradient_pairs its isotropic order-1 pairs) or bending_rows (the bending energy), and row_stencils its rows as a nonlinear solve's residual blocks. The weight's unit: penalty_scale with mean_positive_eigenvalue (m̄). The linear solve on sparse operators, on any KernelAbstractions device: cgls on a PenalizedOperator of CSRMatrixes (Krylov.jl's CGLS on the stacked operator [A; √W D]; on CUDA the products are cuSPARSE's, under its reproducible algorithm), or normal_cg (Krylov.jl's CG) on a normal operator AᵀA + DᵀWD that is only applied (its rows never stored); each Jacobi-preconditioned on request, each with a stopping rule relative to the normal equations' right-hand side ‖Aᵀb + DᵀWh‖. The weight's choice by generalized cross-validation is VisualGeometryToolkitEstimation's (gcv_curve, gcv_select). No penalty fixes a gauge: a direction neither the data nor the penalty holds is held exactly by its caller.
VisualGeometryToolkitCore.CSRMatrix — Type
CSRMatrix(A::SparseMatrixCSC; device = default_device()) -> CSRMatrixA sparse operator in compressed sparse rows on a KernelAbstractions device (default_device() when none is supplied: an accelerator when one is functional, else KA.CPU()). mul!(y, C, x) sums each row in its stored order, no atomics, the same bits on every run: on CUDA the product is cuSPARSE's CSR product of these arrays (its reproducible algorithm; the CUDA extension), elsewhere one work item per row. Adapt.adapt(backend, C) moves it. The operator a device solve (cgls, normal_cg) applies; its transpose is a second CSRMatrix of transpose(A) (PenalizedOperator holds both).
VisualGeometryToolkitCore.PenalizedOperator — Type
PenalizedOperator(A::SparseMatrixCSC, D::SparseMatrixCSC = spzeros(0, size(A, 2)); device = default_device())The operators of a penalized least squares min ‖A x − b‖² + ‖√w ∘ (D x − h)‖² on a KernelAbstractions device, built once for every solve on them (cgls): A, D and their transposes as CSRMatrixes. D is any of the stack's penalty operators (difference_rows, bending_rows as a sparse matrix), its rows' weights w the solve's.
VisualGeometryToolkitCore.cgls — Method
cgls(op::PenalizedOperator, b; w = 1, h = nothing, diagonal = nothing, x0 = nothing, rtol = 1e-6, max_iter = 500)
-> (; x, iterations, converged, relres)The least-squares solution of min_x ‖A x − b‖² + ‖√w ∘ (D x − h)‖² (op's A and D; w a weight per penalty row, or one for all; h the penalty's target, nothing for 0) by CGLS (Krylov.jl's cgls) — conjugate gradients on the normal equations (AᵀA + DᵀWD) x = Aᵀb + DᵀWh applied as the stacked operator S = [A; √W D], never formed — from x0 (a start: the iteration solves for the correction to it, so a direction neither A nor D holds keeps x0's value; nothing: 0), on op's device (the vectors moved there; x returned on it). Every product is a row product of a stored operator (A, D and their transposes as rows of their own: no transpose product, so the result is the same on every run).
diagonal, the normal matrix's diagonal diag(AᵀA + DᵀWD) (positive), preconditions the iteration (Jacobi: the unknowns' columns scaled to unit norm, S C with C = diagonal^-½), nothing none.
The stopping rule is relative to the normal equations' right-hand side, ‖C sₖ‖ ≤ rtol · ‖C (Aᵀb + DᵀWh)‖, sₖ = Aᵀ(b − A xₖ) + DᵀW(h − D xₖ) the normal-equation residual (C the Jacobi scaling, 1 without one): a reference fixed by the problem, not by the start, so a start that is already close needs few or no iterations. converged when the rule is met within max_iter iterations; relres is the left side over the right side's norm.
The rows of A stored; a problem whose normal operator is only applied (its rows never formed) solves with normal_cg.
VisualGeometryToolkitCore.mean_positive_eigenvalue — Method
mean_positive_eigenvalue(D) -> Float64
mean_positive_eigenvalue(Symmetric(DᵀD)) -> Float64The mean of the positive eigenvalues of a penalty operator's normal matrix DᵀD: the trace over the rank, from the operator D itself (‖D‖²_F / rank D; the rank by a sparse QR for a sparse D, so a lattice of 10⁴ nodes needs no dense eigendecomposition), or from the normal matrix given as Symmetric (tr G / rank G). The two forms agree; a plain matrix is read as the operator, so the normal matrix must come wrapped. The m̄ of the penalty weight's normalization (penalty_scale).
VisualGeometryToolkitCore.normal_cg — Method
normal_cg(N, g; diagonal = nothing, x0 = nothing, rtol = 1e-6, max_iter = 500)
-> (; x, iterations, converged, relres)The solution of the normal equations N x = g of a penalized least squares, N = AᵀA + DᵀWD and g = Aᵀb + DᵀWh (cgls's problem), by conjugate gradients (Krylov.jl's cg) on N APPLIED: N a function N(y, x) writing y = N x into y, or an operator with mul!(y, N, x) (a matrix, a CSRMatrix). The problem whose rows are never stored — a panorama's pixels accumulated into a lattice stencil and a Schur block — solves here; one whose rows are stored solves with cgls, which applies the rows and so loses no digits to forming N.
diagonal, N's diagonal (positive), preconditions the iteration (Jacobi: the symmetric scaling C N C, C = diagonal^-½), nothing none; x0 is a start (nothing: 0; the iteration solves for the correction to it). The vectors live on g's KernelAbstractions backend (diagonal and x0 moved there; x returned on it), and N applies there. The stopping rule is relative to the right-hand side, ‖C (g − N xₖ)‖ ≤ rtol · ‖C g‖ (C the Jacobi scaling, 1 without one), so a start at the solution takes 0 iterations. converged when it is met within max_iter iterations; relres is the left side over the right side's norm.
VisualGeometryToolkitCore.penalty_scale — Method
penalty_scale(energy, N, m̄) -> Float64The unit of a penalty weight: λ = β · energy / N / m̄, β relative to the data's energy per parameter (N parameters) and to the penalty operator's mean positive eigenvalue m̄ (mean_positive_eigenvalue), so β means the same for every operator, lattice and region; β = 1 weighs the penalty like the data. The energy is the caller's: a linear solve's data energy (tr AᵀA, or a radiometric Σϕ²), or a nonlinear refine's Gauss–Newton energy tr JᵀJ over its parameters (the residuals of exact data vanish, so they cannot set a scale).
VisualGeometryToolkitCore.row_stencils — Method
row_stencils(D::SparseMatrixCSC) -> Vector{Tuple{Vector{Int}, Vector{Float64}}}The rows of a sparse operator as the columns (nodes) each reads and its coefficients there, in row order, the nodes ascending: the form of a penalty as a nonlinear solve's residual blocks (one block per row on the variables holding its nodes, its residual √λ·Σₘ sₘ xₙₘ, a caller's chart and row weighting applied to it).
Vignetting and lighting
A camera's vignetting fields, the measured radial profile and the scene's light moved to VisualRadiometryToolkitCore in 0.12.0 (radiometry lifts geometry: that package depends on this one), under the same names. The B-spline basis they evaluate (bspline_basis, clamped_knots) and the support protocol of their control points stay here.
Blur
One blur of an imaging model — a camera's in-focus blur (diffraction, the sensor's anti-aliasing filter, demosaicing), a print's own blur — as a GaussianBlur. A point spread function is a probability density (where a photon from a point source lands), and a Gaussian blur is the density N(0, Σ), so it is held by its covariance, a CovMat{2} (cov; var and std per axis): round, axis-aligned (stored as an SDiagonal) or general, px² on a sensor, m² (quantities) on a print. Several blurs compose into a point spread: two Gaussians in sequence add their covariances (convolve), and a linear map of the plane pushes the covariance forward, Σ ↦ J Σ Jᵀ (propagate).
camera = GaussianBlur(0.6) # round, σ = 0.6 px
print = GaussianBlur(0.02mm, 0.03mm) # axis-aligned, on the print: m² quantities
total = convolve(propagate(J, ustrip(mm, print)), camera) # in mm², through the pixel footprint J (px/mm), then the camera's
std(camera) # (0.6, 0.6) pxVisualGeometryToolkitCore.Cameras.AbstractBlur — Type
AbstractBlurA blur kernel of an imaging model on a plane (a sensor, a print). One of several that compose into a point spread; GaussianBlur is the one implementation.
VisualGeometryToolkitCore.Cameras.GaussianBlur — Type
GaussianBlur(σ) # round: Σ = σ² I
GaussianBlur(σx, σy) # axis-aligned: Σ = diag(σx², σy²)
GaussianBlur(Σ::CovMat{2}) # by its covariance, its frame kept
GaussianBlur(Σ::ScaledIdentity{2}) # round
GaussianBlur(Σ::SDiagonal{2}) # axis-aligned
GaussianBlur(Σ::SMatrix{2, 2}) # general: symmetric, positive semidefiniteA Gaussian blur: the point spread density N(0, Σ), where a photon from a point source lands relative to its ideal image. Its parameter is the covariance Σ, a CovMat{2} whose frame tag names the plane the blur lives on: px² for a blur on a sensor, m² (FlexUnits quantities) for one on a print. σ, σx, σy are standard deviations. The storage is the structure: a round blur is stored as a ScaledIdentity (one number, unchanged by a rotation), an axis-aligned one as an SDiagonal, a general one as a full symmetric SMatrix. Isbits.
The accessors are Statistics.jl's: cov(b) is Σ as stored (its frame kept), var(b) the per-axis variances (Σ₁₁, Σ₂₂) and std(b) the per-axis standard deviations (√Σ₁₁, √Σ₂₂). A print's blur holds m² quantities (GaussianBlur(0.02mm)); ustrip(u, b) is its plain form in u² and with_unit(b, u) reads a plain one as in u².
Blurs in sequence add their covariances (convolve); a linear map of the plane pushes the covariance forward (propagate).
VisualGeometryToolkitCore.Cameras.convolve — Method
convolve(a::GaussianBlur, b::GaussianBlur) -> GaussianBlurThe blur of a followed by b (convolution commutes): the sum of two independent displacements, so the covariances add, diagonal when both are. The two must live on one plane (their frames meet; an unlabeled one takes the other's).
Thin-plate spline
f(𝐩) = a₁ + a_x 𝐩_x + a_y 𝐩_y + Σᵢ wᵢ U(𝐩, 𝐩ᵢ) with U = ½ r² ln r², through nodal values on scattered nodes (hoisted from RadiometricCamera, its arithmetic unchanged). ThinPlateSplineSystem is the host-side solver; the evaluator tps_evaluate and its gradient tps_gradient take isbits (nodes, w, a), allocate nothing and run in a KernelAbstractions kernel. tps_coefficients is a loop (no BLAS), and with the pure-bending projection tps_pure_bending it runs in a kernel on an adapted system (Adapt.adapt(backend, tps), a Float32 one by convert_eltype).
tps = ThinPlateSplineSystem(grid_nodes(5)) # 5 × 5 nodes over [-½, ½]²
w, a = tps_coefficients(tps, values) # linear in the 25 nodal values
w, a = tps_coefficients(tps, tps_pure_bending(tps, values)) # the plane dropped first: a HeightField's (w, a)
f = tps_evaluate(tps.nodes, w, a, SVector(0.1, -0.2))
∇f = tps_gradient(tps.nodes, w, a, SVector(0.1, -0.2))VisualGeometryToolkitCore.ThinPlateSplineSystem — Type
ThinPlateSplineSystem(nodes) -> ThinPlateSplineSystem{N}The solver object of a thin-plate spline through N nodes in the plane: the node positions, the (N + 3) × N matrix coeff that maps nodal values to the spline's weights and affine part, (w; a) = coeff · values, and the N × N bending-energy form Q (valuesᵀ Q values = wᵀ K w, K = [U(𝐩ᵢ, 𝐩ⱼ)], see bending_energy). The target calibration kinds' pure-bending rule (is_pure_bending, tps_pure_bending) reads the 3 × N affine rows coeff[N+1:N+3, :] (a = coeff[N+1:N+3, :] · values); the plane fields [1 𝐩_x 𝐩_y] are formed from the nodes.
Everything it holds is in the CALLER's frame (frame in = frame out): the nodes as given, (w, a) of the spline f(𝐩) in those coordinates, and Q the bending form of those coordinates (valuesᵀ Q values = wᵀ K w, K of the given nodes: the values' unit squared over the nodes' unit squared, a fixed multiple of the bending energy J = ∫ f_xx² + 2f_xy² + f_yy² in the given coordinates).
The system is SOLVED in a conditioned frame, as an estimator's private normalization: the nodes are centred on their centroid and scaled by one isotropic factor σ (HartleyNormalization: mean distance √2 from the centroid), 𝐩′ = σ 𝐩 + 𝐭. An interpolating thin-plate spline is the same surface under a similarity of its domain, so only rounding changes: the weights map back as w = σ² w′, the linear part as (a_x, a_y) = σ (a′_x, a′_y), the constant as a₁ = a′₁ + a′ · 𝐭 + ln σ Σᵢ w′ᵢ |𝐩′ᵢ|² (the kernel's σ² ln σ · r² term, a constant under the side conditions Σ w′ᵢ = 0, Σ w′ᵢ 𝐩′ᵢ = 0), and Q = σ² Q′. (A PER-AXIS scaling is not a similarity: the spline through the same values then bends differently, so it is never used here.) Without it, nodes in pixels (thousands) put r² ln r² near 1e8 beside the unit column of P.
coeff is mapped back from the first N columns of the inverse of the conditioned system matrix [K′ P′; P′ᵀ 0], solved for those N right-hand sides by one LU factorization. The matrices live on the host; tps_coefficients turns nodal values into the isbits (w, a) that tps_evaluate and tps_gradient take.
VisualGeometryToolkitCore.bending_energy — Method
bending_energy(tps::ThinPlateSplineSystem, values) -> scalarThe bending energy valuesᵀ Q values of the spline through the nodal values values (= wᵀ K w with K = [U(𝐩ᵢ, 𝐩ⱼ)] the radial basis between the nodes; zero for affine values up to rounding); a quadratic regularizer on the values.
VisualGeometryToolkitCore.bending_factor — Method
bending_factor(tps::ThinPlateSplineSystem{N}) -> Matrix{Float64}A factor L (N × N) of the bending-energy form: ‖L · values‖² =bending_energy(tps, values) to rounding, so L · values are a regularizer's residual rows. Q is positive semidefinite with the affine values in its null space, so it has no Cholesky factor: L = Λ₊^½ Vᵀ from the eigendecomposition Q = V Λ Vᵀ, Λ₊ the eigenvalues clamped at zero (the null space's rounding residue may come out negative). An eigenbasis is not unique, so only LᵀL is defined — never compare the rows one by one. The null space's rows (zero) dropped, it is bending_rows(tps), the penalty operator.
VisualGeometryToolkitCore.grid_nodes — Method
grid_nodes(n; lo = -0.5, hi = 0.5) -> Vector{SVector{2, Float64}}
grid_nodes(n, r::Rect) -> Vector{SVector{2, Float64}}The n × n equally spaced nodes over the square [lo, hi]², or over the rectangle r (its corners included: a sensor's Rect(sensor) in pixels), column-major (x fastest).
VisualGeometryToolkitCore.nodal_weights — Method
nodal_weights(tps::ThinPlateSplineSystem{N}, 𝐩) -> SVector{N, Float64}The sensitivity of the spline at 𝐩 to each nodal value, ∂f(𝐩)/∂valuesᵢ. With f(𝐩) = φ(𝐩)ᵀ (coeff · values) and φ = (U(𝐩, 𝐩₁) … U(𝐩, 𝐩_N), 1, 𝐩_x, 𝐩_y), node i's weight is Σ_r coeff[r, i] φ_r(𝐩) — exact, since the spline is linear in its values. Host-side (it reads coeff), in Float64 for a real point (a Float32 point included); a ForwardDiff Dual point gives Dual weights. Allocation-free at every N.
VisualGeometryToolkitCore.tps_coefficients — Method
tps_coefficients(tps::ThinPlateSplineSystem{N}, values) -> (w::SVector{N}, a::SVector{3})The spline's weights w and affine part a = (a₁, a_x, a_y) through the nodal values values (length N). Linear in values, and generic in their element type: a ForwardDiff.Dual or a Float32 is kept (the product with the Float64 matrix is formed first, then converted to the values' element type).
VisualGeometryToolkitCore.tps_evaluate — Method
tps_evaluate(nodes, w, a, 𝐩) -> scalar
tps_evaluate(tps::ThinPlateSplineSystem, w, a, 𝐩) -> scalarThe spline at 𝐩: a₁ + a_x 𝐩_x + a_y 𝐩_y + Σᵢ wᵢ U(𝐩, 𝐩ᵢ) with (w, a) from tps_coefficients and U the tps_radial_basis. The first form takes isbits inputs only (nodes::SVector{N, <:StaticVector{2}}, w, a, 𝐩): it allocates nothing and runs in a KernelAbstractions kernel on the CPU backend. It is generic in the element type and PROMOTES the nodes with w and 𝐩: a Float32 evaluation with Float64 nodes returns Float64. This differs from tps_gradient, which converts the nodes, as RadiometricCamera's two functions do; supply the nodes in the evaluation's type (Float32 nodes for Float32) and both work in that type.
VisualGeometryToolkitCore.tps_gradient — Method
tps_gradient(nodes, w, a, 𝐩) -> SVector{2}
tps_gradient(tps::ThinPlateSplineSystem, w, a, 𝐩) -> SVector{2}The analytic gradient ∇f(𝐩) of tps_evaluate: (a_x, a_y) + Σᵢ wᵢ (𝐩 − 𝐩ᵢ)(ln r² + 1), since ∇_𝐩 ½ r² ln r² = (𝐩 − 𝐩ᵢ)(ln r² + 1); a node at r = 0 contributes zero. The sum is RadiometricCamera's board_height_grad term for term (its height field is this spline without the affine part, which is added last). Isbits inputs and allocation-free, and it runs in a KernelAbstractions kernel on the CPU backend, as tps_evaluate. Unlike tps_evaluate, which promotes, it CONVERTS the node coordinates to the promoted element type of w and 𝐩 (RadiometricCamera's board_height_grad rule): Float32w and 𝐩 with Float64 nodes give a Float32 gradient. Supply the nodes in the evaluation's type and the two agree.
VisualGeometryToolkitCore.tps_pure_bending — Method
tps_pure_bending(tps::ThinPlateSplineSystem{N}, values) -> SVector{N}The nodal values without the affine term of their interpolant — the pure-bending rule of the thin-plate target calibration kinds (HeightField, ThinPlateSpline): v ↦ v − M a(v), M = [1 u v] the plane fields at the nodes and a(v) the interpolant's affine coefficients (tps_coefficients). The spline through M a is that plane exactly, so the projection keeps the weights w and zeroes a; it is oblique (along the plane fields, not orthogonal to them: orthogonal to the affine rows of coeff) and idempotent. Values whose plane part is within 1e-12 of them (‖M a(v)‖ ≤ 1e-12 ‖v‖, the rule of is_pure_bending) are kept as they are, bit for bit, and a ForwardDiff Dual's partials are projected all the same, so a Dual through it is the derivative of the projection everywhere (the kinds' transform_deriv_params); other values are projected, values and partials. Loops only: allocation-free, generic in the element type and callable in a KernelAbstractions kernel on an adapted system.
VisualGeometryToolkitCore.tps_radial_basis — Method
tps_radial_basis(𝐩, 𝐪) -> scalarThe thin-plate spline's radial basis function U(𝐩, 𝐪) = ½ r² ln r² with r = ‖𝐩 − 𝐪‖ (the same as r² ln r), and U = 0 at r = 0. The zero branch is taken on r² > 0, not by evaluating 0 · log 0, so a ForwardDiff.Dual evaluated on a node gets a zero value with zero partials (the true derivative there) instead of NaN.
Special functions
Device-safe scalar functions — pure arithmetic, generic over Real, so they compile in a KernelAbstractions kernel at any float width and ForwardDiff Duals flow through: log Γ and the digamma/trigamma series of the Student-t adaptation kernels and the normal CDF normcdf_as (Abramowitz & Stegun 7.1.26).
VisualGeometryToolkitCore.digamma_stirling — Method
digamma_stirling(x)ψ(x) by the Stirling series through the x⁻⁴ term — pure arithmetic, for the Newton step on a Student-t's degrees of freedom inside a device kernel.
VisualGeometryToolkitCore.lgamma_lanczos — Method
lgamma_lanczos(x::AbstractFloat)log Γ(x) by the Lanczos approximation (g = 7, n = 9) — branch-free, only + * / log, so it compiles on every KernelAbstractions backend and every float width. ~1 ulp in Float64 for x ≥ 1; no reflection, so callers keep their argument ≥ 1 (every adaptation call site does).
VisualGeometryToolkitCore.normcdf_as — Method
normcdf_as(x::Real)The standard normal CDF Φ(x) by Abramowitz & Stegun 7.1.26 on erf (|error| < 1.5e-7): branch-free but for the sign, only + * / exp, so it compiles in any KernelAbstractions kernel at any float width. A Dual flows through the polynomial, so its derivative is that of the approximation (φ(x) to ≈ 1e-6) — the exact derivative of the evaluated objective.
VisualGeometryToolkitCore.trigamma_stirling — Method
trigamma_stirling(x)ψ′(x) by the Stirling series through the x⁻³ term; companion of digamma_stirling.
Quadrature
The probabilists' Gauss–Hermite rule gauss_hermite_prob (an expectation over N(0, 1), exact to degree 2n − 1) and the Hammersley points hammersley (a kernel's own low-discrepancy sample from its index).
VisualGeometryToolkitCore.gauss_hermite_prob — Method
gauss_hermite_prob(n) -> (nodes::Vector{Float64}, weights::Vector{Float64})The n-point Gauss–Hermite rule for the standard normal weight (probabilists' form): Σ wᵢ f(xᵢ) ≈ E[f(X)], X ~ N(0, 1), the weights summing to one, exact for polynomials of degree 2n − 1. Closed forms for n ≤ 3; beyond, Golub–Welsch — the eigen-decomposition of the Jacobi matrix of the monic Hermite polynomials He_k (off-diagonal √k), the weights the squared first components of its eigenvectors. Ascending nodes. An ArgumentError for n < 1.
For N(μ, σ²), use the nodes μ + σ·xᵢ with the same weights.
VisualGeometryToolkitCore.hammersley — Method
hammersley([T = Float64,] i, N) -> (u₁, u₂)The i-th point (1 ≤ i ≤ N) of the N-point 2-D Hammersley set in [0, 1)²: u₁ = (i − ½)/N, u₂ the base-2 van der Corput radical inverse of i − 1. Scalar and isbits-only (a kernel computes its own point from its index); the type argument leads, as in rand(T, …).
VisualGeometryToolkitCore.square_rule — Method
square_rule(x, w) -> (nodes::Vector{SVector{2, Float64}}, weights::Vector{Float64})The tensor-product rule over a square of a one-dimensional rule (nodes x, weights w, e.g. QuadGK's gauss(n) or gauss_hermite_prob): node (x[i], x[j]) with weight w[i]·w[j], the first coordinate fastest. The weights sum to the square of the 1-D rule's sum; it integrates exactly the products of the polynomials the 1-D rule integrates exactly.
VisualGeometryToolkitCore.unit_disc_rule — Method
unit_disc_rule(n_ρ, n_φ) -> (nodes::Vector{SVector{2, Float64}}, weights::Vector{Float64})The polar product rule over the unit disc ‖x‖ ≤ 1: Gauss–Legendre in the radius ρ ∈ [0, 1] (n_ρ nodes, QuadGK's gauss mapped from [−1, 1]) times n_φ equally spaced angles φ_j = 2π(j − ½)/n_φ; node (ρ cos φ, ρ sin φ) with weight ρ·w_ρ·2π/n_φ (the polar area element), radius-major (node (k − 1)·n_φ + j). The weights sum to π, the disc's area: Σ wᵢ f(xᵢ) approximates ∫ f over the disc, exactly for ρ^a cos(mφ), ρ^a sin(mφ) with a + 1 ≤ 2n_ρ − 1 and |m| < n_φ.
One-dimensional minimization
VisualGeometryToolkitCore.grid_brent_minimum scans an evenly spaced grid and refines its least value by Brent's method: the stack's one 1-D minimizer (public, not exported).
VisualGeometryToolkitCore.grid_brent_minimum — Method
grid_brent_minimum(f, xs::AbstractRange; periodic = false) -> (; x, value, at_end)The minimum of f over the evenly spaced grid xs (evaluated in its order), the least value refined by Brent's method (golden-section steps with parabolic interpolation; Brent 1973, ch. 5) between its neighbours: (; x, value, at_end) of the smallest value evaluated. Brent stops at its own floor: x within √eps·(|x| + step(xs)) (a smooth minimum is resolved no better than √eps relative), so the result needs no tolerance. On a periodic grid the neighbours wrap (f periodic). On a linear one a least value at an end of the grid is refined on its one neighbouring interval, and at_end says the result lies at that end: Brent's final bracket still has the end as its edge (every evaluation descended towards it). A value of Inf or NaN is never the minimum. The stack's one 1-D minimizer: Core's azimuthal minimum (the view-angle limit, the inscribed radius) and Estimation's GCV weight search (gcv_select).
Kernel building blocks
An in-register LU solve for fixed-size systems and a KernelAbstractions workgroup reduction, for code that writes its own kernels (public, not exported).
VisualGeometryToolkitCore.lu_solve! — Method
lu_solve!(A::MMatrix{N, N}, b::MVector{N}) -> Int32Solve A x = b in place by LU with partial pivoting — A is overwritten with its factors and the solution replaces b. Zero-allocation on static mutable arrays (StaticArrays' generic lu falls back to a heap path at these sizes), and pure register arithmetic, so it runs inside a KernelAbstractions kernel on any backend.
Returns 0, or k when the k-th pivot is EXACTLY zero — mirroring LAPACK getrf's info, which a batched driver uses to invalidate degenerate systems rather than propagate Infs. Hoisted verbatim from BlobBoards' 6×6 adaptation solve (2026-09); its op sequence is the one the CPU adaptation path is pinned to.
(_e5_solve10! in the native 5-point essential solver is a multi-right-hand-side elimination that multiplies by the reciprocal pivot on back-substitution, ported from Eigen for fidelity to PoseLib; it stays separate because unifying the two would change one of them at the ulp level.)
VisualGeometryToolkitCore.tree_sum! — Function
tree_sum!(@context, smem, tid, nt, half = Int32(128))Workgroup tree reduction: sums smem[1:nt] into smem[1], thread tid of nt folding smem[tid + s] into smem[tid] for s = half, half/2, …, 1 with a barrier per level (left to right within a level, so the sum is the one every 256-thread reduction of the stack forms). half is HALF the workgroup size (the default serves 256-thread groups). A helper, not a kernel — it takes the kernel's @context so it may @synchronize; every thread in the group must call it, after a barrier that follows its write of smem[tid] (the barriers are convergent only then).
On KernelAbstractions' CPU backend a barrier must be written in the kernel body itself (its CPU transform splits the body at a syntactic @synchronize; one inside a called function is an error), so a CPU kernel calls this with a group of one, nt == 1 — the stack's CPU group size for its per-blob kernels — where there is nothing to fold and no barrier is taken: smem[1] is the sum.
Index
VisualGeometryToolkitCore.CSRMatrixVisualGeometryToolkitCore.Cameras.AbstractBlurVisualGeometryToolkitCore.Cameras.GaussianBlurVisualGeometryToolkitCore.OnBackendVisualGeometryToolkitCore.PenalizedOperatorVisualGeometryToolkitCore.ThinPlateSplineSystemVisualGeometryToolkitCore.Cameras.convolveVisualGeometryToolkitCore.bending_energyVisualGeometryToolkitCore.bending_factorVisualGeometryToolkitCore.bending_gramVisualGeometryToolkitCore.bending_rowsVisualGeometryToolkitCore.bending_rowsVisualGeometryToolkitCore.bending_stencilsVisualGeometryToolkitCore.bspline_4d_partialVisualGeometryToolkitCore.bspline_basisVisualGeometryToolkitCore.bspline_basis_iterativeVisualGeometryToolkitCore.bspline_surface_partialVisualGeometryToolkitCore.cglsVisualGeometryToolkitCore.clamped_knotsVisualGeometryToolkitCore.contract_basis_4d_weightedVisualGeometryToolkitCore.cubic_lattice_basisVisualGeometryToolkitCore.difference_rowsVisualGeometryToolkitCore.difference_stencilVisualGeometryToolkitCore.digamma_stirlingVisualGeometryToolkitCore.evaluate_bspline_4dVisualGeometryToolkitCore.evaluate_bspline_surfaceVisualGeometryToolkitCore.gauss_hermite_probVisualGeometryToolkitCore.gradient_pairsVisualGeometryToolkitCore.grid_brent_minimumVisualGeometryToolkitCore.grid_nodesVisualGeometryToolkitCore.hammersleyVisualGeometryToolkitCore.lgamma_lanczosVisualGeometryToolkitCore.lu_solve!VisualGeometryToolkitCore.mean_positive_eigenvalueVisualGeometryToolkitCore.natural_extrapolationVisualGeometryToolkitCore.nodal_weightsVisualGeometryToolkitCore.normal_cgVisualGeometryToolkitCore.normcdf_asVisualGeometryToolkitCore.penalty_scaleVisualGeometryToolkitCore.pointwise!VisualGeometryToolkitCore.pointwise_kernelVisualGeometryToolkitCore.row_stencilsVisualGeometryToolkitCore.square_ruleVisualGeometryToolkitCore.tps_coefficientsVisualGeometryToolkitCore.tps_evaluateVisualGeometryToolkitCore.tps_gradientVisualGeometryToolkitCore.tps_pure_bendingVisualGeometryToolkitCore.tps_radial_basisVisualGeometryToolkitCore.tree_sum!VisualGeometryToolkitCore.trigamma_stirlingVisualGeometryToolkitCore.unit_disc_rule