API reference

Every docstring of the package: the module, the exported refiners and the LeZach strategy and the PoseLibRefineMetadata result metadata, the public Rigid3 pose type and is_available, and the PoseLibSolver methods this package adds to VisualGeometryToolkitCore's minimal-solver kernels.

PoseLib.PoseLib — Module
PoseLib

Julia bindings to PoseLib, the C++ library of minimal solvers and robust refiners for camera pose estimation.

Loading this package extends VisualGeometryToolkitCore:

  • Its minimal solvers gain a PoseLib backend, PoseLibSolver(): the four-point homography, the seven-point fundamental matrix, the perspective-three-point (P3P) absolute pose and the five-point essential matrix. The eight-point essential matrix, essential_dlt, has no native variant, so its default routes here.
  • PoseLib's robust Levenberg–Marquardt refiners are exported: refine_homography, refine_fundamental, refine_relpose and refine_absolute_pose. Each takes a seed model and its correspondences, and returns the refined model in the caller's frame.

The C++ library and its flat C wrapper ship prebuilt as the binary package PoseLib_jll. The manual's "Platforms and binaries" page says how to run against a library built from source.

source
PoseLib.LeZach — Type
LeZach() <: AbstractOptimizationStrategy

PoseLib's own optimization strategy, given as optimization_strategy = LeZach() or selected from TOML as type = "le_zach" (its tag in POSELIB_STRATEGY_TAGS): the truncated loss ρ = min(r², w²) (VisualGeometryToolkitCore's TLSShape()), optimized by the iteratively reweighted least-squares scheme of Le and Zach (International Conference on 3D Vision, 2021). This is PoseLib's TRUNCATED_LE_ZACH loss. It selects how the truncated loss is optimized, so it takes the same width as Irls{TLSShape}. Like TLS it has no efficiency constant, so it takes width, not scale. A member of the stack's one optimization-strategy family: its loss family is TLS's, its path IRLS and its label TLS-LZ.

source
PoseLib.PoseLibRefineMetadata — Type
PoseLibRefineMetadata

The metadata of a PoseLib refiner's FitResult: the robust-estimation contract's scale record. scale is the noise scale the refiner was given and scale_source where it came from (ScaleGiven for scale = σ, ScaleUpstream for scale = result; ScaleNone, scale = NaN, for width = w and for least squares); width is the loss width PoseLib ran at (NaN for least squares). noise_scale(result) hands the scale on.

source
PoseLib.Rigid3 — Type
Rigid3

The rigid motion x ↦ R·x + t that the pose refiners take as their seed and return: a VisualGeometryToolkitCore EuclideanMap in 3D with a RotMatrix rotation, a Vec3 translation and unlabeled frames. EuclideanMap(RotMatrix{3}(R), Vec3(t)) builds one.

For refine_relpose it maps the first camera's frame to the second's. For refine_absolute_pose it maps the world frame to the camera's. A labeled pose, such as the Extrinsics3 that P3P returns, is converted with VisualGeometryToolkitCore's unlabel.

source
PoseLib.POSELIB_STRATEGY_TAGS — Constant
POSELIB_STRATEGY_TAGS

PoseLib's optimization strategies' registry (a VisualGeometryToolkitCore TagRegistry, built by its @lo_registry): "le_zach" → LeZach. A caller that reads strategies from TOML merges it with the core's, merge(VisualGeometryToolkitCore.STRATEGY_TAGS, PoseLib.POSELIB_STRATEGY_TAGS) (or with VisualGeometryToolkitEstimation's OPTIMIZATION_STRATEGY_TAGS).

source
PoseLib.calibrated_loss_width — Method
calibrated_loss_width(K1, K2, width) -> Float64

The calibrated (bearing-unit) loss width that PoseLib's relative-pose refiner works at when it is handed width in the unit of the points together with the intrinsics K1 and K2 (refine_relpose(pose, K1, K2, cs; width)): width · ½(1/f₁ + 1/f₂), PoseLib's camera-aware rule (robust.cc), with each fᵢ = (Kᵢ₁₁ + Kᵢ₂₂)/2, the focal length of the SIMPLE_PINHOLE camera the binding hands over.

A refiner on calibrated data that must deploy at the same operating point as PoseLib's pixel refiner converts its width with this, so the two cannot drift apart when the camera mapping changes.

source
PoseLib.is_available — Method
is_available() -> Bool

Whether PoseLib_jll provides the PoseLib library on this platform. Without it, every function that calls into the library raises an error stating why; the package itself loads everywhere.

source
PoseLib.refine_absolute_pose — Method
refine_absolute_pose(pose::Rigid3, K, cs; max_iter, optimization_strategy, scale | width) -> FitResult{Rigid3}

Robust Levenberg–Marquardt refinement of an absolute pose from 3D-to-2D correspondences, through PoseLib's C++ absolute-pose bundle adjustment. The residual is the reprojection error project(camera, R X + t) − u.

pose is the seed Rigid3, mapping the world frame to the camera's. K is the intrinsic matrix of the frame the image points are in. cs holds the correspondences X => u of 3D points to 2D image points. Returns the refined Rigid3, as a FitResult.

The width is that of the loss on the reprojection residual, in the unit of u and K (pixels for a pixel-frame K; codimension 2). max_iter, optimization_strategy, scale and width, and the result, are as in refine_homography. Only the six degrees of freedom of the pose are refined; K stays fixed.

K reaches PoseLib as its SIMPLE_PINHOLE camera model, with focal length (K₁₁ + K₂₂)/2 and principal point (K₁₃, K₂₃). Skew and any difference between K₁₁ and K₂₂ are dropped, so the model is exact only for square pixels and zero skew.

It ends in the local minimum its seed leads to, as described for refine_homography.

source
PoseLib.refine_absolute_pose — Method
refine_absolute_pose(pose::Rigid3, cs::…StaticVector{2}…; max_iter, optimization_strategy, width)

This form is refused with an ArgumentError. Its targets would be calibrated points on the z = 1 plane (GnomonicFrame), with a loss width divided by the focal length. That plane has no point for a ray at or beyond 90° from the optical axis.

Use refine_absolute_pose(pose, K, cs) instead: the same correspondences in pixels, the intrinsic matrix K, and the width in pixels.

PoseLib has no absolute-pose refiner on bearings. Its residual is a reprojection error in the image, so the pixel form is the only one, and it reaches PoseLib as a SIMPLE_PINHOLE camera. That model cannot project a ray at or beyond 90° either, so this package offers no correct absolute-pose refinement for a fisheye camera.

source
PoseLib.refine_absolute_pose — Method
refine_absolute_pose(pose::UnitfulEuclideanMap{3}, K, cs; max_iter, optimization_strategy, width)

The physical form of refine_absolute_pose: pose a world→camera pose whose translation is a length (FlexUnits quantities) and cs the correspondences X => u with the world points X lengths. Both are stripped in metres once, the plain refinement runs, and the refined pose comes back with its translation in metres, as a FitResult: equal, after ustrip(m, ·), to the plain refinement's on the inputs' ustrip(m, ·), with the plain refinement's retcode, iterations and scale record. u, K and the width stay in pixels. A labeled seed (an estimate's CameraExtrinsics, world → camera) is read unlabeled; the refined pose comes back unlabeled. A physical seed with plain points, or the reverse, is an ArgumentError.

source
PoseLib.refine_fundamental — Method
refine_fundamental(F, cs; max_iter, optimization_strategy, scale | width) -> FitResult{FundMat}

Robust Levenberg–Marquardt refinement of a fundamental matrix through PoseLib's refine_fundamental. It minimizes the Sampson error over the Bartoli–Sturm parametrization of F, which keeps the matrix at rank 2.

F is the seed 3×3 fundamental matrix. cs holds the correspondences x1 => x2 of 2D image points, in the frame F is expressed in (pixels for a pixel-frame F). The width is that of the loss on the Sampson residual, in the same unit (codimension 1). max_iter, optimization_strategy, scale and width, and the result, are as in refine_homography. Returns a FitResult whose estimate is the refined FundMat. It ends in the local minimum its seed leads to, as described for refine_homography.

source
PoseLib.refine_homography — Method
refine_homography(H, cs; max_iter, optimization_strategy, scale | width) -> FitResult{HomographyMat}

Robust Levenberg–Marquardt refinement of a homography through PoseLib's refine_homography. It minimizes the symmetric transfer error: the forward residual x2 − π(H x1) and the backward residual x1 − π(H⁻¹ x2), each robustified per point, where π divides by the third coordinate.

H is the seed 3×3 homography. cs holds the correspondences x1 => x2 of 2D image points, in the frame H is expressed in (pixels for a pixel-frame H). The width is that of the loss on a one-sided transfer residual, in the same unit. Returns a FitResult whose estimate is the refined HomographyMat.

optimization_strategy is required, like max_iter: the robust estimate's strategy is the caller's configuration (its strategy or TOML table), never a default here. PoseLib's Levenberg–Marquardt weighs each residual by its loss weight ρ′ alone, so its strategies are Irls{K} (VisualGeometryToolkitCore's); a Triggs{K} (the ρ″-corrected LM) is refused, as is any other shape:

optimization_strategyρ(r)note
Irls{TLSShape}()min(r², w²)truncated quadratic: inliers have r < w, outliers get zero weight
Irls{HuberShape}()r² up to w, linear beyond
Irls{CauchyShape}()w² log(1 + r²/w²)
LeZach()min(r², w²)the truncated loss under Le and Zach's scheme
nothingr²plain least squares; takes no scale or width

A robust strategy takes exactly one of:

  • width = w, the width w of VisualGeometryToolkitCore's closed form for that shape. PoseLib's own loss_scale equals it for every loss bound here, so it reaches PoseLib unchanged;
  • scale = σ (or an earlier robust result, read by noise_scale): the loss runs at w = c·σ, with c the strategy's efficiency constant, else VisualGeometryToolkitCore's efficiency_constant(shape, d) at the residual's codimension d (2 here). TLS and Le–Zach have no efficiency constant (a truncation), so they take width.

The retired loss and loss_width keywords are refused with an ArgumentError naming their replacements. max_iter is PoseLib's Levenberg–Marquardt iteration budget; it has no default (an Irls strategy's own max_iter, its reweighting count, is not read by PoseLib).

The result records PoseLib's iterations and its termination record (retcode, stop_rule; VisualGeometryToolkitCore's, SciMLBase's semantics): (ReturnCode.Success, NoRule) when PoseLib stopped on its own tolerances — PoseLib reports no stop rule, so none is claimed — and (ReturnCode.MaxIters, IterationCap) when it used the whole budget. Its metadata (PoseLibRefineMetadata) is the scale record: noise_scale(result) hands a given scale on to the next stage. PoseLib reports no residuals, weights or cost, so rms is NaN and n is the number of correspondences.

A refiner ends in the local minimum its seed leads to: under a truncated loss, a correspondence whose residual at the seed exceeds the width gets no weight, so a seed must be close enough that its inliers fall within the width. Without the library on this platform (is_available false), a call that reaches it raises an error that says so; a refused form (calibrated 2-vectors, mixed units, non-unit bearings) raises its own ArgumentError first.

julia> using PoseLib, VisualGeometryToolkitCore

julia> H = HomographyMat([1.0 0.0 5.0; 0.0 1.0 -3.0; 0.0 0.0 1.0]);

julia> x1 = [Point2(a, b) for a in 0.0:100.0:300.0 for b in 0.0:100.0:300.0];

julia> x2 = [Point2(x[1] + 5.0, x[2] - 3.0) for x in x1];

julia> r = refine_homography(HomographyMat([1.0 0.0 6.0; 0.0 1.0 -3.0; 0.0 0.0 1.0]), Correspondences(x1, x2);
                              max_iter = 25, optimization_strategy = Irls{TLSShape}(), width = 2.0);

julia> H1 = estimate(r); H1 / H1[3, 3] ≈ H
true

julia> converged(r), iterations(r) < 25
(true, true)
source
PoseLib.refine_relpose — Method
refine_relpose(pose::Rigid3, K1, K2, cs; max_iter, optimization_strategy, scale | width) -> FitResult{Rigid3}

Robust Levenberg–Marquardt refinement of a relative pose from pixel correspondences and the two cameras' intrinsics.

pose is the seed Rigid3, mapping the first camera's frame to the second's. cs holds the correspondences x1 => x2, each point in its own image's frame. K1 and K2 are the intrinsic matrices of those frames: pixel matrices for pixel points, or matrices in whatever frame the points are in. Returns a FitResult whose estimate is the refined Rigid3, as a FitResult.

The width is in the unit of the points. PoseLib back-projects through each camera, so one width serves both images even when their focal lengths differ. max_iter, optimization_strategy, scale and width, and the result, are as in refine_homography; the residual's codimension is 1.

Each K reaches PoseLib as its SIMPLE_PINHOLE camera model, with focal length (K₁₁ + K₂₂)/2 and principal point (K₁₃, K₂₃). Skew and any difference between K₁₁ and K₂₂ are dropped, so the model is exact only for square pixels and zero skew.

It ends in the local minimum its seed leads to, as described for refine_homography.

source
PoseLib.refine_relpose — Method
refine_relpose(pose::Rigid3, cs::…StaticVector{2}…; max_iter, optimization_strategy, width)

This form is refused with an ArgumentError. Calibrated 2-vectors are coordinates on the z = 1 plane (GnomonicFrame). That plane has no point for a ray at or beyond 90° from the optical axis, which excludes fisheye and omnidirectional cameras. Relative pose is calibrated geometry, so in this stack it takes unit bearings (unit rays).

Use one of the two supported forms:

  • refine_relpose(pose, cs), with cs of unit bearings f₁ => f₂ that carry their back-projection Jacobians M₁ and M₂. This is PoseLib's tangent-Sampson refiner.
  • refine_relpose(pose, K1, K2, cs), with pixel points and the two intrinsic matrices. PoseLib back-projects through the cameras and runs the same refiner.

backproject_with_jacobian(camera, u) returns a bearing and its Jacobian together.

source
PoseLib.refine_relpose — Method
refine_relpose(pose::Rigid3, cs; max_iter, optimization_strategy, scale | width) -> FitResult{Rigid3}

Robust Levenberg–Marquardt refinement of a relative pose from unit bearings, through PoseLib's tangent-Sampson refiner.

pose is the seed Rigid3, mapping the first camera's frame to the second's. cs holds the correspondences f₁ => f₂ of unit bearings (StaticVector{3}). They carry their 3×2 back-projection Jacobians as the attribute columns M₁ and M₂, attached with withattrs(cs; M₁, M₂). VisualGeometryToolkitCore's backproject_with_jacobian(camera, u) returns each bearing together with its Jacobian. Returns a FitResult whose estimate is the refined Rigid3, as a FitResult.

No image plane is involved, so a ray at or beyond 90° from the optical axis is an ordinary input. A 3-vector that is not of unit length is refused with an ArgumentError: a point on the z = 1 plane lifted to (x, y, 1) has the right type but is not a bearing.

The unit of the width

width is the width w of the loss on the tangent-Sampson residual

r = (f₂ᵀ E f₁) / √(‖M₂ᵀ E f₁‖² + ‖M₁ᵀ Eᵀ f₂‖²)

The denominator is the gradient norm of the epipolar constraint with respect to the coordinates that M differentiates against. The residual r is in the unit of those coordinates, and width (or scale) must be given in the same unit. The Jacobians set the unit; the call cannot.

M isr and width are in
∂f/∂u from backproject_with_jacobianpixels of u
an orthonormal basis of the tangent plane of the sphere at fradians

A width in the wrong unit is off by a factor of the focal length. Nothing reports this: a truncated loss simply keeps the wrong inliers.

All arguments are in the caller's frame, and the refined pose comes back in it. It ends in the local minimum its seed leads to, as described for refine_homography.

source
PoseLib.refine_relpose — Method
refine_relpose(pose::UnitfulEuclideanMap{3}, args...; kw...)

This form is refused with an ArgumentError. A calibrated relative pose is determined only up to scale, so its translation is a direction, not a length: there is no unit to strip at entry or to reattach at exit. Hand over the plain seed, ustrip(m, pose) converted to a Rigid3.

source
VisualGeometryToolkitCore.essential_5pt — Method
essential_5pt(::PoseLibSolver, x1, x2) -> FixedModels{10, EssentialMat{Float64}}
essential_5pt(::PoseLibSolver, cs, idx) -> FixedModels{10, EssentialMat{Float64}}

PoseLib's five-point essential-matrix solver, the PoseLibSolver backend of VisualGeometryToolkitCore's essential_5pt kernel. x1 and x2 are five unit bearings each (an NTuple or an SVector{5} of StaticVector{3, Float64}); the second form reads the five correspondences cs[idx] of bearings f₁ => f₂. A 2-vector is a MethodError, never lifted to the z = 1 plane. Returns every real solution that VisualGeometryToolkitCore's is_valid_solution accepts, up to ten; a degenerate sample returns fewer, possibly none. The public entry point is solve(Essential5pt(PoseLibSolver()), cs, idx).

source
VisualGeometryToolkitCore.essential_dlt — Method
essential_dlt(::PoseLibSolver, cs) -> Union{EssentialMat{Float64}, Nothing}
essential_dlt(cs) -> Union{EssentialMat{Float64}, Nothing}

PoseLib's eight-point essential matrix: the direct linear transform over eight or more correspondences cs of unit bearings f₁ => f₂, every one taken as an inlier. VisualGeometryToolkitCore has no native eight-point solver, so the form without a backend runs this one. An algorithm, so it can only be degenerate: it returns nothing when PoseLib's solution is not finite, but a degenerate set (a pure rotation, repeated points) can also give a finite, meaningless E that nothing flags. Fewer than eight correspondences throw an ArgumentError naming the count.

source
VisualGeometryToolkitCore.fundamental_7pt — Method
fundamental_7pt(::PoseLibSolver, cs, idx) -> FixedModels{3, FundMat{Float64}}

PoseLib's seven-point fundamental-matrix solver, the PoseLibSolver backend of VisualGeometryToolkitCore's fundamental_7pt kernel. cs holds 2D image correspondences x1 => x2 in the caller's frame and idx names the seven of the sample. Returns every real solution, up to three, in that frame; a degenerate sample returns fewer, possibly none. The public entry point is solve(Fundamental7pt(PoseLibSolver()), cs, idx).

source
VisualGeometryToolkitCore.homography_4pt — Method
homography_4pt(::PoseLibSolver, s₁, s₂, s₃, s₄, d₁, d₂, d₃, d₄) -> Union{HomographyMat, Nothing}

PoseLib's four-point homography solver, the PoseLibSolver backend of VisualGeometryToolkitCore's homography_4pt kernel. The points are 2-vectors: sources sᵢ and targets dᵢ with dᵢ ≃ H sᵢ. Returns the homography, sign normalized, or nothing for a degenerate sample. The public entry point is solve(Homography4pt(PoseLibSolver()), cs, idx).

source
VisualGeometryToolkitCore.p3p — Method
p3p(::PoseLibSolver, x, X) -> Union{FixedModels{4, Extrinsics3}, Nothing}

PoseLib's perspective-three-point solver, the PoseLibSolver backend of VisualGeometryToolkitCore's p3p kernel. x holds three unit bearings (StaticVector{3, Float64}) and X the three world points they see. Returns every real solution, up to four, as world-to-camera extrinsics in the frame of X (plain Float64 points; the translation is in their unit); a degenerate sample returns fewer, possibly none, and inputs of any length other than three return nothing. The public entry point is solve(P3P(PoseLibSolver()), cs, idx) with cs the correspondences X => x.

source