Getting started
The examples on this page are doctests: the documentation build runs them and checks their output. They share one scene. A camera at the origin looks down the z axis at a grid of points on a gently curved surface, about six units away. Its intrinsic matrix K has a focal length of 800 pixels.
julia> using PoseLib, VisualGeometryToolkitCore
julia> K = [800.0 0.0 320.0; 0.0 800.0 240.0; 0.0 0.0 1.0];
julia> X = [Point3(x, y, 6.0 + 0.5x * y) for x in -1.0:0.5:1.0 for y in -1.0:0.5:1.0];
julia> pixel(K, x) = (y = K * x; Point2(y[1] / y[3], y[2] / y[3]));
julia> u = [pixel(K, x) for x in X];A minimal solver
The P3P solver estimates a camera pose from three world points and their bearings, the unit rays from the camera centre towards them. Selecting the backend PoseLibSolver() runs PoseLib's solver in place of VisualGeometryToolkitCore's native one. P3P has up to four real solutions; one of them is the true pose, the identity.
julia> f = [Vec3(x / sqrt(sum(abs2, x))) for x in X];
julia> poses = solve(P3P(PoseLibSolver()), Correspondences(X, f), [1, 13, 25]);
julia> 1 <= length(poses) <= 4
true
julia> any(p -> maximum(abs, p.t) < 1e-8, poses)
trueRefining an absolute pose
A refiner starts from a seed and polishes it. It returns a FitResult, whose estimate is the refined model; the pose refiners take and return a Rigid3, which EuclideanMap(R, t) builds from a rotation and a translation. Here the seed is a few pixels off.
The strategy is optimization_strategy = Irls{TLSShape}(), the truncated loss, and its width is in the unit of the image points, pixels here. A point whose residual exceeds it counts as an outlier and gets zero weight. A seed must therefore be close enough that the inliers fall within the width.
julia> seed = EuclideanMap(RotationVec(0.002, -0.001, 0.0), Vec3(0.005, 0.0, 0.01));
julia> r = refine_absolute_pose(seed, K, Correspondences(X, u); max_iter = 50, optimization_strategy = Irls{TLSShape}(), width = 2.0);
julia> pose = estimate(r); maximum(abs, pose.t) < 1e-8
true
julia> converged(r)
trueA P3P solution is a labeled Extrinsics3, a map from the world frame to the camera frame. unlabel turns it into a Rigid3 seed.
julia> best = poses[argmin([maximum(abs, p.t) for p in poses])];
julia> estimate(refine_absolute_pose(unlabel(best), K, Correspondences(X, u); max_iter = 50, optimization_strategy = Irls{TLSShape}(), width = 2.0)) isa PoseLib.Rigid3
trueRefining a relative pose
A second camera, rotated and moved sideways by one unit, sees the same points. The relative pose maps the first camera's frame to the second's. Its translation is known only up to scale, and the refiner returns it at unit length.
julia> truth = EuclideanMap(RotationVec(0.0, 0.1, 0.0), Vec3(1.0, 0.0, 0.0));
julia> x2 = [pixel(K, truth(x)) for x in X];
julia> seed = EuclideanMap(RotationVec(0.0, 0.101, 0.001), Vec3(0.99995, 0.01, 0.0));
julia> rel = estimate(refine_relpose(seed, K, K, Correspondences(u, x2); max_iter = 50, optimization_strategy = Irls{TLSShape}(), width = 2.0));
julia> maximum(abs, rel.R - truth.R) < 1e-6
trueThe same refinement runs on unit bearings, the form that also serves fisheye cameras. Each bearing carries the Jacobian of the back-projection, which sets the unit of the width: pixels for these Jacobians. See Refiners for why.
julia> using VisualGeometryToolkitCore: backproject_with_jacobian
julia> camera = CameraModel(LogicalIntrinsics(CameraCalibrationMat(K)), Pinhole());
julia> b1 = backproject_with_jacobian.(Ref(camera), u);
julia> b2 = backproject_with_jacobian.(Ref(camera), x2);
julia> cs = withattrs(Correspondences(first.(b1), first.(b2)); M₁ = last.(b1), M₂ = last.(b2));
julia> maximum(abs, estimate(refine_relpose(seed, cs; max_iter = 50, optimization_strategy = Irls{TLSShape}(), width = 2.0)).R - rel.R) < 1e-12
trueRefining a homography
The homography refiner takes the seed matrix and the point correspondences it should explain. The points here are exact, so the refined matrix matches the true one up to scale.
julia> H = HomographyMat([1.1 0.02 5.0; -0.03 0.95 -3.0; 1e-4 2e-4 1.0]);
julia> x1 = [Point2(a, b) for a in 0.0:100.0:400.0 for b in 0.0:100.0:300.0];
julia> y = [(h = H * [x; 1.0]; Point2(h[1] / h[3], h[2] / h[3])) for x in x1];
julia> H0 = HomographyMat([1.1 0.02 6.0; -0.03 0.96 -3.0; 1e-4 2e-4 1.0]);
julia> H1 = estimate(refine_homography(H0, Correspondences(x1, y); max_iter = 25, optimization_strategy = Irls{TLSShape}(), width = 2.0));
julia> H1 / H1[3, 3] ≈ H
true