PanoramaStitching.jl
PanoramaStitching — Module
PanoramaStitchingThe panorama of a camera that only rotated (a sweep of a distant scene), seen through a calibrated lens of any field of view: the rotations of its frames (register_rotations — features, pairwise rotations, the view graph's rotation averaging, the tracks and the rotation and direction bundle), the frames sampled at directions of the sphere (sample_frames, the observations of a vignetting validation), and the panorama by the reverse warp of the frames through the camera into any view — the cylindrical family, a virtual camera, a cube map — unblended (render_panorama, render_cube_map); the camera's vignetting (VisualGeometryToolkitCore's model) and the frames' exposure gains from the frames alone, interpolation free, or the gains alone under a given vignetting (fit_vignetting), and the seams a vignetting leaves (score_seams).
Built on the VisualGeometryToolkit stack: VisualGeometryToolkitFeatures' ASIFT detector and LoMa descriptor and matcher, VisualGeometryToolkitEstimation's find_rotation, ViewGraph and fit_rotation_averaging, VisualGeometryToolkitCore's cameras, vignetting models, rotation charts, panorama projections and warping machinery (image_warp, image_sample!, pointwise!: KernelAbstractions kernels on GPU arrays, the same code on the host); KernelAbstractions.jl and Atomix.jl (the vignetting fit's normal equations), Adapt.jl (a camera's forward map moved to the frames' backend), SparseArrays (the fit's host Cholesky solve), NLLSsolver.jl (the rotation bundle) and StableRNGs.jl (the pairs' RANSAC samplers, seeded by VisualGeometryToolkitCore's stable_seed).
PanoramaStitching estimates the rotations of a camera that only rotated from its frames, and renders the panorama in any view, through a calibrated lens of any field of view (fisheyes past 90° included). With no calibration target, the same frames also refine the camera itself (autocalibration), and they fit its vignetting with no flat field. All geometry is on unit bearings, and every residual is in the camera's pixels.
| Step | Function | Result |
|---|---|---|
| the rotation of every frame | register_rotations | a Registration: rotations, view graph, tracks, bundle |
| the camera and the rotations together, no target (autocalibration) | refine_camera_and_rotations | a FitResult: the refined camera and rotations |
| the frames sampled at directions | sample_frames | FrameSamples |
| the panorama in a view | panorama_view, render_panorama, render_cube_map | (; image, owner, coverage, extent) |
| the vignetting from the frames alone, or the gains under a given one | fit_vignetting | a FitResult of a VisualGeometryToolkitCore vignetting model |
| the seams a vignetting leaves | score_seams | (; ratio, rms, mad, n) |
| the model-free vignetting per ring of a field angle (the pair equation), the frames' additive level | ring_correspondences, ring_pair_vignetting, fit_additive_level | a FitResult: ln V per ring; the level |
The sampling, the render and the vignetting fit run as KernelAbstractions kernels (the first two through VisualGeometryToolkitCore's warping machinery): GPU arrays run on the device, host arrays run the same code on the CPU.
Installation
PanoramaStitching and the packages it is built on are registered in General. Julia 1.13 or later. Add the package and the packages the example also loads:
pkg> add PanoramaStitching VisualGeometryToolkitCore StaticArraysA first example
A fisheye sweeps a synthetic scene in four frames 35° apart. The example estimates the frames' rotations from the frames alone and renders their cylindrical panorama.
using PanoramaStitching
using VisualGeometryToolkitCore: CameraModel, BoundedCamera, Size2, KannalaBrandt, Circle, Point2, Equirectangular,
Cylindrical, CauchyShape, Triggs, LinearRaw, raw_frame_map, backproject, project, bilinear_sample, exp_so3, log_so3, px
using StaticArrays: SA
using LinearAlgebra: norm
using Random: Xoshiro
# The scene: random Gaussian blobs on an equirectangular texture (columns the longitude, rows the latitude).
function blob_scene(rng; width = 2048, height = 1024, n = 20_000)
scene = fill(0.15f0, height, width, 3)
for _ in 1:n
x, y, s, a, c = width * rand(rng), height * rand(rng), 1 + 3rand(rng), 0.6rand(rng), 0.3 .+ 0.7rand(rng, 3)
sx = s / max(sinpi(y / height), 0.05) # round on the sphere, not on the texture
for j in floor(Int, x - 3sx):ceil(Int, x + 3sx), i in max(1, floor(Int, y - 3s)):min(height, ceil(Int, y + 3s))
scene[i, mod1(j, width), :] .+= a * exp(-((j - x)^2 / sx^2 + (i - y)^2 / s^2) / 2) .* c
end
end
return d -> (p = project(Equirectangular(), d); bilinear_sample(scene, Val(3), SA[(p[1] + π) / 2π, 0.5 - p[2] / π]))
end
radiance = blob_scene(Xoshiro(7))
# A Kannala–Brandt fisheye (300 px focal length, 1000 × 1000 sensor, imaging to 100° off axis, its image circle of
# radius 480 px the lens coverage: every mask reads the camera's imaged region) turned 35° between frames, each frame
# 2 × 2 binned as a RAW's :bin2 frames are: the frames' map onto the camera's pixels is that mode's, x ↦ 2x
# (VisualGeometryToolkitCore's raw_frame_map), so frame pixel (i, j), centred at (j − ½, i − ½), is camera pixel (2j − 1, 2i − 1).
frame_map = raw_frame_map(LinearRaw{:bin2})
camera = BoundedCamera(CameraModel(300.0px, SA[500.0px, 500.0px]; lens = KannalaBrandt(-0.02)), Size2(1000, 1000);
lens_coverage = Circle(Point2(500.0, 500.0), 480.0))
truth = [exp_so3(SA[0.0, deg2rad(35.0 * k), 0.0]) for k in 0:3] # d = R_k·f, the first frame the common frame
frames = map(truth) do R
rgb = [(u = frame_map(SA[j - 0.5, i - 0.5]); hypot(u[1] - 500, u[2] - 500) < 480 ? radiance(R * backproject(camera, u)) :
SA[0f0, 0f0, 0f0]) for i in 1:500, j in 1:500]
Float32[rgb[i, j][c] for i in 1:500, j in 1:500, c in 1:3] # H × W × 3, linear
end
# The rotations from the frames alone, then the cylindrical panorama at 0.25° per pixel.
reg = register_rotations(frames, camera; frame_map, tone_percentile = 99.5, detector_margin_px = 20.0,
asift_tilts = [(1.0, 0.0)], max_detections = 1024, descriptor = :b, matcher = :b128,
min_score = 0.0, pairs = :all, min_inliers = 40, seed = 1,
max_trials = 10_000, min_trials = 100,
bundle_optimization_strategy = Triggs{CauchyShape}(efficiency_constant = 2.5), bundle_max_iter = 1000,
bundle_ftol = 1e-10, bundle_xtol = 1e-10)
errors = [rad2deg(norm(log_so3(reg.rotations[k]' * truth[k]))) for k in 1:4] # degrees, each below 0.05
reg.bundle.rms # the bundle's rms reprojection error: 0.24 px here, below 0.5 px
cyl = panorama_view(Cylindrical(), (-100.0, 205.0, -45.0, 45.0), 0.25)
pano = render_panorama(camera, frames, reg.rotations, cyl; vignetting = (u, f) -> 1.0, frame_map, rim_margin_px = 12.0,
fov_deg = 89.0)
size(pano.image) # (458, 1220, 3); NaN where no frame sees the pixelDocumentation
- Start here: Getting started continues from the example, step by step: the inputs, the registration, sampling the frames at directions, the panorama views and the vignetting fit.
- Reference: the API, every public function and type.
- Worked example: the repository's
examples/panorama_vignetting/runs the whole chain, from RAW decode to corrected panoramas, on two RAW sweeps from configs of record.
Role in the stack
PanoramaStitching is part of the VisualGeometryToolkit stack. It composes VisualGeometryToolkitFeatures' ASIFT detector and LoMa descriptor and matcher, VisualGeometryToolkitEstimation's find_rotation and fit_rotation_averaging, and VisualGeometryToolkitCore's cameras, panorama projections and warping. Its one radiometric estimate is the vignetting a rotating camera's own frames determine; exposure, response and colour are left to the caller.
Citing
PanoramaStitching is written by James Pritts. Its CITATION.cff is the source of this entry:
@software{pritts2026panoramastitching,
author = {Pritts, James},
title = {{PanoramaStitching.jl}: Panoramas and Vignetting of a Rotating Camera},
url = {https://github.com/VisualGeometryToolkit/PanoramaStitching.jl},
year = {2026}
}MIT licence.