The [tuning] schema
This is the one document of the stack's cross-validation specification. Every parameter that influences a tuned result is a field of a strategy type and has its default in configs/tuning.toml. The two are joined by one parser, Tuning.tuning_spec. A tuning run persists the spec it ran under, so a deployed value can be traced back to the exact protocol that produced it.
Module: VisualGeometryToolkitEval.Tuning (src/tuning/*.jl). The protocol these specs serve is The experimental protocol.
Vocabulary
Each quantity has one name, used in types, fields, TOML keys, grid axes and artifact keys:
| name | meaning |
|---|---|
efficiency_constant | a width as a multiple of the estimated inlier scale ŝ (dimensionless); width = efficiency_constant · ŝ |
loss_width | a robust loss's width, supplied directly (no scale estimate): the width of a lifted loss, of a loss minimized by iteratively reweighted least squares (IRLS), or of a threshold score (a threshold is its truncated loss's width) |
outlier_halfwidth | the Student-t marginal's uniform outlier half-width |
efficiency_constant_window | the standardized window [lo, hi] of efficiency constants every search window is cut from |
Where the parameters live: three levels
A config states [tuning.<name>] tables. Each names its scheme with type and states only what differs from the defaults. A key it leaves out is inherited through three levels:
[tags.<family>.<tag>]: the default parameters of a tagged strategy (for example,log_counthasn = 200). The untagged strategies use[types.<type>]instead: the efficiency-constant window, the grid extension, the cross-validation and the budget sampling.[schemes.<scheme>]: the scheme's own defaults, where they differ from level 1.- The config's
[tuning.<name>]table.
If a tagged table names a different type than the table it would inherit, it starts again from its own tag's defaults: a strategy never inherits the parameters of a different strategy. An unknown key or tag is an ArgumentError that names the key path, for example [tuning.miscal.coarse.spacing] has unknown key(s) ["per_decad"].
layer_tuning(lower, upper) layers a paper's protocol-level table under a cell's overrides with the same tag-aware rule; the parser does not layer, so a paper layers its two tables before parsing. resolve_tuning!(cfg) replaces every entry with its complete table. The Experiments run_key and write_run_records both resolve before hashing or writing, so a run key covers every tuning parameter, including defaults the config only inherited.
The writer tuning_table(x) emits the complete table. The round trips TOML → type → TOML and type → persisted table (read_tuning) → type are both lossless under ==. A persisted table inherits nothing: a missing key is an error.
Tag families
Every tagged table carries type = "<tag>". The registry is TUNING_TAGS.
spacing: GridSpacing
grid(lo, hi, spacing) is the one grid constructor: a Base range (linear) or logrange (log) of npoints(lo, hi, spacing) points. Both endpoints are exactly lo and hi, and every grid has at least 2 points.
| tag | type | parameters | points |
|---|---|---|---|
log_per_decade | LogPerDecade | per_decade | round(log10(hi/lo)·per_decade) + 1 |
log_per_octave | LogPerOctave | per_octave | round(log2(hi/lo)·per_octave) + 1 |
log_count | LogCount | n | n log-spaced points |
linear_count | LinearCount | n | n linear points |
The +1 is the fencepost: d decades at p per decade make d·p intervals, so they have d·p + 1 points.
window: WindowRule
window_bounds(rule; efficiency_constant_window, native, s2, center) returns (lo, hi). A rule refuses a context it needs and is not given.
| tag | type | parameters | window |
|---|---|---|---|
fixed | FixedWindow | lo, hi (required) | [lo, hi] in native units |
standardized | StandardizedWindow | none | the run's efficiency-constant window |
coverage | CoverageWindow | q_lo, q_hi, log_margin, zeta_octaves | [native(lo, ŝ_min/ζ)·e^-m, native(hi, ŝ_max·ζ)·e^m], where (ŝ_min, ŝ_max) = scale_bounds(s2, (q_lo, q_hi)), ζ = 2^zeta_octaves and m = log_margin |
centered | CenteredWindow | factor | [center/factor, center·factor] around a centre the axis supplies |
The coverage window depends on the data: on the validation pool's σ̂² and on the codimension d_g inside native. The realized window is persisted in the grid spec and the rule in the spec. d_g is not a tuning parameter: it is the problem's codimension, has no default anywhere, and is persisted as the artifact key d_g. q_lo == q_hi is the one-anchor case; at 0.5 it is the reference scale, so { q_lo = 0.5, q_hi = 0.5, zeta_octaves = 0.0 } is the window-matched grid.
axis: TuningAxis
| tag | type | fields |
|---|---|---|
spaced | SpacedAxis | window (a rule), spacing |
listed | ListedAxis | values (finite, strictly increasing) |
grid_spec(axis; context...) realizes an axis as a persisted GridSpec.
grid: GridSpec (persisted, never authored)
| tag | type | fields |
|---|---|---|
spaced | SpacedGrid | lo, hi, spacing, n_lo, n_hi |
listed | ListedGrid | values |
grid(spec) rebuilds the realized grid: one range over the base window's step continued n_lo grid points below lo and n_hi above hi (a log step is a ratio). extend(spec, sides; npts) -> (spec′, n_lo) is the one extension: every point of grid(spec) reappears, to rounding, at index + n_lo, so computed values stay valid by index. extension_offset(base, sup) compares specs, never realized values: it returns the offset when sup is base or an extension of it, and nothing otherwise. nearest_column(grid, x) is the evaluation-only lookup of a corrupted native (never a tuning's).
criterion: the objective (TuningCriterion)
| tag | type | parameters | objective (minimized) |
|---|---|---|---|
median_error | MedianError | none | median, failures as Inf |
finite_median | FiniteMedian | none | median of the finite errors (Inf when none) |
scene_mean_lower_median | SceneMeanLowerMedian | none | mean over scenes of each scene's lower median |
scene_mean_median | SceneMeanMedian | none | mean over scenes of each scene's finite median (Stats scene_mean_median) |
mean_average_accuracy | MeanAverageAccuracy | max_error (required) | minus the mean average accuracy (mAA) up to max_error, in the errors' own units (Stats eligible_maa) |
scene_split: SceneSplit
| tag | type | parameters |
|---|---|---|
registered | RegisteredSceneSplit | name (required); validation_scenes, test_scenes filled in from configs/splits.toml |
unit_sampling: UnitSampling
| tag | type | parameters |
|---|---|---|
exhaustive | ExhaustiveSampling | none |
stratified | StratifiedSampling | fraction, cap_per_scene (0 = no cap), seed, rng |
uniform | UniformSampling | fraction, cap (0 = no cap), seed, rng |
The retired tags all and simple_random (now exhaustive and uniform) are refused as unknown.
Untagged strategies ([types.<key>])
| key | type | fields |
|---|---|---|
efficiency_constant_window | EfficiencyConstantWindow | lo, hi |
grid_extension | GridExtension | npts (per clipped side per round), max_rounds |
cross_validation | CrossValidation | scene_split (no default), validation, test (both exhaustive by default) |
budget_sampling | BudgetSampling | sizes, n_draws, seed, rng |
Randomness
Each rule below is the definition. Every seed of a sample is derived by Stats stable_seed(seed, keys...): the first 8 bytes of the SHA-256 hash of the seed and the keys as text. The algorithm is fixed, unlike Base.hash, which changes across Julia versions, and the seed is keyed by a name rather than an index:
- Stratified sampling. Scene
sdraws fromrng(stable_seed(seed, s)). - Uniform sampling. The pooled draw is
rng(stable_seed(seed, "simple_random"))(the salt keeps its original spelling, so a sample did not move with the type's rename). - Budget draws. Draw
rat budgetnisrng(stable_seed(seed, "budget", n, r))(budget_rng,budget_units); the interval over a budget's draws resamples fromrng(stable_seed(seed, "resampling", n))(resampling_rng). - Scene-clustered bootstrap rows (Stats
scene_mean_median_rows!): pointjresamples fromStableRNG(stable_seed(seed, "scene_mean_median", j))(the mAA's: key"scene_mean_maa").
rng names an algorithm in RNG_ALGORITHMS: "StableRNG" only (StableRNGs.jl), whose streams are fixed across Julia versions; a permutation is drawn with shuffle, which StableRNGs guarantees (randperm it does not). The bias-corrected and accelerated (BCa) bootstrap itself (Bootstrap.jl) resamples from Julia's default stream seeded by the row's rng, so a reported interval also depends on the Julia version, recorded in every artifact's code identity.
The cross-validation
Every scheme holds a cross_validation::CrossValidation: the data a tuning reads and the data its evaluation reports on.
CrossValidation(scene_split::SceneSplit, validation::UnitSampling, test::UnitSampling)- The scene split.
RegisteredSceneSplit(name, validation_scenes, test_scenes): a split ofconfigs/splits.toml, the one file of record of every scene list in the stack. A config names it,scene_split = { type = "registered", name = "eth3d" }, and never lists scenes (a config that does must list exactly the registry's, else anArgumentError). The parser fills the lists into the complete table, sotuning_hash, the run key and every artifact carry the exact lists used: editing a list moves every hash that names the split. The constructor refuses an empty side, a duplicate and any scene on both sides; the lists are kept sorted. There is no default split. - The unit samplings.
ExhaustiveSampling();StratifiedSampling(fraction, cap_per_scene, seed, rng), each scene sampled separately (the scene mix is preserved);UniformSampling(fraction, cap, seed, rng), the units of all the side's scenes pooled. A sampler always permutes, keepsmax(1, round(fraction·n))and applies the cap. Only then does it sort back into the data's order, so a cap never keeps a prefix of the data. Adding, removing or reordering other scenes leaves a stratified scene's sample unchanged. - The sampling unit is a property of the data, never of the spec:
sampling_unit(::Type{D})isPairUnit()(an image pair: the stereo datasets, the scoring pools) orQueryUnit()(a query image of an absolute-pose dataset), declared by each data type with no fallback. The tuning artifact records it.
validation_units(cv, units_by_scene) and test_units(cv, units_by_scene) apply a side: units_by_scene maps scene names to their units in the data's order and must hold every scene of that side; the result is scene => kept units, scenes sorted. sample_units(sampling, units_by_scene) is the sampler itself.
A driver states which samplings it supports in the signature of the method it calls, never with a runtime check, so an unsupported spec is a MethodError:
two_view_units(cv::CrossValidation{<:SceneSplit,<:Union{ExhaustiveSampling,StratifiedSampling}}, pools) = …
query_units(cv::CrossValidation{<:SceneSplit,<:Union{ExhaustiveSampling,UniformSampling}}, queries) = …The budget draws of a validation-budget run are the scheme's budget_sampling (BudgetSampling(sizes, n_draws, seed, rng)): for each budget n, n_draws uniform samples of n of the eligible validation units, budget_units(b, n_units) -> [(n, r, indices)].
[tuning.miscal.cross_validation]
scene_split = { type = "registered", name = "eth3d" }
validation = { type = "stratified", fraction = 0.25 }
test = { type = "exhaustive" }Schemes
joint_coarse_fine: JointCoarseFine (Evaluation coarse_fine_search)
The coarse-to-fine search over the product of named axes. It runs in four phases:
- the coarse product grid, every cell scored;
- while the joint argmin lies within
edge_marginpoints of a face, that axis extends on that side (extension.nptspoints, at mostextension.max_roundsper side) and only the new slab is scored; - the product of each axis's
finespacing over the coarse argmin's two neighbours on that axis; its free argmin is the selection; bootstrap_argminover the evaluated cells'unit_statisticsgives the joint selection's sampling distribution, read per axis.
An axis still within the margin after its budget is a boundary hit, refused by the one reader deployed_setting. An expensive axis is asked for one value at a time: the objective is called once per round per distinct expensive value, so a caller that keeps its expensive run (a detection run) per value pays one run per value visited.
| field | meaning |
|---|---|
axes | name → CoarseFineAxis: coarse (window rule + spacing), extension (npts, max_rounds per side), fine (spacing over the bracket), expensive |
edge_margin | extend when the argmin lies within this many points of a face |
criterion | the objective (the errors at a cell; the criterion aggregates them) |
cross_validation | the scene split and the validation / test unit samplings |
A new search backend (a surrogate model, say) is a new scheme whose search returns the same record (selected, boundary_hit, evaluated); the artifact, the reader and the bootstrap serve it unchanged.
coarse_fine: CoarseFine (Evaluation coarse_fine_cv)
The one-axis case of joint_coarse_fine, its axis spelled flat (the same search; coarse_fine_cv and its batched form return its record in the scalar keys).
| field | meaning |
|---|---|
coarse | axis: the coarse probe's window rule and spacing |
extension | npts per extension, max_rounds per side |
edge_margin | extend when the argmin lies within this many points of an edge |
fine | the fine spacing over the coarse argmin's bracket |
criterion | the objective (evalfn(x) returns the errors; the criterion aggregates them) |
cross_validation | the scene split and the validation / test unit samplings |
windowed_bootstrap: WindowedBootstrap (Evaluation WindowedGridMethod)
| field | meaning |
|---|---|
efficiency_constant_window | the standardized window every search window is cut from |
outlier_halfwidth, loss_width | the native grids (axes) |
efficiency_constant | the standardized display axis |
criterion | the objective of each draw's windowed free argmin |
cross_validation | the scene split and the validation / test unit samplings |
budget_sampling | the validation budgets and draws (compute_method_rows) |
factorial: Factorial
Every combination of named settings of a method's configuration keys: listed axes, no reference scale, no boundary extension.
| field | meaning |
|---|---|
axes | named axes (configuration key → TuningAxis, usually listed) |
criterion | the objective |
cross_validation | the scene split and the validation / test unit samplings |
Examples
A scalar probe:
[tuning.polish]
type = "coarse_fine"
coarse = { window = { type = "fixed", lo = 0.5, hi = 16.0 }, spacing = { type = "log_per_decade", per_decade = 4 } }
extension = { npts = 4, max_rounds = 2 }
fine = { type = "log_count", n = 200 }
cross_validation = { scene_split = { type = "registered", name = "phototourism" } }The bootstrap harness:
[tuning.lo_sensitivity]
type = "windowed_bootstrap"
efficiency_constant_window = { lo = 1.0, hi = 3.3 }
efficiency_constant = { spacing = { type = "linear_count", n = 17 } }
budget_sampling = { n_draws = 120 }
cross_validation = { scene_split = { type = "registered", name = "phototourism" } }A joint detector tuning (one detection run per peak threshold):
[tuning.detection]
type = "joint_coarse_fine"
criterion = { type = "mean_average_accuracy", max_error = 10.0 }
axes.peak_threshold = { coarse = { type = "spaced", window = { type = "fixed", lo = 0.005, hi = 0.02 }, spacing = { type = "log_per_octave", per_octave = 2 } }, fine = { type = "log_count", n = 3 }, expensive = true }
axes.min_similarity = { coarse = { type = "spaced", window = { type = "fixed", lo = 0.3, hi = 0.8 }, spacing = { type = "linear_count", n = 6 } }, fine = { type = "linear_count", n = 5 } }
cross_validation = { scene_split = { type = "registered", name = "<a registered split>" } }The tuning artifact
save_tuning_artifact(path, spec; grids, unit, code, data...) is the one writer of every artifact a tuning run persists (a tuning sidecar or its tune-phase checkpoint, a probe CV artifact, a bundle of validation matrices); load_tuning_artifact(path) -> TuningArtifact is the one reader.
| key | content |
|---|---|
tuning_format | the :tuning artifact format (src/artifact_format.jl): the constant, not a number copied here |
tuning_name | the [tuning.<name>] entry |
tuning_spec | its complete scheme table, the cross-validation's resolved scene lists included |
tuning_hash | tuning_hash(spec): SHA-256 (16 hex) of the scheme table's canonical JSON (VisualGeometryToolkitCore's canonical_json; format 8, a v6/v7 file's sorted-TOML hash is verified as stored and re-hashed on read); names and spelling do not move it, any parameter does |
grid_specs | axis → GridSpec table; the realized grids are rebuilt from these |
sampling_unit | "pair" or "query": the data's sampling_unit (art.unit) |
code_identity | the version of this package and of every module the caller names as the code its result came from; julia: its version and the rng algorithms. The code's git state is the run's code_trees (its run records and _provenance.toml) |
| … | the run's own data (the arms, their tuned points as native values, the boundary hits, d_g, …) |
The reader refuses a file of another format first, with an ArgumentError naming the file and the format change (regenerate it by re-running its tuning). The same mechanism stamps pools, with pool_format. The reader then refuses a stored spec whose hash does not match its table, and finally rebuilds every grid: art.grids[:axis] is the realized grid.
Format 6 is still read. Format 7 (2026-11-11) names the efficiency constant efficiency_constant (the axis, its grid spec, the scheme's efficiency_constant axis and efficiency_constant_window) and stores it as the PLAIN multiple, width = c·ŝ, for every loss. A format-6 sidecar keeps its old keys on disk (a sidecar is immutable provenance, never rewritten): the reader carries it over in memory through its version step, registered with VisualGeometryToolkitIO's migrate_schema and run by its migrate_record. The step checks the file's recorded hash against its STORED table, renames the keys of the spec and the grid specs and the axis names in the data (boundary_hits, kinds, a coarse-fine artifact's axes, selected, boundary_hit and record.param_name), and records the renamed table's hash. art.format is the format the file was written in and art.recorded_hash its own hash. A format-6 efficiency constant is the value its arm's ctor took THEN, in whatever currency that ctor read it: a bare Geman–McClure Irls took a half-weight depth, while a roster whose own map composed the conversion (or a GNC schedule stated in the target width) took the plain value already. The loss shape cannot tell the two apart, so the ROSTER declares it, per arm: ComposedSpec(…; format6_constant = f), the identity by default (f = m -> decision_width(GemanMcclureShape(), m) for a ctor that took the half-weight depth). The hook is applied once, in Evaluation.tuned_width(art, arm), the one reader of a tuned width, so the deployed LO is the one the sidecar was tuned with: a caller that needs a width itself (to refit, render, or carry it into a resumed run's artifact) calls it, never art["tuned_width"], so a format-6 width carried into a format-7 file is stored plain.
Provenance: from a deployed constant to the run
- A deployed cross-validated constant carries
tuning = { config, name, hash, arm }: the config of record holding the[tuning.<name>]spec, the spec'stuning_hash, and the arm it was tuned for. A transferred constant names the constant it transferred from. - Parsing the config's
[tuning.<name>](layered as the paper layers it) gives the spec; its hash must equal the constant's. - The run's artifact records the same spec and hash, its grid specs, its code identity and its run key, and the tuned point.
- The run key (
run_key: the SHA-256, 16 hex, of the resolved config's canonical JSON) names the directory that holds the artifact and its_config.toml/_provenance.tomlrecords.