Camera RAW API
The sensor image: a decoded Bayer RAW as RawMosaic (an AbstractMatrix{UInt16} of its photosites that knows its colour phase, its levels and where it sits on the sensor), its photosites channel-first as a method of ImageCore's channelview (a view, on the mosaic's device), and LinearRaw, its exact linear values on the 2×2 cells, with the maps of every plane and channel onto the sensor. Decoding a file is VisualGeometryToolkitFeatures' (load_raw_mosaic, LibRaw), and so is the display demosaic. A cell mode's frame map is also read without an image, raw_frame_map(LinearRaw{:green_cells}), and its provenance record, raw_provenance(LinearRaw{:green_cells}), for a frame with no RAW behind it. See the Raw images guide.
The mosaic and its photosite views
VisualGeometryToolkitCore.BayerCellFrame — Type
BayerCellFrame{C} <: Frame{2}The COLMAP pixel frame of one colour plane C of the half-resolution grid of 2×2 Bayer cells: pixel (row, column) of the plane is centred at (column − ½, row − ½) of its frame. The planes are indexed alike but sampled at different points of the cell, so each is a frame of its own:
:R,:B— a cell's red and blue photosite (a plane of a:bin2LinearRaw, a channel of a mosaic'schannelview);:G1,:G2— a cell's two green photosites, in the cell's column-major order (the two green channels ofchannelview(m));:G— the mean of the two greens, which sits at the cell centre in every phase: the cell grid itself (the:green_cellsimage), whose map israw_frame_map.
bayer_plane_map maps each onto the visible raw's PixelFrame{:colmap}, and a composition that joins one plane's frame to another's is refused (MethodError), as at every frame junction of Core.
VisualGeometryToolkitCore.LinearRawView — Type
LinearRawView{T, A, D, L} <: AbstractMatrix{T}The lazy linear view of a mosaic's photosites on the raw grid, linearview(m): element (i, j) is the linear value of the held array's code there, with its channel's black level and span, computed when read. Under a sensor model whose noise has a variance law (law: a PoissonGaussian or a RelativeNoise) the element is an Uncertain{V}, the value with its scalar variance computed on the same read (owner ruling D14: the default uncertain image is this lazy one; copy materialises it as a StructArray); otherwise T is V. V is the value type linearview(V, m) was asked for (Float64 by default): each element is computed in Float64 and rounded once to V (its variance to Float32 for a Float16 view; see LinearRaw, "The element type"). It holds the photosites (parent, the mosaic's held array, a device array too), the four channels' levels and, from a sensor model's Dark, the offset map over the visible raw (dark, on the same device; nothing without one) read at the mosaic's window_origin; similar, a broadcast and copy are the held array's kind (a broadcast over a device view runs on the device), adapt moves it, and a KernelAbstractions kernel reads it as it is (its indexing is two parities, the one black rule and one linearisation).
VisualGeometryToolkitCore.PhotositeColors — Type
PhotositeColors <: AbstractMatrix{UInt8}The colour index of every photosite of a mosaic, lazily, as an isbits value: the size, the colour pattern (at most 4 × 4, in an NTuple) and its phase at the mosaic's first photosite. Indexing is the one phase rule; nothing is stored per photosite, so it is the same on the host and inside a GPU kernel or a fused broadcast with a device mosaic's arrays (it holds no array, and a device array in the same broadcast decides where it runs).
VisualGeometryToolkitCore.RawMosaic — Type
RawMosaic{A <: AbstractMatrix{UInt16}, M <: NamedTuple} <: AbstractMatrix{UInt16}A RAW file decoded and nothing more: the visible area of the sensor, as the decoder reports it (VisualGeometryToolkitFeatures' load_raw_mosaic, LibRaw). It IS its photosite matrix: size, indexing and parent are the held array's, so an ordinary array or image function reads it unchanged, and what such a function makes (similar, copy, a broadcast, a reduction) is a plain array of the held array's kind — the colour pattern and the metadata stay with the mosaic. copy, ==, isequal and isapprox are the held array's own (no element read through the wrapper, so a device mosaic compares on the device), and equality is the photosite matrix's: two mosaics with the same photosites are == whatever their colour pattern, window origin or metadata (compare photosite_colors, window_origin and metadata for those), and a mosaic is == to its held array. It is read-only (no setindex!): write to parent(m). The held array may be a device array (adapt(CuArray, m), or adapt(backend, m) with a KernelAbstractions backend, moves the photosites and keeps the rest); a kernel receives parent(m), never the wrapper (nor, on a GPU, the nested view channelview(m): read its photosites by its index rule).
A unit-range index m[rows, columns] (a copy) or view(m, rows, columns) (no copy) is a window: a mosaic of its own whose colours are the parent's at those photosites (the colour phase shifts with an odd start, as a circshift of the pattern would) and whose raw_frame_map is its offset into the visible raw. Any other index is a plain array.
raw::A— the photosites,raw[row, column], in the sensor's orientation (no EXIF flip): pixel(row, column)is the raw's COLMAP pixel centred at(column − ½, row − ½)— the visible raw'sPixelFrame{:colmap}, the frame every calibration of the raw is in (a window's pixels are offset bywindow_origin).pattern::Matrix{UInt8}— the smallest repeating colour pattern, square, at most 4 × 4, from the full raw frame's origin (LibRaw'sCOLORread from there), in LibRaw's colour indices:0R,1G,2B,3the second G of an RGBG pattern. Each photosite's index isphotosite_colors(mosaic), the pattern tiled from that origin.black::NTuple{4, Float64}— the black level per colour index (the common and per-channel parts summed, LibRaw'sadjust_bl).white::Float64— the white level (LibRaw'smaximum): the code at which a photosite is saturated.visible_origin::NTuple{2, Int}—(top_margin, left_margin), the visible area's offset into the full raw frame, which israw_size((height, width)).metadata::NamedTuple— what the decoder read: the EXIF fields (VisualGeometryToolkitFeatures types them:exposure_settings,lens_settings,body_identity,lens_identity,capture_time), the daylight white balance, the camera's declared linear-response limit, the decoder and its library's version. Core reads none of it except inraw_provenance.window_origin::NTuple{2, Int}—(rows, columns), the offset ofraw[1, 1]from the visible area's first photosite:(0, 0)as decoded (the keyword of the constructor), a window's start less one otherwise.
VisualGeometryToolkitCore.SaturationView — Type
SaturationView{A} <: AbstractMatrix{Bool}The lazy saturation flags of a mosaic's photosites on the raw grid, saturationview(m): element (i, j) is raw[i, j] ≥ white, read when indexed. The match of LinearRawView: the same grid, the same held array, and the same device behaviour (similar, a broadcast and copy are the held array's kind; adapt moves it; a kernel reads it as it is).
ImageCore.channelview — Method
channelview(m::RawMosaic) -> AbstractArray{UInt16, 3}ImageCore's channelview, for a Bayer mosaic: the photosites channel-first, 4 × H ÷ 2 × W ÷ 2, without a copy. The Bayer interpretation of "channels": each 2×2 cell is a pixel and its four photosites — spatially interleaved, each at its own position on the sensor — are its channels, so v[k, i, j] is photosite k of cell (i, j), the cell's photosites in column-major order (k = di + 2dj + 1 for photosite (2i − 1 + di, 2j − 1 + dj)): the mosaic's own phase order, whose colours are bayer_channels(m) (R, G1, G2, B for an RGGB cell; another phase permutes them, since reordering channels is not a view). It is built as ImageCore builds a view: a reshape of the photosite matrix (H × W → 2 × H/2 × 2 × W/2), its dimensions permuted (→ 2 × 2 × H/2 × W/2, a PermutedDimsArray) and reshaped (→ 4 × H/2 × W/2), views of parent(m): writing through it writes the mosaic's held array, a device mosaic's view is a view of its device array (KernelAbstractions' get_backend is the device's, adapt(Array, v) moves it whole), and a broadcast or a Cartesian copy over a whole mosaic's view runs on the device. An odd last row or column (no 2×2 cell) is left out, as in LinearRaw.
In a GPU kernel read the photosites by the view's index rule instead, parent(m)[2i − 1 + di, 2j − 1 + dj] (or the 4-D parent(v)[di + 1, dj + 1, i, j]): Base's ReshapedArray indexing compiles in a CUDA kernel only over a whole contiguous array, at about a third of the held matrix's bandwidth (509 against 1391 GB/s on a B200, 2848 × 4256), and not at all over a window or an odd-sized mosaic (an InvalidIRError); benchmark/raw_channelview.jl measures it, and LinearRaw's cell kernel reads the held matrix.
A plain AbstractMatrix{UInt16} keeps ImageCore's own method (a gray image's channel view is the array itself): this method is the mosaic's, whose phase says which photosite is which. Channel k's pixel frame is BayerCellFrame{C} (C its colour), and bayer_plane_map(m, k) maps it onto the visible raw, exactly, for every Bayer phase and either parity of a window's origin. Its inverse is photositeview. The linear values channel-first are channelview(linearview(m)).
VisualGeometryToolkitCore.bayer_channels — Method
bayer_channels(m::RawMosaic) -> NTuple{4, Symbol}The colour of each channel of channelview(m): :R, :G1, :G2, :B, in the channels' order, the cell's photosites column-major ((0, 0), (1, 0), (0, 1), (1, 1) from the cell's top-left). (:R, :G1, :G2, :B) for an RGGB cell, (:G1, :B, :R, :G2) for GRBG; the greens are numbered in that order. The cell's phase is the mosaic's own at its first photosite, so a window with an odd start reads another phase (from the same pattern, the one phase rule of photosite_colors). A pattern that is not a 2×2 Bayer cell (one R, two G, one B) is an ArgumentError.
VisualGeometryToolkitCore.bayer_plane_map — Method
bayer_plane_map(m::RawMosaic, k::Integer) -> SimilarityMap{2, …, BayerCellFrame{C}, PixelFrame{:colmap}}The map of channel k of channelview(m) onto the visible raw's PixelFrame{:colmap}: pixel (i, j) of the channel, at (j − ½, i − ½) of its frame BayerCellFrame{C} (C = bayer_channels(m)[k]), goes onto its photosite's centre, (2j − 1 + dj − ½, 2i − 1 + di − ½) plus a window's offset, exactly, for every Bayer phase and either parity of the visible or window origin. It is the cell grid's map x ↦ 2x after the photosite's translation from the cell centre ((±¼, ±¼) cell pixels): the rule the planes of a LinearRaw share, so the R channel's map IS the R plane's of the mosaic's :bin2 image. An isbits value, usable inside a kernel. A channel past the fourth is an ArgumentError.
VisualGeometryToolkitCore.cfa_string — Method
cfa_string(mosaic::RawMosaic) -> StringThe colour pattern as letters, row by row ("RGGB" for [0 1; 3 2]): LibRaw's indices read through "RGBG", as convert_raw.py records it.
VisualGeometryToolkitCore.linear_quantum — Method
linear_quantum(v::LinearRawView, i::Integer, j::Integer) -> Float64The quantum of photosite (i, j) of a linear view: the resolution of its stored value in linear units, for a caller that derives a resolution from it (an LM's residual_resolution). It is one RAW code, 1 / (white − black[c]), c its colour, the step between two adjacent codes of linearview's value there (the channel's span; a Dark replaces the black per photosite but not the span), or the spacing of the view's element type at the value, eps(value_of(v[i, j])), where that is coarser: a Float16 view near the white level steps by 2⁻¹¹, span/2048 codes (≈ 7.75 on a 14-bit span). For the Float64 default the spacing is always finer (eps(x) < 1/65535 for every value), and the quantum is the code's, as before.
VisualGeometryToolkitCore.linearview — Method
linearview([T = Float64], m::RawMosaic; sensor_model::SensorModel = SensorModel()) -> LinearRawViewThe mosaic's linear values on the raw grid, lazily, as ImageCore's normedview reads codes as numbers without a copy: element (i, j) is (raw − black[c]) / (white − black[c]) of photosite (i, j), c its colour — the one linearisation, the one LinearRaw's cell kernel applies. Not clipped, no gain. It composes with ImageCore's channelview: channelview(linearview(m)) is the linear values channel-first (4 × H/2 × W/2, the channels of channelview(m)), so the green-cell value of cell (i, j) is the mean of its two green channels, bit for bit (LinearRaw(m; mode = :green_cells).image[i, j, 1], at gain 1). An odd last row or column is in the view (its photosites' colours by the same rule) and left out of the channel view.
Not normedview(N0f16, …) itself, whose code / 65535 ignores the black levels, nor a MappedArrays.mappedarray of the codes, whose function sees the value alone: the black level and span depend on the photosite's position (its colour in the phase), so the view reads its index — one parity rule, the cell sites' (k = di + 2dj + 1).
sensor_model, the frame's SensorModel, sets the levels as it does LinearRaw's, by the same resolution: its black slot's per-colour-index black (the span white − black[c]), and a Dark's offset in place of the black per photosite. Its defects are not values: they join LinearRaw's mask, not this view. Its noise is: under a PoissonGaussian or a RelativeNoise each element is an Uncertain{T}, the value and its scalar variance (Statistics.var(x), computed on the read by the law LinearRaw's kernel writes per plane, one site at gain 1: owner ruling D14, the default uncertain image of the photosites; the binned cell image, LinearRaw's, is materialised once by its kernel and never re-derived by a consumer), and copy of the view is the materialised form, a StructArray of a value plane and a variance plane (on the view's device). The layout is the raw grid, H × W; channelview of it is the 4 × H/2 × W/2 cells, uncertain alike. With the absent model, the default, the view is the mosaic's own levels, plain values.
T (a Base.IEEEFloat) is the elements' value type, as LinearRaw's (the same one default): each element is computed in Float64 and rounded once to T, so copy(linearview(Float16, m)) is Float16.(copy(linearview(m))) bit for bit, and an uncertain element's variance is Float32 beside a Float16 value. A lazy view stores nothing; the type is what a copy, a broadcast or a kernel reading it holds.
VisualGeometryToolkitCore.photosite_colors — Method
photosite_colors(mosaic::RawMosaic) -> PhotositeColors <: AbstractMatrix{UInt8}Each photosite's colour index (LibRaw's: 0 R, 1 G, 2 B, 3 the second G), the size of mosaic: the pattern tiled from the full raw frame's origin, so photosite (row, column) reads pattern[mod(top + row − 1, n) + 1, mod(left + column − 1, n) + 1] with (top, left) = visible_origin .+ window_origin. Lazy and isbits (PhotositeColors): no matrix is built, and it broadcasts with a device mosaic's arrays on the device, fused, so a photosite mask is one broadcast with no host round-trip:
col = photosite_colors(m)
valid = ((col .== 0x01) .| (col .== 0x03)) .& (parent(m) .!= 0) .& .!saturationview(m)(collect gives a Matrix{UInt8}.)
VisualGeometryToolkitCore.photositeview — Method
photositeview(v::AbstractArray{T, 3}) -> AbstractMatrix{T}The inverse of a mosaic's channelview, in the role ImageCore's colorview plays for an image's (a name of its own: colorview makes colorants of channels, and this makes the photosite matrix of cells): a channel-first array of 2×2 cells (4 × h × w, each column a cell's photosites in column-major order) as the 2h × 2w photosite matrix, without a copy. Of a mosaic's channelview it is the mosaic's (even-sized) held array again; of any other array (per-photosite values computed channel-first, say) it is a view in the mosaic's layout, photositeview(v)[2i − 1 + di, 2j − 1 + dj] == v[di + 2dj + 1, i, j]. A first dimension other than 4 is an ArgumentError.
VisualGeometryToolkitCore.raw_frame_map — Method
raw_frame_map(m::RawMosaic) -> SimilarityMap{2, …, PixelFrame{:colmap}, PixelFrame{:colmap}}The map of the mosaic's pixels onto the visible raw's PixelFrame{:colmap}: the identity as decoded, a window's offset (x ↦ x + (columns, rows) of its window_origin) for a window.
VisualGeometryToolkitCore.saturationview — Method
saturationview(m::RawMosaic) -> SaturationViewThe mosaic's saturation flags on the raw grid, lazily: true at a photosite whose code is at or above the white level, raw ≥ white with no margin — THE saturation rule, the one LinearRaw's flags apply per cell (a cell is flagged when any photosite its planes read is; under a sensor model holding a DefectList LinearRaw also masks those photosites). The Bool match of linearview(m): the same grid, and channelview of it is the flags channel-first. A mask of the usable photosites of one colour is a plain broadcast with photosite_colors:
col = photosite_colors(m)
usable_green = ((col .== 0x01) .| (col .== 0x03)) .& .!saturationview(m)Linear values on the 2×2 cells
VisualGeometryToolkitCore.LinearRaw — Type
LinearRawA RAW image as linear values: (raw − black[c]) / (white − black[c]) × gain (each photosite's linearview value), camera raw colour, white balance 1, 1.0 = the white level (before the gain). Not clipped: a value above 1 (a gain above 1) or below 0 (noise under the black level) is kept.
image—image[row, column, plane], on the mosaic's device (anArrayon the host); a plane's frame is asked of the image with its index (bayer_plane_map(L, c)). Under a sensor model whose noise defines a variance (PoissonGaussian,RelativeNoise) it is aStructArrayofUncertain{T}values (owner ruling D11): itsvaluecomponent the values of element typeT(below), itsattributescomponent each value's variance as a 1×1CovMat(cov(L.image[i, j, p])[1], ofFloat32forFloat16values, else ofT), both planes from the one kernel. This cell image is the MATERIALISED uncertain image (owner ruling D14, (a)): its binned variances are computed once, in the one kernel, from the one variance formula, and a consumer never re-derives them; the photosites' uncertain image,linearview(m; sensor_model), is the lazy default. Otherwise (no noise, aReadNoisealone) it is a plainAbstractArray{T, 3}. A consumer dispatches on the image's element type.saturated::AbstractMatrix{Bool}(aBitMatrixon the host) — the MASKED pixels: those whose value includes a photosite at the white level (raw ≥ white) or, under a sensor model holding aDefectList, a defective photosite: in:bin2a cell is masked when ANY of its four photosites is, so a cell whose two greens straddle the white level is flagged rather than averaged into a plausible value; in:green_cellswhen either green is. Masked, never interpolated.geometry— where the samples are,(; plane_maps, green_offsets, plane_sites):plane_maps, per plane, its map onto the visible raw (bayer_plane_map, which reads it);green_offsets, the cell's two green photosites,(Δx, Δy)from the cell centre in RAW pixels, in the cell's column-major order (G1,G2ofbayer_channels;green_cell_sites);plane_sites, per plane, the photosites its value is the mean of, each(; color, dn): its CFA colour index (LibRaw's0R,1G,2B,3the second G; the index of a per-colour component's entry,σ[color + 1]) and itswhite − black[c], the codes that span0 … 1(before the gain) — what a value's photon and read noise scale by. The green plane's are in the order ofgreen_offsets.black[c]is the channel's constant (the sensor model's black slot's) also when aDarkreplaces it per photosite: the span is a scale, the same for every photosite of the channel, so a value is stillcodes / dnabove its own black.
nothingfor:demosaic, whose pixels are interpolated, not samples.gain(positive and finite,ArgumentErrorotherwise) and the mosaic'smetadata(the decoder and its library's version among it).
The element typeT of the values is chosen at construction, LinearRaw(T, mosaic; mode): Float64 by default (one constant, _LINEAR_RAW_ELTYPE), Float32 or Float16. Each value is computed in Float64 (the black subtraction, the cell's mean, the gain) and rounded ONCE, to nearest, when it is stored, and its variance is computed from the unrounded value, plus the variance the store adds, spacing²/12 at the value (0 for Float64: the stored value's variance, see the bright end below); so LinearRaw(Float16, m; mode).image == Float16.(LinearRaw(m; mode).image), bit for bit, and each value is within eps(Float16)·|x|/2 of the Float64 one (half the subnormal step, 2⁻²⁵, below 2⁻¹⁴). The mosaic stays its UInt16 codes. Why Float16 holds a RAW's values (the Sony FX3: white 16383, black 512, so the span is d = 15871 codes, and K = 11.33 e⁻/code):
- The bright end. A value is its codes above black over
d. Float16 keeps 11 significant bits, so on[½, 1)— 7936 to 15871 codes above black — its step is2⁻¹¹·d = d/2048 ≈ 7.75codes, about 8. The shot noise there isσ = √(μ/K)codes:√(16000/11.33) ≈ 37.6at 16k codes and√(7936/11.33) ≈ 26.5at the binade's bottom. The rounding adds its variance,step²/12 ≈ 5.0codes² per value, which an uncertain image's variance plane includes (a consumer that pools the stored values' spread, a photon-transfer fit over Float16 values, sees it: on the FX3 it lowers K from 11.30 to 11.18 e⁻/code, about 22 times K's counts standard error, the added variance per point measured overstep²/120.98 to 1.03, and 11.297 with it subtracted). Against the shot varianceμ/Kthat is largest at the bottom of the binade,μ = d/2:(d/2048)²·K/(12·d/2) = d·K/(6·2048²) ≈ 0.71%of the variance (0.36% of σ). Each binade below halves it: the step halves, so its square quarters, while μ only halves. - The darks. Near the black level Float16 is relative:
2⁻¹¹ ≈ 4.9e-4of the value,log₁₀ 2048 ≈ 3.3significant digits, and below2⁻¹⁴(d·2⁻¹⁴ ≈ 0.97codes) a fixed step of2⁻²⁴·d ≈ 9.5e-4codes. A 16-bit integer PNG of the codes held whole codes, a step of 1. - The variance plane is not Float16 (
Float32beside Float16 values). In the values' units² a dark green cell's variance, its read noise alone, isσ_r²/(2d²)(two greens, gain 1):≈ 7.9e-9for σ_r = 2 codes. That is below half Float16's smallest subnormal (2⁻²⁴ ≈ 6.0e-8), so it would be stored as0, and a whitened row would divide by it. At the white level the shot variance1/(2Kd) ≈ 2.8e-6is subnormal (below2⁻¹⁴ ≈ 6.1e-5), held to about 2% (its spacing 6.0e-8). Float32's smallest normal is 1.2e-38.
A Float16 image takes 2 bytes per value (6 with its variance) against 8 (16). A value must be finite in T: a gain at which a code could exceed floatmax(T) (65504 for Float16: a gain past ≈ 16000 on a 14-bit span) is refused (ArgumentError). Reduce it in Float64: sum(Float64, A), mean(Float64, A). A Float16 accumulation overflows past 65504 (a million green cells near the white level sum to ≈ 8e5) and drops low bits long before. Core's estimators (master_dark, photon_transfer, defect_onsets) read the mosaics' UInt16 codes and accumulate in Float64, so the element type never reaches them.
The mode is the type's first parameter, LinearRaw{:green_cells}, LinearRaw{:bin2} or LinearRaw{:demosaic}: the geometric accessors are methods of the cell images, and the display image's are its refusals. raw_provenance gives the mode, the gain, the metadata and the frame map as plain data for a result to record.
VisualGeometryToolkitCore.LinearRaw — Method
LinearRaw([T = Float64], mosaic::RawMosaic; mode::Symbol, gain::Real = 1, sensor_model::SensorModel = SensorModel()) -> LinearRawThe mosaic's linear values, (raw − black[c]) / (white − black[c]) × gain, on the half-resolution grid of 2×2 Bayer cells (H ÷ 2 × W ÷ 2; an odd last row or column is dropped), arranged by mode (required: there is no neutral choice between them):
:green_cells— THE image for geometry and radiometry, one plane: each cell's two green photosites' mean. The greens of a cell are always diagonal, so in every Bayer phase their mean sits at the cell centre (offset(0, 0)), and the cell grid maps onto the raw exactly byraw_frame_map(x ↦ 2xin COLMAP coordinates). The footprint of a value is its two photosites, at(±½, ∓½)raw pixels from the cell centre for an RGGB or BGGR cell,(±½, ±½)for GRBG or GBRG (green_cell_sites).:bin2— the same cells with their red and blue,× 3planes: R, the mean of the two greens (the:green_cellsimage, bit for bit), B. R and B sit at their photosites,(±¼, ±¼)cell pixels from the centre: each plane's map onto the raw isbayer_plane_map.
Every value is from the mosaic's linearview, read channel-first (channelview(linearview(mosaic)), the mosaic's channelview): the green-cell value of cell (i, j) is the mean of the view's two green channels there, times the gain, bit for bit (the Float64 view's, at the Float64 default; for another element type T the value is that Float64 mean rounded once to T, not the mean of two rounded values). T, a Base.IEEEFloat (Float64, Float32, Float16), is the values' storage: see LinearRaw, "The element type", for the default, the rounding and why Float16 suffices. :demosaic (LibRaw's interpolation, for display figures) is VisualGeometryToolkitFeatures': load_raw(path or mosaic; mode = :demosaic).
The values are those convert_raw.py writes (--mode bin2), in the same Float64 arithmetic and order, before its clip to [0, 1] and 16-bit rounding.
THE black subtraction is here, once, for every caller: sensor_model, the frame's SensorModel (sensor_model resolves it from a characterisation), each slot by its component:
- black:
FileBlack(the absent model's) the mosaic's own per-channelmosaic.black— with the absent modelSensorModel(), the default, bit for bit the values without the keyword;ChannelBlackits values (the spanwhite − black[c]); aDarkits per-photositeoffsetin place of the channel constant in the subtraction ((raw − offset) / (white − black[c]),blackits channel black: the per-channel scalar is the constant-map case of the same kernel path); - defects: a
DefectList's photosites join the saturation flags (saturatedis then "masked: saturated or defective");NoDefectsadds none; - noise:
PoissonGaussianorRelativeNoisemakeimagean uncertain image, each value with its variance from the same kernel (D11):g·max(x, 0)/(m²K)·Σ 1/d + g²/m²·Σ s/d²for a valuexofmphotosites (spansd, gaing,sthe photosite'sσ_r[c]², or under aDarkits dark variance times1 + 1/frames: its read noise and dark current, and the error of the offset subtracted),(σ_rel·x)²under aRelativeNoise;NoNoiseandReadNoiseleave it plain (a read-noise variance alone underestimates a well-exposed pixel's, ruling A).
The maps cover the visible raw and are read at the mosaic's window_origin (DimensionMismatch when they do not reach the mosaic's photosites); they are moved to the mosaic's device.
Saturation is raw ≥ white with no margin: on the Sony ARW sets this was written for, codes pile up exactly at the white level (16383) while the histogram runs smoothly through the camera's declared linear limit (15360, metadata.linear_max), so any margin would be a tuned constant. A caller who wants that stricter limit thresholds mosaic.raw itself.
VisualGeometryToolkitCore.bayer_plane_map — Method
bayer_plane_map(L::LinearRaw, c::Integer) -> SimilarityMap{2, …, BayerCellFrame{C}, PixelFrame{:colmap}}The Bayer transform of plane c of the image: the map of the plane's own frame (BayerCellFrame{C}, C its colour — :R, :G, :B for :bin2, :G for :green_cells) onto the visible raw's PixelFrame{:colmap}. It is the plane's translation onto the cell grid (where its samples sit from the cell centre: (±¼, ±¼) cell pixels for R and B, none for G) composed with raw_frame_map(L), so pixel (row, column) of the plane, at (column − ½, row − ½) of its frame, goes onto the centre of the photosites its value is the mean of — R's or B's one, the two greens' midpoint — exactly, for every Bayer phase and either parity of the visible origin. The green plane's map IS raw_frame_map(L), and the R and B planes' maps are the R and B channels' of the mosaic's channelview (bayer_plane_map(m, k)): one rule.
A warp reads plane c of an image through inv(bayer_plane_map(L, c)) ∘ math, math landing in the raw's pixels, like any other map (image_warp; one warp per plane: image_warp!(dst, warps, L.image) with a tuple of them). A composition that joins two planes' frames is refused (MethodError). A plane past the image's, or a :demosaic image, is an ArgumentError.
VisualGeometryToolkitCore.green_cell_sites — Method
green_cell_sites(L::LinearRaw, row::Integer, column::Integer) -> NTuple{2, SVector{2, Float64}}The footprint of the green-cell value of pixel (row, column): the centres of its two green photosites in the visible raw's COLMAP frame, G1 then G2 (the cell's column-major order). They are diagonal, at (±½, ∓½) raw pixels from the cell centre raw_frame_map(L)((column − ½, row − ½)) in an RGGB or BGGR cell and (±½, ±½) in a GRBG or GBRG one, and the value is the mean of the two — what a renderer models to predict it exactly. Each is the green channel's bayer_plane_map of the mosaic at that pixel. A :demosaic image is refused (ArgumentError).
VisualGeometryToolkitCore.green_plane — Method
green_plane(L::LinearRaw) -> AbstractMatrix (of the image's element type)The image's green cells, a view: each cell's two greens' mean — L.image's plane whose frame IS the cell grid (raw_frame_map(L)): a :green_cells image's one plane, a :bin2 image's second (G). The plane every green-cell consumer reads (a detector, VisualGeometryToolkitFeatures' variance_stabilize); its photosites' colours and spans are L.geometry.plane_sites of that plane. An uncertain image's green plane is uncertain too (a StructArray view: values and variances). A :demosaic image is refused (ArgumentError).
VisualGeometryToolkitCore.poisson_slope — Method
poisson_slope(L::LinearRaw, model::SensorModel) -> NTuple{P, Float64}Per plane of L (L.image[:, :, p]), the Poisson slope α of a value's variance under model: the α of α·x + σ², gain/(m²K)·Σ 1/d over the plane's m photosites (L.geometry.plane_sites[p]), 0 without a Poisson term (a model whose noise is not PoissonGaussian). The one expression the kernel writes the variance plane with; the generalized Anscombe transform reads it beside the variance plane (VisualGeometryToolkitFeatures' variance_stabilize). A :demosaic image is refused (ArgumentError).
poisson_slope(L::LinearRaw, model::SensorModel, plane) -> Float64The slope of one plane: plane its index p, or its colour, :R, :G or :B (:G the green plane, green_plane(L)'s; a :green_cells image has :G alone). A colour the image does not hold is an ArgumentError.
VisualGeometryToolkitCore.raw_coordinates — Method
raw_coordinates(img, x) -> SVector{2, Float64}Where the point x of an image's pixel frame (COLMAP, (column − ½, row − ½) the centre of pixel (row, column); any 2-vector or tuple) lies on the sensor: in the visible raw's PixelFrame{:colmap}, the frame of every calibration of the raw. THE accessor of an image's sensor-pixel coordinates, raw_frame_map(img)(x):
- a
RawMosaic: the identity as decoded, a window's offset added for a window; - a
LinearRawcell image (:green_cells,:bin2): the cell grid's map,x ↦ 2x(and a window's offset) — a detection on the green cells reports its position through it, never as raw pixels unmapped (half the scale); a position on another plane of a:bin2image isbayer_plane_map(L, c)(x), and on a channel of a mosaic'schannelview(m)it isbayer_plane_map(m, k)(x); - a
:demosaicimage is refused (ArgumentError), and a plain matrix, which states no frame, has no method.
VisualGeometryToolkitCore.raw_frame_map — Method
raw_frame_map(L::LinearRaw) -> SimilarityMap{2, …, BayerCellFrame{:G}, PixelFrame{:colmap}}THE map of the image's pixel frame (the cell grid, which is its green plane's frame) onto the visible raw's PixelFrame{:colmap} (the frame of RawMosaic.raw and of every calibration of the raw): x ↦ 2x, exactly, for every Bayer phase and either parity of the visible origin (the cells start at the visible area's first photosite; an odd margin changes only the cell's colour phase); for an image made from a window of the mosaic, x ↦ 2x then the window's offset. Its inverse is inv(raw_frame_map(L)), and a calibration matrix moves with it through Core, reframed_calibration(raw_frame_map(L), K_cells), K_cells in the cell image's pixels (BayerCellFrame{:G}), giving K in the raw's pixels (and back with the inverse; K carries no frame label, so which frame it is in is the caller's statement). A :demosaic image is refused (ArgumentError): it is for display.
VisualGeometryToolkitCore.raw_frame_map — Method
raw_frame_map(::Type{LinearRaw{mode}}) -> SimilarityMap{2, …, BayerCellFrame{:G}, PixelFrame{:colmap}}The frame map of a mode image made from a whole mosaic (no window), without an image: x ↦ 2x, exactly, for :green_cells and :bin2 alike (the cell grid; a map that depends on the mode alone, not on the photograph, its Bayer phase or its visible origin). THE source of the cell images' map: every LinearRaw is made with this map after its mosaic's own (raw_frame_map(mosaic), a window's offset), so raw_frame_map(LinearRaw(m; mode)) == raw_frame_map(m) ∘ raw_frame_map(LinearRaw{mode}) bit for bit, and for an unwindowed mosaic the two are equal. For a frame with no RAW behind it (a decoded image on the cell grid) whose map is needed, e.g. scale(raw_frame_map(LinearRaw{:green_cells})) == 2. LinearRaw{:demosaic} is refused (ArgumentError), as its images are.
VisualGeometryToolkitCore.raw_provenance — Method
raw_provenance(L::LinearRaw) -> NamedTupleHow the image was made, as plain data a result records: decoder, libraw_version (from the mosaic's metadata; nothing where it states none), mode, gain and frame_map ((; from, to, scale, translation) of raw_frame_map(L), nothing for :demosaic). Its mode is the mode's own record's (raw_provenance(LinearRaw{mode})), and both records' frame_map are written by one rule from the map, so an image of a whole mosaic records the mode's frame_map; a window's records its offset in translation.
VisualGeometryToolkitCore.raw_provenance — Method
raw_provenance(::Type{LinearRaw{mode}}) -> (; mode, frame_map)The part of raw_provenance(L) that the mode alone determines, without an image: mode ("green_cells", "bin2", "demosaic") and frame_map, the record of raw_frame_map(LinearRaw{mode}) ((; from, to, scale, translation); nothing for :demosaic, which holds no geometry). An image made from a whole mosaic (no window) records the same mode and frame_map, == field for field, whatever its Bayer phase or visible origin. For a result whose frames have no RAW behind them that records the frame they are on.
The sensor model: the black level, the noise and the defects
Three levels: the geometric Sensor (part of a camera), a SensorCharacterisation (ONE body's components per ISO, held beside the camera like CaptureSettings) and a SensorModel (ONE frame's black, noise and defect components). Each slot's component is a type, absence included (FileBlack, NoNoise, NoDefects), and each present one states its source (SensorSource: measured, estimated, specified, file). The resolver sensor_model is a lookup: the frame's body and ISO pick an entry, and with none the absent model SensorModel() linearises bit for bit as the RAW alone. LinearRaw(mosaic; mode, sensor_model) is THE one black subtraction; under a PoissonGaussian or RelativeNoise its image carries each value's variance. The estimators build a characterisation's components from frames, offline: photon_transfer, master_dark, defect_onsets.
VisualGeometryToolkitCore.DarkOutOfBand — Constant
DarkOutOfBand: a ModelReason — this body at this ISO, but the frame's exposure is outside the Dark's band (or every band of its Darks): its ChannelBlack, the entry's other components.
VisualGeometryToolkitCore.Matched — Constant
Matched: a ModelReason — an entry of this body at the frame's ISO gave the model.
VisualGeometryToolkitCore.NoCharacterisation — Constant
NoCharacterisation: a ModelReason — no characterisation was given; the absent model.
VisualGeometryToolkitCore.OtherBody — Constant
OtherBody: a ModelReason — the characterisations given are of other bodies; the absent model.
VisualGeometryToolkitCore.UnmatchedISO — Constant
UnmatchedISO: a ModelReason — this body is characterised, but not at the frame's ISO; the absent model.
VisualGeometryToolkitCore.BlackComponent — Type
BlackComponent, NoiseComponent, DefectComponentThe three slots of a SensorModel and of a SensorCharacterisation entry: what a code's black level is (FileBlack, ChannelBlack, Dark), its noise (NoNoise, ReadNoise, PoissonGaussian, RelativeNoise) and which photosites are defects (NoDefects, DefectList).
VisualGeometryToolkitCore.ChannelBlack — Type
ChannelBlack(values, std_error = nothing; source::SensorSource.T)A per-channel black level: values per CFA colour index (LibRaw's 0 R, 1 G, 2 B, 3 the second G: values[c + 1], RawMosaic.black's layout), in codes, finite; std_error its standard error per channel (nonnegative), or nothing where none is stated (a specified value). source is measured, estimated or specified (the RAW's own is FileBlack). Every value is below the white level where it is used (ArgumentError there).
VisualGeometryToolkitCore.Dark — Type
Dark(black, offset, variance, exposure_band, frames; source::SensorSource.T)A master dark of one body at one ISO, for one exposure-time band — the black slot's per-photosite form (D2: all dark information in ONE component):
black— its per-channel black (ChannelBlackorFileBlack): the spanwhite − black[c]of every photosite of the channel, and the black a frame whose exposure the band does not serve is linearised with;offset— per photosite of the visible raw (raw[row, column], the size of the decoded mosaic), the mean code of the lens-capped frames (the black, the DSNU and the dark current together):LinearRaw's kernel subtracts it in place of the channel's black;variance— its temporal variance (codes²: the read noise with the rounding, and the dark current's shot noise), the Gaussian part of aPoissonGaussianmodel's variance at that photosite;exposure_band = (shortest, longest), time quantities, the exposures it serves (stored in seconds);frames, the number of frames averaged (at least 2).
Made by master_dark (SensorSource.measured). Moved to a device by Adapt.adapt, as LinearRawView is.
VisualGeometryToolkitCore.Darks — Type
Darks(black, darks; source::SensorSource.T)The master darks of one body at one ISO, one Dark per exposure band — the black slot's form for darks taken at several exposures (Core 0.10.3):
black— the shared per-channel black (ChannelBlackorFileBlack), the black a frame whose exposure NO band serves is linearised with;darks— the Darks, any order, stored in increasing band; their bands are DISJOINT (two bands sharing an exposure, an endpoint included, are anArgumentError: which would serve it?), their maps of one size, at least one, each ofsource.
sensor_model resolves a frame to the ONE Dark whose band serves its exposure (Matched: the model is the one that Dark alone gives), else to black with the reason DarkOutOfBand. The selection is inside the one component: the slot's precedence of sources is unchanged. A single band is a Darks of one; a bare Dark stays a Dark. Each Dark keeps its own channel black (its span when it serves a frame: the black measured at its exposure); black is the fallback alone. A Darks is a characterisation's, never a frame's: a SensorModel refuses one (resolve it with sensor_model), and the model sensor_model returns, holding one Dark, is what moves to a device (Adapt.adapt of a Darks moves every band's maps).
VisualGeometryToolkitCore.DefectComponent — Type
BlackComponent, NoiseComponent, DefectComponentThe three slots of a SensorModel and of a SensorCharacterisation entry: what a code's black level is (FileBlack, ChannelBlack, Dark), its noise (NoNoise, ReadNoise, PoissonGaussian, RelativeNoise) and which photosites are defects (NoDefects, DefectList).
VisualGeometryToolkitCore.DefectList — Type
DefectList(x, y, onset; size, source::SensorSource.T)The defective photosites of a sensor: photosite (x[i], y[i]) — x the column and y the row of the visible raw, 1-based (raw[y, x]) — is a defect at every exposure of at least onset[i] (a time quantity, stored in seconds: the shortest exposure at which it is hot). size = (rows, columns) is the visible raw's (every site inside it, each listed once). Made from per-exposure masks by defect_onsets. sensor_model keeps the sites whose onset is at most the frame's exposure, and LinearRaw masks them (masked, never interpolated). Moved to a device by Adapt.adapt.
VisualGeometryToolkitCore.FileBlack — Type
FileBlack()The black slot's absent default: the RAW file's own per-channel black level (RawMosaic.black, LibRaw's), today's behaviour bit for bit. Its source is SensorSource.file.
VisualGeometryToolkitCore.ModelReason — Type
ModelReasonWhy sensor_model gave a frame the model it did (an isbits @enum, SensorModel.reason):
Matched: a characterisation of this body holds an entry at the frame's ISO; its components are the model's (an absent slot of the entry is the absent type);NoCharacterisation: none was given (also the absent defaultSensorModel());UnmatchedISO: characterisations of this body exist, none at this ISO (no extrapolation across ISO);OtherBody: the characterisations given are of other bodies;DarkOutOfBand(Core 0.10.2): the body and the ISO matched, but the frame's exposure is outside the entry'sDark's exposure band, so the model's black is the Dark'sChannelBlack(the entry's other components kept): the fallback is visible, never silent. OfDarks(0.10.3): no band serves the exposure, and the model's black is their shared channel black.
NoCharacterisation, UnmatchedISO and OtherBody give the absent model; Matched and DarkOutOfBand the entry's components. A caller that matches the values exhaustively handles DarkOutOfBand too.
VisualGeometryToolkitCore.NoDefects — Type
NoDefects()The defect slot's absent default: no photosite is masked beyond the saturated ones.
VisualGeometryToolkitCore.NoNoise — Type
NoNoise()The noise slot's absent default: no noise is stated. A LinearRaw image is plain, and each detector applies its own documented policy.
VisualGeometryToolkitCore.NoiseComponent — Type
BlackComponent, NoiseComponent, DefectComponentThe three slots of a SensorModel and of a SensorCharacterisation entry: what a code's black level is (FileBlack, ChannelBlack, Dark), its noise (NoNoise, ReadNoise, PoissonGaussian, RelativeNoise) and which photosites are defects (NoDefects, DefectList).
VisualGeometryToolkitCore.PoissonGaussian — Type
PoissonGaussian(gain, σ, std_error = nothing; source::SensorSource.T)The Poisson–Gaussian noise: gain K, electrons per code (positive, finite), and the read noise σ per CFA colour index, in codes, the rounding included; a photosite's variance about its black-subtracted mean μ (codes) is μ/K + σ[c+1]². std_error is (; gain, σ) (a number and a 4-tuple) or nothing. source is measured, estimated (photon_transfer) or specified. Under it a LinearRaw image carries each value's variance (an Uncertain image).
VisualGeometryToolkitCore.ReadNoise — Type
ReadNoise(σ, std_error = nothing; source::SensorSource.T)The read noise alone: σ per CFA colour index (σ[c + 1]), in codes, the rounding included (nonnegative, finite), and its standard error per channel or nothing. Stored and recorded, but it defines no pixel variance: a read-noise variance alone badly underestimates the noise of a well-exposed pixel, so a LinearRaw image under it is plain, as under NoNoise (ruling A).
VisualGeometryToolkitCore.RelativeNoise — Type
RelativeNoise(σ_rel; source::SensorSource.T)A noise proportional to the value: a value x has variance (σ_rel·x)² (σ_rel positive, finite). Its source is SensorSource.specified only, stated by a person (owner ruling on D14): no estimator or measurement makes one, so any other source is an ArgumentError naming it — the one rule, at the source (an IO reader of a stored one meets it here too). The model of a processed (tone-mapped) intensity, not of the photosites' physics; a LinearRaw image under it carries each value's variance (an Uncertain image), with no read-noise or dark term.
VisualGeometryToolkitCore.SensorCharacterisation — Type
SensorCharacterisation(body::DeviceIdentity; entries)The description of ONE physical sensor, host-side: body, the key (serial if known, else make and model), held beside the camera like CaptureSettings, never in a camera or lens model; and entries, per ISO its components — an iterable of iso => (; black, noise, defects), each slot optional (an absent slot IS the absent type: FileBlack, NoNoise, NoDefects), so a partial characterisation is a mix of components. c.entries is a Dict{Float64, SensorEntry} (ISO → the three slots). An ISO listed twice, a slot not among black, noise and defects (a dark map is the black slot's Dark), or a non-positive ISO is an ArgumentError.
sensor_model resolves a frame's model from it. Written and read by VisualGeometryToolkitIO's vgio/sensor_characterisation kind.
VisualGeometryToolkitCore.SensorEntry — Type
SensorEntry = @NamedTuple{black::BlackComponent, noise::NoiseComponent, defects::DefectComponent}One ISO's entry of a SensorCharacterisation: its three slots, each a component (an absent slot its absent type).
VisualGeometryToolkitCore.SensorModel — Type
SensorModel(black::BlackComponent, noise::NoiseComponent, defects::DefectComponent;
reason::ModelReason = Matched, specified_override::Bool = false)
SensorModel()The sensor model of ONE frame: the black, noise and defects components sensor_model picked for its ISO and exposure (a Dark only when its band serves the exposure, a DefectList holding only the sites whose onset is at most the exposure), reason why (a ModelReason) and specified_override, whether a specified component displaced a measured one (the run record notes the override).
SensorModel() is the absent model, SensorModel(FileBlack(), NoNoise(), NoDefects(); reason = NoCharacterisation): LinearRaw and linearview under it are bit for bit the RAW's own linearisation. A model of isbits components is isbits; one holding a dark map or a defect list moves to a device with Adapt.adapt. content_digest is its identity.
VisualGeometryToolkitCore.component_sources — Method
component_sources(m::SensorModel) -> NamedTuple{(:black, :noise, :defects, :reason, :specified_override)}Where each of a frame's sensor components came from — the provenance a run record states (ruling A): black, noise and defects each the component's SensorSource.T (a FileBlack's is file), or nothing for an absent slot (NoNoise, NoDefects); reason why the frame got this model (a ModelReason: Matched, NoCharacterisation, UnmatchedISO, OtherBody, or DarkOutOfBand when the entry's Dark did not serve the frame's exposure (or no Dark of its Darks did) and the ChannelBlack stands in); specified_override whether a specified component won over a measured one. The one public view of a model's provenance: a consumer reads it, never the components' fields.
VisualGeometryToolkitCore.content_digest — Method
content_digest(model::SensorModel) -> StringThe identity of a frame's sensor model: the SHA-256 (lower-case hex) of the canonical JSON (canonical_json, the stack's one content-identity encoding) of a tree of its components — each component's type, values, standard errors and source, a map (a Dark's offset and variance) by the SHA-256 of its Float32 values' bytes and its size, a DefectList by its sites — its reason and specified_override. Equal models have equal digests, on any device (a device map is read to the host); a cache keyed by a model (VisualGeometryToolkitFeatures' decode cache) keys on it.
VisualGeometryToolkitCore.defect_mask — Method
defect_mask(model::SensorModel, mosaic::RawMosaic; backend = the mosaic's) -> AbstractMatrix{Bool} or nothing
defect_mask(defects::DefectComponent, mosaic::RawMosaic; backend = the mosaic's)The defect slot as a mask over the VISIBLE raw (raw[y, x], the DefectList's size), on backend: true at each site of the list (sensor_model has cut a model's to the frame's exposure), the mask LinearRaw joins its saturation mask with. nothing for NoDefects: no photosite is masked. Indexed by the visible raw, not the mosaic's window: a window's photosite (i, j) is mask[i + o[1], j + o[2]], o = mosaic.window_origin. A list that does not cover the mosaic's photosites is a DimensionMismatch.
VisualGeometryToolkitCore.sensor_model — Method
sensor_model(characterisation, capture::CaptureSettings, t::Integer) -> SensorModelTHE resolver of frame t's SensorModel: a LOOKUP (owner ruling D1; it never estimates — photon_transfer makes an estimated entry offline). characterisation is a SensorCharacterisation, a vector or tuple of them, or nothing; the frame's body is capture.body, its ISO and exposure capture[t].exposure.
- The characterisations of this body (serial if both state one, else make and model;
DeviceIdentity), and their entries at exactly the frame's ISO (no extrapolation across ISO). - Per slot, independently, the component those entries hold; where several hold one, the specified one over the measured over the estimated (D5), and
specified_overriderecords a specified one displacing a measured one; two of the same source are anArgumentError, as are two different bodies matching a capture that states no serial. - A
Darkonly when its exposure band serves the frame's exposure (otherwise its channel black, and the reasonDarkOutOfBand); ofDarks, the one Dark whose band serves it (otherwise their shared channel black, andDarkOutOfBand); aDefectListcut to the sites whose onset is at most the exposure (none left:NoDefects).
With no characterisation (NoCharacterisation), none of this body (OtherBody) or none at this ISO (UnmatchedISO) the result is the absent model SensorModel(FileBlack(), NoNoise(), NoDefects()) with that ModelReason: today's linearisation, bit for bit. Otherwise its reason is Matched, or DarkOutOfBand when the entry's Dark does not serve the frame's exposure (or no Dark of its Darks does). A frame the capture does not hold is an ArgumentError. An explicit model a caller holds is used as it is: this resolver is where a characterisation becomes one, never a substitute for one given.
VisualGeometryToolkitCore.value_variance — Method
value_variance(v::LinearRawView, i::Integer, j::Integer, x::Real) -> Float64The variance the view's noise law gives a linear value x at photosite (i, j) (gain 1, the law linearview's element there carries at its own value, the store's rounding aside): for a CENSORED photosite — saturated, its value clipped at the white level — the variance of that clipped value, value_variance(v, i, j, 1.0), never 0 under a law. 0 for a view without a variance law (no noise, or read noise alone: a plain view). Kernel-lawful.
VisualGeometryToolkitCore.NoiseEstimateFailed — Type
NoiseEstimateFailed(reason)The exception photon_transfer and master_dark throw when the data cannot give the estimate (too few usable samples, a degenerate fit), reason a sentence. The resolver sensor_model never runs an estimator (a lookup): a characterisation without the estimate lacks the component, and a frame resolves to its absent type.
VisualGeometryToolkitCore.PhotonTransferMetadata — Type
PhotonTransferMetadataThe FitResult metadata of photon_transfer(groups, darks) (its estimate a PoissonGaussian), read through the result's forwarded properties:
level_range::NTuple{2, Float64}— the fit's lowest and highest point (block means, codes above black);level_fractionthe same as fractions of the white level above black (the greens' span);intercept::Float64— σ₀² (codes²), the variance at zero signal, fixed from the darks at each group's own exposure (the median over the fitted points);curvature,curvature_t— the μ² coefficientcofv = σ₀² + μ/K + cμ²over the range and itst(cover its standard error);tests,test_level— the μ² tests the lower end's search can perform (m, the candidate lower ends) and each test's two-sided level,significance / m(Bonferroni: the search's family-wise rate issignificance);rollover_tests,rollover_levelthe same for the upper end's rollover tests (one-sided);birge_ratio— R = √(χ²/dof) of the fitted points about the line (each in its median's standard deviation from the counts; dof the points less the one slope): the scatter the counts explain is R ≤ 1; when R > 1, K's standard error is multiplied by R (the Birge ratio: never shrunk);points_offered— the points before the range search (the result'snis the points fitted).
VisualGeometryToolkitCore.defect_onsets — Method
defect_onsets(masks; source::SensorSource.T = SensorSource.measured) -> DefectListONE DefectList from per-exposure defect masks: masks is a vector of exposure => mask (a time quantity and a Bool matrix over the visible raw, master_dark's defects with the shortest exposure of its band, say), all masks of one size. Each photosite flagged in any mask is listed once, its onset the shortest exposure whose mask flags it (the resolver masks it at every exposure from there on: a site flagged at a short exposure and missed at a longer one is a defect at both). The sites are in column-major order of the visible raw. An empty vector or masks of different sizes are an ArgumentError.
VisualGeometryToolkitCore.master_dark — Method
master_dark(frames::AbstractVector{<:RawMosaic}; exposure_band, false_defects = 0.01)
-> (; dark::Dark, read_noise::ReadNoise, defects::BitMatrix)THE dark-frame estimator: the lens-capped frames of one body at one ISO and one exposure band (whole mosaics of one size, at least 2) into the components of a characterisation's entry, all SensorSource.measured:
dark— theDark: per photosite the temporal mean (the offset: black, DSNU and dark current) and sample variance, one KernelAbstractions work item each, on the frames' device; its channel black (dark.black) aChannelBlack, per colour index (LibRaw's0–3) the mean offset of its photosites (defects left out), with its standard error from their spread and count;read_noise— aReadNoise: per colour index, the root mean temporal variance (the read noise with the rounding, and at a long exposure the dark current's shot noise), its standard error from theχ²spread of the frame count and the photosite count (√(2/(n − 1)/N)·σ²for the mean variance, by first order);defects— aBitMatrix, the photosites whose offset leaves their channel's median by more thanzrobust standard deviations (the median absolute deviation over the normal's,mad_scale's rule for codimension 1, about the channel's median),zthe normal quantile at whichfalse_defectsdefect-free photosites are expected to be flagged over the whole sensor ofNphotosites (1 − false_defects / (2N), two-sided, for offsets that are Gaussian about their channel's black). The masks of several exposures make oneDefectListbydefect_onsets.
A colour index the pattern lacks (a pattern with one green index) has black and read noise 0 (no photosite reads it).
exposure_band is (shortest, longest), time quantities, the exposures the dark serves. A frame of another size, or fewer than 2, is an ArgumentError; a channel without a usable photosite throws NoiseEstimateFailed.
VisualGeometryToolkitCore.photon_transfer — Method
photon_transfer(groups::AbstractDict{<:Quantity, <:AbstractVector{<:RawMosaic}}, darks::Darks;
defects::DefectComponent = NoDefects(), block = 8, normalize = true, significance = 0.001)
-> FitResult{PoissonGaussian, PhotonTransferMetadata}
photon_transfer(groups::AbstractDict{<:Quantity, <:AbstractVector{<:RawMosaic}},
darks::AbstractDict{<:Quantity, <:ReadNoise}; sensor_model::SensorModel = SensorModel(),
block = 8, normalize = true, significance = 0.001) -> FitResult{PoissonGaussian, PhotonTransferMetadata}
photon_transfer(mosaic::RawMosaic; sensor_model::SensorModel = SensorModel(), block = 8, significance = 0.001)
-> PoissonGaussianTHE estimator of a sensor's Poisson–Gaussian noise for a characterisation's entry, built offline (owner ruling D1: the resolver sensor_model never estimates): the conversion gain K (electrons per code), by a photon-transfer fit of variance against mean, v = σ₀² + μ/K.
The photon-transfer series (groups, exposure => the frames at that exposure, at least 2 each, of one body at one ISO), a MEASURED PoissonGaussian (SensorSource.measured, a dedicated capture) in a FitResult (its estimate), its metadata a PhotonTransferMetadata: per green photosite its TEMPORAL variance over the group's frames, which cancels the scene and every fixed pattern, about its mean μ. Pairs are the case of 2 frames. With normalize (the default) each frame is first scaled to its group's mean (its greens' black-subtracted mean over the group's: a source that flickers between frames adds no μ² term). Per block, the photosites' means and their pooled variance give a point, v̂/v following χ²_{p(N−1)}/(p(N − 1)) over a whole block of p photosites and N frames; every group gives its first N frames, N the smallest group's, so every block follows that one law. The darks come in two forms:
a
Darks(Core 0.10.4;master_dark's Darks, one per exposure band): each group is served by THE Dark whose band holds its exposure (the Darks' own rule, assensor_modelresolves a frame: an exposure of any number type,10 * msand0.01 salike). Each group is linearised with its own Dark — its offset map per photosite and its channel black — and its interceptσ₀²(t)is that Dark's mean green variance over the photositesdefectsleaves (cut to the group's exposure: a site is masked from its onset); σr's standard error is carried from the Dark's frame and photosite counts,√(2/((n − 1)N))·σ₀²(masterdark's rule). A group whose exposure no band serves is anArgumentError: never the shared black, never another band's Dark;a
DictofReadNoise(exposure=>masterdark's ReadNoise over the darks of that exposure; scheduled for removal in the next breaking wave, use theDarksform): every group linearised with `sensormodel's black and defects (ONE model for every exposure), its intercept the greens'σ[c + 1]²ofdarks[t]. A group whose exposure has no entry indarks(EQUAL in seconds:1//100 sis not theFloat640.01 s) is anArgumentError: never interpolated, never substituted. σ_r's standard error is the ReadNoise's (0` when it states none).THE INTERCEPT is fixed at each group's own exposure t; the fit is the slope alone,
v − σ₀²(t) = μ/K.THE RANGE is derived from the data, at least 3 points: the upper end below the variance's rollover (the top point dropped while it lies significantly below the line fitted to the points under it, one-sided), then the lower end, the lowest point from which the μ² test on the line succeeds (the lowest point dropped until it does). Each search runs its tests at
significance / m,mthe tests it can perform (Bonferroni: the search's family-wise false-alarm rate issignificance); the range (in codes and as fractions of the white level),mand the levels are in the result's metadata.THE STANDARD ERROR of K is the fitted line's from the counts, multiplied by the Birge ratio R = √(χ²/dof) of the fitted points when R > 1 (the points scatter more than the counts explain); when R ≤ 1 it is unchanged, never shrunk. R is in the result's metadata.
The result's σr is
√σ₀²of the fitted points' darks (their median), its standard error the darks' at that median (the mean of the two middle points' when it falls between two exposures), by first order; a characterisation states σr per colour frommaster_dark'sReadNoise.The result's return code and stop rule are the weighted line's:
SuccessandIncrementSizewhen its weights reached their fixed point (iterationsthe reweightings),MaxItersandIterationCapwhen they did not.
One frame (mosaic), in the line of Foi et al. (2008), an ESTIMATED PoissonGaussian (SensorSource.estimated): unreliable on flats, scheduled for removal in the next breaking wave; use the multi-frame method. A single flat frame cannot certify its own K (its brightness range gives the μ² test little power); for flat fields use the multi-frame photon-transfer method. Each 2×2 cell's two greens see the same light, so their difference d carries no scene where the scene is flat across the cell, and Var d = 2(μ/K + σ_r²) about their black-subtracted mean μ. Per block, the cells' mean and the difference's sample variance give a point (μ̂, v̂ = s²/2), v̂/v following χ²_{n−1}/(n − 1) over a whole block of n usable samples (cells); the line's intercept σr² is fitted, over every point, with its standard error (σr stated for every colour index, the greens'). A fixed pattern between the two greens (their gain or response mismatch) does NOT cancel in d: it adds a term in μ², which the μ² test REFUSES; a frame that gives fewer than 4 points, on which the μ² test cannot run, is refused too (its K is never returned untested).
Both: per block of block × block cells (one KernelAbstractions work item each, on the frames' device), a sample with a photosite at the white level, or under the model's defect mask, is left out (CLIPPED, never modelled), and only whole blocks (the most usable samples any block has) enter the fit. The blocks are sorted by μ̂ into points of equal count (min(32, blocks ÷ 8); the series': per exposure, so a point never mixes two exposures' intercepts, an exposure of fewer than 8 whole blocks left out); each point's median μ̂ and median v̂ / q (q that law's median, so the ratio is median-unbiased for v). The line is fitted by weighted least squares, each point weighted by the inverse of its median's variance from the counts, v²/(4 m f(q)² q²) over a point of m blocks (f the law's density: the sample median's asymptotic variance), with v the line's own prediction (iterated to a fixed point from a start weighted at each point's own value, at most 100 reweightings). The median is robust to the blocks a scene edge raises. The standard errors are the fitted line's covariance from those counts; K's by first order.
THE μ² TEST: the points fitted again with a μ² term, v = … + μ/K + cμ², at the same weights, and c significant when |ĉ / se(ĉ)| exceeds the two-sided normal quantile at the test's level (one frame: significance, 3.29 at the default 0.001, over every point; the series: the lower end's search). It never returns a K biased by a μ² term it detects. The default significance is configs/default.toml's [photon_transfer] significance.
sensor_model gives what is subtracted and left out, as it does for LinearRaw: its black slot's per-colour black (a Dark's offset per photosite in its place) and its defect slot's photosites; its noise slot is not read. Throws NoiseEstimateFailed with the reason when fewer than 16 whole blocks of at least 8 usable samples remain (a saturated frame), the points hold one level only (no slope), one frame gives fewer than 4 points, one frame's μ² term is significant, one frame's weighted line does not converge, the series' μ² test fails on every contiguous range of at least 3 points, or the fit is not physical (K, or one frame's σ_r², not positive). A group of fewer than 2 frames, frames of another size, pattern or white level, or a group's exposure without darks, is an ArgumentError; dark maps that do not cover the frames a DimensionMismatch.
VisualGeometryToolkitCore.SensorSource — Module
SensorSourceWhere a sensor component's values came from, SensorSource.T (an isbits @enum): SensorSource.measured (a record of dark frames, flats or a photon-transfer capture), SensorSource.estimated (fitted by an estimator, photon_transfer, with its standard error), SensorSource.specified (stated by a person: a datasheet, a config, TOML literals) or SensorSource.file (the RAW file's own metadata: FileBlack's). string(s) is its tag ("measured", …). Each present component holds one in its source field. A specified component wins over a measured one at the same slot (sensor_model).
VisualGeometryToolkitCore.SensorSource.T — Type
SensorSource.T: the SensorSource enum type, a sensor component's source field (an isbits @enum).
VisualGeometryToolkitCore.SensorSource.measured — Constant
SensorSource.measured: a SensorSource — the values were measured (dark frames, flats, a photon-transfer capture).
VisualGeometryToolkitCore.SensorSource.estimated — Constant
SensorSource.estimated: a SensorSource — the values were fitted by an estimator, with their standard errors.
VisualGeometryToolkitCore.SensorSource.specified — Constant
SensorSource.specified: a SensorSource — the values were stated by a person (a datasheet, a config, TOML literals).
VisualGeometryToolkitCore.SensorSource.file — Constant
SensorSource.file: a SensorSource — the values are the RAW file's own metadata (a FileBlack's).