pyarts3.retrieval

For examples and interpretation, see Correlating temperature and log-water on the same grid. The mathematical definitions are in Information carried by the measurements.

Covariance checks and local information analysis before or after OEM.

The information spectrum follows Nesser et al. (2021), Sect. 2.2, https://doi.org/10.5194/amt-14-5521-2021. These functions neither execute an agenda nor change a workspace, its covariance matrices, or its settings.

class pyarts3.retrieval.InformationReport(singular_values: ndarray, mode_variance_reduction: ndarray, degrees_of_freedom: float, information_bits: float, prior_standard_deviation: ndarray, posterior_standard_deviation: ndarray, variance_reduction: ndarray, state_modes: ndarray, measurement_modes: ndarray, state_labels: tuple[str, ...], prior_correlation_condition: float, measurement_correlation_condition: float, innovation_chi_square: float | None, innovation_expected_mean: int | None, _noise: _CovarianceFactor, state_blocks: tuple[tuple[str, int, int], ...] = ())[source]

Snapshot of information implied by a Jacobian and assumed covariances.

Reports compare and hash by object identity; compare their arrays explicitly when checking numerical agreement. All arrays are independent, read-only snapshots. singular_values has one entry per state mode, including unobserved null modes. Columns of state_modes are perturbations in the supplied state coordinates: state_modes @ state_modes.T equals the prior covariance. Mode signs and bases within repeated singular values are not unique.

measurement_modes contains the left singular vectors in whitened measurement coordinates, not physical measurement units. Its number of columns is the smaller of the measurement and state dimensions.

Standard deviations and marginal variance_reduction refer to the original state coordinates. mode_variance_reduction refers to the independent prior-normalized modes. Correlated priors can improve a parameter indirectly through information about other parameters.

state_labels identify individual state coordinates. Workspace reports also include state_blocks tuples of (field name, x_start, x_size). These indices describe flattened retrieval coordinates, not physical grids.

Innovation statistics are present only when both measurements and an explicit prior prediction were supplied. Their interpretation assumes independent Gaussian prior and observation errors and a valid linear approximation around the prior.

__init__(singular_values: ndarray, mode_variance_reduction: ndarray, degrees_of_freedom: float, information_bits: float, prior_standard_deviation: ndarray, posterior_standard_deviation: ndarray, variance_reduction: ndarray, state_modes: ndarray, measurement_modes: ndarray, state_labels: tuple[str, ...], prior_correlation_condition: float, measurement_correlation_condition: float, innovation_chi_square: float | None, innovation_expected_mean: int | None, _noise: _CovarianceFactor, state_blocks: tuple[tuple[str, int, int], ...] = ()) → None

degrees_of_freedom: float

describe(max_states=10)[source]: Explain the report, showing at most max_states state entries.

information_bits: float

innovation_chi_square: float | None

innovation_expected_mean: int | None

measurement_correlation_condition: float

measurement_modes: ndarray

mode_variance_reduction: ndarray

plot()[source]

Return (figure, axes) with mode and state uncertainty diagnostics.

Both panels are dimensionless, so quantities with different physical units can be compared. This does not call matplotlib.pyplot.show.

posterior_standard_deviation: ndarray

prior_correlation_condition: float

prior_standard_deviation: ndarray

reduction(rank=None, *, max_lost_dofs=None, max_lost_information_bits=None)[source]

Select leading modes for ReducedOEM by rank or absolute loss limits.

Supply either an explicit integer rank in 1..n, or one or both finite, nonnegative loss limits. Limits select the smallest rank satisfying all supplied limits, retaining at least one column even for zero information. Limits are absolute DOFS/bits, not fractions. Nothing is rounded from the total DOFS; a broad weak spectrum can require many more modes than that total suggests. Zero limits discard only modes with zero information in the computed spectrum.

The returned matrices are B = L_a V_r and C = U_q.T L_e^-1, where q = min(rank, m). The reduced prior and noise covariances are identity. C contains weighted combinations, not a selection of physical channels.

singular_values: ndarray

state_blocks: tuple[tuple[str, int, int], ...] = ()

state_labels: tuple[str, ...]

state_modes: ndarray

variance_reduction: ndarray

class pyarts3.retrieval.ReductionReport(model_state_basis_mat: ndarray, measurement_basis_mat: ndarray, retained_degrees_of_freedom: float, discarded_degrees_of_freedom: float, retained_information_bits: float, discarded_information_bits: float, _state_modes: ndarray, _posterior_factors: ndarray)[source]

Fixed state/measurement reductions and their local linear information loss.

Assign both reduction matrices to ws.oem before calling ws.oemCalcReduced(). Losses describe the Jacobian used for the information report, not a bound on nonlinear retrieval error. The underlying state modes are shared with that read-only report. Reports compare and hash by object identity; compare their arrays explicitly when checking numerical agreement.

__init__(model_state_basis_mat: ndarray, measurement_basis_mat: ndarray, retained_degrees_of_freedom: float, discarded_degrees_of_freedom: float, retained_information_bits: float, discarded_information_bits: float, _state_modes: ndarray, _posterior_factors: ndarray) → None

discarded_degrees_of_freedom: float

discarded_information_bits: float

measurement_basis_mat: ndarray

model_state_basis_mat: ndarray

posterior_covariance()[source]

Return the local full-state covariance, retaining discarded priors.

This allocates an n by n matrix. It is exact for a linear Gaussian model when all nonzero modes are retained. Otherwise it is the posterior of the truncated linear model. It is not recomputed at the state returned by a subsequent ReducedOEM call.

property rank: Number of retained columns, and hence reduced state variables.

retained_degrees_of_freedom: float

retained_information_bits: float

pyarts3.retrieval.information(measurement_jac, model_state_covmat, measurement_vec_error_covmat, *, state_labels=None, measurement=None, prior_prediction=None, relative_tolerance=1e-10, max_dense_elements=10000000)[source]

Validate covariance inputs and calculate local retrieval information.

Parameters:

measurement_jac (array_like) – Jacobian with shape (m, n) in the actual retrieved coordinates and measurement units. It must describe the state at which this local analysis is intended.
model_state_covmat – Prior and observation-error covariances, each a CovarianceMatrix or array_like. Square arrays are full covariances; one-dimensional arrays contain independent variances, not standard deviations. Native blocks and stored inverses are checked.
measurement_vec_error_covmat – Prior and observation-error covariances, each a CovarianceMatrix or array_like. Square arrays are full covariances; one-dimensional arrays contain independent variances, not standard deviations. Native blocks and stored inverses are checked.
state_labels (sequence of str) – Optional labels, one per state coordinate in Jacobian-column order.
measurement (array_like) – Optional arrays with shape (m,). Supply both to compare measurements with an explicit \(F(\vec{x}_a)\). The Jacobian must then be appropriate around \(\vec{x}_a\). A spectrum fitted at another state is not a prior prediction.
prior_prediction (array_like) – Optional arrays with shape (m,). Supply both to compare measurements with an explicit \(F(\vec{x}_a)\). The Jacobian must then be appropriate around \(\vec{x}_a\). A spectrum fitted at another state is not a prior prediction.
relative_tolerance (float) – Relative, scaled tolerance for covariance symmetry/inverse checks.
max_dense_elements (int) – Limit on estimated dense analysis storage and on dense covariance factors. The initial implementation uses an exact SVD and returns all n state modes. LAPACK can require additional temporary workspace. Independent measurement variances do not allocate an m-by-m matrix.

Returns:

Dimensionless information spectrum, physical-coordinate uncertainty estimates, and optional prior innovation statistics. This function does not evaluate the forward model or modify any of its inputs.

Return type:

InformationReport

Notes

With covariance factors \(\mathbf{S}_a=\mathbf{L}_a\mathbf{L}_a^{\top}\) and \(\mathbf{S}_\epsilon=\mathbf{L}_\epsilon\mathbf{L}_\epsilon^{\top}\), the singular values of the whitened Jacobian

\[\widetilde{\mathbf{J}} = \mathbf{L}_\epsilon^{-1}\mathbf{J}\mathbf{L}_a\]

are the mode signal-to-noise ratios. The posterior mode variance fractions are \(1/(1+s_i^2)\). Values apply to the assumed linear Gaussian model; high information does not establish correct uncertainties. No explicit covariance or posterior-precision inverse is formed.

pyarts3.retrieval.information_from_workspace(ws, *, prior_prediction=None, **options)[source]

Analyze the Jacobian and covariances already present in a workspace.

No agenda is executed. Supply prior_prediction explicitly to enable innovation checking using ws.oem.measurement_vec. The caller must ensure that the Jacobian describes the intended state and coordinates. Field labels and state_blocks (name, start, size) are inferred from jac_targets. Offsets refer to the supplied Jacobian’s columns; field indices are flattened target indices, not altitude or physical units. Keys alone do not identify logarithmic or other coordinate transforms. Missing target metadata leaves generic x[i] labels. Explicit state_labels override the automatic per-coordinate labels. Remaining options are forwarded to information().

class pyarts3.retrieval._CovarianceFactor(components, size, standard_deviation, condition)[source]: Square-root factors on disjoint coordinate subsets, with diagonal fast paths.