OptimalEstimationMethod

class pyarts3.arts.OptimalEstimationMethod(*args, **kwargs)

Algorithm and linear solver used by optimal estimation.

Every option minimizes the same cost function, so the choice is about how the minimum is approached and not about what is being minimized:

\[\chi^2 = \chi^2_y + \chi^2_x,\]

\[\chi^2_y = \frac{1}{m}\left(\vec{y}-\vec{y}_f\right)^\top \mathbf{S}_\epsilon^{-1}\left(\vec{y}-\vec{y}_f\right), \qquad \chi^2_x = \frac{1}{m}\left(\vec{x}-\vec{x}_a\right)^\top \mathbf{S}_a^{-1}\left(\vec{x}-\vec{x}_a\right).\]

The a priori term belongs to the cost for every option, and no option changes the statistical meaning of the two covariance matrices. What an option selects is how the state is stepped and which linear system produces that step. Both are built from the half-gradient and the Gauss–Newton approximation of the half-Hessian of \(m\chi^2\):

\[\vec{g} = \mathbf{J}^\top\mathbf{S}_\epsilon^{-1} \left(F\left(\vec{x}\right)-\vec{y}\right) + \mathbf{S}_a^{-1}\left(\vec{x}-\vec{x}_a\right), \qquad \mathbf{H} = \mathbf{J}^\top\mathbf{S}_\epsilon^{-1}\mathbf{J} + \mathbf{S}_a^{-1}.\]

There are two ways to step. li and gn take the undamped Gauss–Newton step

\[\mathbf{H}\Delta\vec{x} = -\vec{g},\]

which li takes exactly once and gn repeats. lm instead takes the damped step of Levenberg–Marquardt damping

\[\left(\mathbf{H}+\gamma\mathbf{D}\right)\Delta\vec{x} = -\vec{g}, \qquad \mathbf{D} = \operatorname{diag} \left(\operatorname{diag}\left(\mathbf{S}_a^{-1}\right)\right),\]

where \(\gamma\) is raised on a rejected trial and lowered on an accepted one. At \(\gamma=0\) this is the Gauss–Newton step again.

There are also two spaces to solve in. The plain options solve the \(n \times n\) state-space system written above. The _m options solve the \(m \times m\) measurement-space system instead:

\[\mathbf{M} = \mathbf{J}\mathbf{S}_a\mathbf{J}^\top + \mathbf{S}_\epsilon, \qquad \vec{r} = \vec{y} - F\left(\vec{x}_i\right) + \mathbf{J}\left(\vec{x}_i-\vec{x}_a\right),\]

\[\mathbf{M}\vec{u} = \vec{r}, \qquad \vec{x}_{i+1} = \vec{x}_a + \mathbf{S}_a\mathbf{J}^\top\vec{u}.\]

The two systems have the same solution in exact arithmetic, so the choice is one of size and conditioning: the state-space system is the smaller one when there are more measurements than state elements, and the measurement-space system when there are fewer. Finally, the _cg options replace the direct solve of the selected system with a conjugate-gradient solve, which never forms the inverse and is worth its tolerance only when the direct solve is the bottleneck.

where:

\(\vec{y}\) is the measurement vector,
\(\vec{y}_f = F\left(\vec{x}\right)\) is the simulated measurement,
\(\vec{x}\) is the model state and \(\vec{x}_a\) its a priori,
\(F\) is the forward model, run through inversion_iterate_agenda,
\(\mathbf{J}\) is the Jacobian of \(F\) at the current state,
\(\mathbf{S}_\epsilon\) is the measurement error covariance,
\(\mathbf{S}_a\) is the a priori covariance of the model state,
\(m\) is the number of measurements, and
\(n\) is the number of retrieved state elements.

Four limits end a retrieval, and which of them can be reached depends on the option. An iteration budget stops the outer loop of the iterated options and reports IterationLimit; the linear options report it after their single step. A convergence limit compares the state-step measures of State-step convergence measures against stop_dx, which is the only way to report Converged. A conjugate-gradient budget applies to the _cg options only: an unconverged step is rejected, and li and gn then stop with LinearSolverLimit while lm first retries with more damping. A damping limit applies to lm alone, which reports DampingLimit when it runs out of damping without an acceptable step. A starting cost above max_start_cost skips the inversion for every option and reports StartCostLimit.

Note

lm has no measurement-space form, as the damping is applied to \(\mathbf{H}\) in state space.

ml and ml_cg are historical spellings of lm and lm_cg. They select neither maximum-likelihood estimation nor a retrieval without the a priori term.

Valid options:

"li" or "Linear": A single undamped step, solving the state-space system directly.
"li_m" or "LinearMeasurementSpace": A single undamped step, solving the measurement-space system directly.
"li_cg" or "LinearConjugateGradient": A single undamped step, solving the state-space system by conjugate gradient.
"li_cg_m" or "LinearConjugateGradientMeasurementSpace": A single undamped step, solving the measurement-space system by conjugate gradient.
"gn" or "GaussNewton": Iterated undamped steps, solving the state-space system directly.
"gn_m" or "GaussNewtonMeasurementSpace": Iterated undamped steps, solving the measurement-space system directly.
"gn_cg" or "GaussNewtonConjugateGradient": Iterated undamped steps, solving the state-space system by conjugate gradient.
"gn_cg_m" or "GaussNewtonConjugateGradientMeasurementSpace": Iterated undamped steps, solving the measurement-space system by conjugate gradient.
"lm" or "ml": Iterated damped steps, solving the state-space system directly.
"lm_cg" or "ml_cg": Iterated damped steps, solving the state-space system by conjugate gradient.

Overview

Method	`readxml()`	Read variable from file.
Method	`savexml()`	Saves variable to file.
Static Method	`fromxml()`	Create variable from file.
Static Method	`get_options()`	Get a list of all options
Static Method	`get_options_as_strings()`	Get a list of all options as strings
Static Data	`pyarts3.arts.OptimalEstimationMethod.GaussNewton`	“gn” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.GaussNewtonConjugateGradient`	“gn_cg” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.GaussNewtonConjugateGradientMeasurementSpace`	“gn_cg_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.GaussNewtonMeasurementSpace`	“gn_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.Linear`	“li” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.LinearConjugateGradient`	“li_cg” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.LinearConjugateGradientMeasurementSpace`	“li_cg_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.LinearMeasurementSpace`	“li_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.gn`	“gn” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.gn_cg`	“gn_cg” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.gn_cg_m`	“gn_cg_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.gn_m`	“gn_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.li`	“li” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.li_cg`	“li_cg” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.li_cg_m`	“li_cg_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.li_m`	“li_m” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.lm`	“lm” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.lm_cg`	“lm_cg” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.ml`	“lm” (`OptimalEstimationMethod`)
Static Data	`pyarts3.arts.OptimalEstimationMethod.ml_cg`	“lm_cg” (`OptimalEstimationMethod`)
Operator	`__eq__()`	self == other
Operator	`__format__()`	Default object formatter.
Operator	`__ge__()`	self >= other
Operator	`__gt__()`	self > other
Operator	`__hash__()`	Allows hashing
Operator	`__init__()`	`__init__(self, arg: str, /) -> None`
Operator	`__le__()`	self <= other
Operator	`__lt__()`	self < other
Operator	`__ne__()`	self != other
Operator	`__repr__()`	`__repr__(self) -> str`
Operator	`__str__()`	`__str__(self) -> str`

Constructors

__init__(self) → None
__init__(self, arg: OptimalEstimationMethod) → None
__init__(self, arg: str, /) → None

Methods

readxml(self, file: str) → str

Read variable from file.

Parameters:: file (str) – A file that can be read.
Raises:: RuntimeError – For any failure to read.
Returns:: file – The file path found (may differ from input due to environment variables).
Return type:: str

savexml(self, file: str, type: str = 'ascii', clobber: bool = True) → str

Saves variable to file.

Parameters:

file (str) – The path to which the file is written. Note that several of the options might modify the name or write more files.
type (str, optional) – Type of file to save. See FileType for options. Defaults is “ascii”.
clobber (bool, optional) – Overwrite existing files or add new file with modified name? Defaults is True.

Raises:

RuntimeError – For any failure to write.

Returns:

file – The file saved. May differ from input.

Return type:

str

Static Methods

fromxml(file: str) → OptimalEstimationMethod

Create variable from file.

Parameters:: file (str) – A file that can be read
Raises:: RuntimeError – For any failure to read.

get_options() → list[OptimalEstimationMethod]: Get a list of all options

get_options_as_strings(i: int = 0) → list[str]: Get a list of all options as strings

Operators

__eq__(self, arg: OptimalEstimationMethod, /) → bool: self == other

__format__(format_spec, /)

Default object formatter.

Return str(self) if format_spec is empty. Raise TypeError otherwise.

__ge__(self, arg: OptimalEstimationMethod, /) → bool: self >= other

__gt__(self, arg: OptimalEstimationMethod, /) → bool: self > other

__hash__(self) → int: Allows hashing

__init__(self) → None
__init__(self, arg: OptimalEstimationMethod) → None
__init__(self, arg: str, /) → None

__le__(self, arg: OptimalEstimationMethod, /) → bool: self <= other

__lt__(self, arg: OptimalEstimationMethod, /) → bool: self < other

__ne__(self, arg: OptimalEstimationMethod, /) → bool: self != other

__repr__(self) → str

__str__(self) → str