Scoring¶
Proper scoring rules let us evaluate predictive observation distributions with respect to data in ways beyond marginal likelihood.
dynestyx.evaluation.scoring defines the score objects themselves: BaseObservationScore, GaussianLogProbScore, DawidSebastianiScore, ObservationWiseCRPSScore, and EnergyScore. These scores currently operate on the one-step-ahead predictive observation distributions produced by the continuous-time CD-Dynamax Gaussian filters (ContinuousTimeKFConfig, ContinuousTimeEKFConfig, ContinuousTimeUKFConfig, and ContinuousTimeEnKFConfig). ObservationScoringConfig is documented on the companion Scoring Configs page.
Scoring rules for predictive observation distributions.
BaseObservationScore
dataclass
¶
Bases: ABC
Base class for predictive-observation scoring rules.
Subclasses define a per-time score array. site_name is available for
integrations that want to record score arrays into named trace sites.
GaussianLogProbScore
dataclass
¶
Bases: BaseObservationScore
Per-time multivariate Gaussian log-probability score.
Uses predictive Gaussian moments and returns a score array of shape
(*plate, time, 1). Higher values are better.
DawidSebastianiScore
dataclass
¶
Bases: BaseObservationScore
Per-time Dawid-Sebastiani score under Gaussian predictive moments.
Uses predictive Gaussian moments and returns a score array of shape
(*plate, time, 1). Lower values are better.
ObservationWiseCRPSScore
dataclass
¶
Bases: BaseObservationScore
Per-observation-component CRPS under Gaussian predictive marginals.
Applies the scalar Gaussian CRPS to each observation component separately
and returns a score array of shape (*plate, time, observation_dim).
Lower values are better.
EnergyScore
dataclass
¶
Bases: BaseObservationScore
Per-time ensemble energy score with exponent beta.
If an explicit predictive observation ensemble is unavailable, this score
can approximate one by drawing n_samples observations from the Gaussian
predictive observation moments. Returns a score array of shape
(*plate, time, 1). Lower values are better.
When synthetic sampling is needed, pass sample_seed to compute.
vectorized_pairwise=True is faster for moderate ensemble sizes but
materializes the full pairwise distance tensor. Setting it to False
uses a lower-memory lax.scan path at the cost of extra compute.