Why Rare Events Need More Than Just Data

An intuitive look at why rare events are difficult to emulate from limited data, and how local instability can provide the missing dynamical context.

By Isabella Thiel · September 30, 2026 · 10 min read

Based on our NeurIPS 2026 oral paper

Mechanism-Aware Ensemble Conditioning for Data-Limited Emulation of Extreme Events

Isabella S. Thiel, Juan Bello-Rivas, Yannis G. Kevrekidis, and Themistoklis P. Sapsis

Read on arXivPDFGitHub code

A model of a complex dynamical system may reproduce its frequently observed behavior with very small prediction error, while giving an inaccurate description of its most critical excursions. This distinction is particularly important for extreme events, whose low probability makes them difficult to represent in training data, even when their consequences are substantial.

Learning the bulk does not ensure accurate tails

Extreme events occur in natural, engineered, and economic systems. Examples include rogue ocean waves, heat waves, unusually large structural loads, cascading failures in power grids, and extreme financial losses. In a dynamical systems setting, such events are typically defined through an observable of the system, for example the wave height or the load acting on a structure, that exceeds a prescribed high threshold. Such systems may spend most of their time in a frequently visited regime and occasionally enter regions where finite-time amplification produces large excursions before the dynamics return to their usual behavior. Extreme values generally occupy the low-probability tails of the observable's distribution. [Sapsis, 2021] [Farazmand & Sapsis, 2019]

In our example, the training data come from two simulations of the same fluid flow. The high-resolution simulation is our reference and the cheaper coarse model simulation misses some small-scale motion. Those missing scales can affect extreme events even when the two simulations look similar most of the time. Generating a sufficiently long high-resolution trajectory to capture these rare extremes directly is prohibitively expensive.

This motivates the central question of our work: Can we extract and make use of more of the dynamical information already contained in the limited data available to us, rather than relying on increasingly expensive high-resolution simulations? Our theoretical contribution shows how a small ensemble of perturbed trajectories acts as a sensor of local finite-time instability. In this article, we illustrate how that information can be used in supervised learning to improve the long-time statistics of a coarsely simulated model, with particular emphasis on extreme events.

As a representative example, we consider a two-layer quasi-geostrophic (QG) flow. [Paper, Appendix C.1] Similarly, we have an expensive high-resolution reference and a cheaper coarsely resolved model. Before comparing these models, two difficulties need to be addressed. Their spatial resolutions differ, so we project the high-resolution state onto the coarse grid to express both in a common state space. Moreover, because the flow is chaotic, two freely evolving trajectories separate even when their initial conditions are close. A direct pointwise comparison would then mix the structural error of the coarse model with chaotic divergence of the trajectories, leaving the learning algorithm to fit this misalignment and noise instead of the unresolved dynamics. We therefore use nudging during training to keep the nudged-coarse trajectory in a neighborhood of the projected reference. [Paper, §2]

More details on nudging The equations for comparable training trajectories

Let the high-resolution state satisfy the evolution equation with vector field FF, and let ff denote the coarse-model vector field. We use P\mathcal{P} to project the reference onto the coarse state space. The three relevant evolutions are the reference, the freely evolving coarse model, and the deterministically nudged coarse model.

u˙=F(u),u=Pu,v˙=f(v),v˙τ=f(vτ)−τ−1(vτ−u).\begin{aligned} \dot{\mathbf {u}}&=F(\mathbf u),&u&=\mathcal P\mathbf u,\\ \dot v&=f(v),&\dot v_\tau&=f(v_\tau)-\tau^{-1}(v_\tau-u). \end{aligned}

Here the high-resolution state is denoted by u\mathbf{u}, its projection by u=Puu=\mathcal{P}\mathbf{u}, the unconstrained coarse trajectory by vv, and the nudged trajectory by vτv_\tau. The nudging timescale τ\tau controls the strength of relaxation toward the projected reference and is chosen to balance two competing requirements: keeping the coarse trajectory sufficiently close to the reference while preserving the intrinsic dynamics and statistical characteristics of the coarse model's attractor. Projection makes the numerical states spatially comparable, while nudging reduces the effect of chaotic phase separation over the finite training interval. Neither operation supplies the missing fine-scale physics, and nudging is not itself the learned correction. [Paper, §2] [Paper, Appendix D]

Explore the flow

One system, four views

Quasi-geostrophic (QG) flow subject to bottom layer topography
High-resolution reference128 × 128 grid
Projected referenceThe reference on a 24 × 24 grid
Free-running coarse model24 × 24 grid
Nudged coarse modelKept near the projected reference

Watch the how the reference, its projection, the low-fidelity coarse model, and the respective nudged streamfunctions evolve. Freely evolving chaotic trajectories separate. Nudging keeps the coarse trajectory near the reference so that training pairs and a learned correction remain meaningful. Nudging alone is not the learned correction, but a deterministic operation. Compare evolution and structure of the different trajectories for a synchronized time window.

Our objective with this QG example is to seek an inexpensive emulator that reproduces the long-time distribution of the reference, including the frequency and spatial extent of events with return periods longer than those observed during training.

01 / The idea

Using an ensemble to sense transient instability

When nearby trajectories separate rapidly, the flow is temporarily sensitive to small disturbances. Finite-time Lyapunov exponents, Cauchy–Green deformation tensors, and optimally time-dependent (OTD) modes provide well-established means of describing transient amplification. Their direct computation, however, requires the Jacobian, tangent-equation integrations, or additional evolution equations, which are computationally infeasible in high-dimensional settings. We instead apply small random perturbations to the nudged coarse model and track how the resulting trajectories spread.

In detail we evolve independent ensemble deviations Q(m)Q^{(m)} driven by small stochastic perturbations. The ensemble members satisfy the following stochastic differential equation:

dQ(m)(t)=[f(u(t)+Q(m)(t))−PF(u(t))−τ−1Q(m)(t)] dt+σ dBt (m).\begin{aligned} \mathrm{d}Q^{(m)}(t) &=\biggl[f\Bigl(u(t)+Q^{(m)}(t)\Bigr) -\mathcal{P}F\Bigl(\mathbf{u}(t)\Bigr)-\tau^{-1}Q^{(m)}(t)\biggr] \,\mathrm{d}t+\sqrt{\sigma}\,\mathrm{d}B_t \,^{(m)}. \end{aligned}

Here B(m)B^{(m)} are independent Brownian motions, and σ\sigma is the noise intensity. The deterministic terms describe the nudged-coarse evolution, while the stochastic forcing continuously excites nearby directions. When these perturbations are small, the way the ensemble expands and contracts provides information about the local dynamics, rather than merely a measure of forecast uncertainty. [Paper, §2.2]

The training data

From two simulators to a local ensemble

RQE figure: Reference and coarse model dynamics
Step 1 of 4Reference and coarse model dynamicsThe expensive high-resolution reference (red) and inexpensive coarse model (blue) live on different attractors and potentially follow different trajectories.
How does this look in the QG data?

For the QG system considered here, the resulting ensemble is illustrated below. We show representative stochastically perturbed trajectories for different noise intensities, together with their deviations from the deterministically nudged coarse solution. These deviations provide information about the local response of the dynamics to small perturbations.

Reference and coarse zonally averaged lower-layer streamfunctions feed a deterministically nudged baseline. Five representative ensemble members at increasing noise levels are shown with their differences from the baseline-nudged solution.
Response of the nudged coarse model to stochastic forcing. The projected reference, freely evolving coarse solution, and deterministically nudged baseline are shown for the zonally averaged lower-layer streamfunction ψ2\psi_2. Representative ensemble members at increasing noise intensities illustrate how the stochastic trajectories differ from the nudged baseline. Notice the domain of instability even in the low noise case σ=10−3\sigma=10^{-3}. These deviations provide a local probe of dynamical sensitivity. Click to enlarge.

The important point is that the resulting ensemble spread is not arbitrary. Perturbations are damped along contracting directions and can grow in directions of finite-time instability, so the evolving distribution of nearby trajectories contains information about local deformation. We show mathematically that, in the small-noise regime, the ensemble covariance is governed by the same finite-time deformation kernels that characterize local amplification. This provides a dynamical interpretation of the ensemble as a non-intrusive instability sensor.

The mathematical connection Covariance evolution, assumptions, and finite-ensemble guarantees

We denote by Q‾(t)=E[Q(t)]\overline{Q}(t)=\mathbb{E}[Q(t)] the ensemble mean and define the covariance of the centered fluctuations as

Σ(t)=E[(Q(t)−Q‾(t))(Q(t)−Q‾(t))T].\Sigma(t)=\mathbb{E}\left[\bigl(Q(t)-\overline{Q}(t)\bigr)\bigl(Q(t)-\overline{Q}(t)\bigr)^{\mathsf T}\right].

Here, Q(t)Q(t) denotes a solution of the stochastic ensemble equation. The covariance characterizes both the magnitude and the directional structure of the ensemble spread.

Linearizing the drift of the stochastic ensemble around its mean gives a matrix Lyapunov equation, perturbed by a remainder that accounts for the nonlinear terms.

Σ˙(t)=Lτ(t)Σ(t)+Σ(t)Lτ(t)T+σ2I+RΣ(t).\dot{\Sigma}(t)=L_{\tau}(t)\Sigma(t)+\Sigma(t)L_{\tau}(t)^{\mathsf T}+\sigma^2I+R_{\Sigma}(t).

The linearized operator is evaluated along the mean perturbed coarse state and includes the stabilizing contribution from nudging.

Lτ(t)=Df(u(t)+Q‾(t))−τ−1I.L_{\tau}(t)=Df\bigl(u(t)+\overline{Q}(t)\bigr)-\tau^{-1}I.

The terms involving LτL_\tau describe the local stretching and contraction of the ensemble, while the isotropic forcing injects perturbations continuously. On a fixed finite interval, this relation requires the stated smoothness conditions and second- and fourth-moment bounds on the deviations. The small-noise bounds have the following scaling.

sup⁡t∈[0,T]E∥Q′(t)∥2≤C2σ2,sup⁡t∈[0,T]E∥Q′(t)∥4≤C4σ4.\sup_{t\in[0,T]}\mathbb E\|Q'(t)\|^2\le C_2\sigma^{2},\qquad\sup_{t\in[0,T]}\mathbb E\|Q'(t)\|^4\le C_4\sigma^{4}.

Under these bounds and the stated regularity conditions, the covariance remainder is of order O(σ3)\mathcal O(\sigma^{3}) as σ\sigma increases.

Let Φ(t,s)\Phi(t,s) be the fundamental solution of the linearized nudged dynamics. Integrating the covariance equation gives

Σ(t)=Φ(t,0)Σ(0)Φ(t,0)T+σ2∫0tΦ(t,s)Φ(t,s)T ds+OT(σ3).\Sigma(t)=\Phi(t,0)\Sigma(0)\Phi(t,0)^{\mathsf T}+\sigma^2\int_0^t\Phi(t,s)\Phi(t,s)^{\mathsf T}\,\mathrm ds+\mathcal O_T(\sigma^{3}).

The covariance therefore combines the deformation of the initial spread with deformation accumulated from the stochastic forcing over the interval. These kernels contain the same local stretching information used in finite-time instability analysis, although their time integral is not identical to a single Cauchy–Green tensor. Relating their leading subspaces further requires separation of the relevant growth directions.

For a finite ensemble of MM independent centered samples, Theorem 2.2 quantifies the error in the estimated leading covariance subspace. If the fourth moment is bounded by κ\kappa and the relevant eigenvalues are separated by a gap δ\delta, the expected projector error is bounded by 2κ/(δM)2\sqrt{\kappa}/(\delta\sqrt M). This is a finite-sample statement about the inferred geometry, not a theorem asserting that all rare-event statistics can be recovered from a short dataset. [Paper, §2.2] [Paper, Appendix A]

02 / The method

An instability signal for existing learning architectures

The ensemble provides information about the local dynamics, but this information becomes useful for emulation only when the learning algorithm can respond to it. An important feature of our framework is that the conditioning is independent of the particular neural-network backbone. Instead of replacing an established corrector or redesigning its internal architecture, we introduce a small interface that makes the ensemble-derived dynamical context available to its existing hidden features.

For this purpose we use feature-wise linear modulation (FiLM) [Perez et al., 2018].

The FiLM implementationBackbone agnostic introduction of dynamical context

A small conditioning network transforms the dynamical context into scale and shift coefficients, which modulate the hidden representation of the original model. If hth_t is a hidden feature vector and ctc_t is the ensemble-derived context, the modulation takes the form

h~t=(1+γ(ct))⊙ht+b(ct).\widetilde h_t=\bigl(1+\gamma(c_t)\bigr)\odot h_t+b(c_t).

The coefficients depend on the instantaneous dynamical context, so the learned corrector can adapt its internal response when the flow enters a sensitive regime. This is a lightweight addition to an existing learning pipeline rather than a new dynamical model. In our comparisons we retain the original backbones and baseline training setups, including their architecture and hyperparameter choices, and learn the additional conditioning within the supervised training procedure. [Paper, §2.3]

High-resolution referenceAlign the coarse dynamics

Projection and nudging provide comparable training states.

Perturbed coarse ensembleProbe finite-time amplification

Ensemble response supplies dynamical context.

FiLM and an existing backboneCondition its hidden features

Learn context-dependent gains and offsets without replacing the corrector.

The additional ensemble integrations are performed using the inexpensive coarse model, whereas the high-resolution reference provides the supervised targets. FiLM injects the dynamical information into an otherwise unchanged correction architecture.

We demonstrate this interface on different types of neural-network models, including a Transformer-style residual-attention corrector and a probabilistic recurrent STORN model. Improvements in both settings show that the conditioning is not specific to a particular backbone architecture. We compare each conditioned model with its unconditioned counterpart, keeping the underlying backbone, its hyperparameters, and the baseline training setup the same. This makes the effect of the added dynamical context easier to assess. [Paper, §3] [Paper, §4]

The following animation shows for the QG example how this FiLM context is introduced into the existing STORN pipeline.

The QG correction model

Building FiLM-STORN, one stage at a time

RQE figure: Start with STORN
Step 1 of 5Start with STORNThe STORN training pipeline takes nudged coarse fields as input and learns to reconstruct the projected reference.
Original STORN baseline — Sorensen et al. (2024)

During long-time deployment, the QG corrector uses statistics of the freely evolving coarse trajectory as an approximate context signal. This is a deliberate choice that ensures the method operates independently of the high-resolution reference. While generating the nudged ensemble during training requires reference data, neither nudging nor additional reference information is needed at deployment. The corrected coarse simulation can therefore be evaluated over arbitrarily long horizons without introducing information from the reference trajectory into the correction process. [Paper, Appendix D]

03 / The results

Extreme-event statistics beyond the training horizon

We examine the sensing mechanism in a low-dimensional chaotic system, where independent OTD modes provide a characterization of transient instability. The leading ensemble-covariance direction becomes active during the same instability episodes as the OTD mode, supporting the interpretation of stochastic ensemble spread as a probe of finite-time amplification. [Paper, §3]

For the QG flow the high-resolution reference is generated on a 128 × 128 grid and the coarse model evolves on a 24 × 24 grid. In the strongly data-limited experiment, our FiLM-conditioned model is trained on just 50 time units of nudged-coarse and high-resolution reference data and a few-member ensemble. This is a particularly restrictive setting because the training trajectory is short even for characterizing the bulk of the distribution dynamics, while extreme events with long return periods may not occur within the training horizon at all. This places us in a substantially more data-limited regime than previously demonstrated for successful correction of the same QG system. Earlier work found that approximately 500 to 1,000 time units were necessary to reproduce the relevant long-time statistics accurately. [Barthel Sorensen et al., 2024] [Barthel Sorensen et al., 2026] With only 50 time units, the available record is therefore insufficient not only for resolving extremes but also for providing a well-sampled representation of the frequently visited dynamics.

We then evaluate the corrected model for 50,000 time units, a horizon 1,000 times longer than the training length . The purpose is to determine whether the learned model reproduces the reference distribution, including events whose characteristic return times are much longer than the duration of its training data. The reference simulation over the evaluation interval is used to construct the benchmark statistics and is not supplied to the deployed corrector.

We compare against an unconditioned STORN trained on the same 50-unit record and a second unconditioned STORN trained on 1,000 time units of high-resolution data, which represents the state-of-the-art in this literature [Barthel Sorensen et al., 2024] [Barthel Sorensen et al., 2026]. Alongside the overall probability density, we examine the frequency of high-threshold exceedances and the spatial area affected by these events. These two diagnostics are important because accurate bulk statistics alone do not establish that the model reproduces how frequently extreme episodes occur or how extensively they develop within the flow. [Paper, §4]

Explore the reported results

How often do extremes occur?

Paper, Table 2

Mean absolute error in the probability that the field’s magnitude exceeds a high threshold somewhere in the domain, averaged over both layers and three extreme thresholds.

Absolute frequency error · lower is better

About 41% less frequency error than the baseline trained on the same 50 time units, and about 39% less than the baseline trained on 1,000.

STORN is the baseline; FiLM-STORN adds ensemble conditioning. Values are from the fixed main-text comparison (σ=0.01\sigma=0.01, Nadd=2N_{\mathrm{add}}=2), not the best value selected separately for each metric. Frequency and area errors average thresholds c∈{1.0,1.5,1.75}c\in\{1.0,1.5,1.75\} and both layers. These are errors, not the event probabilities themselves. Percentage reductions are calculated from Table 2.
See every value and its source
Table 2 of the paper. Lower is better in every column.
MethodTraining horizonFrequency errorArea errorLog-density errorKL
STORN500.121180.0157411.9700.07079
FiLM-STORN500.071990.010478.29590.03646
STORN1,0000.117260.011955.34540.00393

Original table · Metric definitions

With the same short training record, ensemble conditioning improves all four reported diagnostic errors relative to the unconditioned model. More strikingly, even with just 50 time units of training data, the conditioned model has lower average errors for how often high-threshold events occur and how much area they cover than the baseline trained on 1,000 time units. The longer-trained baseline remains better on the overall KL divergence and log-density errors. These results are expected as both the KL-divergence and log-density are dominated by the frequently observed dynamics, which are better represented in a longer training record. The interactive comparison above shows the results reported in Table 2.

This result also motivates the computational comparison. Generating the perturbed ensemble requires additional coarse simulations, but their cost is small relative to extending an expensive high-resolution reference from 50 to 1,000 time units. Importantly, we find that only a small ensemble is needed to obtain substantial improvements. The results presented here use just a two-member ensemble. The conditioned approach therefore achieves more accurate averaged high-threshold statistics using substantially less high-resolution supervision and a much shorter training trajectory. In this regime, enriching the training information with a description of the dynamics can be more effective than obtaining many more examples of the same frequently observed behavior.

The big take-away is that local dynamical information can be extracted from inexpensive numerical trajectories and supplied to existing learning architectures without redesigning the underlying numerical solver or neural-network backbone. The effectiveness of the approach ultimately depends on the quality of the underlying coarse model. Although the coarse dynamics need not reproduce the reference accurately, they must retain the essential dynamical structures and amplification mechanisms of the original system. If these mechanisms are absent or fundamentally misrepresented, the ensemble cannot recover the missing information through perturbations alone. [Paper, §5]

When high-resolution data are limited, the response of a stochastically evolved ensemble can provide valuable information about extreme-event mechanisms that is not contained in the observed trajectory alone.

If this connects with a system you work on, I would love to hear about it. For questions, possible applications, or a more detailed presentation, get in touch.

From LinkedIn

Conversation around the paper

Open on LinkedIn

If LinkedIn blocks the embedded preview in your browser, read the original post directly.

Read further

The paper & the citation

Mechanism-Aware Ensemble Conditioning for Data-Limited Emulation of Extreme Events

Isabella S. Thiel, Juan Bello-Rivas, Yannis G. Kevrekidis, and Themistoklis P. Sapsis
arXiv:2609.30746 · 2026 · Accepted to NeurIPS 2026 (oral)

Read on arXivPDFGitHub code

BibTeX

@misc{thiel2026mechanismaware,
  title = {Mechanism-Aware Ensemble Conditioning for
           Data-Limited Emulation of Extreme Events},
  author = {Thiel, Isabella S. and Bello-Rivas, Juan and
            Kevrekidis, Yannis G. and Sapsis, Themistoklis P.},
  year = {2026},
  eprint = {2609.30746},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  doi = {10.48550/arXiv.2609.30746},
  url = {https://arxiv.org/abs/2609.30746}
}

This essay accompanies version 1 of the paper. Quantitative results follow the paper, which is available under CC BY 4.0.