rareflow: Rare-event modeling with normalizing flows

Description

Tools for variational inference, rare-event estimation, Girsanov tilting, and Freidlin–Wentzell quasi-potentials.

Author(s)

Maintainer: Pietro Piu pietro.piu.si@gmail.com

Evidence Lower Bound (ELBO) for Flow-Based Variational Inference

Description

Computes the ELBO:

Usage

ELBOflow(flow, Qobs, pxgivenz, nmc = 256)

Arguments

Details

\log p(x \| z) + \log p(z) - \log q(z)

where:

• q(z) is the flow-based variational posterior

• p(z) is a standard Gaussian prior

• p(x | z) is provided by the user as pxgivenz

The expectation is approximated via Monte Carlo sampling.

Value

A numeric ELBO value.

Examples

f <- makeflow("planar", list(u = 0.1, w = 0.2, b = 0))
px <- function(z) c(0.3, 0.4, 0.3)
ELBOflow(f, Qobs = c(0.2, 0.5, 0.3), pxgivenz = px, nmc = 100)

Freidlin-Wentzell Quasi-Potential via Path Minimization

Description

Computes an approximate Freidlin-Wentzell quasi-potential between two points x_0 and x_1 by minimizing the FW action functional over discretized paths.

Usage

FW_quasipotential(
  x0,
  x1,
  drift,
  T = 200,
  dt = 0.01,
  niter = 200,
  stepsize = 0.1
)

Arguments

Details

The algorithm:

1. Initializes a straight-line path between x_0 and x_1.

2. Performs simple gradient descent on the FW action.

This is a naive but effective illustrative method for low-dimensional systems. More advanced solvers (string method, MAM, etc.) can be plugged in.

Value

A list with:

• path: matrix of size T \times d

• action: FW action of the optimized path

Freidlin-Wentzell Action Functional

Description

Computes the discrete Freidlin-Wentzell action for a path \phi(t) represented as a matrix of size T \times d. The continuous action is:

Usage

Freidlin_Wentzell_action(phi, drift, dt)

Arguments

Details

I[\phi] = \frac{1}{2} \int_0^T \lVert \dot{\phi}(t) - b(\phi(t)) \rVert^2 \, dt

and the discrete approximation is:

I \approx \frac{1}{2} \sum_{t=1}^{T-1} \lVert (\phi_{t+1} - \phi_t)/dt - b(\phi_t) \rVert^2 \, dt

Value

Numeric action value.

Kullback–Leibler Divergence

Description

Computes the KL divergence D(Q || P) between two discrete probability distributions Q and P.

Usage

KLdiv(Q, P)

Arguments

Value

A numeric KL divergence value.

Fit Flow Using Freidlin-Wentzell Quasi-Potential

Description

Computes the Freidlin-Wentzell quasi-potential between x_0 and x_1, constructs a tilted likelihood proportional to \exp(-V/\mathrm{eps}), and fits a flow-based variational posterior.

Usage

fitflow_FW(
  observed,
  states = NULL,
  flowtype = "maf",
  flowspec = list(),
  inittheta = NULL,
  drift,
  x0,
  x1,
  T = 200,
  dt = 0.01,
  eps = 0.1,
  nmc = 256,
  control = list()
)

Arguments

Details

This is useful for rare-event inference in small-noise diffusions, where the quasi-potential acts as an effective energy landscape.

Value

Output of fitflowvariational().

Fit Flow with Girsanov-Tilted Likelihood

Description

Applies a Girsanov change of measure to tilt the likelihood and then fits a flow-based variational posterior using fitflowvariational().

Usage

fitflow_girsanov(
  observed,
  states = NULL,
  flowtype = "maf",
  flowspec = list(),
  inittheta = NULL,
  base_pxgivenz,
  theta_path,
  Winc,
  dt,
  nmc = 256,
  control = list()
)

Arguments

Details

This is useful when the target distribution arises from a drift-tilted diffusion process, where the Radon-Nikodym derivative is given by the Girsanov theorem.

Value

Output of fitflowvariational().

Fit a Flow-Based Variational Posterior for Sanov Inference

Description

This function fits a variational posterior q(z) using a chosen normalizing flow. The objective is the ELBO:

Usage

fitflowvariational(
  observed,
  states = NULL,
  flowtype = c("maf", "splinepwlin", "planar", "radial"),
  flowspec = list(),
  inittheta = NULL,
  pxgivenz,
  nmc = 256,
  control = list()
)

Arguments

Details

\log p(x | z) + \log p(z) - \log q(z)

The flow parameters (theta) are optimized via optim() when applicable (MAF and spline flows). Planar and radial flows have no trainable parameters.

This function performs generic variational inference using a chosen normalizing flow and a user-provided likelihood pxgivenz.

For specialized rare-event inference using:

• Girsanov change of measure

• Freidlin–Wentzell quasi-potential

see the wrapper functions:

• fitflow_girsanov()

• fitflow_FW()

These wrappers construct a tilted likelihood and then call fitflowvariational() internally.

Value

A list containing:

• flow: the fitted flow model

• elbo: final ELBO value

• theta: optimized parameter vector (if applicable)

• convergence: optim() convergence code

Girsanov Log-Ratio for Drift-Tilted Diffusions

Description

Computes the Radon-Nikodym derivative (log form) associated with a Girsanov change of measure for an SDE:

Usage

girsanov_logratio(theta_path, Winc, dt)

Arguments

Details

dX_t = b(X_t)\, dt + dW_t

tilted by an alternative drift:

dX_t = (b(X_t) + \theta_t)\, dt + dW_t.

The log-likelihood ratio is:

\log \frac{dQ}{dP} = \sum_t \left( \theta_t W_t - \frac{1}{2}\theta_t^2\, dt \right)

This function returns the log-ratio for a given path of drift tilts theta_path, Brownian increments Winc, and time step dt.

Value

A numeric log-likelihood ratio.

Logit Transform

Description

Computes log(u / (1 - u)).

Usage

logit(u)

Arguments

Value

Numeric logit value.

Masked Autoregressive Flow (MAF)

Description

A readable and structured implementation of a Masked Autoregressive Flow. This version supports:

• arbitrary dimension d

• K sequential flow steps

• a single parameter vector theta containing all weights

Usage

mafflowmodel(d = 3, K = 2, theta = NULL)

Arguments

Value

A flow model object with methods:

• sampleq(n)

• logq(z0)

• applyflow(z0)

Biological Two-Separator Likelihood

Description

Same structure as makeneurolik(), but with a different default separation parameter. This can be used to model simple biological switching systems or coarse-grained gene expression states.

Usage

makebiolik(a = 0.2)

Arguments

Value

A function mapping a latent vector z to a probability vector.

Flow Factory

Description

A unified constructor for all flow models in the rareflow package. This function dispatches to the appropriate flow implementation based on the flowtype argument.

Usage

makeflow(flowtype, params = list())

Arguments

Details

Supported flow types:

• "planar" -> planarflowmodel()

• "radial" -> radialflowmodel()

• "maf" -> mafflowmodel()

• "splinepwlin" -> splinepwlinflowmodel()

Value

A flow model object with methods:

• sampleq(n)

• logq(z0)

• applyflow(z0)

Examples

# Create a planar flow
f <- makeflow("planar", list(u = 0.1, w = 0.2, b = 0))
s <- f$sampleq(10)

# Create a 2D spline flow
f2 <- makeflow("splinepwlin", list(d = 2, K = 8))

Neural Two-Separator Likelihood

Description

Constructs a simple 3-category likelihood model based on a latent vector z. The likelihood is defined by two logistic separators:

Usage

makeneurolik(a = 0.3)

Arguments

Details

p1 = sigmoid(mean(z) - a) p2 = sigmoid(mean(z) + a)

producing a 3-class probability vector:

(1 - p1, p1 - p2, p2)

This likelihood is useful for toy neural classification models or simple latent-to-categorical mappings.

Value

A function mapping a latent vector z to a probability vector.

Default Operator %||%

Description

Returns b if a is NULL, otherwise returns a.

Usage

a %||% b

Arguments

Value

Either a or b.

Planar Normalizing Flow (1D)

Description

A simple and readable implementation of a 1-dimensional planar flow: z_K = z_0 + u * h(wz_0 + b)

Usage

planarflowmodel(u, w, b)

Arguments

Details

where:

• h is a smooth activation (tanh)

• the log-determinant is computed analytically

This flow is mainly useful for pedagogical purposes or as a lightweight building block in variational inference.

Value

A flow model object with methods:

• sampleq(n)

• logq(z0)

• applyflow(z0)

Radial Normalizing Flow (1D)

Description

A readable implementation of a 1-dimensional radial flow:

Usage

radialflowmodel(z_ref, alpha, beta)

Arguments

Details

z_K = z_0 + beta / (alpha + |z_0 - z_ref|) * (z_0 - z_ref)

where:

• z_ref is a reference point

• alpha > 0 ensures numerical stability

• beta controls the strength of the radial deformation

The log-determinant is computed analytically for the 1D case.

Value

A flow model object with methods:

• sampleq(n)

• logq(z0)

• applyflow(z0)

Sanov Probability Bound

Description

Computes the classical Sanov upper bound:

Usage

sanovprob(Q, P, n)

Arguments

Details

P(Q_n \approx Q) \le \exp\{-n \, KL(Q \| P)\}

where Q is the empirical distribution and P is the true distribution.

Value

A numeric upper bound.

Sigmoid Activation

Description

Standard logistic sigmoid function.

Standard logistic sigmoid function.

Usage

sigmoid(x)

sigmoid(x)

Arguments

Value

Numeric value between 0 and 1.

Numeric output in (0,1).

Softmax Function

Description

Computes a numerically stable softmax.

Usage

softmax(x)

Arguments

Value

Probability vector summing to 1.

Piecewise-Linear Spline Flow (Monotone)

Description

A readable implementation of a monotone piecewise-linear spline flow. Each dimension is transformed independently using learned spline parameters.

Usage

splinepwlinflowmodel(d = 2, K = 8, theta = NULL)

Arguments

Details

The spline flow uses:

• K bins with learned widths w and heights h

• a softmax transformation to ensure positivity and normalization

• a sigmoid reparameterization for numerical stability

The flow is invertible and differentiable almost everywhere.

Value

A flow model object with methods:

• sampleq(n)

• logq(z0)

• applyflow(z0)