BayesFluxR: Implementation of Bayesian Neural Networks

Description

Implementation of 'BayesFlux.jl' for R; It extends the famous 'Flux.jl' machine learning library to Bayesian Neural Networks. The goal is not to have the fastest production ready library, but rather to allow more people to be able to use and research on Bayesian Neural Networks.

Author(s)

Maintainer: Enrico Wegner e.wegner@student.maastrichtuniversity.nl

Installs Julia packages if needed

Description

Installs Julia packages if needed

Usage

.install_pkg(...)

Arguments

Obtain the status of the current Julia project

Description

Obtain the status of the current Julia project

Usage

.julia_project_status()

Set a seed both in Julia and R

Description

Set a seed both in Julia and R

Usage

.set_seed(seed)

Arguments

Value

No return value, called for side effects.

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  .set_seed(123)

## End(Not run)

Loads Julia packages

Description

Loads Julia packages

Usage

.using(...)

Arguments

Create a Bayesian Neural Network

Description

Create a Bayesian Neural Network

Usage

BNN(x, y, like, prior, init)

Arguments

Value

List with the following content

• 'juliavar' - the julia variable containing the BNN

• 'juliacode' - the string representation of the BNN

• 'x' - x

• 'juliax' - julia variable holding x

• 'y' - y

• 'juliay' - julia variable holding y

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGLD()
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Obtain the total parameters of the BNN

Description

Obtain the total parameters of the BNN

Usage

BNN.totparams(bnn)

Arguments

Value

The total number of parameters in the BNN

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  BNN.totparams(bnn)

## End(Not run)

Set up of the Julia environment needed for BayesFlux

Description

This will set up a new Julia environment in the current working directory or another folder if provided. This environment will then be set with all Julia dependencies needed.

Usage

BayesFluxR_setup(
  pkg_check = TRUE,
  nthreads = 4,
  seed = NULL,
  env_path = getwd(),
  installJulia = FALSE,
  ...
)

Arguments

Value

No return value, called for side effects.

Examples

## Not run: 
  ## Time consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)

## End(Not run)

Chain various layers together to form a network

Description

Chain various layers together to form a network

Usage

Chain(...)

Arguments

Value

List with the following content

• juliavar - the julia variable containing the network

• specification - the string representation of the network

• nc - the julia variable for the network constructor

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  Chain(LSTM(5, 5))
  Chain(RNN(5, 5, "tanh"))
  Chain(Dense(1, 5))

## End(Not run)

Create a Dense layer with 'in_size' inputs and 'out_size' outputs using 'act' activation function

Description

Create a Dense layer with 'in_size' inputs and 'out_size' outputs using 'act' activation function

Usage

Dense(in_size, out_size, act = c("identity", "sigmoid", "tanh", "relu"))

Arguments

Value

A list with the following content

• in_size - Input Size

• out_size - Output Size

• activation - Activation Function

• julia - Julia code representing the Layer

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 5, "relu"))

## End(Not run)

Create a Gamma Prior

Description

Creates a Gamma prior in Julia using Distributions.jl

Usage

Gamma(shape = 2, scale = 2)

Arguments

Value

A list with the following content

• juliavar - julia variable containing the distribution

• juliacode - julia code used to create the distribution

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))

## End(Not run)

Create an Inverse-Gamma Prior

Description

Creates and Inverse Gamma prior in Julia using Distributions.jl

Usage

InverseGamma(shape = 2, scale = 2)

Arguments

Value

A list with the following content

• juliavar - julia variable containing the distribution

• juliacode - julia code used to create the distribution

See Also

Gamma

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, InverseGamma(2.0, 0.5))

## End(Not run)

Create an LSTM layer with 'in_size' input size, and 'out_size' hidden state size

Description

Create an LSTM layer with 'in_size' input size, and 'out_size' hidden state size

Usage

LSTM(in_size, out_size)

Arguments

Value

A list with the following content

• in_size - Input Size

• out_size - Output Size

• julia - Julia code representing the Layer

See Also

Dense

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(LSTM(5, 5))

## End(Not run)

Create a Normal Prior

Description

Creates a Normal prior in Julia using Distributions.jl. This can then be truncated using Truncated to obtain a prior that could then be used as a variance prior.

Usage

Normal(mu = 0, sigma = 1)

Arguments

Value

see Gamma

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Truncated(Normal(0, 0.5), 0, Inf))

## End(Not run)

Create a RNN layer with 'in_size' input, 'out_size' hidden state and 'act' activation function

Description

Create a RNN layer with 'in_size' input, 'out_size' hidden state and 'act' activation function

Usage

RNN(in_size, out_size, act = c("sigmoid", "tanh", "identity", "relu"))

Arguments

Value

A list with the following content

• in_size - Input Size

• out_size - Output Size

• activation - Activation Function

• julia - Julia code representing the Layer

See Also

Dense

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(RNN(5, 5, "tanh"))

## End(Not run)

Truncates a Distribution

Description

Truncates a Julia Distribution between 'lower' and 'upper'.

Usage

Truncated(dist, lower, upper)

Arguments

Value

see Gamma

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Truncated(Normal(0, 0.5), 0, Inf))

## End(Not run)

Use Bayes By Backprop to find Variational Approximation to BNN.

Description

This was proposed in Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015, June). Weight uncertainty in neural network. In International conference on machine learning (pp. 1613-1622). PMLR.

Usage

bayes_by_backprop(
  bnn,
  batchsize,
  epochs,
  mc_samples = 1,
  opt = opt.ADAM(),
  n_samples_convergence = 10
)

Arguments

Value

a list containing

• 'juliavar' - julia variable storing VI

• 'juliacode' - julia representation of function call

• 'params' - variational family parameters for each iteration

• 'losses' - BBB loss in each iteration

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(RNN(5, 1))
  like <- likelihood.seqtoone_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  data <- matrix(rnorm(10*1000), ncol = 10)
  # Choosing sequences of length 10 and predicting one period ahead
  tensor <- tensor_embed_mat(data, 10+1)
  x <- tensor[1:10, , , drop = FALSE]
  # Last value in each sequence is the target value
  y <- tensor[11,,]
  bnn <- BNN(x, y, like, prior, init)
  vi <- bayes_by_backprop(bnn, 100, 100)
  vi_samples <- vi.get_samples(vi, n = 1000)

## End(Not run)

Find the MAP of a BNN using SGD

Description

Find the MAP of a BNN using SGD

Usage

find_mode(bnn, optimiser, batchsize, epochs)

Arguments

Value

Returns a vector. Use posterior_predictive to obtain a prediction using this MAP estimate.

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  find_mode(bnn, opt.RMSProp(), 10, 100)

## End(Not run)

Creates a random string that is used as variable in julia

Description

Creates a random string that is used as variable in julia

Usage

get_random_symbol()

Initialises all parameters of the network, all hyper parameters of the prior and all additional parameters of the likelihood by drawing random values from 'dist'.

Description

Initialises all parameters of the network, all hyper parameters of the prior and all additional parameters of the likelihood by drawing random values from 'dist'.

Usage

initialise.allsame(dist, like, prior)

Arguments

Value

A list containing the following

• 'juliavar' - julia variable storing the initialiser

• 'juliacode' - julia code used to create the initialiser

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  BNN.totparams(bnn)

## End(Not run)

Use a Normal likelihood for a Feedforward network

Description

This creates a likelihood of the form

y_i \sim Normal(net(x_i), \sigma)\;\forall i=1,...,N

where the x_i is fed through the network in a standard feedforward way.

Usage

likelihood.feedforward_normal(chain, sig_prior)

Arguments

Value

A list containing the following

• juliavar - julia variable containing the likelihood

• juliacode - julia code used to create the likelihood

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  BNN.totparams(bnn)

## End(Not run)

Use a t-Distribution likelihood for a Feedforward network

Description

This creates a likelihood of the form

\frac{y_i - net(x_i)}{\sigma} \sim T_\nu\;\forall i=1,...,N

where the x_i is fed through the network in the standard feedforward way.

Usage

likelihood.feedforward_tdist(chain, sig_prior, nu = 30)

Arguments

Value

see likelihood.feedforward_normal

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_tdist(net, Gamma(2.0, 0.5), nu=8)
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  BNN.totparams(bnn)

## End(Not run)

Use a Normal likelihood for a seq-to-one recurrent network

Description

This creates a likelihood of the form

y_i \sim Normal(net(x_i), \sigma), i=1,...,N

Here x_i is a subsequence which will be fed through the recurrent network to obtain the final output net(x_i) = \hat{y}_i. Thus, if one has a single time series, and splits the single time series into subsequences of length K which are then used to predict the next output of the time series, then each x_i consists of K consecutive observations of the time series. In a sense one constraints the maximum memory length of the network this way.

Usage

likelihood.seqtoone_normal(chain, sig_prior)

Arguments

Value

see likelihood.feedforward_normal

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(RNN(5, 1))
  like <- likelihood.seqtoone_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- array(rnorm(5*100*10), dim=c(10,5,100))
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  BNN.totparams(bnn)

## End(Not run)

Use a T-likelihood for a seq-to-one recurrent network.

Description

See likelihood.seqtoone_normal and likelihood.feedforward_tdist for details,

Usage

likelihood.seqtoone_tdist(chain, sig_prior, nu = 30)

Arguments

Value

see likelihood.feedforward_normal

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(RNN(5, 1))
  like <- likelihood.seqtoone_tdist(net, Gamma(2.0, 0.5), nu=5)
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- array(rnorm(5*100*10), dim=c(10,5,100))
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  BNN.totparams(bnn)

## End(Not run)

Use the diagonal of sample covariance matrix as inverse mass matrix.

Description

Use the diagonal of sample covariance matrix as inverse mass matrix.

Usage

madapter.DiagCov(adapt_steps, windowlength, kappa = 0.5, epsilon = 1e-06)

Arguments

Value

list containing 'juliavar' and 'juliacode' and all given arguments.

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  madapter <- madapter.DiagCov(100, 10)
  sampler <- sampler.GGMC(madapter = madapter)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Use a fixed mass matrix

Description

Use a fixed mass matrix

Usage

madapter.FixedMassMatrix(mat = NULL)

Arguments

Value

list with 'juliavar' and 'juliacode' and given matrix or 'NULL'

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  madapter <- madapter.FixedMassMatrix()
  sampler <- sampler.GGMC(madapter = madapter)
  ch <- mcmc(bnn, 10, 1000, sampler)


  # Providing a non-sense weight matrix
  weight_matrix <- matrix(runif(BNN.totparams(bnn)^2, 0, 1),
                          nrow = BNN.totparams(bnn))
  madapter2 <- madapter.FixedMassMatrix(weight_matrix)
  sampler2 <- sampler.GGMC(madapter = madapter2)
  ch2 <- mcmc(bnn, 10, 1000, sampler2)

## End(Not run)

Use the full covariance matrix as inverse mass matrix

Description

Use the full covariance matrix as inverse mass matrix

Usage

madapter.FullCov(adapt_steps, windowlength, kappa = 0.5, epsilon = 1e-06)

Arguments

Value

see madapter.DiagCov

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  madapter <- madapter.FullCov(100, 10)
  sampler <- sampler.GGMC(madapter = madapter)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Use RMSProp to adapt the inverse mass matrix.

Description

Use RMSProp as a preconditions/mass matrix adapter. This was proposed in Li, C., Chen, C., Carlson, D., & Carin, L. (2016, February). Preconditioned stochastic gradient Langevin dynamics for deep neural networks. In Thirtieth AAAI Conference on Artificial Intelligence for the use in SGLD and related methods.

Usage

madapter.RMSProp(adapt_steps, lambda = 1e-05, alpha = 0.99)

Arguments

Value

list with 'juliavar' and 'juliacode' and all given arguments

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  madapter <- madapter.RMSProp(100)
  sampler <- sampler.GGMC(madapter = madapter)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Sample from a BNN using MCMC

Description

Sample from a BNN using MCMC

Usage

mcmc(
  bnn,
  batchsize,
  numsamples,
  sampler = sampler.SGLD(stepsize_a = 1),
  continue_sampling = FALSE,
  start_value = NULL
)

Arguments

Value

a list containing the 'samples' and the 'sampler' used.

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGNHTS(1e-3)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

ADAM optimiser

Description

ADAM optimiser

Usage

opt.ADAM(eta = 0.001, beta = c(0.9, 0.999), eps = 1e-08)

Arguments

Value

see opt.Descent

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  find_mode(bnn, opt.ADAM(), 10, 100)

## End(Not run)

Standard gradient descent

Description

Standard gradient descent

Usage

opt.Descent(eta = 0.1)

Arguments

Value

list containing

• 'julivar' - julia variable holding the optimiser

• 'juliacode' - string representation

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  find_mode(bnn, opt.Descent(1e-5), 10, 100)

## End(Not run)

RMSProp optimiser

Description

RMSProp optimiser

Usage

opt.RMSProp(eta = 0.001, rho = 0.9, eps = 1e-08)

Arguments

Value

see opt.Descent

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  find_mode(bnn, opt.RMSProp(), 10, 100)

## End(Not run)

Draw from the posterior predictive distribution

Description

Draw from the posterior predictive distribution

Usage

posterior_predictive(bnn, posterior_samples, x = NULL)

Arguments

Value

A matrix whose columns are the posterior predictive draws.

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGLD()
  ch <- mcmc(bnn, 10, 1000, sampler)
  pp <- posterior_predictive(bnn, ch$samples)

## End(Not run)

Use an isotropic Gaussian prior

Description

Use a Multivariate Gaussian prior for all network parameters. Covariance matrix is set to be equal 'sigma * I' with 'I' being the identity matrix. Mean is zero.

Usage

prior.gaussian(chain, sigma)

Arguments

Value

a list containing the following

• 'juliavar' the julia variable used to store the prior

• 'juliacode' the julia code

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGLD()
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Scale Mixture of Gaussian Prior

Description

Uses a scale mixture of Gaussian for each network parameter. That is, the prior is given by

\pi_1 Normal(0, sigma1) + (1-\pi_1) Normal(0, sigma2)

Usage

prior.mixturescale(chain, sigma1, sigma2, pi1)

Arguments

Value

a list containing the following

• 'juliavar' the julia variable used to store the prior

• 'juliacode' the julia code

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.mixturescale(net, 10, 0.1, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGLD()
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Sample from the prior predictive of a Bayesian Neural Network

Description

Sample from the prior predictive of a Bayesian Neural Network

Usage

prior_predictive(bnn, n = 1)

Arguments

Value

matrix of prior predictive samples; Columns are the different samples

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  pp <- prior_predictive(bnn, n = 10)

## End(Not run)

Use a constant stepsize in mcmc

Description

Use a constant stepsize in mcmc

Usage

sadapter.Const(l)

Arguments

Value

list with 'juliavar', 'juliacode' and the given arguments

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sadapter <- sadapter.Const(1e-5)
  sampler <- sampler.GGMC(sadapter = sadapter)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Use Dual Averaging like in STAN to tune stepsize

Description

Use Dual Averaging like in STAN to tune stepsize

Usage

sadapter.DualAverage(
  adapt_steps,
  initial_stepsize = 1,
  target_accept = 0.65,
  gamma = 0.05,
  t0 = 10,
  kappa = 0.75
)

Arguments

Value

list with 'juliavar', 'juliacode', and all given arguments

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sadapter <- sadapter.DualAverage(100)
  sampler <- sampler.GGMC(sadapter = sadapter)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Adaptive Metropolis Hastings as introduced in

Description

Haario, H., Saksman, E., & Tamminen, J. (2001). An adaptive Metropolis algorithm. Bernoulli, 223-242.

Usage

sampler.AdaptiveMH(bnn, t0, sd, eps = 1e-06)

Arguments

Value

a list with 'juliavar', 'juliacode', and all given arguments

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.AdaptiveMH(bnn, 10, 1)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Gradient Guided Monte Carlo

Description

Proposed in Garriga-Alonso, A., & Fortuin, V. (2021). Exact langevin dynamics with stochastic gradients. arXiv preprint arXiv:2102.01691.

Usage

sampler.GGMC(
  beta = 0.1,
  l = 1,
  sadapter = sadapter.DualAverage(1000),
  madapter = madapter.FixedMassMatrix(),
  steps = 3
)

Arguments

Value

a list with 'juliavar', 'juliacode' and all provided arguments.

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sadapter <- sadapter.DualAverage(100)
  sampler <- sampler.GGMC(sadapter = sadapter)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Standard Hamiltonian Monte Carlo (Hybrid Monte Carlo).

Description

Allows for the use of stochastic gradients, but the validity of doing so is not clear.

Usage

sampler.HMC(
  l,
  path_len,
  sadapter = sadapter.DualAverage(1000),
  madapter = madapter.FixedMassMatrix()
)

Arguments

Details

This is motivated by parts of the discussion in Neal, R. M. (1996). Bayesian Learning for Neural Networks (Vol. 118). Springer New York. https://doi.org/10.1007/978-1-4612-0745-0

Value

a list with 'juliavar', 'juliacode', and all given arguments

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sadapter <- sadapter.DualAverage(100)
  sampler <- sampler.HMC(1e-3, 3, sadapter = sadapter)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Stochastic Gradient Langevin Dynamics as proposed in Welling, M., & Teh, Y. W. (n.d.). Bayesian Learning via Stochastic Gradient Langevin Dynamics. 8.

Description

Stepsizes will be adapted according to

a(b+t)^{-\gamma}

Usage

sampler.SGLD(
  stepsize_a = 0.1,
  stepsize_b = 0,
  stepsize_gamma = 0.55,
  min_stepsize = -Inf
)

Arguments

Value

a list with 'juliavar', 'juliacode', and all given arguments

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGLD()
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Stochastic Gradient Nose-Hoover Thermostat as proposed in

Description

Proposed in Leimkuhler, B., & Shang, X. (2016). Adaptive thermostats for noisy gradient systems. SIAM Journal on Scientific Computing, 38(2), A712-A736.

Usage

sampler.SGNHTS(
  l,
  sigmaA = 1,
  xi = 1,
  mu = 1,
  madapter = madapter.FixedMassMatrix()
)

Arguments

Details

This is similar to SGNHT as proposed in Ding, N., Fang, Y., Babbush, R., Chen, C., Skeel, R. D., & Neven, H. (2014). Bayesian sampling using stochastic gradient thermostats. Advances in neural information processing systems, 27.

Value

a list with 'juliavar', 'juliacode' and all arguments provided

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGNHTS(1e-3)
  ch <- mcmc(bnn, 10, 1000, sampler)

## End(Not run)

Print a summary of a BNN

Description

Print a summary of a BNN

Usage

## S3 method for class 'BNN'
summary(object, ...)

Arguments

Embed a matrix of timeseries into a tensor

Description

This is used when working with recurrent networks, especially in the case of seq-to-one modelling. Creates overlapping subsequences of the data with length 'len_seq'. Returned dimensions are seq_len x num_vars x num_subsequences.

Usage

tensor_embed_mat(mat, len_seq)

Arguments

Value

A tensor of dimension: len_seq x num_vars x num_subsequences

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(RNN(5, 1))
  like <- likelihood.seqtoone_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  data <- matrix(rnorm(5*1000), ncol = 5)
  # Choosing sequences of length 10 and predicting one period ahead
  tensor <- tensor_embed_mat(data, 10+1)
  x <- tensor[1:10, , , drop = FALSE]
  # Last value in each sequence is the target value
  y <- tensor[11,1,]
  bnn <- BNN(x, y, like, prior, init)
  BNN.totparams(bnn)

## End(Not run)

Convert draws array to conform with 'bayesplot'

Description

BayesFluxR returns draws in a matrix of dimension params x draws. This cannot be used with the 'bayesplot' package which expects an array of dimensions draws x chains x params.

Usage

to_bayesplot(ch, param_names = NULL)

Arguments

Value

Returns an array of dimensions draws x chains x params.

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  sampler <- sampler.SGLD()
  ch <- mcmc(bnn, 10, 1000, sampler)
  ch <- to_bayesplot(ch)
  library(bayesplot)
  mcmc_intervals(ch, pars = paste0("param_", 1:10))

## End(Not run)

Draw samples form a variational family.

Description

Draw samples form a variational family.

Usage

vi.get_samples(vi, n = 1)

Arguments

Value

a matrix whose columns are draws from the variational posterior

Examples

## Not run: 
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(RNN(5, 1))
  like <- likelihood.seqtoone_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  data <- matrix(rnorm(10*1000), ncol = 10)
  # Choosing sequences of length 10 and predicting one period ahead
  tensor <- tensor_embed_mat(data, 10+1)
  x <- tensor[1:10, , , drop = FALSE]
  # Last value in each sequence is the target value
  y <- tensor[11,,]
  bnn <- BNN(x, y, like, prior, init)
  vi <- bayes_by_backprop(bnn, 100, 100)
  vi_samples <- vi.get_samples(vi, n = 1000)
  pp <- posterior_predictive(bnn, vi_samples)

## End(Not run)