VBMC

Note

The VBMC class implements the Variational Bayesian Monte Carlo (VBMC) algorithm.

VBMC computes a variational approximation of the full posterior and a lower bound on the log normalization constant (log marginal likelhood or log model evidence) for a provided unnormalized log posterior.

To perform inference, first initialize a VBMC object and then call vbmc.optimize() on the instance.

By default VBMC assumes noiseless evaluations of the log posterior, but noisy likelihoods can also be handled. See PyVBMC Example 6: Noisy log-likelihood evaluations for more details.

See below for more details on the VBMC class methods and interface. The primary entry-points for users are the VBMC class, which initializes the algorithm, and the VariationalPosterior class, which represents the returned variational solution. The Basic options may also be useful.

class pyvbmc.VBMC(log_density: callable, x0: ndarray = None, lower_bounds: ndarray = None, upper_bounds: ndarray = None, plausible_lower_bounds: ndarray = None, plausible_upper_bounds: ndarray = None, options: dict = None, options_path: PathLike | None = None, prior: Prior = None, log_prior: callable = None, sample_prior: callable = None)

Posterior and model inference via Variational Bayesian Monte Carlo (VBMC).

VBMC computes a variational approximation of the full posterior and a lower bound on the normalization constant (marginal likelhood or model evidence) for a provided unnormalized log posterior.

Initialize a VBMC object to set up the inference problem, then run optimize(). See the examples for more details.

Parameters:

log_densitycallable

A given target log-posterior or log-likelihood. If log_prior is None, log_density accepts input x and returns the value of the target log-joint, that is, the unnormalized log-posterior density at x. If log_prior is not None, log_density should return the unnormalized log-likelihood. In either case, if options["specify_target_noise"] is true, log_density should return a tuple where the first element is the noisy log-density, and the second is an estimate of the standard deviation of the noise.

x0np.ndarray, optional

Starting point for the inference. Ideally x0 is a point in the proximity of the mode of the posterior. Default is None.

lower_bounds, upper_boundsnp.ndarray, optional

lower_bounds (LB) and upper_bounds (UB) define a set of strict lower and upper bounds for the coordinate vector, x, so that the posterior has support on LB < x < UB. If scalars, the bound is replicated in each dimension. Use None for LB and UB if no bounds exist. Set LB [i] = -inf and UB [i] = inf if the i-th coordinate is unbounded (while other coordinates may be bounded). Note that if LB and UB contain unbounded variables, the respective values of PLB and PUB need to be specified (see below), by default None.

plausible_lower_bounds, plausible_upper_boundsnp.ndarray, optional

Specifies a set of plausible_lower_bounds (PLB) and plausible_upper_bounds (PUB) such that LB < PLB < PUB < UB. Both PLB and PUB need to be finite. PLB and PUB represent a “plausible” range, which should denote a region of high posterior probability mass. Among other things, the plausible box is used to draw initial samples and to set priors over hyperparameters of the algorithm. When in doubt, we found that setting PLB and PUB using the topmost ~68% percentile range of the prior (e.g, mean +/- 1 SD for a Gaussian prior) works well in many cases (but note that additional information might afford a better guess). Both are by default None.

optionsdict, optional

Additional options can be passed as a dict. Please refer to the VBMC options page for the default options. If no options are passed, the default options are used.

options_pathpathlike, optional

Additional options can also be specified by a .ini file. The options dict keyword takes precedence over options set in the .ini file. options_path should be either absolute or relative to the pyvbmc/vbmc/ directory.

prior, optional

An optional separate prior. It can be a pyvbmc.priors.Prior subclass, an appropriate scipy.stats distribution, or a list of one-dimensional PyVBMC Prior and/or one-dimenional continuous scipy.stats distributions. (see the documentation on priors for more details). If prior is not None, the argument log_density is assumed to represent the log-likelihood (otherwise it is assumed to represent the log-joint).

log_priorcallable, optional

An optional separate log-prior function, which should accept a single argument x and return the log-density of the prior at x. If log_prior is not None, the argument log_density is assumed the log-joint).

sample_priorcallable, optional

An optional function which accepts a single argument n and returns an array of samples from the prior, of shape (n, D), where D is the problem dimension. Currently unused.

Raises:

ValueError

When neither x0 or (plausible_lower_bounds and plausible_upper_bounds) are specified.

ValueError

When various checks for the bounds (LB, UB, PLB, PUB) of VBMC fail.

References

[1]

Huggins, B., Li, C., Tobaben, M., Aarnos, M., & Acerbi, L. (2023). “PyVBMC: Efficient Bayesian inference in Python”. Journal of Open Source Software 8(86), 5428, https://doi.org/10.21105/joss.05428.

[2]

Acerbi, L. (2018). “Variational Bayesian Monte Carlo”. In Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pp. 8213-8223.

[3]

Acerbi, L. (2020). “Variational Bayesian Monte Carlo with Noisy Likelihoods”. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020).

Examples

For VBMC usage examples, please look up the Jupyter notebook tutorials in the PyVBMC documentation: https://acerbilab.github.io/pyvbmc/_examples/pyvbmc_example_1.html

optimize(self)

Run inference on an initialized VBMC object.

VBMC computes a variational approximation of the full posterior and the ELBO (evidence lower bound), a lower bound on the log normalization constant (log marginal likelhood or log model evidence) for the provided unnormalized log posterior.

Returns:

vpVariationalPosterior

The VariationalPosterior computed by VBMC.

resultsdict

A dictionary with additional information about the VBMC run.

save(self, file, overwrite=False)

Save the VBMC instance to a file.

Note

Complex attributes of a VBMC instance (such as the stored log_joint callable) may not behave correctly if they have been saved and loaded by different minor versions of Python, due to differing dependency versions. Basic (static) data should remain legible across versions.

Parameters:

filepath-like

The file name or path to write to. Default file extension .pkl will be added if no extension is specified.

overwritebool

Whether to allow overwriting existing files. Default False.

Raises:

FileExistsError

If the file already exists and overwrite is False.

OSError

If the file cannot be opened for other reasons (e.g., the directory is not found, the disk is full, etc.).

load(file, new_options=None, iteration=None, set_random_state=False)

Load a VBMC instance from a file.

Note

Complex attributes of a VBMC instance (such as the stored log_joint callable) may not behave correctly if they have been saved and loaded by different minor versions of Python, due to differing dependency versions. Basic (static) data should remain legible across versions.

Parameters:

filepath-like

The file name or path to write to.

new_optionsdict or None

A dictionary of options to change when loading the stored VBMC instance. Useful, for example, to continue a previous run with a larger budget of function evaluations and/or iterations. See the documentation on PyVBMC’s options for more details.

iterationint or None

The iteration at which to initialize the stored VBMC instance. Default is None, meaning initialize to the last recorded iteration.

set_random_statebool

Whether to set the global random state to the stored random state of the VBMC object, for reprodicibility. Default False.

Returns:

vbmcVBMC

The loaded VBMC instance, initialized at the specified iteration.

Raises:

ValueError

If the specified iteration is less than zero or larger than the last stored iteration.

OSError

If the file cannot be found, or cannot be opened for other reasons.

determine_best_vp(max_idx: int = None, safe_sd: float = 5, frac_back: float = 0.25, rank_criterion_flag: bool = False)

Return the best VariationalPosterior found during the optimization of VBMC as well as its ELBO, ELBO_SD and the index of the iteration.

Parameters:

max_idxint, optional

Check up to this iteration, by default None which means last iter.

safe_sdfloat, optional

Penalization for uncertainty, by default 5.

frac_backfloat, optional

If no past stable iteration, go back up to this fraction of iterations, by default 0.25.

rank_criterion_flagbool, optional

If True use new ranking criterion method to pick best solution. It finds a solution that combines ELCBO, stability, and recency, by default False.

Returns:

vpVariationalPosterior

The VariationalPosterior found during the optimization of VBMC.

elbofloat

The ELBO of the iteration with the best VariationalPosterior.

elbo_sdfloat

The ELBO_SD of the iteration with the best VariationalPosterior.

idx_bestint

The index of the iteration with the best VariationalPosterior.

final_boost(vp: VariationalPosterior, gp: GP)

Perform a final boost of variational components.

Parameters:

vpVariationalPosterior

The VariationalPosterior that should be boosted.

gpGaussianProcess

The corresponding GaussianProcess of the VariationalPosterior.

Returns:

vpVariationalPosterior

The VariationalPosterior resulting from the final boost.

elboVariationalPosterior

The ELBO of the VariationalPosterior resulting from the final boost.

elbo_sdVariationalPosterior

The ELBO_SD of the VariationalPosterior resulting from the final boost.

changed_flagbool

Indicates if the final boost has taken place or not.

Additional functions

pyvbmc.vbmc.optimize_vp(options: Options, optim_state: dict, vp: VariationalPosterior, gp: GP, fast_opts_N: int, slow_opts_N: int, K: int = None)

Optimize variational posterior.

Parameters:

optionsOptions

Options from the VBMC instance we are calling this from.

optim_statedict

Optimization state from the VBMC instance we are calling this from.

vpVariationalPosterior

The variational posterior we want to optimize.

gpgpyreg.GaussianProcess

The Gaussian process surrogate of the log-posterior, against which to optimize the VP.

fast_opts_Nint

Number of fast optimizations.

slow_opts_Nint

Number of slow optimizations.

Kint, optional

Number of mixture components. If not given defaults to the number of mixture components the given VP has.

Returns:

vpVariationalPosterior

The optimized variational posterior.

var_ssint

Estimated variance of the ELBO, due to variance of the expected log-joint, for each GP hyperparameter sample.

prunedint

Number of pruned components.

pyvbmc.vbmc.train_gp(hyp_dict: dict, optim_state: dict, function_logger: FunctionLogger, iteration_history: IterationHistory, options: Options, plb_tran: ndarray, pub_tran: ndarray)

Train Gaussian process model.

Parameters:

hyp_dictdict

Hyperparameter summary statistics dictionary. If it does not contain the appropriate keys they will be added automatically.

optim_statedict

Optimization state from the VBMC instance we are calling this from.

function_loggerFunctionLogger

Function logger from the VBMC instance which we are calling this from.

iteration_historyIterationHistory

Iteration history from the VBMC instance we are calling this from.

optionsOptions

Options from the VBMC instance we are calling this from.

plb_tranndarray, shape (1, D)

Transformed lower plausible bounds, used to set GP hyperparameters.

pub_tranndarray, shape (1, D)

Transformed upper plausible bounds, used to set GP hyperparameters.

Returns:

gpGP

The trained GP.

gp_s_Nint

The number of samples for fitting.

sn2_hpdfloat

An estimate of the GP noise variance at high posterior density.

hyp_dictdict

The updated summary statistics.

pyvbmc.vbmc.update_K(optim_state: dict, iteration_history: IterationHistory, options: Options)

Update number of variational mixture components.

Parameters:

optim_statedict

Optimization state from the VBMC instance we are calling this from.

iteration_historyIterationHistory

Iteration history from the VBMC instance we are calling this from.

optionsOptions

Options from the VBMC instance we are calling this from.

Returns:

K_newint

The new number of variational mixture components.