PyBADS Example 2: Non-box constraints#

In this example, we will show how to set more complex constraints in PyBADS, besides a simple bounded box.

This notebook is Part 2 of a series of notebooks in which we present various example usages for BADS with the PyBADS package. The code used in this example is available as a script here.

import numpy as np
from pybads import BADS

0. Constrained optimization#

PyBADS naturally supports box constraints lb and ub, as we saw in the previous example. However, some optimization problems might have more complex constraints over the variables. Formally, we may wish to solve the problem

\[ \mathbf{x}^\star = \arg\min_{\mathbf{x} \in \mathcal{X}} f(\mathbf{x}) \]

where \(\mathcal{X} \subseteq \mathbb{R}^D\) is the admissible region for the optimization.

We can do this in PyBADS by providing a function non_box_cons that defines constraints violation, that is a function \(g(\mathbf{x})\) which returns True if \(\mathbf{x} \notin \mathcal{X}\) (and False otherwise), as demonstrated below.

1. Problem setup#

We optimize Rosenbrock’s banana function in 2D as in the previous example, but here we force the input to stay within a circle with unit radius.

Since we know the optimization region, we set tight box bounds lb and ub around the circle. This step is not necessary, but it is recommended as it further helps the search.

The function passed to non_box_cons:

takes as input an array \(\mathbf{x}_1, \ldots, \mathbf{x}_M\) with shape (M, D), where each \(\mathbf{x}_m \in \mathbb{R}^D\);
outputs a bool array with shape (M, 1), where the \(m\)-th value is True if \(\mathbf{x}_m\) violates the constraint, False otherwise;

where \(M\) is an arbitrary number of inputs.

def rosenbrocks_fcn(x):
    """Rosenbrock's 'banana' function in any dimension."""
    x_2d = np.atleast_2d(x)
    return np.sum(100 * (x_2d[:, 0:-1]**2 - x_2d[:, 1:])**2 + (x_2d[:, 0:-1]-1)**2, axis=1)

x0 = np.array([0, 0]);      # Starting point
lower_bounds = np.array([-1, -1])
upper_bounds = np.array([1, 1])

def circle_constr(x):
    """Return constraints violation outside the unit circle."""
    x_2d = np.atleast_2d(x)
    # Note that nonboxcons assumes the function takes a 2D input 
    return np.sum(x_2d**2, axis=1) > 1.

2. Run the optimization#

We initialize bads with the non-box constraints defined by non_box_cons. Note that we also still specify standard box constraints lower_bounds and upper_bounds, as this will help the search.

Here BADS will complain because we did not specify the plausible bounds explicitly. In the absence of plausible bounds, BADS will create them based on the lower/upper bounds instead. As a general rule, it is strongly recommended to specify the plausible bounds.

bads = BADS(rosenbrocks_fcn, x0, lower_bounds, upper_bounds, non_box_cons=circle_constr)
optimize_result = bads.optimize()

bads:TooCloseBounds: For each variable, hard and plausible bounds should not be too close. Moving plausible bounds.
Beginning optimization of a DETERMINISTIC objective function

 Iteration    f-count         f(x)           MeshScale          Method             Actions
         2               1               1                                 Uncertainty test
         5               1               1         Initial mesh            Initial points
         9               1             0.5         Refine grid             Train
        13         0.71573             0.5     Incremental search (ES-wcm)        
        17         0.71573            0.25         Refine grid             Train
        18        0.213085            0.25     Successful search (ES-wcm)        
        20       0.0866235            0.25     Successful search (ES-wcm)        
        22       0.0750055            0.25     Incremental search (ES-wcm)        
        23       0.0555838            0.25     Incremental search (ES-ell)        
        24       0.0503648            0.25     Incremental search (ES-ell)        
        27       0.0503648           0.125         Refine grid             
        28       0.0473246           0.125     Incremental search (ES-wcm)        
        29       0.0460316           0.125     Incremental search (ES-wcm)        
        30       0.0460089           0.125     Incremental search (ES-ell)        
        33       0.0460089          0.0625         Refine grid             
        36       0.0459778          0.0625     Incremental search (ES-wcm)        
        39       0.0459778         0.03125         Refine grid             Train
        40       0.0457596         0.03125     Incremental search (ES-ell)        
        43       0.0456915         0.03125     Incremental search (ES-ell)        
        45       0.0456915        0.015625         Refine grid             
        48       0.0456879        0.015625     Incremental search (ES-ell)        
        51       0.0456879      0.00390625         Refine grid             Train
        55       0.0456831      0.00390625     Incremental search (ES-ell)        
        57       0.0456831     0.000976562         Refine grid             
Optimization terminated: change in the function value less than options['tol_fun'].
Function value at minimum: 0.04568314807326722

3. Results and conclusions#

x_min = optimize_result['x']
fval = optimize_result['fval']

print(f"BADS minimum at: x_min = {x_min.flatten()}, fval = {fval:.4g}")
print(f"total f-count: {optimize_result['func_count']}, time: {round(optimize_result['total_time'], 2)} s")
print(f"Problem type: {optimize_result['problem_type']}")

BADS minimum at: x_min = [0.78639246 0.6176717 ], fval = 0.04568
total f-count: 58, time: 1.03 s
Problem type: non-box constraints

The true global minimum of the Rosenbrock function under these constraints is at \(\textbf{x}^\star = [0.786,0.618]\), where \(f^\star = 0.0457\).

Remarks#

While in theory non_box_cons can receive any arbitrary constraints, in practice PyBADS will likely work well only within relatively simple domains (e.g., simple convex regions), as the current version of (Py)BADS uses a simple heuristic to reject samples outside the admissible region.
In particular, PyBADS does not support equality constraints (e.g., of the form \(x_1 + x_2 + x_3 = 1\)).

PyBADS Example 2: Non-box constraints

Contents