Adding a New Test Function Implementation

Adding a New Test Function Implementation#

In this guide, we will explain how to implement a new UQ test function into the UQTestFuns code base. A test function may be added on runtime using the UQTestFun class as illustrated here. However, adding the test function directly to the code base is advantageous that it becomes exposed to the high-level convenient functionalities (such as list_functions()). Furthermore, once merged, the test function will become available for all.

Note

Before moving on, make sure you’ve set up a local development environment as explained here.

Similar to creating a new test function on runtime, we are going to use the Branin function [DS78] as the motivating problem. The function is defined as follows:

\[ \mathcal{M}(x_1, x_2) = a \left( x_2 - b x_1^2 + c x_1 - r \right)^2 + s \left(1 - t \right) \cos{(x_1}) + s \]

where \(x_1\) and \(x_2\) are the input variables and \(\{ a, b, c, r, s, t \}\) are the parameters.

The input variables are defined in the table below.

No.

Name

Distribution

Parameters

1

\(x_1\)

uniform

\([-5, 10]\)

2

\(x_2\)

uniform

\([0, 15]\)

The typical values for the parameters are shown in the table below.

No.

Parameter

Value

1

\(a\)

\(1.0\)

2

\(b\)

\(\frac{5.1}{(2 \pi)^2}\)

3

\(c\)

\(\frac{5}{\pi}\)

4

\(r\)

\(6\)

5

\(s\)

\(10\)

6

\(t\)

\(\frac{1}{8 \pi}\)

We are going to implement this test function into the UQTestFuns code base step-by-step.

Step 0: Putting things in the right place#

A built-in test function in UQTestFuns is implemented as a Python module and stored inside src/uqtestfuns/test_functions (with respect to the source root directory). If you have a look at the directory you’ll see the following (or something similar):

├── core
├── test_functions              <- Sub-package that contains all UQ test function modules
│   ├── __init__.py
│   ├── ackley.py               <- An implementation of the Ackley function
│   ├── borehole.py             <- An implementation of the borehole function
│   ├── damped_oscillator.py    <- An implementation of the damped oscillator model
│   ├── ...
│   └── wing_weight.py          <- An implementation of the wing weight function
├── ...
└── utils.py

Say you’ve (aptly) named the Python module implementing the Branin function branin.py. You must put that file inside the directory.

Step 1: Implementing the function#

Now you’re ready to implement the test function inside branin.py. Let’s start with module-level docstring.

Adding module-level docstring#

Describe the module and add some information regarding the test function, where it was first used and in which context. Don’t forget to add the proper bibliographic information here (under the heading References).

So for example, we may write the following as the module-level docstring.

"""
Module with an implementation of the two-dimensional Branin function.

The Branin function is a two-dimensional scalar-valued function.
The function is commonly used as a test function for optimization algorithms [1].

References
----------

1. L. C. W. Dixon and G. P. Szegö. Towards global optimization 2,
   chapter The global optimization problem: an introduction, pages 1–15.
   North-Holland, Amsterdam, 1978.
"""

Importing relevant packages, classes, and functions#

Next, there are a few packages, classes, and utility functions that are commonly used in every test function implementation. Depending on a particular test function, you might import more. Be aware, however, we don’t arbitrarily add external dependencies to UQTestFuns (outside NumPy and SciPy).

Here are the few things we usually use:

import numpy as np

from uqtestfuns.core.custom_typing import ProbInputSpecs, FunParamSpecs
from uqtestfuns.core.uqtestfun_abc import UQTestFunFixDimABC

__all__ = ["Branin"]

Here are some explanations:

  • NumPy is usually a must, especially for implementing the evaluation function.

  • Because Branin test function is a test function with a fixed number of (input) dimensions, the class will be derived from the abstract base class UQTestFunFixDimABC.

  • To assist specifying the data for probabilistic input model the custom type ProbInputSpecs can be used; these are supposed to help you specifying all the required data via the typechecker.

  • Similarly, the custom type FunParamSpecs is used for specifying the function parameters.

Implementing a concrete evaluation function#

For an implementation of a test function, create a top module-level function (conventionally named evaluate() if there is only one test function in the module):

def evaluate(xx: np.ndarray, a: float, b: float, c: float, r: float, s: float, t: float):
    """Evaluate the Branin function on a set of input values.
    
    Parameters
    ----------
    xx : np.ndarray
        2-Dimensional input values given by an N-by-2 array where
        N is the number of input values.
    a : float
        Parameter 'a' of the Branin function.
    b : float
        Parameter 'b' of the Branin function.
    c : float
        Parameter 'c' of the Branin function.
    r : float
        Parameter 'r' of the Branin function.
    s : float
        Parameter 's' of the Branin function.
    t : float
        Parameter 't' of the Branin function.
    
    Returns
    -------
    np.ndarray
        The output of the Branin function evaluated on the input values.
        The output is a 1-dimensional array of length N.    
    """
    yy = (
        a * (xx[:, 1] - b * xx[:, 0]**2 + c * xx[:, 0] - r)**2
        + s * (1 - t) * np.cos(xx[:, 0]) 
        + s
    )
    
    return yy

Notice that for a test function with parameters, the signature should also include the parameters.

Specifying probabilistic input model#

The specification of the probabilistic model is stored in a module-level dictionary. While you’re free to name the dictionary anything you like, the convention is AVAILABLE_INPUTS. We define the variable as follows:

AVAILABLE_INPUT_SPECS: ProbInputSpecs = {
    "Dixon1978": {
        "name": "Branin",
        "description": (
            "Search domain for the Branin function from Dixon and Szegö (1978)."
        ),
        "marginals": [
            {
                "name": "x1",
                "distribution": "uniform",
                "parameters": [-5, 10],
                "description": None,
            },
            {
                "name": "x2",
                "distribution": "uniform",
                "parameters": [0.0, 15.0],
                "description": None,
            },
        ],
        "copulas": None,
    },
}

Each key-value pair of this dictionary contains the specification to create an input model as an instance of the ProbInput class. For this particular example, we only have one input specification available. If there were more, we would add them here in the dictionary each with a unique keyword.

Note

The keyword is, by convention, the citation key of the particular reference as listed in the BibTeX file.

Also notice that in the code above, we store the specifications of the marginals in a list of marginal specifications. Each element of this list is used to create an instance of Marginal.

With that, we have completed the input specification of the Branin function.

Specifying parameters#

For a parameterized test function, we also need to define a module-level dictionary that stores the parameter sets. Conventionally, we name this variable AVAILABLE_PARAMETERS:

AVAILABLE_PARAMETERS: FunParamSpecs = {
    "Dixon1978": {
      "function_id": "Branin",
      "description": "Parameter set for the Branin function from Dixon (1978)",
      "declared_parameters": [
          {
              "keyword": "a",
              "value": 1.0,
              "type": float,
          },
          {
              "keyword": "b",
              "value": 5.1 / (2 * np.pi) ** 2,
              "type": float,
          },
          {
              "keyword": "c",
              "value": 5 / np.pi,
              "type": float,
          },
          {
              "keyword": "r",
              "value": 6.0,
              "type": float,
          },
          {
              "keyword": "s",
              "value": 10.0,
              "type": float,
          },
          {
              "keyword": "t",
              "value": 1 / 8 / np.pi,
              "type": float,
          },
      ],
    },
}

This is a nested dictionary, where each top key-value pair contains one set of parameters from the literature.

The value of the parameters in the set can be of any type, as long as it is consistent with how the parameters are going to be consumed by the evaluate() function.

As before, if there are multiple parameter sets available in the literature, additional key-value pair should be added here.

Implementing a concrete class#

Each built-in test function is an implementation of the abstract base class UQTestFunABC. A concrete implementation of this base class requires the following:

  • a static method named evaluate()

  • several class-level properties, namely: _tags, _description, _available_inputs, _available_parameters, _default_input_id, and _default_parameters_id.

The full definition of the class for the Branin test function is shown below.

class Branin(UQTestFunABC):
    """A concrete implementation of the Branin test function."""
  
    _tags = ["optimization"]  # Application tags
    _description = "Branin function from Dixon and Szegö (1978)"  # Short description
    _available_inputs = AVAILABLE_INPUTS          # As defined above 
    _available_parameters = AVAILABLE_PARAMETERS  # As defined above
    _default_input_id = "Dixon1978"       # Optional, if only one input is available
    _default_parameters_id = "Dixon1978"  # Optional, if only one set of parameters is available

    evaluate = staticmethod(evaluate)  # assuming `evaluate()` has been defined

There is no need to define an __init__() method. We will use the default __init__() from the base class.

Notice the two last class properties: _default_input and _default_parameters. In case of only one input specification (resp. set of parameters) is available, these properties are optional. With more than one specification (resp. set), you must explicitly tell UQTestFuns which specification and set should be used by default (i.e., when not specified).

With this, the test function implementation is complete. We now need to import the test function at the package level.

Step 2: Making the function available upstream#

To make your new test function available upstream, you need to modify the file src/uqtestfuns/test_functions/__init__.py with the following:

...
from .branin import Branin
...

__all__ = [
  ...,
  "Branin",
  ...,
]

Now you can check if all has been correctly set up by listing the available built-in functions from a Python terminal.

>>> import uqtestfuns as uqtf
>>> uqtf.list_functions()
+-------+-------------------------------+-----------+------------+----------+---------------+--------------------------------+
|  No.  |          Constructor          |  # Input  |  # Output  |  Param.  |  Application  | Description                    |
+=======+===============================+===========+============+==========+===============+================================+
|   1   |           Ackley()            |     M     |     1      |   True   | optimization, | Optimization test function     |
|       |                               |           |            |          | metamodeling  | from Ackley (1987)             |
+-------+-------------------------------+-----------+------------+----------+---------------+--------------------------------+
|   2   |        Alemazkoor20D()        |    20     |     1      |  False   | metamodeling  | High-dimensional low-degree    |
|       |                               |           |            |          |               | polynomial from Alemazkoor &   |
|       |                               |           |            |          |               | Meidani (2018)                 |
+-------+-------------------------------+-----------+------------+----------+---------------+--------------------------------+
|   3   |        Alemazkoor2D()         |     2     |     1      |  False   | metamodeling  | Low-dimensional high-degree    |
|       |                               |           |            |          |               | polynomial from Alemazkoor &   |
|       |                               |           |            |          |               | Meidani (2018)                 |
+-------+-------------------------------+-----------+------------+----------+---------------+--------------------------------+
|   4   |          Borehole()           |     8     |     1      |  False   | metamodeling, | Borehole function from Harper  |
|       |                               |           |            |          |  sensitivity  | and Gupta (1983)               |
+-------+-------------------------------+-----------+------------+----------+---------------+--------------------------------+
|   5   |            Branin()           |     2     |     1      |  False   | optimization  | Branin function from Dixon     |
|       |                               |           |            |          |               | and Szegö (1978)               |
+-------+-------------------------------+-----------+------------+----------+---------------+--------------------------------+
...

Step 3: Implementing a test#

Although a generic test suite has been included that applies to all UQ test functions. Some test functions may have particular behaviors that must be tested separately. In the case of the Branin function, as an optimization test function, the optimum value and their (three) locations are known (analytically):

\[\begin{split} \begin{align} \mathcal{M}(\boldsymbol{x}^*) & = 0.397887 \\ \boldsymbol{x}^*_1 & = (-\pi, 12.275) \\ \boldsymbol{x}^*_2 & = (\pi, 2.275)\\ \boldsymbol{x}^*_3 & = (9.42478, 2.475)\\ \end{align} \end{split}\]

An additional test can (and should) be added to the suite to test this. Create a new test file specifically for the Branin function in tests/uq_test_functions/test_branin.py and write:

"""
Test module for the Branin test function.

Notes
-----
- The tests defined in this module deal with
  the correctness of the evaluation.
"""
import numpy as np
import uqtestfuns as uqtf

def test_optimum_value():
    """Test the optimum values of the Branin function"""
    # Create a test instance
    branin_fun = uqtf.Branin()

    yy_opt = 0.397887
    xx_opt = np.array([[-np.pi, 12.275], [np.pi, 2.275], [9.42478, 2.475]])

    # Assertions
    assert np.allclose(branin_fun(xx_opt), yy_opt)

Then run pytest from the root source directory to make sure that the function, especially its evaluation function, has been implemented properly.

Step 4: Adding the documentation of the test function#

Each test function in UQTestFuns has its dedicated page in the docs. Your new test function is no exception! Check out this guide on how to add the documentation for a test function.

Congratulations you just successfully implemented your first UQ test function and add it to the code base!

References#

[DS78]

L. C. W. Dixon and G. P. Szegö. Towards global optimization 2, chapter The global optimization problem: an introduction, pages 1–15. North-Holland, Amsterdam, 1978.