The SUMO toolbox  2017a
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Pages
Public Member Functions | Static Public Member Functions | Public Attributes | Protected Member Functions | Protected Attributes | List of all members
BasicGaussianProcess Class Reference

A kriging surrogate model (also known as a Gaussian Process) More...

Inheritance diagram for BasicGaussianProcess:
Inheritance graph
[legend]

Public Member Functions

function BasicGaussianProcess (var varargin)
 Class constructor. More...
 
function getHyperparameters ()
 Returns the hyperparameters. More...
 
function getRho ()
 Returns the rho parameter (only valid for CoKriging) More...
 
function getProcessVariance ()
 Returns the process variance (sigma2) More...
 
function getCorrelationMatrix ()
 Returns the full extrinsic correlation matrix. More...
 
function getSigma ()
 Returns the intrinsic covariance matrix. More...
 
function getRegressionMatrix ()
 Returns the model matrix (Vandermonde matrix) More...
 
function getSamples ()
 Returns the input sample matrix. More...
 
function getValues ()
 Returns the output value matrix. More...
 
function setOption (var key, var value)
 Sets a value in the options structure. More...
 
function getOption (var key)
 Gets a value in the options structure. More...
 
function cleanup ()
 Clears some unused variables. More...
 
function display ()
 Returns user-friendly description of the class instance. More...
 
function fit (var samples, var values)
 Fits a gaussian process for a dataset. More...
 
function predict (var points)
 Predict the mean and/or variance for one or more points x. More...
 
function predictCovarianceMatrix (var points1, var points2, var outputIdx)
 Predict the full posterior covariance matrix. More...
 
function predict_derivatives (var point)
 Predict the derivatives of the mean and/or variance for a points. More...
 
function predict_limit (var points)
 Limit predictor of kriging (EXPERIMENTAL) More...
 
function regressionFunction (var options)
 Returns the regression function. More...
 
function correlationFunction (var options)
 Returns the internal correlation function handle and a symbolic expression of the the correlation part. More...
 
function getExpression (var options)
 Returns the Matlab expression of this Gaussian Process model. More...
 
function cvpe ()
 Calculates the log of the leave-one-out cross validation error (LOO-CV) More...
 
function marginalLikelihood (var dpsi, var dsigma2)
 Marginal likelihood function. More...
 
function pseudoLikelihood (var dpsi, var dsigma2)
 Leave-one-out predictive log probability (pseudo-likelihood) More...
 
function mseTestset (var testx, var testy)
 Calculates error on holdout set. More...
 
function imse ()
 Calculates the log of the integrated mean squared error. More...
 
function rcValues ()
 Quantifies magnification of noise (lower is better) More...
 
function plotVariogram ()
 Variogram plot (EXPERIMENTAL) More...
 
function getSensitivityIndices (var model, var order, var type, var method)
 Calculate sensitivity indices. More...
 
function extrinsicCorrelationMatrix (var points1, var points2)
 Constructs extrinsic correlation matrix. More...
 
function intrinsicCovarianceMatrix (var points1, var points2)
 Constructs intrinsic covariance matrix (stochastic kriging/regression kriging) More...
 

Static Public Member Functions

static function getDefaultOptions ()
 Returns a default options structure. More...
 

Public Attributes

const var RHO
 index of the rho parameter More...
 
const var LAMBDA
 index of the lambda parameter More...
 
const var SIGMA2
 index of the sigma2 parameter More...
 
const var HP
 index of the correlation function parameters More...
 

Protected Member Functions

function setData (var samples, var values)
 Sets samples and values matrix. More...
 
function updateModel (var F, var hp)
 Constructs model. More...
 
function updateRegression (var F, var hp)
 Constructs regression part. More...
 
function updateStochasticProcess (var hp)
 Constructs correlation part. More...
 
function regressionMatrix (var points)
 Constructs regression matrix. More...
 
function tuneParameters (var F)
 Hyperparameter optimization. More...
 
function generateDegrees (var dj)
 Generate degrees matrix from individual ones. More...
 

Protected Attributes

var options
 
var regressionFcn
 degrees matrix (strings are converted) More...
 
var correlationFcn
 string -> function handle More...
 
var hyperparameters0
 initial hp values OR the real ones (if optimization is done outside the class) More...
 
var dist
 sample inter-distance More...
 
var distIdxPsi
 indexing needed to calculate psi from D More...
 
var optimIdx
 logical indices to parameters that are optimized More...
 
var optimNrParameters
 number of optimization parameter (vector; one entry per type of parameter) More...
 
var alpha
 Regression coefficients. More...
 
var gamma
 'Correlation part' coefficients More...
 
var hyperparameters
 correlation parameters More...
 
var rho
 
var C
 Choleski Decomposition of extrinsic + intrinsic matrices. More...
 
var sigma2
 process variance of the GP (extrinsic variance ) More...
 
var tau2
 intrinsic variance More...
 
var Sigma
 intrinsic covariance matrix (amount of regression of stochastic part) More...
 
var Ft
 decorrelated model matrix More...
 
var R
 from QR decomposition of regression part More...
 
var sigma2_reinterp
 Reinterpolation version of sigma2. More...
 
var C_reinterp
 Reinterpolation version of C. More...
 
var Ft_reinterp
 Reinterpolation version of Ft. More...
 
var R_reinterp
 Reinterpolation version of R. More...
 

Detailed Description

A kriging surrogate model (also known as a Gaussian Process)

Papers:

Todo:

Refactor correlation functions into proper basis function class hierarchy.

solve the correlation matrix vs covariance matrix issue

Constructor & Destructor Documentation

function BasicGaussianProcess ( var  varargin)
inline

Class constructor.

Parameters
optionsOptions structure
hyperparameters0Initial theta values
regressionFcnThe type of trend function to use ('regpoly0', 'regpoly1', ...)
correlationFcnFunction handle to the correlation function (@corrgauss, ...)
Returns
instance of the basicGaussianProcess class.

Member Function Documentation

function cleanup ( )
inline

Clears some unused variables.

function correlationFunction ( var  options)

Returns the internal correlation function handle and a symbolic expression of the the correlation part.

Example: [correlationFcn expression] = correlationFunction( this,struct( 'latex', true, 'includeCoefficients', false) )

The expression is based on the scaled data

Parameters
optionsOptions struct
Return values
correlationFcnString of correlation function
expressionSymbolic expression
function cvpe ( )

Calculates the log of the leave-one-out cross validation error (LOO-CV)

Error function used is the Mean Squared Error (MSE)

Return values
outlog(loo-cv) score

Papers:

  • "Blind Kriging: A New Method for Developing Metamodels", V.R. Joseph and Y. Hung and A. Sudjianto, ASME Journal of Mechanical Design, 2008
  • "Predictive Approaches for Choosing Hyperparameters in Gaussian Process", S. Sundararajan, S.S. Keerthu, 1999
function display ( )
inline

Returns user-friendly description of the class instance.

function extrinsicCorrelationMatrix ( var  points1,
var  points2 
)

Constructs extrinsic correlation matrix.

Generate correlation matrix of points1 vs points2 using the current hyperparameters as followed:

Parameters
points1input point matrix (optional)
points2input point matrix (optional)
Return values
psiCorrelation matrix
dpsiDerivative of correlation matrix w.r.t. the hyperparameters
  • extrinsicCorrelationMatrix(this)
    • Assume points1 = points2 = samples
  • extrinsicCorrelationMatrix(this, points1)
    • Assume points2 = samples
  • extrinsicCorrelationMatrix(this, points1, points2)

NOTE: The first case returns derivatives w.r.t. X The latter two cases returns derivatives w.r.t. the hyperparameters

function fit ( var  samples,
var  values 
)

Fits a gaussian process for a dataset.

Need to be invoked before calling any of the prediction methods.

Parameters
samplesinput sample matrix
valuesoutput value matrix
Note
Kriging can't do a cleanup automatically, if needed cleanup() can be called manually.
function generateDegrees ( var  dj)
protected

Generate degrees matrix from individual ones.

Generates a (full) degree matrix based on one or more degree matrices for variable j The option 'regressionMaxLevelInteractions' determines the maximum level of interactions (BasicGaussianProcess::getDefaultOptions)

Parameters
djCell array of degree matrices (for each dimension)
Return values
degreesFull degrees matrix
idxCell array of indices (used by blind kriging)
function getCorrelationMatrix ( )
inline

Returns the full extrinsic correlation matrix.

static function getDefaultOptions ( )
inlinestatic

Returns a default options structure.

Available options:

options = struct( ...
'generateHyperparameters0', false, ...
'hpBounds', [], ... % hyperparameter bounds
'hpOptimizer', [], ... % optimizer class
'hpLikelihood', @marginalLikelihood, ...
'sigma20', NaN, ... % initial value for sigma2
'sigma2Bounds', [0.001 ; 10], ... % sigma2 parameter bounds
'lambda0' ,-Inf, ... % initial lambda values
'lambdaBounds', [-10 ; 0], ... % lambda parameter bounds (in log scale)
'Sigma', [], ... % intrinsic covariance matrix (stochastic kriging)
'reinterpolation', false, ... % reinterpolate error (replaces standard error)
'lowRankApproximation', false, ... % enable low rank approximation of correlation matrix
'rankTol', 1e-12, ... % tolerance for lowRankApprox.
'rankMax', Inf, ... % maximum rank to achieve for lowRankApprox.
'regressionMaxLevelInteractions', 2, ... % consider maximal two-level interactions
'debug', false, ... % enables debug plot of the likelihood function
... % Cokriging specific
'rho0', -Inf, ... % initial scaling factor between datasets
'rhoBounds', [1 ; 5] ... % scaling factor bounds
);
Return values
optionsOptions structure
function getExpression ( var  options)

Returns the Matlab expression of this Gaussian Process model.

Example: expression = getExpression( this )

Parameters
optionsOptions struct
Return values
expressionSymbolic expression
function getHyperparameters ( )
inline

Returns the hyperparameters.

function getOption ( var  key)
inline

Gets a value in the options structure.

Parameters
keyoption name
function getProcessVariance ( )
inline

Returns the process variance (sigma2)

function getRegressionMatrix ( )
inline

Returns the model matrix (Vandermonde matrix)

function getRho ( )
inline

Returns the rho parameter (only valid for CoKriging)

function getSamples ( )
inline

Returns the input sample matrix.

function getSensitivityIndices ( var  model,
var  order,
var  type,
var  method 
)

Calculate sensitivity indices.

Return values
sensitivityIndicesstruct
function getSigma ( )
inline

Returns the intrinsic covariance matrix.

function getValues ( )
inline

Returns the output value matrix.

function imse ( )

Calculates the log of the integrated mean squared error.

Only supported for 1D and 2D problems.

Return values
outlog(imse) score

Papers:

  • See Sacks 1989
  • See pechiny paper...
Todo:
Implement generic monte carlo integration
function intrinsicCovarianceMatrix ( var  points1,
var  points2 
)

Constructs intrinsic covariance matrix (stochastic kriging/regression kriging)

Generate covariance matrix matrix of points1 vs points2 using the current hyperparameters as followed:

Parameters
points1input point matrix (optional)
points2input point matrix (optional)
Return values
psiCovariance matrix
dpsiDerivative of covariance matrix w.r.t. the hyperparameters OR the input points
  • intrinsicCovarianceMatrix(this)
    • Assume points1 = points2 = samples
  • intrinsicCovarianceMatrix(this, points1)
    • Not used (and not implemented yet)
  • intrinsicCovarianceMatrix(this, points1, points2)
    • Not used (and not implemented yet)
function marginalLikelihood ( var  dpsi,
var  dsigma2 
)

Marginal likelihood function.

Used for Maximum Likelihood Estimation (MLE)

Parameters
dpsicell array of derivative matrices (optional; for internal use only)
Return values
outscore
doutDerivatives w.r.t. hyperparameters

Papers:

  • "Gaussian Processes for Machine Learning" (Chapter 5), C. E. Rasmussen and C. K. I. Williams, MIT Press, 2006
  • "An adjoint for likelihood maximization" D.J.J. Toal, A.I.J. Forrester, N.W. Bressloff, A.J. Keane, C. Holden, Proc. of the Royal Society, 2009
Todo:
Adjoint derivatives work, but are very slow due to naive implementation
function mseTestset ( var  testx,
var  testy 
)

Calculates error on holdout set.

Error function used is the Mean Squared Error (MSE) function.

Parameters
testxinput samples of the test set
testyoutput samples of the test set
Return values
outmse error on test set
function plotVariogram ( )

Variogram plot (EXPERIMENTAL)

Plots the experimental (semi-)variogram (based on the data) as well as the theoretical kriging (semi-)variogram (defined by the correlation function).

Return values
hFigure handle

Empirical variogram:

  • Methods of moments estimator Matheron (Cressie, 1993)
  • More robust estimators: Cressie-Hawkins (1980), Genton (1998) In general: use median instead of mean
function predict ( var  points)

Predict the mean and/or variance for one or more points x.

Parameters
pointsMatrix of input points to be predicted
Return values
valuespredicted output values
sigma2predicted variance of the output values (optional)
Precondition
The kriging object should be fitted using a dataset (BasicGaussianProcess::fit)
function predict_derivatives ( var  point)

Predict the derivatives of the mean and/or variance for a points.

NOTE:

Parameters
pointinput point to calculate the derivative of
Return values
dvaluesDerivative w.r.t. the output
sigma2Derivative w.r.t. the output variance (optional)
  • limited to one point at a time (x is a vector)
    Precondition
    The kriging object should be fitted using a dataset (BasicGaussianProcess::fit)
function predict_limit ( var  x)

Limit predictor of kriging (EXPERIMENTAL)

Limited to ordinary kriging (prediction variance sigma2 not supported).

Parameters
pointsMatrix of input points to be predicted
Return values
valuespredicted output values
sigma2predicted variance of the output values (optional)

Cancels out the mean. Should reduce the 'spikyness' of the response surface somewhat.

Papers:

  • "Limit Kriging" V. Roshan Joseph (submitted 2005)
function predictCovarianceMatrix ( var  points1,
var  points2,
var  outputIdx 
)

Predict the full posterior covariance matrix.

The diagonal is the same as the variance of predict

Parameters
pointsMatrix of input points to be predicted
outputIdxthe output to predict for (default is 1)
Return values
sigmaposterior covariance matrix
Precondition
The kriging object should be fitted using a dataset (Kriging::fit)
function pseudoLikelihood ( var  dpsi,
var  dsigma2 
)

Leave-one-out predictive log probability (pseudo-likelihood)

Calculates the leave-one-out predictive log probability.

Parameters
dpsicell array of derivative matrices (optional; for internal use only)
Return values
outscore
doutDerivatives w.r.t. hyperparameters

Papers:

  • "Predictive Approaches for Choosing Hyperparameters in Gaussian Process", S. Sundararajan, S.S. Keerthu, 1999
function rcValues ( )

Quantifies magnification of noise (lower is better)

Robustness-criterion (In theory useful only for ordinary kriging).

Return values
rcrobustness-criterion

Returns a 2xn matrix:

  • the first row contains the absolute robustness
  • the second row contains the relative robustness Papers:
  • "Kriging models that are robust w.r.t. simulation errors" A.Y.D. Siem, D. den Hertog (tech report)
function regressionFunction ( var  options)

Returns the regression function.

Example: [regressionFcn expression terms] = regressionFunction( this, struct('latex', true, 'precision', '%.5g') )

The expression is based on the scaled data

Parameters
optionsOptions struct
Return values
regressionFcnDegree matrix representing the regression function
expressionSymbolic expression
termsCell array of the individual terms
function regressionMatrix ( var  points)
protected

Constructs regression matrix.

Regression matrix (model matrix, Vandermonde matrix, ...) for a set of points Based on this.regressionFcn.

Parameters
pointsinput point matrix (optional)
Return values
FModel matrix
dFDerivative of model matrix w.r.t. the hyperparameters OR the input points
function setData ( var  samples,
var  values 
)
inlineprotected

Sets samples and values matrix.

Parameters
samplesinput sample matrix
valuesoutput value matrix
function setOption ( var  key,
var  value 
)
inline

Sets a value in the options structure.

Parameters
keyoption name
valueoption value
function tuneParameters ( var  F)
protected

Hyperparameter optimization.

Setups and invokes the optimizer.

Parameters
Fmodel matrix
Return values
optimHpoptimized hyperparameters
perfPerformance score (likelihood score)
function updateModel ( var  F,
var  hp 
)
protected

Constructs model.

Full update of the model (regression + correlation part)

Parameters
Fmodel matrix
hpnew hyperparameters
function updateRegression ( var  F,
var  hp 
)
protected

Constructs regression part.

Updates regression part of the model.

Parameters
Fmodel matrix
hpnew hyperparameters
Return values
errerror string (if any)
Todo:
Rho is only used by co-kriging, can we abstract this somehow ?
function updateStochasticProcess ( var  hp)
protected

Constructs correlation part.

Updates correlation part of the model.

Parameters
hphyperparameters
Return values
errerror string (if any)
dpsiDerivative of correlation matrix w.r.t. the hyperparameters

Member Data Documentation

var alpha
protected

Regression coefficients.

var C
protected

Choleski Decomposition of extrinsic + intrinsic matrices.

var C_reinterp
protected

Reinterpolation version of C.

var correlationFcn
protected

string -> function handle

var dist
protected

sample inter-distance

var distIdxPsi
protected

indexing needed to calculate psi from D

var Ft
protected

decorrelated model matrix

var Ft_reinterp
protected

Reinterpolation version of Ft.

var gamma
protected

'Correlation part' coefficients

const var HP

index of the correlation function parameters

var hyperparameters
protected

correlation parameters

var hyperparameters0
protected

initial hp values OR the real ones (if optimization is done outside the class)

const var LAMBDA

index of the lambda parameter

var optimIdx
protected

logical indices to parameters that are optimized

var optimNrParameters
protected

number of optimization parameter (vector; one entry per type of parameter)

var options
protected
var R
protected

from QR decomposition of regression part

var R_reinterp
protected

Reinterpolation version of R.

var regressionFcn
protected

degrees matrix (strings are converted)

const var RHO

index of the rho parameter

var rho
protected
var Sigma
protected

intrinsic covariance matrix (amount of regression of stochastic part)

const var SIGMA2

index of the sigma2 parameter

var sigma2
protected

process variance of the GP (extrinsic variance )

var sigma2_reinterp
protected

Reinterpolation version of sigma2.

Note
reinterpolation: might be nicer to just construct and keep a sub-GP... takes more space, some calculations are done twice but performance shouldn't take a very big hit...
var tau2
protected

intrinsic variance


The documentation for this class was generated from the following files: