# Gaussian process modeling with inequality constraints

Sébastien Da Veiga^{[1]}; Amandine Marrel^{[2]}

- [1] IFP Energies nouvelles 1 & 4 avenue de Bois Préau, 92852 Rueil-Malmaison, France
- [2] CEA, DEN, DER, F-13108 Saint-Paul-lez-Durance, France

Annales de la faculté des sciences de Toulouse Mathématiques (2012)

- Volume: 21, Issue: 3, page 529-555
- ISSN: 0240-2963

## Access Full Article

top## Abstract

top## How to cite

topDa Veiga, Sébastien, and Marrel, Amandine. "Gaussian process modeling with inequality constraints." Annales de la faculté des sciences de Toulouse Mathématiques 21.3 (2012): 529-555. <http://eudml.org/doc/250989>.

@article{DaVeiga2012,

abstract = {Gaussian process modeling is one of the most popular approaches for building a metamodel in the case of expensive numerical simulators. Frequently, the code outputs correspond to physical quantities with a behavior which is known a priori: Chemical concentrations lie between 0 and 1, the output is increasing with respect to some parameter, etc. Several approaches have been proposed to deal with such information. In this paper, we introduce a new framework for incorporating constraints in Gaussian process modeling, including bound, monotonicity and convexity constraints. We also extend this framework to any type of linear constraint. This new methodology mainly relies on conditional expectations of the truncated multinormal distribution. We propose several approximations based on correlation-free assumptions, numerical integration tools and sampling techniques. From a practical point of view, we illustrate how accuracy of Gaussian process predictions can be enhanced with such constraint knowledge. We finally compare all approximate predictors on bound, monotonicity and convexity examples.},

affiliation = {IFP Energies nouvelles 1 & 4 avenue de Bois Préau, 92852 Rueil-Malmaison, France; CEA, DEN, DER, F-13108 Saint-Paul-lez-Durance, France},

author = {Da Veiga, Sébastien, Marrel, Amandine},

journal = {Annales de la faculté des sciences de Toulouse Mathématiques},

language = {eng},

month = {4},

number = {3},

pages = {529-555},

publisher = {Université Paul Sabatier, Toulouse},

title = {Gaussian process modeling with inequality constraints},

url = {http://eudml.org/doc/250989},

volume = {21},

year = {2012},

}

TY - JOUR

AU - Da Veiga, Sébastien

AU - Marrel, Amandine

TI - Gaussian process modeling with inequality constraints

JO - Annales de la faculté des sciences de Toulouse Mathématiques

DA - 2012/4//

PB - Université Paul Sabatier, Toulouse

VL - 21

IS - 3

SP - 529

EP - 555

AB - Gaussian process modeling is one of the most popular approaches for building a metamodel in the case of expensive numerical simulators. Frequently, the code outputs correspond to physical quantities with a behavior which is known a priori: Chemical concentrations lie between 0 and 1, the output is increasing with respect to some parameter, etc. Several approaches have been proposed to deal with such information. In this paper, we introduce a new framework for incorporating constraints in Gaussian process modeling, including bound, monotonicity and convexity constraints. We also extend this framework to any type of linear constraint. This new methodology mainly relies on conditional expectations of the truncated multinormal distribution. We propose several approximations based on correlation-free assumptions, numerical integration tools and sampling techniques. From a practical point of view, we illustrate how accuracy of Gaussian process predictions can be enhanced with such constraint knowledge. We finally compare all approximate predictors on bound, monotonicity and convexity examples.

LA - eng

UR - http://eudml.org/doc/250989

ER -

## References

top- Abrahamsen (P.) and Benth (F.E.).— Kriging with inequality constraints. Mathematical Geology, 33(6), p. 719-744 (2001). Zbl1011.86007MR1956391
- Azaïs (J.-M.) and Wschebor (M.).— Level sets and extrema of random processes and fields. New York: Wiley (2009). Zbl1168.60002MR2478201
- Bigot (J.) and Gadat (S.).— Smoothing under diffeomorphic constraints with homeomorphic splines. SIAM Journal on Numerical Analysis, 48(1), p. 224-243 (2010). Zbl1330.62187MR2608367
- Chopin (N.).— Fast simulation of truncated Gaussian distributions. Statistics and Computing, 21, p. 275-288 (2011). Zbl1284.65015MR2774857
- Cozman (F.) and Krotkov (E.).— Truncated Gaussians as Tolerance Sets. Fifth Workshop on Artificial Intelligence and Statistics, Fort Lauderdale Florida (1995).
- Cramér (H.) and Leadbetter (M.R.).— Stationary and Related Stochastic Processes: Sample Function Properties and Their Applications. New York: Wiley (1967). Zbl0162.21102MR217860
- Da Veiga (S.), Wahl (F.) and Gamboa (F.).— Local Polynomial Estimation for Sensitivity Analysis on Models With Correlated Inputs Technometrics, 51(4), p. 452-463 (2009). MR2756480
- Dette (H.) and Scheder (R.).— Strictly monotone and smooth nonparametric regression for two or more variables. The Canadian Journal of Statistics, 34(44), p. 535-561 (2006). Zbl1115.62039MR2345035
- Ellis (N.) and Maitra (R.).— Multivariate Gaussian Simulation Outside Arbitrary Ellipsoids. Journal of Computational and Graphical Statistics, 16(3), p. 692-798 (2007). MR2351086
- Fernandez (P.J.), Ferrari (P.A.) and Grynberg (S.P.).— Perfectly random sampling of truncated multinormal distributions. Adv. in Appl. Probab., 39(4), p. 973-990 (2007). Zbl1137.65011MR2381584
- Genz (A.).— Numerical Computation of Multivariate Normal Probabilities. J. Comp. Graph Stat., 1, p. 141-149 (1992). Zbl1204.62088
- Genz (A.).— Comparison of Methods for the Computation of Multivariate Normal Probabilities. Computing Science and Statistics, 25, p. 400-405 (1993).
- Genz (A.) and Bretz (F.).— Computation of Multivariate Normal and t Probabilities. Lecture Notes in Statistics, Vol. 195, Springer-Verlag, Heidelberg (2009). Zbl1204.62088MR2840595
- Geweke (J.).— Efficient simulation from the multivariate normal and student t-distribution subject to linear constraints. Computing Science and Statistics: Proceedings of the Twenty-Third Symposium on the Interface, p. 571-578 (1991).
- Ginsbourger (D.), Bay (X.) and Carraro (L.).— Noyaux de covariance pour le Krigeage de fonctions symétriques. submitted to C. R. Acad. Sci. Paris, section Maths (2009).
- Griffiths (W.).— A Gibbs’ sampler for the parameters of a truncated multivariate normal distribution. Working Paper, http://ideas.repec.org/p/mlb/wpaper/856.html (2002). Zbl1084.62045
- Hall (P.) and Huang (L.-S.).— Nonparametric kernel regression subject to monotonicity constraints. The Annals of Statistics, 29(3), p. 624-647 (2001). Zbl1012.62030MR1865334
- Hazelton (M.L.) and Turlach (B.A.).— Semiparametric regression with shape-constrained penalized slpines. Computational Statistics and Data Analysis, 55, p. 2871-2879 (2011). Zbl1218.62032MR2811872
- Horrace (W.C.).— Some results on the multivariate truncated normal distribution. Journal of Multivariate Analysis, 94, p. 209-221 (2005). Zbl1065.62098MR2161218
- Kleijnen (J.P.C.) and van Beers (W.C.M.).— Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations. Working Paper, http://www.tilburguniversity.edu/research/institutes-and-research-groups/center/staff/kleijnen/monotoneKriging.pdf (2010).
- Kotecha (J.H.) and Djuric (P.M.).— Gibbs sampling approach for generation of truncated multivariate gaussian random variables. IEEE Computer Society, p. 1757-1760 (1999).
- Kotz (S.), Balakrishnan (N.) and Johnson (N.L.).— Continuous multivariate distributions, Volume 1: models and applications New York: Wiley (2000). Zbl0946.62001MR1788152
- Lee (L.-F.).— On the first and second moments of the truncated multi-normal distribution and a simple estimator. Economics Letters, 3, p. 165-169 (1979). MR554496
- Lee (L.-F.).— The determination of moments of the doubly truncated multivariate tobit model. Economics Letters, 11, p. 245-250 (1983). Zbl1273.91378
- Marrel (A.), Iooss (B.), Van Dorpe (F.) and Volkova (E.).— An efficient methodology for modeling complex computer codes with Gaussian processes. Computational Statistics and Data Analysis, 52, p. 4731-4744 (2008). Zbl05565053MR2521618
- Michalak (A.M.).— A Gibbs sampler for inequality-constrained geostatistical interpolation and inverse modeling. Water Resour. Res., 44, W09437, doi:10.1029/2007WR006645 (2008).
- Muthén (B.).— Moments of the censored and truncated bivariate normal distribution. British Journal of Mathematical and Statistical Psychology, 43, p. 131-143 (1990). Zbl0718.62031MR1065201
- Oakley (JE.) and O’Hagan (A.).— Probabilistic sensitivity analysis of complex models: A Bayesian approach. Journal of the Royal Statistical Society, Series B, 66, p. 751-769 (2004). Zbl1046.62027MR2088780
- Philippe (A.) and Robert (C.).— Perfect simulation of positive Gaussian distributions. Statistics and Computing, 13(2), p. 179-186 (2003). MR1963334
- Racine (J.S.), Parmeter (C.F.) and Du (P.).— Constrained nonparametric kernel regression: Estimation and inference. Working Paper, http:/economics.ucr.edu/spring09/Racine paper for 5 8 09.pdf (2009).
- Ramsay (J.O.) and Silverman (B.W.).— Functional Data Analysis. Springer Series in Statistics, Springer-Verlag (2005). Zbl1079.62006MR2168993
- Rasmussen (C.E.) and Williams (C.K.I.).— Gaussian Processes for Machine Learning (2006). The MIT Press. Zbl1177.68165MR2514435
- Robert (C.P.).— Simulation of truncated normal variables. Statistics and Computing, 5, p. 121-125 (1995).
- Rodriguez-Yam (G.), Davis (R.A.) and Scharf (L.).— Efficient Gibbs Sampling of Truncated Multivariate Normal with Application to Constrained Linear Regression. Working Paper, http://www.stat.columbia.edu/ rdavis/papers/CLR.pdf (2004).
- Sacks (J.), Welch (W.), Mitchell (T.) and Wynn (H.).— Design and analysis of computer experiments. Statistical Science, 4, p. 409-435 (1989). Zbl0955.62619MR1041765
- Saltelli (A.), Chan (K.) and Scott (E.M.) (Eds.).— Sensitivity Analysis. Wiley (2000). Zbl0961.62091MR1886391
- Santner (T.), Williams (B.) and Notz (W.).— The design and analysis of computer experiments. Springer (2003). Zbl1041.62068MR2160708
- Tallis (G.M.).— The moment generating function of the truncated multinormal distribution. Journal of the Royal Statistical Society, Series B, 23(1), p. 223-229 (1961). Zbl0107.14206MR124077
- Tallis (G.M.).— Elliptical and radial truncation in normal populations. Ann. Math. Statist., 34, p. 940-944 (1963). Zbl0142.16104MR152081
- Tallis (G.M.).— Plane truncation in normal populations. Journal of the Royal Statistical Society, Series B, 27(2), p. 301-307 (1965). Zbl0137.36602MR198522
- Yoo (E.-H.) and Kyriakidis (P.C.).— Area-to-point Kriging with inequality-type data. Journal of Geographical Systems, 8(4), p. 357 (2006).

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.