Gaussian process modeling with inequality constraints

Sébastien Da Veiga[1]; Amandine Marrel[2]

  • [1] IFP Energies nouvelles 1 & 4 avenue de Bois Préau, 92852 Rueil-Malmaison, France
  • [2] CEA, DEN, DER, F-13108 Saint-Paul-lez-Durance, France

Annales de la faculté des sciences de Toulouse Mathématiques (2012)

  • Volume: 21, Issue: 3, page 529-555
  • ISSN: 0240-2963

Abstract

top
Gaussian process modeling is one of the most popular approaches for building a metamodel in the case of expensive numerical simulators. Frequently, the code outputs correspond to physical quantities with a behavior which is known a priori: Chemical concentrations lie between 0 and 1, the output is increasing with respect to some parameter, etc. Several approaches have been proposed to deal with such information. In this paper, we introduce a new framework for incorporating constraints in Gaussian process modeling, including bound, monotonicity and convexity constraints. We also extend this framework to any type of linear constraint. This new methodology mainly relies on conditional expectations of the truncated multinormal distribution. We propose several approximations based on correlation-free assumptions, numerical integration tools and sampling techniques. From a practical point of view, we illustrate how accuracy of Gaussian process predictions can be enhanced with such constraint knowledge. We finally compare all approximate predictors on bound, monotonicity and convexity examples.

How to cite

top

Da Veiga, Sébastien, and Marrel, Amandine. "Gaussian process modeling with inequality constraints." Annales de la faculté des sciences de Toulouse Mathématiques 21.3 (2012): 529-555. <http://eudml.org/doc/250989>.

@article{DaVeiga2012,
abstract = {Gaussian process modeling is one of the most popular approaches for building a metamodel in the case of expensive numerical simulators. Frequently, the code outputs correspond to physical quantities with a behavior which is known a priori: Chemical concentrations lie between 0 and 1, the output is increasing with respect to some parameter, etc. Several approaches have been proposed to deal with such information. In this paper, we introduce a new framework for incorporating constraints in Gaussian process modeling, including bound, monotonicity and convexity constraints. We also extend this framework to any type of linear constraint. This new methodology mainly relies on conditional expectations of the truncated multinormal distribution. We propose several approximations based on correlation-free assumptions, numerical integration tools and sampling techniques. From a practical point of view, we illustrate how accuracy of Gaussian process predictions can be enhanced with such constraint knowledge. We finally compare all approximate predictors on bound, monotonicity and convexity examples.},
affiliation = {IFP Energies nouvelles 1 & 4 avenue de Bois Préau, 92852 Rueil-Malmaison, France; CEA, DEN, DER, F-13108 Saint-Paul-lez-Durance, France},
author = {Da Veiga, Sébastien, Marrel, Amandine},
journal = {Annales de la faculté des sciences de Toulouse Mathématiques},
language = {eng},
month = {4},
number = {3},
pages = {529-555},
publisher = {Université Paul Sabatier, Toulouse},
title = {Gaussian process modeling with inequality constraints},
url = {http://eudml.org/doc/250989},
volume = {21},
year = {2012},
}

TY - JOUR
AU - Da Veiga, Sébastien
AU - Marrel, Amandine
TI - Gaussian process modeling with inequality constraints
JO - Annales de la faculté des sciences de Toulouse Mathématiques
DA - 2012/4//
PB - Université Paul Sabatier, Toulouse
VL - 21
IS - 3
SP - 529
EP - 555
AB - Gaussian process modeling is one of the most popular approaches for building a metamodel in the case of expensive numerical simulators. Frequently, the code outputs correspond to physical quantities with a behavior which is known a priori: Chemical concentrations lie between 0 and 1, the output is increasing with respect to some parameter, etc. Several approaches have been proposed to deal with such information. In this paper, we introduce a new framework for incorporating constraints in Gaussian process modeling, including bound, monotonicity and convexity constraints. We also extend this framework to any type of linear constraint. This new methodology mainly relies on conditional expectations of the truncated multinormal distribution. We propose several approximations based on correlation-free assumptions, numerical integration tools and sampling techniques. From a practical point of view, we illustrate how accuracy of Gaussian process predictions can be enhanced with such constraint knowledge. We finally compare all approximate predictors on bound, monotonicity and convexity examples.
LA - eng
UR - http://eudml.org/doc/250989
ER -

References

top
  1. Abrahamsen (P.) and Benth (F.E.).— Kriging with inequality constraints. Mathematical Geology, 33(6), p. 719-744 (2001). Zbl1011.86007MR1956391
  2. Azaïs (J.-M.) and Wschebor (M.).— Level sets and extrema of random processes and fields. New York: Wiley (2009). Zbl1168.60002MR2478201
  3. Bigot (J.) and Gadat (S.).— Smoothing under diffeomorphic constraints with homeomorphic splines. SIAM Journal on Numerical Analysis, 48(1), p. 224-243 (2010). Zbl1330.62187MR2608367
  4. Chopin (N.).— Fast simulation of truncated Gaussian distributions. Statistics and Computing, 21, p. 275-288 (2011). Zbl1284.65015MR2774857
  5. Cozman (F.) and Krotkov (E.).— Truncated Gaussians as Tolerance Sets. Fifth Workshop on Artificial Intelligence and Statistics, Fort Lauderdale Florida (1995). 
  6. Cramér (H.) and Leadbetter (M.R.).— Stationary and Related Stochastic Processes: Sample Function Properties and Their Applications. New York: Wiley (1967). Zbl0162.21102MR217860
  7. Da Veiga (S.), Wahl (F.) and Gamboa (F.).— Local Polynomial Estimation for Sensitivity Analysis on Models With Correlated Inputs Technometrics, 51(4), p. 452-463 (2009). MR2756480
  8. Dette (H.) and Scheder (R.).— Strictly monotone and smooth nonparametric regression for two or more variables. The Canadian Journal of Statistics, 34(44), p. 535-561 (2006). Zbl1115.62039MR2345035
  9. Ellis (N.) and Maitra (R.).— Multivariate Gaussian Simulation Outside Arbitrary Ellipsoids. Journal of Computational and Graphical Statistics, 16(3), p. 692-798 (2007). MR2351086
  10. Fernandez (P.J.), Ferrari (P.A.) and Grynberg (S.P.).— Perfectly random sampling of truncated multinormal distributions. Adv. in Appl. Probab., 39(4), p. 973-990 (2007). Zbl1137.65011MR2381584
  11. Genz (A.).— Numerical Computation of Multivariate Normal Probabilities. J. Comp. Graph Stat., 1, p. 141-149 (1992). Zbl1204.62088
  12. Genz (A.).— Comparison of Methods for the Computation of Multivariate Normal Probabilities. Computing Science and Statistics, 25, p. 400-405 (1993). 
  13. Genz (A.) and Bretz (F.).— Computation of Multivariate Normal and t Probabilities. Lecture Notes in Statistics, Vol. 195, Springer-Verlag, Heidelberg (2009). Zbl1204.62088MR2840595
  14. Geweke (J.).— Efficient simulation from the multivariate normal and student t-distribution subject to linear constraints. Computing Science and Statistics: Proceedings of the Twenty-Third Symposium on the Interface, p. 571-578 (1991). 
  15. Ginsbourger (D.), Bay (X.) and Carraro (L.).— Noyaux de covariance pour le Krigeage de fonctions symétriques. submitted to C. R. Acad. Sci. Paris, section Maths (2009). 
  16. Griffiths (W.).— A Gibbs’ sampler for the parameters of a truncated multivariate normal distribution. Working Paper, http://ideas.repec.org/p/mlb/wpaper/856.html (2002). Zbl1084.62045
  17. Hall (P.) and Huang (L.-S.).— Nonparametric kernel regression subject to monotonicity constraints. The Annals of Statistics, 29(3), p. 624-647 (2001). Zbl1012.62030MR1865334
  18. Hazelton (M.L.) and Turlach (B.A.).— Semiparametric regression with shape-constrained penalized slpines. Computational Statistics and Data Analysis, 55, p. 2871-2879 (2011). Zbl1218.62032MR2811872
  19. Horrace (W.C.).— Some results on the multivariate truncated normal distribution. Journal of Multivariate Analysis, 94, p. 209-221 (2005). Zbl1065.62098MR2161218
  20. Kleijnen (J.P.C.) and van Beers (W.C.M.).— Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations. Working Paper, http://www.tilburguniversity.edu/research/institutes-and-research-groups/center/staff/kleijnen/monotoneKriging.pdf (2010). 
  21. Kotecha (J.H.) and Djuric (P.M.).— Gibbs sampling approach for generation of truncated multivariate gaussian random variables. IEEE Computer Society, p. 1757-1760 (1999). 
  22. Kotz (S.), Balakrishnan (N.) and Johnson (N.L.).— Continuous multivariate distributions, Volume 1: models and applications New York: Wiley (2000). Zbl0946.62001MR1788152
  23. Lee (L.-F.).— On the first and second moments of the truncated multi-normal distribution and a simple estimator. Economics Letters, 3, p. 165-169 (1979). MR554496
  24. Lee (L.-F.).— The determination of moments of the doubly truncated multivariate tobit model. Economics Letters, 11, p. 245-250 (1983). Zbl1273.91378
  25. Marrel (A.), Iooss (B.), Van Dorpe (F.) and Volkova (E.).— An efficient methodology for modeling complex computer codes with Gaussian processes. Computational Statistics and Data Analysis, 52, p. 4731-4744 (2008). Zbl05565053MR2521618
  26. Michalak (A.M.).— A Gibbs sampler for inequality-constrained geostatistical interpolation and inverse modeling. Water Resour. Res., 44, W09437, doi:10.1029/2007WR006645 (2008). 
  27. Muthén (B.).— Moments of the censored and truncated bivariate normal distribution. British Journal of Mathematical and Statistical Psychology, 43, p. 131-143 (1990). Zbl0718.62031MR1065201
  28. Oakley (JE.) and O’Hagan (A.).— Probabilistic sensitivity analysis of complex models: A Bayesian approach. Journal of the Royal Statistical Society, Series B, 66, p. 751-769 (2004). Zbl1046.62027MR2088780
  29. Philippe (A.) and Robert (C.).— Perfect simulation of positive Gaussian distributions. Statistics and Computing, 13(2), p. 179-186 (2003). MR1963334
  30. Racine (J.S.), Parmeter (C.F.) and Du (P.).— Constrained nonparametric kernel regression: Estimation and inference. Working Paper, http:/economics.ucr.edu/spring09/Racine paper for 5 8 09.pdf (2009). 
  31. Ramsay (J.O.) and Silverman (B.W.).— Functional Data Analysis. Springer Series in Statistics, Springer-Verlag (2005). Zbl1079.62006MR2168993
  32. Rasmussen (C.E.) and Williams (C.K.I.).— Gaussian Processes for Machine Learning (2006). The MIT Press. Zbl1177.68165MR2514435
  33. Robert (C.P.).— Simulation of truncated normal variables. Statistics and Computing, 5, p. 121-125 (1995). 
  34. Rodriguez-Yam (G.), Davis (R.A.) and Scharf (L.).— Efficient Gibbs Sampling of Truncated Multivariate Normal with Application to Constrained Linear Regression. Working Paper, http://www.stat.columbia.edu/ rdavis/papers/CLR.pdf (2004). 
  35. Sacks (J.), Welch (W.), Mitchell (T.) and Wynn (H.).— Design and analysis of computer experiments. Statistical Science, 4, p. 409-435 (1989). Zbl0955.62619MR1041765
  36. Saltelli (A.), Chan (K.) and Scott (E.M.) (Eds.).— Sensitivity Analysis. Wiley (2000). Zbl0961.62091MR1886391
  37. Santner (T.), Williams (B.) and Notz (W.).— The design and analysis of computer experiments. Springer (2003). Zbl1041.62068MR2160708
  38. Tallis (G.M.).— The moment generating function of the truncated multinormal distribution. Journal of the Royal Statistical Society, Series B, 23(1), p. 223-229 (1961). Zbl0107.14206MR124077
  39. Tallis (G.M.).— Elliptical and radial truncation in normal populations. Ann. Math. Statist., 34, p. 940-944 (1963). Zbl0142.16104MR152081
  40. Tallis (G.M.).— Plane truncation in normal populations. Journal of the Royal Statistical Society, Series B, 27(2), p. 301-307 (1965). Zbl0137.36602MR198522
  41. Yoo (E.-H.) and Kyriakidis (P.C.).— Area-to-point Kriging with inequality-type data. Journal of Geographical Systems, 8(4), p. 357 (2006). 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.