Machine-learning in optimization of expensive black-box functions

Yoel Tenne

International Journal of Applied Mathematics and Computer Science (2017)

  • Volume: 27, Issue: 1, page 105-118
  • ISSN: 1641-876X

Abstract

top
Modern engineering design optimization often uses computer simulations to evaluate candidate designs. For some of these designs the simulation can fail for an unknown reason, which in turn may hamper the optimization process. To handle such scenarios more effectively, this study proposes the integration of classifiers, borrowed from the domain of machine learning, into the optimization process. Several implementations of the proposed approach are described. An extensive set of numerical experiments shows that the proposed approach improves search effectiveness.

How to cite

top

Yoel Tenne. "Machine-learning in optimization of expensive black-box functions." International Journal of Applied Mathematics and Computer Science 27.1 (2017): 105-118. <http://eudml.org/doc/288102>.

@article{YoelTenne2017,
abstract = {Modern engineering design optimization often uses computer simulations to evaluate candidate designs. For some of these designs the simulation can fail for an unknown reason, which in turn may hamper the optimization process. To handle such scenarios more effectively, this study proposes the integration of classifiers, borrowed from the domain of machine learning, into the optimization process. Several implementations of the proposed approach are described. An extensive set of numerical experiments shows that the proposed approach improves search effectiveness.},
author = {Yoel Tenne},
journal = {International Journal of Applied Mathematics and Computer Science},
keywords = {simulations; metamodels; classifiers; machine learning},
language = {eng},
number = {1},
pages = {105-118},
title = {Machine-learning in optimization of expensive black-box functions},
url = {http://eudml.org/doc/288102},
volume = {27},
year = {2017},
}

TY - JOUR
AU - Yoel Tenne
TI - Machine-learning in optimization of expensive black-box functions
JO - International Journal of Applied Mathematics and Computer Science
PY - 2017
VL - 27
IS - 1
SP - 105
EP - 118
AB - Modern engineering design optimization often uses computer simulations to evaluate candidate designs. For some of these designs the simulation can fail for an unknown reason, which in turn may hamper the optimization process. To handle such scenarios more effectively, this study proposes the integration of classifiers, borrowed from the domain of machine learning, into the optimization process. Several implementations of the proposed approach are described. An extensive set of numerical experiments shows that the proposed approach improves search effectiveness.
LA - eng
KW - simulations; metamodels; classifiers; machine learning
UR - http://eudml.org/doc/288102
ER -

References

top
  1. Arlot, S. (2010). A survey of cross-validation procedures for model selection, Statistics Survey 4: 40-79. Zbl1190.62080
  2. Benoudjit, N., Archambeau, C., Lendasse, A., Lee, J. and Verleysen, M. (2002). Width optimization of the Gaussian Kernels in radial basis function networks, Proceedings of the 10th European Symposium on Artificial Neural Networks, ESANN 2002, Bruges, Belgium, pp. 425-432. 
  3. Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V. and Trosset, M.W. (1999). A rigorous framework for optimization of expensive functions by surrogates, Structural Optimization 17(1): 1-13. 
  4. Büche, D., Schraudolph, N.N. and Koumoutsakos, P. (2005). Accelerating evolutionary algorithms with Gaussian process fitness function models, IEEE Transactions on Systems, Man, and Cybernetics C 35(2): 183-194. 
  5. Chipperfield, A., Fleming, P., Pohlheim, H. and Fonseca, C. (1994). Genetic Algorithm TOOLBOX for Use with MATLAB, Version 1.2, Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield. 
  6. Conn, A.R., Gould, N.I.M. and Toint, P.L. (2000). Trust Region Methods, SIAM, Philadelphia, PA. Zbl0958.65071
  7. Conn, A.R., Scheinberg, K. and Toint, P.L. (1998). A derivative free optimization algorithm in practice, Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, USA, Paper No. AIAA-1998-4718. Zbl1042.90617
  8. de Jong, K.A. (2006). Evolutionary Computation: A Unified Approach, MIT Press, Cambridge, MA. Zbl1106.68093
  9. Drela, M. and Youngren, H. (2001). Xfoil 6.9 user primer, Technical report, MIT, Cambridge, MA. 
  10. Emmerich, M.T.M., Giotis, A., Özedmir, M., Bäck, T. and Giannakoglou, K.C. (2002). Metamodel-assisted evolution strategies, in J.J. Merelo Guervós (Ed.), 7th International Conference on Parallel Problem Solving from Nature- PPSN VII, Lecture Notes in Computer Science, Vol. 2439, Springer, Berlin, pp. 361-370. 
  11. Forrester, A.I.J. and Keane, A.J. (2008). Recent advances in surrogate-based optimization, Progress in Aerospace Science 45(1-3): 50-79. 
  12. Golberg, M.A., Chen, C.S. and Karur, S.R. (1996). Improved multiquadric approximation for partial differential equations, Engineering Analysis with Boundary Elements 18(1): 9-17. 
  13. Gorissen, D., De Tommasi, L., Croon, J. and Dhaene, T. (2008). Automatic model type selection with heterogeneous evolution: An application to RF circuit block modeling, Proceedings of the IEEE Congress on Evolutionary Computation (CEC), Hong Kong, China, pp. 989-996. 
  14. Handoko, S., Kwoh, C.K. and Ong, Y.-S. (2010). Feasibility structure modeling: An effective chaperon for constrained memetic algorithms, IEEE Transactions on Evolutionary Computation 14(5): 740-758. 
  15. Hansen, N. and Ostermeier, A. (2001). Completely derandomized self-adaptation in evolution strategies, Evolutionary Computation 9(2): 159-195. 
  16. Hicks, R.M. and Henne, P.A. (1978). Wing design by numerical optimization, Journal of Aircraft 15(7): 407-412. 
  17. Jin, Y., Olhofer, M. and Sendhoff, B. (2002). A framework for evolutionary optimization with approximate fitness functions, IEEE Transactions on Evolutionary Computation 6(5): 481-494. 
  18. Jones, D.R., Schonlau, M. and Welch, W.J. (1998). Efficient global optimization of expensive black-box functions, Journal of Global Optimization 13(4): 455-492. Zbl0917.90270
  19. McKay, M.D., Beckman, R.J. and Conover, W.J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics 21(2): 239-245. Zbl0415.62011
  20. Molinaro, A.M., Simon, R. and Pfeiffer, R.M. (2005). Prediction error estimation: A comparison of resampling methods, Biometrika 21(15): 3301-3307. 
  21. Muller, J. and Shoemaker, C.A. (2014). Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems, Journal of Global Optimization 60(2): 123-144. Zbl1312.90064
  22. Okabe, T. (2007). Stabilizing parallel computation for evolutionary algorithms on real-world applications, Proceedings of the 7th International Conference on Optimization Techniques and Applications (ICOTA 7), Kobe, Japan, pp. 131-132. 
  23. Poloni, C., Giurgevich, A., Onseti, L. and Pediroda, V. (2000). Hybridization of a multi-objective genetic algorithm, a neural network and a classical optimizer for a complex design problem in fluid dynamics, Computer Methods in Applied Mechanics and Engineering 186(2-4): 403-420. Zbl0956.76023
  24. Powell, M.J.D. (2001). Radial basis function methods for interpolation of functions of many variables, Proceedings of the 5th Hellenic-European Conference on Computer Mathematics and Its Applications (HERCMA-01), Athens, Greece, pp. 2-24. Zbl1048.65502
  25. Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidyanathan, R. and Tucker, K.P. (2005). Surrogate-based analysis and optimization, Progress in Aerospace Science 41(1): 1-28. 
  26. Rasheed, K., Hirsh, H. and Gelsey, A. (1997). A genetic algorithm for continuous design space search, Artificial Intelligence in Engineering 11(3): 295-305. 
  27. Ratle, A. (1999). Optimal sampling strategies for learning a fitness model, 1999 IEEE Congress on Evolutionary Computation-CEC 1999, Washington, DC, USA, pp. 2078-2085. 
  28. Regis, R.G. (2014). Particle swarm with radial basis function surrogates for expensive black-box optimization, Journal of Computational Science 5(1): 12-23. 
  29. Regis, R.G. and Shoemaker, C.A. (2013). A quasi-multistart framework for global optimization of expensive functions using response surface models, International Journal of Global Optimization 56(4): 1719-1753. Zbl1275.90068
  30. Sacks, J., Welch, W.J., Mitchell, T.J. and Wynn, H.P. (1989). Design and analysis of computer experiments, Statistical Science 4(4): 409-435. Zbl0955.62619
  31. Sheskin, D.J. (2007). Handbook of Parametric and Nonparametric Statistical Procedures, 4th Edn., Chapman and Hall, Boca Raton, FL. Zbl1118.62001
  32. Smoczek, J. (2013). Evolutionary optimization of interval mathematics-based design of a TSK fuzzy controller for anti-sway crane control, International Journal of Applied Mathematics and Computer Science 23(4): 749-759, DOI: 10.2478/amcs-2013-0056. Zbl1284.93143
  33. Smołka, M., Schaefer, R., Paszyński, M., Pardo, D. and Álvarez Aramberri, J. (2015). An agent-oriented hierarchic strategy for solving inverse problems, International Journal of Applied Mathematics and Computer Science 25(3): 483-498, DOI: 10.1515/amcs-2015-0036. 
  34. Sobieszczanski-Sobieski, J. and Haftka, R.T. (1997). Multidisciplinary aerospace design optimization: Survey of recent developments, Structural Optimization 14(1): 1-23. 
  35. Tenne, Y. (2013). An optimization algorithm employing multiple metamodels and optimizers, International Journal of Automation and Computing 10(3): 227-241. 
  36. Tenne, Y. (2015). An adaptive-topology ensemble algorithm for engineering optimization problems, Optimization and Engineering 16(2): 303-334. 
  37. Tenne, Y. and Armfield, S.W. (2008). A versatile surrogate-assisted memetic algorithm for optimization of computationally expensive functions and its engineering applications, in A. Yang et al. (Eds.), Success in Evolutionary Computation, Studies in Computational Intelligence, Vol. 92, Springer-Verlag, Berlin/Heidelberg, pp. 43-72. 
  38. Tenne, Y. and Goh, C.K. (Eds.) (2010). Computational Intelligence in Expensive Optimization Problems, Springer, Berlin. Zbl1187.90020
  39. Tenne, Y., Izui, K. and Nishiwaki, S. (2010). Handling undefined vectors in expensive optimization problems, in C. Di Chio (Ed.), Proceedings of the 2010 EvoStar Conference, Lecture Notes in Computer Science, Vol. 6024, Springer, Berlin, pp. 582-591. 
  40. Tenne, Y., Izui, K. and Nishiwaki, S. (2011). A classifier-assisted framework for expensive optimization problems: A knowledge-mining approach, in C.A. Coello-Coello (Ed.), Proceedings of the 5th Learning and Intelligent Optimization Conference (LION 5), Lecture Notes in Computer Science, Vol. 6683, Springer, Berlin/Heidelberg, pp. 161-175. 
  41. Viana, F.A.C., Haftka, R.T. and Watson, L.T. (2013). Efficient global optimization algorithm assisted by multiple surrogate technique, Journal of Global Optimization 56(2): 669-689. Zbl1275.90072
  42. Wortmann, T., Costa, A., Nannicini, G. and Schroepfer, T. (2015). Advantages of surrogate models for architectural design optimization, Artificial Intelligence for Engineering Design, Analysis and Manufacturing 29(4): 471-481. 
  43. Wu, H.-Y., Yang, S., Liu, F. and Tsai, H.-M. (2003). Comparison of three geometric representations of airfoils for aerodynamic optimization, Proceedings of the 16th AIAA Computational Fluid Dynamics Conference, Orlando, FL, USA, pp. 1-11, Paper no. AIAA 2003-4095. 
  44. Wu, X., Kumar, V., Quinlan, R.J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.-H., Steinbach, M., Hand, D.J. and Steinberg, D. (2008). Top 10 algorithms in data mining, Knowledge and Information Systems 14(1): 1-37. 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.