Improving the generalization ability of neuro-fuzzy systems by ε-insensitive learning

Jacek Łęski

International Journal of Applied Mathematics and Computer Science (2002)

  • Volume: 12, Issue: 3, page 437-447
  • ISSN: 1641-876X

Abstract

top
A new learning method tolerant of imprecision is introduced and used in neuro-fuzzy modelling. The proposed method makes it possible to dispose of an intrinsic inconsistency of neuro-fuzzy modelling, where zero-tolerance learning is used to obtain a fuzzy model tolerant of imprecision. This new method can be called ε-insensitive learning, where, in order to fit the fuzzy model to real data, the ε-insensitive loss function is used. ε-insensitive learning leads to a model with minimal Vapnik-Chervonenkis dimension, which results in an improved generalization ability of this system. Another advantage of the proposed method is its robustness against outliers. This paper introduces two approaches to solving ε-insensitive learning problem. The first approach leads to a quadratic programming problem with bound constraints and one linear equality constraint. The second approach leads to a problem of solving a system of linear inequalities. Two computationally efficient numerical methods for ε-insensitive learning are proposed. Finally, examples are given to demonstrate the validity of the introduced methods.

How to cite

top

Łęski, Jacek. "Improving the generalization ability of neuro-fuzzy systems by ε-insensitive learning." International Journal of Applied Mathematics and Computer Science 12.3 (2002): 437-447. <http://eudml.org/doc/207600>.

@article{Łęski2002,
abstract = {A new learning method tolerant of imprecision is introduced and used in neuro-fuzzy modelling. The proposed method makes it possible to dispose of an intrinsic inconsistency of neuro-fuzzy modelling, where zero-tolerance learning is used to obtain a fuzzy model tolerant of imprecision. This new method can be called ε-insensitive learning, where, in order to fit the fuzzy model to real data, the ε-insensitive loss function is used. ε-insensitive learning leads to a model with minimal Vapnik-Chervonenkis dimension, which results in an improved generalization ability of this system. Another advantage of the proposed method is its robustness against outliers. This paper introduces two approaches to solving ε-insensitive learning problem. The first approach leads to a quadratic programming problem with bound constraints and one linear equality constraint. The second approach leads to a problem of solving a system of linear inequalities. Two computationally efficient numerical methods for ε-insensitive learning are proposed. Finally, examples are given to demonstrate the validity of the introduced methods.},
author = {Łęski, Jacek},
journal = {International Journal of Applied Mathematics and Computer Science},
keywords = {neural networks; robust methods; fuzzy systems; tolerant learning; generalization control},
language = {eng},
number = {3},
pages = {437-447},
title = {Improving the generalization ability of neuro-fuzzy systems by ε-insensitive learning},
url = {http://eudml.org/doc/207600},
volume = {12},
year = {2002},
}

TY - JOUR
AU - Łęski, Jacek
TI - Improving the generalization ability of neuro-fuzzy systems by ε-insensitive learning
JO - International Journal of Applied Mathematics and Computer Science
PY - 2002
VL - 12
IS - 3
SP - 437
EP - 447
AB - A new learning method tolerant of imprecision is introduced and used in neuro-fuzzy modelling. The proposed method makes it possible to dispose of an intrinsic inconsistency of neuro-fuzzy modelling, where zero-tolerance learning is used to obtain a fuzzy model tolerant of imprecision. This new method can be called ε-insensitive learning, where, in order to fit the fuzzy model to real data, the ε-insensitive loss function is used. ε-insensitive learning leads to a model with minimal Vapnik-Chervonenkis dimension, which results in an improved generalization ability of this system. Another advantage of the proposed method is its robustness against outliers. This paper introduces two approaches to solving ε-insensitive learning problem. The first approach leads to a quadratic programming problem with bound constraints and one linear equality constraint. The second approach leads to a problem of solving a system of linear inequalities. Two computationally efficient numerical methods for ε-insensitive learning are proposed. Finally, examples are given to demonstrate the validity of the introduced methods.
LA - eng
KW - neural networks; robust methods; fuzzy systems; tolerant learning; generalization control
UR - http://eudml.org/doc/207600
ER -

References

top
  1. Bezdek J.C. (1982): Pattern Recognition with Fuzzy Objective Function Algorithms. - New York: Plenum Press. Zbl0503.68069
  2. Box G.E.P. and Jenkins G.M. (1976): Time Series Analysis. Forecasting and Control. - San Francisco: Holden-Day. Zbl0363.62069
  3. Cauwenberghs G. and Poggio T. (2001): Incremental and decremental support vector machine learning. - Proc. IEEE Neural Information Processing Systems Conference, Cambridge MA: MIT Press, Vol. 13, pp. 175-181. 
  4. Chen J.-Q., Xi Y.-G. and Zhang Z.-J. (1998): Aclustering algorithm for fuzzy model identification. - Fuzzy Sets Syst., Vol. 98, No. 3, pp. 319-329. 
  5. Czogała E. and Łęski J. (2001): Fuzzyand Neuro-Fuzzy Intelligent Systems. - Heidelberg: Physica-Verlag, Springer-Verlag Comp. Zbl0953.68122
  6. Gantmacher F.R. (1959): The Theory of Matrices. -New York: Chelsea Publ. Zbl0085.01001
  7. Haykin S. (1999): Neural Networks. A Comprehensive Foundation. - Upper Saddle River: Prentice-Hall. Zbl0934.68076
  8. Ho Y.-C. and Kashyap R.L. (1965): An algorithm for linear inequalities and its applications. - IEEE Trans. Elec.Comp., Vol. 14, No. 5, pp. 683-688. Zbl0173.17902
  9. Ho Y.-C. and Kashyap R.L. (1966): A class of iterative procedures for linear inequalities. - SIAM J. Contr., Vol. 4, No. 2, pp. 112-115. Zbl0143.37503
  10. Huber P.J. (1981): Robust Statistics. - New York: Wiley. Zbl0536.62025
  11. Jang J.-S.R., Sun C.-T. and Mizutani E. (1997): Neuro-Fuzzy and Soft Computing. A Computational Approach to Learning and Machine Intelligence. - Upper Saddle River: Prentice-Hall. 
  12. Joachims T. (1999): Making large-scale support vector machine learning practical, In: Advances in Kernel Methods-SupportVector Learning (B. Scholkopf, J.C. Burges and A.J. Smola, Eds.). - NewYork: MIT Press. 
  13. Łęski J. (2001): An ε-insensitive approach to fuzzy clustering. - Int. J. Appl. Math. Comp. Sci., Vol. 11, No. 4, pp. 993-1007. Zbl1004.94043
  14. Osuna E., Freund R. and Girosi F. (1997): An improved training algorithm for support vector machines. - Proc. IEEE Workshop Neural Networks for Signal Processing, Breckenridge, Colorado, pp. 276-285. 
  15. Pedrycz W. (1984): An identification algorithmin fuzzy relational systems. - Fuzzy Sets Syst., Vol. 13, No. 1, pp. 153-167. Zbl0554.93070
  16. Platt J. (1999): Sequential minimal optimization: A fast algorithm for training support vector machines, In: Advances in Kernel Methods-Support Vector Learning (B. Scholkopf, J.C. Burges and A.J. Smola, Eds.). - New York: MIT Press. 
  17. Rutkowska D. (2001): Neuro-Fuzzy Architectures and Hybrid Learning. - Heidelberg: Physica-Verlag, Springer-Verlag Comp. Zbl1005.68127
  18. Rutkowska D. and Hayashi Y. (1999): Neuro-fuzzysystems approaches. - Int. J. Adv. Comp.Intell., Vol. 3, No. 3, pp. 177-185. 
  19. Rutkowska D. and Nowicki R. (2000): Implication-based neuro-fuzzy architectures. - Int. J. Appl. Math. Comp. Sci., Vol. 10, No. 4, pp. 675-701. Zbl0972.68134
  20. Setnes M. (2000): Supervised fuzzy clustering forrule extraction. - IEEE Trans. Fuzzy Syst., Vol. 8, No. 4, pp. 416-424. 
  21. Sugeno M. and Kang G.T. (1988): Structure identification of fuzzy model. - Fuzzy Sets Syst., Vol. 28, No. 1, pp. 15-33. Zbl0652.93010
  22. Takagi H. and Sugeno M. (1985): Fuzzy identification of systems and its application to modeling and control. -IEEE Trans. Syst. Man Cybern., Vol. 15, No. 1, pp. 116-132. Zbl0576.93021
  23. Vapnik V. (1998): Statistical Learning Theory. -New York: Wiley. Zbl0935.62007
  24. Vapnik V. (1999): An overview of statistical learning theory. - IEEE Trans. Neural Netw., Vol. 10, No. 5, pp. 988-999. 
  25. Wang L.-X. (1998): A Course in Fuzzy Systems and Control. - New York: Prentice-Hall. 
  26. Weigend A.S., Huberman B.A. and Rumelhart D.E. (1990): Predicting the future: A connectionist approach. - Int. J. Neural Syst., Vol. 1, No. 2, pp. 193-209. 
  27. Yen J., Wang L. and Gillespie C.W. (1998): Improving the interpretability of TSK fuzzy models by combining global learning and local learning. - IEEE Trans. Fuzzy Syst., Vol. 6, No. 4, pp. 530-537. 
  28. Zadeh L.A. (1964): Fuzzy sets. - Inf. Contr., Vol. 8, No. 4, pp. 338-353. Zbl0139.24606
  29. Zadeh L.A. (1973): Outline of a new approach to the analysis of complex systems and decision processes. - IEEE Trans. Syst. Man Cybern., Vol. 3, No. 1, pp. 28-44. Zbl0273.93002

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.