The UD RLS algorithm for training feedforward neural networks

Jarosław Bilski

International Journal of Applied Mathematics and Computer Science (2005)

  • Volume: 15, Issue: 1, page 115-123
  • ISSN: 1641-876X

Abstract

top
A new algorithm for training feedforward multilayer neural networks is proposed. It is based on recursive least squares procedures and U-D factorization, which is a well-known technique in filter theory. It will be shown that due to the U-D factorization method, our algorithm requires fewer computations than the classical RLS applied to feedforward multilayer neural network training.

How to cite

top

Bilski, Jarosław. "The UD RLS algorithm for training feedforward neural networks." International Journal of Applied Mathematics and Computer Science 15.1 (2005): 115-123. <http://eudml.org/doc/207720>.

@article{Bilski2005,
abstract = {A new algorithm for training feedforward multilayer neural networks is proposed. It is based on recursive least squares procedures and U-D factorization, which is a well-known technique in filter theory. It will be shown that due to the U-D factorization method, our algorithm requires fewer computations than the classical RLS applied to feedforward multilayer neural network training.},
author = {Bilski, Jarosław},
journal = {International Journal of Applied Mathematics and Computer Science},
keywords = {learning algorithms; neural networks; UD factorization; recursive least squares method},
language = {eng},
number = {1},
pages = {115-123},
title = {The UD RLS algorithm for training feedforward neural networks},
url = {http://eudml.org/doc/207720},
volume = {15},
year = {2005},
}

TY - JOUR
AU - Bilski, Jarosław
TI - The UD RLS algorithm for training feedforward neural networks
JO - International Journal of Applied Mathematics and Computer Science
PY - 2005
VL - 15
IS - 1
SP - 115
EP - 123
AB - A new algorithm for training feedforward multilayer neural networks is proposed. It is based on recursive least squares procedures and U-D factorization, which is a well-known technique in filter theory. It will be shown that due to the U-D factorization method, our algorithm requires fewer computations than the classical RLS applied to feedforward multilayer neural network training.
LA - eng
KW - learning algorithms; neural networks; UD factorization; recursive least squares method
UR - http://eudml.org/doc/207720
ER -

References

top
  1. Abid S., Fnaiech F. and Najim M. (2001): A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm. - IEEE Trans. Neural Netw., Vol. 12, No. 2, pp. 424-434. 
  2. Ampazis N. and Perantonis J. (2002): Two highly efficient second-order algorithms for training feedforward networks. - IEEE Trans. Neural Netw., Vol. 13, No. 5, pp. 1064-1074. 
  3. Azimi-Sadjadi M.R. and Liou R.J. (1992): Fast learning process of multi-layer neural network using recursive least squares method. - IEEE Trans. Signal Process., Vol. 40, No. 2, pp. 443-446. 
  4. Bilski J. (1995): Fast learning procedures for neural networks. - Ph.D. Thesis, AGH University of Science and Technology, (in Polish). 
  5. Bilski J. and Rutkowski L. (1996): The recursive least squares method versus the backpropagation learning algorithms. - Second Conf. Neural Networks and Their Applications, Szczyrk, Poland, pp. 25-31. 
  6. Bilski J. and Rutkowski L. (1998): A fast training algorithm for neural networks. - IEEE Trans. Circuits Syst. II, Vol. 45, No. 6, pp. 749-753, 
  7. Bilski J. and Rutkowski L. (2003): A family of the RLS neural network learning algorithms. - Techn. Report, Dept. Comp. Eng., Technical University of Częstochowa, Poland. 
  8. Bishop C.M. (1995): Neural Networks for Pattern Recognition. -Oxford: Clarendon Press. Zbl0868.68096
  9. Bojarczak O.S.P. and Stodolski M. (1996): Fast second-order learning algorithm for feedforward multilayer neural networks and its application. - Neural Netw. Vol. 9, No. 9, pp. 1583-1596. 
  10. Chen X.- H.Y.G.- A. (1992): Efficient backpropagation learning using optimal learning rate and momentum. - Neural Netw., Vol. 10, No. 3, pp. 517-527. 
  11. Joost M. and Schiffmann W. (1998): Speeding up backpropagation algorithms by using cross-entropy combined with pattern normalization. - Int. J.Uncert. Fuzz. Knowledge-Based Syst., Vol. 6, No. 2, pp. 117-126. Zbl1087.68617
  12. Karayiannis N.B. and Venetsanopoulos A.N. (1993): Efficient Learning Algorithms for Neural Networks (ELEANNE). - IEEE Trans. Syst. Man Cybern., Vol. 23, No. 5, pp. 1372-1383. 
  13. Kitano H. (1994): Neurogenetic learning: an integrated method of designing and training neural networks using genetic algorithms. - Physica D., Vol. 75, No. 1-3, pp. 225-238. Zbl0860.68087
  14. Korbicz J., Obuchowicz A., Uciński D. (1994): Artificial Neural Networks. Fundamentals and Applications. - Warsaw: Akademicka Oficyna Wydawnicza PLJ, (in Polish). 
  15. Lera G. and Pinzolas M. (2002): Neighbourhood based Levenberg-Marquardt algorithm for neural network training. - IEEE Trans. Neural Netw., Vol. l3, No. 5, pp. 1200-1203. 
  16. Leung Ch.S., Tsoi Ah.Ch. and Chan L. W. (2001): Two regularizers for recursive least squared algorithms in feedforward multilayered neural networks. - IEEE Trans. Neural Netw., Vol. 12, No. 6, pp. 1314-1332. 
  17. Moller M. (1993): A scaled conjugate gradient algorithm for fast supervised learning. - Neural Netw., Vol. 6, No. 4, pp. 525-533. 
  18. Perantonis S. and Karras D. (1995): An efficient constrained learning algorithm with momentum acceleration. - Neural Netw., Vol. 8, No. 2, pp. 237-249. 
  19. Rutkowski L. (1994): Adaptive Signal Processing: Theory and Applications. -Warsaw: WNT, (in Polish). 
  20. Strobach P. (1990): Linear Prediction Theory - A Mathematical Basis for Adaptive Systems. - New York: Springer-Verlag. Zbl0727.62093
  21. Sum J., Chan L.W., Leung C.S. and Young G. (1998): Extended Kalman filter-based pruning method for recurrent neural networks. - Neural Comput., Vol. 10, No. 6, pp. 1481-1505. 
  22. Sum J., Leung C., Young G.H. and Kan W. (1999): On the Kalman filtering method in neural-network training and pruning. - IEEE Trans. Neural Netw., Vol. 10, No. 1, pp. 161-166. 
  23. Wellstead P.E. and Zarrop M.B. (1991): Self-Tuning Systems Control and Signal Processing. - Chichester Wiley. Zbl0505.93047
  24. Yao X. (1999): Evolving artificial neural networks. - Proc. IEEE, Vol. 87, No. 9, pp. 1423-1447. 
  25. Zhang Y. and Li R. (1999): A fast U-D factorization-based learning algorithm with applications to nonlinear system modelling and identification. - IEEE Trans. Neural Netw., Vol. 10, No. 4, pp. 930-938. 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.