An improved version of backpropagation algorithm with effective dynamic learning rate and momentum

Mammadov M., Tas E.

WSEAS Transactions on Mathematics, cilt.5, sa.7, ss.872-877, 2006 (Scopus) identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 5 Sayı: 7
  • Basım Tarihi: 2006
  • Dergi Adı: WSEAS Transactions on Mathematics
  • Derginin Tarandığı İndeksler: Scopus
  • Sayfa Sayıları: ss.872-877
  • Anahtar Kelimeler: Backpropagation algorithm with momentum, Local quadratic approximation, Momentum, Multilayer neural networks, Optimization, Supervised learning
  • Anadolu Üniversitesi Adresli: Evet

Özet

An improvement of backpropagation algorithm with momentum is introduced. Local quadratic approximation of the error function is performed at every stage of the learning process and the Hessian matrix of the quadratic error function is approximated [1]. Efficient learning rate and momentum factor is determined at every stage of the learning process by means of maximum and minimum eigenvalues of the Hessian matrix. The effective performance of this new approach is demonstrated on three examples.