An improved version of backpropagation algorithm with effective dynamic learning rate and momentum

Mammadov M., Tas E.

WSEAS Transactions on Mathematics, vol.5, no.7, pp.872-877, 2006 (Scopus) identifier

  • Publication Type: Article / Article
  • Volume: 5 Issue: 7
  • Publication Date: 2006
  • Journal Name: WSEAS Transactions on Mathematics
  • Journal Indexes: Scopus
  • Page Numbers: pp.872-877
  • Keywords: Backpropagation algorithm with momentum, Local quadratic approximation, Momentum, Multilayer neural networks, Optimization, Supervised learning
  • Anadolu University Affiliated: Yes

Abstract

An improvement of backpropagation algorithm with momentum is introduced. Local quadratic approximation of the error function is performed at every stage of the learning process and the Hessian matrix of the quadratic error function is approximated [1]. Efficient learning rate and momentum factor is determined at every stage of the learning process by means of maximum and minimum eigenvalues of the Hessian matrix. The effective performance of this new approach is demonstrated on three examples.