Accelerating backpropagation using effective parameters at each step and an experimental evaluation

Mammadov M., Tas E., Omay R. E.

JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION, cilt.78, sa.11, ss.1055-1064, 2008 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 78 Sayı: 11
  • Basım Tarihi: 2008
  • Doi Numarası: 10.1080/00949650701496172
  • Dergi Adı: JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Sayfa Sayıları: ss.1055-1064
  • Anahtar Kelimeler: Multilayer neural networks, Faster learning, Learning rate, Momentum, Local quadratic approximation, QUADRATIC-FUNCTIONS, STEEPEST DESCENT, MOMENTUM TERM
  • Anadolu Üniversitesi Adresli: Evet

Özet

An acceleration of backpropagation algorithm with momentum (BPM) is introduced. At every stage of the learning process, local quadratic approximation of the error function is performed and the Hessian matrix of the quadratic function is approximated. Effective learning rate and momentum factor are determined by means of maximum and minimum eigenvalues of the approximated Hessian matrix at each step. BPM algorithm is modified so as to work automatically with these effective parameters. Performance of this new approach is demonstrated in comparison with well-known training algorithms on conventional problems by an experimental evaluation.