JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION, vol.78, no.11, pp.1055-1064, 2008 (SCI-Expanded)
An acceleration of backpropagation algorithm with momentum (BPM) is introduced. At every stage of the learning process, local quadratic approximation of the error function is performed and the Hessian matrix of the quadratic function is approximated. Effective learning rate and momentum factor are determined by means of maximum and minimum eigenvalues of the approximated Hessian matrix at each step. BPM algorithm is modified so as to work automatically with these effective parameters. Performance of this new approach is demonstrated in comparison with well-known training algorithms on conventional problems by an experimental evaluation.