The influence of parameter initialization on the training time and accuracy of a nonlinear regression model

E. V. Burnaev, P. D. Erofeev

Research output: Contribution to journalArticlepeer-review

19 Citations (Scopus)

Abstract

In order to construct a nonlinear regression model we have to accurately (in some sense) initialize parameters of the model. In this work we performed comparison of several widely used methods and several newly developed approached for initialization of parameters of a regression model, represented as a decomposition in a linear dictionary of some parametric functions (sigmoids). We proposed a general deterministic approach for initialization, providing repeatability of results, reduction of a learning time and in some cases increase of a regression model accuracy; we developed two new algorithms (based on a piecewise-linear approximation and based on local properties of approximable dependency) in the framework of the proposed approach; we developed randomized initialization algorithm (spherical initialization) for effective approximation of high-dimensional dependencies; we improved the classical initialization method SCAWI (by locating centers of sigmoids in sample points), providing a regression model accuracy improvement on specific classes of dependencies (smooth functions and discontinuous functions with a number of local peculiarities in an input domain) when using RProp algorithm for learning; we performed comparison of classical and newly proposed initialization methods and highlighted the most efficient ones.

Original languageEnglish
Pages (from-to)646-660
Number of pages15
JournalJournal of Communications Technology and Electronics
Volume61
Issue number6
DOIs
Publication statusPublished - 1 Jun 2016
Externally publishedYes

Keywords

  • approximation
  • error back propagation algorithm
  • neural networks
  • nonlinear regression
  • parameter initialization
  • RProp algorithm
  • SCAWI algorithm

Fingerprint

Dive into the research topics of 'The influence of parameter initialization on the training time and accuracy of a nonlinear regression model'. Together they form a unique fingerprint.

Cite this