An hybrid training method for B-spline neural networks

New approaches of particle swarm optimisation algorithm based on Gaussian and Cauchy distributions to adjust the control points of B-spline neural networks are proposed. B-spline networks are trained by gradient-based methods, which may fall into local minimum during the learning procedure. To overcome the problems encountered by the conventional learning methods, particle swarm optimisation  a swarm intelligence methodology  can provide a stochastic search for global optimisation of B-spline networks for nonlinear system identification. Simulation results show the potential of the proposed optimisation with particle swarm of B-spline networks for the identification of Rössler system.

Studies in Classification, Data Analysis, and Knowledge Organization, 2003

The success of an Artificial Neural Network (ANN) strongly depends on its training process. Gradient-based techniques have been satisfactorily used in the ANN training. However, in many cases, these algorithms are very slow and susceptible to the local minimum problem. In our work, we implemented a hybrid learning algorithm that integrates Genetic Algorithms(GAs) and the Levenberg-Marquardt(LM) algorithm, a second order gradient-based technique. The GA-LM algorithm was used to train a Time-Delay Neural Network for river flow prediction. In our experiments, the GA-LM hybrid algorithm obtained low prediction errors within a short execution time.

Applied Soft Computing, 2008

B-Spline Neural Network (BSNN), a type of basis function neural network, is trained by gradient-based methods which may fall into local minima during the learning procedure. To overcome the limitations encountered by gradient-based optimization methods, we propose differential evolution (DE) -an evolutionary computation methodology -which can provide a stochastic search to adjust the control points of a BSNN. In this paper, we propose six DE approaches using chaotic sequences based on logistic mapping to train a BSNN. Chaos describes the complex behavior of a nonlinear deterministic system. The application of chaotic sequences instead of random sequences in DE is a powerful strategy to diversify the DE population and improve the DE's performance in preventing premature convergence to local minima. The numerical results presented here indicate that chaotic DE was effective for building a good BSNN model for the nonlinear identification of an experimental nonlinear yo-yo motion control system. # 2007 Published by Elsevier B.V.

IFAC Proceedings Volumes (IFAC-PapersOnline), 2012

When used for function approximation purposes, neural networks belong to a class of models whose parameters can be separated into linear and nonlinear, according to their influence in the model output. This concept of parameter separability can also be applied when the training problem is formulated as the minimization of the integral of the (functional) squared error, over the input domain. Using this approach, the computation of the gradient involves terms that are dependent only on the model and the input domain, and terms which are the projection of the target function on the basis functions and on their derivatives with respect to the nonlinear parameters, over the input domain. These later terms can be numerically computed with the data.

IFAC Proceedings Volumes, 2001

Complete supervised training algorithms for B-spline neurJJ networks and fuzzy rule-based systems are discu.~•;ed. By introducing the relationships between B-spline neural networks and certain types of l'uzzy models, training algorithms developed initially for neural networks can be adapted to fuzzy systems. The standard training criterion is reformulated, by separating its parameters into linear and non linear ones. By employing this reformulation with the Levenberg-Marquardt algorithm, a new training method. olfering a last rate of convergence, and a robust perlormance compared with the standard Error-Back Propagation algorithm. Copyright t) 20021FAC

Neural Networks, 1998

In this paper, we study the theoretical properties of a new kind of artificial neural network, which is able to adapt its activation functions by varying the control points of a Catmull-Rom cubic spline. Most of all, we are interested in generalization capability, and we can show that our architecture presents several advantages. First of all, it can be seen as a sub-optimal realization of the additive spline based model obtained by the reguralization theory. Besides, simulations confirm that the special learning mechanism allows to use in a very effective way the network's free parameters, keeping their total number at lower values than in networks with sigmoidal activation functions. Other notable properties are a shorter training time and a reduced hardware complexity, due to the surplus in the number of neurons. ᭧

The 2012 International Joint Conference on Neural Networks (IJCNN), 2012

When used for function approximation purposes, neural networks belong to a class of models whose parameters can be separated into linear and nonlinear, according to their influence in the model output. This concept of parameter separability can also be applied when the training problem is formulated as the minimization of the integral of the (functional) squared error, over the input domain. Using this approach, the computation of the gradient involves terms that are dependent only on the model and the input domain, and terms which are the projection of the target function on the basis functions and on their derivatives with respect to the nonlinear parameters, over the input domain. This paper extends the application of this formulation to B-splines, describing how the Levenberg-

1996

This paper describes two algorithms based on cooperative evolution of internal hidden network representations and a combination of global evolutionary and local search procedures. The obtained experimental results are better in comparison with prototype methods. It is demonstrated, that the applications of pure gradient or pure genetic algorithms to the network training problem is much worse than hybrid procedures, which reasonably combine the advantages of global as well as local search.

The Levenberg-Marquardt (LM) gradient descent algorithm is used extensively for the training of Artificial Neural Networks (ANN) in the literature, despite its limitations, such as susceptibility to the local minima that undermine its robustness. In this paper, a bio-inspired algorithm referring to the Bat algorithm was proposed for training the ANN, to deviate from the limitations of the LM. The proposed Bat algorithm-based LM (BALM) was simulated on 10 benchmark datasets. For evaluation of the proposed BALM, comparative simulation experiments were conducted. The experimental results indicated that the BALM was found to deviate from the limitations of the LM to advance the accuracy and convergence speed of the ANN. Also, the BALM performs better than the back-propagation algorithm, artificial bee colony trained back-propagation ANN, and artificial bee colony trained LM ANN. The results of this research provide an alternative ANN training algorithm that can be used by researchers and industries to solve complex real-world problems across numerous domains of applications.

Int. Arab J. Inf. Technol., 2018

Levenberg-Marquardt back-propagation algorithm, as a Feed forward Neural Network (FNN) training method, has some limitations associated with over fitting and local optimum problems. Also Levenberg-Marquardt back-propagation algorithm is opted only for small network. This research uses hybrid evolutionary algorithm based on Particle Swarm Optimization (PSO) in FNN training. This algorithm includes a number of components that gives advantage in the experimental study. Variants such as size of the swarm, acceleration coefficients, coefficient constriction factor and velocity of the swarm are proposed to improve convergence speed as well as to improve accuracy. The integration of components in different ways in hybrid algorithm produces effective optimization of back propagation algorithm. Also, this hybrid evolutionary algorithm based on PSO can be used for complex neural network structure.