Improved Nelder-Mead Optimization Method in Learning Phase of Artificial Neural Network
AbstractIt is difficult to find the optimum weight values of artificial neural networks for optimization problem. In this study, Nelder-Mead optimization method  has been improved and used for determining the optimal values of weights. The results of the proposed improved Nelder-Mead method are compared with results of the standard Nelder-Mead method which is used in ANNs learning algorithm. The most common data sets are taken from UCI machine learning repository. According to the experimental results, in this study better results are achieved in terms of speed and performance.
APA Kecman, V. (2001). Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. MIT press.
Alhamdoosh, M., & Wang, D. (2014). Fast decorrelated neural network ensembles with random weights. Information Sciences, 264, 104-117.
Slowik, A. (2011). Application of an adaptive differential evolution algorithm with multiple trial vectors to artificial neural network training. IEEE Transactions on Industrial Electronics, 58(8), 3160-3167.
Mohaghegi, S., del Valle, Y., Venayagamoorthy, G. K., & Harley, R. G. (2005, June). A comparison of PSO and backpropagation for training RBF neural networks for identification of a power system with STATCOM. In Swarm Intelligence Symposium, 2005. SIS 2005. Proceedings 2005 IEEE (pp. 381-384).
Montana, D. J., & Davis, L. (1989, August). Training Feedforward Neural Networks Using Genetic Algorithms. In IJCAI (Vol. 89, pp. 762-767).
Malinak, P., &Jaksa, R. (2007, September). Simultaneous gradient and evolutionary neural network weights adaptation methods. In Evolutionary Computation, 2007. CEC 2007. IEEE Congress on (pp. 2665-2671). IEEE.
Kennedy, J. (2011). Particle swarm optimization. In Encyclopedia of machine learning (pp. 760-766). Springer US.
Carvalho, M., &Ludermir, T. B. (2007, September). Particle swarm optimization of neural network architectures and weights. In Hybrid Intelligent Systems, 2007. HIS 2007. 7th International Conference on (pp. 336-339). IEEE.
Dorigo, M., Birattari, M., &Stutzle, T. (2006). Ant colony optimization. IEEE computational intelligence magazine, 1(4), 28-39.
Hu, Liang, et al. "Optimization of neural network by genetic algorithm for flowrate determination in multipath ultrasonic gas flowmeter." IEEE Sensors Journal 16.5 (2016): 1158-1167.
Kapanova, K. G., Dimov, I., &Sellier, J. M. A genetic approach to automatic neural network architecture optimization. Neural Computing and Applications, 1-12.
Han, J., Pei, J., &Kamber, M. (2011). Data mining: concepts and techniques. Elsevier.
Schalkoff, R. J. (1997). Artificial neural networks (Vol. 1). New York: McGraw-Hill.
Kelley, C. T. (1999). Detection and remediation of stagnation in the nelder-mead algorithm using a sufficient decrease condition. SIAM journal on optimization, 10(1), 43-55.
Lagarias J. C., J. A. Reeds, M. H. Wright, and P. E. Wright, 1998. Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM Journal on Optimization, 9(1), p. 112- 147.
Asuncion, A., & Newman, D. (2007). UCI machine learning repository.
Nelder, John A., R. Mead (1965). A simplex method for function minimization. Computer Journal. 7: 308–313.
Copyright (c) 2018 International Journal of Intelligent Systems and Applications in Engineering
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.