Biogeography-Based Optimization for Weight Optimization in Elman Neural Network Compared with Meta-Heuristics Methods
DOI:
https://doi.org/10.18662/brain/11.2/76Keywords:
Biogeography-Based Optimization, Time series predictions, classification.Abstract
In this paper, we present a learning algorithm for the Elman Recurrent Neural Network (ERNN) based on Biogeography-Based Optimization (BBO). The proposed algorithm computes the weights, initials inputs of the context units and self-feedback coefficient of the Elman network. The method applied for four benchmark problems: Mackey Glass and Lorentz equations, which produce chaotic time series, and to real life classification; iris and Breast Cancer datasets. Numerical experimental results show improvement of the performance of the proposed algorithm in terms of accuracy and MSE eror over many heuristic algorithms.
References
Baldi, P. & Pollastri, G. (2003). The Principled Design of Large-Scale Recursive Neural Network Architectures-DAG-RNNs and the Protein Structure Prediction Problem. Journal of Machine Learning Research 4(4), 575-602.
https://doi.org/10.1162/153244304773936054
Cao, Q. Ewinga, B. T. & Thompson, M. A. (2012). Forecasting wind speed with recurrent neural networks. European Journal of Operational Research, 221(1), 148-154. https://doi.org/10.1016/j.ejor.2012.02.042
Chandra, R. (2015). Competition and Collaboration in Cooperative Coevolution of Elman Recurrent Neural Networks for Time-Series Prediction. IEEE Trans Neural Netw Learn Syst, 26(12), 3123-3136. https://doi.org/10.1109/TNNLS.2015.2404823
Elman, J. L. (1990). Finding structure in time. Cognitive science, 4(2), 179-211. https://doi.org/10.1016/0364-0213(90)90002-E
Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of human genetics, 7(2), 179-188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79(8), 2554–2558. https://doi.org/10.1073/pnas.79.8.2554
Hordri, N. F., Yuhaniz, S. S. & Nasien, D. (2013). A Comparison Study of Biogeography based. Optimization for Optimization Problems. International Journal of Advances in Soft Computing and its Applications, 5(1), 1-16.
Hori, T., Hori, C., Watanabe, S. & Hershey, J. R. (2016). Minimum word error training of long short-term memory recurrent neural network language models for speech recognition. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, 5990-5994. https://doi.org/10.1109/ICASSP.2016.7472827
Jordan. M. I. (1997). A parallel distributed processing approach. Advances in Psychology, 121, 471-495. https://doi.org/10.1016/S0166-4115(97)80111-2
Karaboga, D., & Basturk, B. (2007). A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. Journal of Global Optimization, 39, 459–471. https://doi.org/10.1007/s10898-007-9149-x
Kawada, K., Yamamoto, T., & Mada, Y. (2004). A design of evolutionary recurrent neural-net based controllers for an inverted pendulum. 5th Asian Control Conference (IEEE Cat. No.04EX904), 3, 1419-1422.
Kechriotis, G., Zervas, E. & Manolakos, E. S. (1994). Using recurrent neural networks for adaptive communication channel equalization. IEEE Transactions on Neural Networks, 5(2), 267-278. https://doi.org/10.1109/72.279190
Kennedy, J. & Eberhart, R. (1995). Particle swarm optimization. Proceedings of ICNN'95 - International Conference on Neural Networks, Perth, WA, Australia, 4, 1942-1948. https://doi.org/10.1109/ICNN.1995.488968
Koskela, T., Lehtokangas, M., Saarinen, J. & Kaski, K. (1996). Time Series Prediction with Multilayer Perceptron, FIR and Elman Neural Networks. Proceedings of the World Congress on Neural Networks, 491-496. https://pdfs.semanticscholar.org/82c8/e5d0cd4a7467f7f54ad823b2136b973eeb6e.pd
Kosko, B. (1988). Bidirectional associative memories. IEEE Transactions on Systems, Man, and Cybernetics, 18(1), 49-60. https://doi.org/10.1109/21.87054.
Li, X. L., Shao, Z. J. & Qian, J. X. (2002). An Optimizing Method Based on Autonomous Animats: Fish-swarm Algorithm. Systems Engineering - Theory & Practice, 22(11), 32-38. https://doi.org/10.12011/1000-6788(2002)11-32
Lorenz, E. N. (1963). Deterministic Nonperiodic Flow. Journal of the Atmospheric Sciences, 20, 130–141. https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2
Ma, H., Fei, M., Simon, D., & Chen, Z. (2015). Biogeography-based optimization in noisy environments. Transactions of the Institute of Measurement and Control, 37(2), 190–204. https://doi.org/10.1177/0142331214537015
Mackey, M. C. & Glass, L. (1977). Oscillation and chaos in physiological control systems. Science, 197(4300), 287-289. https://doi.org/10.1126/science.267326
Marvuglia, A. & Messineo, A. (2012). Using Recurrent Artificial Neural Networks to Forecast Household Electricity Consumption. Energy Procedia, 14, 45-55.
Mehrabian, A. R., & Lucas, C. (2006). A novel numerical optimization algorithm inspired from weed colonization. Ecological Informatics, 1(4), 355-366. https://doi.org/10.1016/j.ecoinf.2006.07.003
Mirjalili, S., Mirjalili, S. & Lewis, A. (2014). Let a biogeography-based optimizer train your multi-layer perceptron. Information Sciences, 269, 188-209. https://doi.org/10.1016/j.ins.2014.01.038
Müller-Navarra, M., Lessmann, S. & Voß, S. (2015). Sales Forecasting with Partial Recurrent Neural Networks: Empirical Insights and Benchmarking Results. 48th Hawaii International Conference on System Sciences, Kauai, HI, 1108-1116. https://doi.org/10.1109/HICSS.2015.135
Nawi, N. M. & Khan, A. & Rehman, G., Syed, M., Chiroma, H. & Herawan, T. (2014). Weight Optimization in Recurrent Neural Networks with Hybrid Metaheuristic Cuckoo Search Techniques for Data Classification. Mathematical Problems in Engineering. https://doi.org/10.1155/2015/868375
Osman, I. H., & Laporte, G. (1996). Metaheuristics: A bibliography. Annals of Operations Research, 63, 511–623. https://doi.org/10.1007/BF02125421
Palafox, L. & Iba, H. (2012). On the use of Population Based Incremental Learning to do Reverse Engineering on Gene Regulatory Networks. IEEE Congress on Evolutionary Computation, Brisbane, QLD, 1-8. https://doi.org/10.1109/CEC.2012.6256580
Pan, W-S. (2012). A new fruit fly optimization algorithm: taking the financial distress model as an example. Knowledge-Based Systems, 26, 69-74. https://doi.org/10.1016/j.knosys.2011.07.001
Passino, K. M. (2002). Biomimicry of bacterial foraging for distributed optimization and control. IEEE Control Systems Magazine, 22(3), 52-67. https://doi.org/10.1109/MCS.2002.1004010
Pham, D. T. & Karaboga, D. (1999). Training Elman and Jordan networks for system identification using genetic algorithms. Artificial Intelligence in Engineering, 13(2), 107-117. https://doi.org/10.1016/S0954-1810(98)00013-2
Robinson, T. & Fallside, F. (1991). A recurrent error propagation network speech recognition system. Computer Speech & Language, 5(3), 259-274. https://doi.org/10.1016/0885-2308(91)90010-N
Rodan, A., Faris, H. & Alqatawna, J. (2016). Optimizing Feedforward Neural Networks Using Biogeography Based Optimization for E-Mail Spam Identification. International Journal of Communications, Network and System Sciences, 9, 19-28. https://doi.org/10.4236/ijcns.2016.91002
Senjyu, T., Yona, A., Urasaki, N. & Funabashi, T. (2006). Application of Recurrent Neural Network to Long-Term-Ahead Generating Power Forecasting for Wind Power Generator. IEEE PES Power Systems Conference and Exposition, Atlanta, GA, 1260-1265. https://doi.org/10.1109/PSCE.2006.296487
Shamsuddin, M. (2004). Lecture note advanced artificial intelligence: Number of hidden neurons [Unpublished Doctoral Thesis]. Universiti Teknologi Malaysia.
Sharmaa, R., Kumar, V., Gaur, P. & Mittal, A. P. (2016). An adaptive PID like controller using mix locally recurrent neural network for robotic manipulator with variable payload. ISA Transactions, 62, 258-267. https://doi.org/10.1016/j.isatra.2016.01.016
Simon, D. (2008). Biogeography-based optimization. 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, TX (702-713). https://doi.org/10.1109/TEVC.2008.919004
Simon, D., Ergezer, M. & Du, D. (2009). Population distributions in biogeography-based optimization algorithms with elitism. 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, TX, 991-996. https://doi.org/10.1109/ICSMC.2009.5346058
Simon, D., Rarick, R., Ergezer, M. & Du, D. (2011). Analytical and numerical comparisons of biogeography-based optimization and genetic algorithms. Information Sciences,181(7), 1224-1248. https://doi.org/10.1016/j.ins.2010.12.006
Storn, R., & Price, K. (1997). Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. Journal of Global Optimization, 11, 341–359. https://doi.org/10.1023/A:1008202821328
Venayagamoorthy, G. A., Welch, R. L. & Ruffing, S. M. (2009). Comparison of Feedforward and Feedback Neural Network. Architectures for Short Term Wind Speed Prediction. IJCNN'09: Proceedings of the 2009 international joint conference on Neural Networks pp. 3141–3146. https://doi.org/10.1109/IJCNN.2009.5179034
Wolberg, W. H., & Mangasarian, O. L. (1990). Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences of the United States of America, 87(23), 9193–9196. https://doi.org/10.1073/pnas.87.23.9193
Xiao, P., Venayagamoorthy, G. K. & Corzine, K. A. (2007). Combined Training of Recurrent Neural Networks with Particle Swarm Optimization and Backpropagation Algorithms for Impedance Identification. IEEE Swarm Intelligence Symposium, Honolulu, HI, 9-15. https://doi.org/10.1109/SIS.2007.368020
Yang, X-S. (2009). Firefly algorithms for multimodal optimization. International Symposium on Stochastic Algorithms, 169-178. https://doi.org/10.1007/978-3-642-04944-6_14
Yang, X. S. (2010). A New Metaheuristic Bat-Inspired Algorithm. In J. R. González, D.A. Pelta, C. Cruz, G. Terrazas, & N. Krasnogor (Eds.), Nature Inspired Cooperative Strategies for Optimization (NICSO 2010). Studies in Computational Intelligence, vol 284. Springer.
Yang, X-S. & Deb, S. (2009). Cuckoo search via Levy flights. World Congress on Nature & Biologically Inspired Computing (NaBIC), Coimbatore, 210-214. https://doi.org/10.1109/NABIC.2009.5393690
Yoo, D. G., & Kim, J. H. (2014). Meta-heuristic algorithms as tools for hydrological science. Geoscience Letters, 1, 4. https://doi.org/10.1186/2196-4092-1-4
Zhang, X., Kang, Q., Tu, Q., Cheng, J. & Wang, X. (2019). Efficient and merged biogeography-based optimization algorithm for global optimization problems. Soft Computing, 23, 4483–4502. https://doi.org/10.1007/s00500-018-3113-1
Zhipeng, Y. Minfang, P., Hao, H. & Xianfeng, L. (2012). Fault Locating of Grounding Grids Based on Ant colony Optimizing Elman Neural Network. 2012 Third International Conference on Digital Manufacturing & Automation, GuiLin, 406-409. https://doi.org/10.1109/ICDMA.2012.97
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant this journal right of first publication, with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work, with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g. post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as an earlier and greater citation of published work (See The Effect of Open Access).
BRAIN. Broad Research in Artificial Intelligence and Neuroscience Journal has an Attribution-NonCommercial-NoDerivs
CC BY-NC-ND
