Biogeography-Based Optimization for Weight Optimization in Elman Neural Network Compared with Meta-Heuristics Methods

Authors

  • Habib DHAHRI Assistant Professor @ King Saud University, Email: hdhahri@ksu.edu.sa Google Scholar: http://scholar.google.com/citations?user=UVDWessAAAAJ&hl=fr DBLP: http://www.informatik.uni-trier.de/~ley/pers/hd/d/Dhahri:Habib.html

DOI:

https://doi.org/10.18662/brain/11.2/76

Keywords:

Biogeography-Based Optimization, Time series predictions, classification.

Abstract

In this paper, we present a learning algorithm for the Elman Recurrent Neural Network (ERNN) based on Biogeography-Based Optimization (BBO). The proposed algorithm computes the weights, initials inputs of the context units and self-feedback coefficient of the Elman network. The method applied for four benchmark problems: Mackey Glass and Lorentz equations, which produce chaotic time series, and to real life classification; iris and Breast Cancer datasets. Numerical experimental results show improvement of the performance of the proposed algorithm in terms of accuracy and MSE eror over many heuristic algorithms.

References

Baldi, P. & Pollastri, G. (2003). The Principled Design of Large-Scale Recursive Neural Network Architectures-DAG-RNNs and the Protein Structure Prediction Problem. Journal of Machine Learning Research 4(4), 575-602.

https://doi.org/10.1162/153244304773936054

Cao, Q. Ewinga, B. T. & Thompson, M. A. (2012). Forecasting wind speed with recurrent neural networks. European Journal of Operational Research, 221(1), 148-154. https://doi.org/10.1016/j.ejor.2012.02.042

Chandra, R. (2015). Competition and Collaboration in Cooperative Coevolution of Elman Recurrent Neural Networks for Time-Series Prediction. IEEE Trans Neural Netw Learn Syst, 26(12), 3123-3136. https://doi.org/10.1109/TNNLS.2015.2404823

Elman, J. L. (1990). Finding structure in time. Cognitive science, 4(2), 179-211. https://doi.org/10.1016/0364-0213(90)90002-E

Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of human genetics, 7(2), 179-188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x

Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79(8), 2554–2558. https://doi.org/10.1073/pnas.79.8.2554

Hordri, N. F., Yuhaniz, S. S. & Nasien, D. (2013). A Comparison Study of Biogeography based. Optimization for Optimization Problems. International Journal of Advances in Soft Computing and its Applications, 5(1), 1-16.

Hori, T., Hori, C., Watanabe, S. & Hershey, J. R. (2016). Minimum word error training of long short-term memory recurrent neural network language models for speech recognition. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, 5990-5994. https://doi.org/10.1109/ICASSP.2016.7472827

Jordan. M. I. (1997). A parallel distributed processing approach. Advances in Psychology, 121, 471-495. https://doi.org/10.1016/S0166-4115(97)80111-2

Karaboga, D., & Basturk, B. (2007). A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. Journal of Global Optimization, 39, 459–471. https://doi.org/10.1007/s10898-007-9149-x

Kawada, K., Yamamoto, T., & Mada, Y. (2004). A design of evolutionary recurrent neural-net based controllers for an inverted pendulum. 5th Asian Control Conference (IEEE Cat. No.04EX904), 3, 1419-1422.

Kechriotis, G., Zervas, E. & Manolakos, E. S. (1994). Using recurrent neural networks for adaptive communication channel equalization. IEEE Transactions on Neural Networks, 5(2), 267-278. https://doi.org/10.1109/72.279190

Kennedy, J. & Eberhart, R. (1995). Particle swarm optimization. Proceedings of ICNN'95 - International Conference on Neural Networks, Perth, WA, Australia, 4, 1942-1948. https://doi.org/10.1109/ICNN.1995.488968

Koskela, T., Lehtokangas, M., Saarinen, J. & Kaski, K. (1996). Time Series Prediction with Multilayer Perceptron, FIR and Elman Neural Networks. Proceedings of the World Congress on Neural Networks, 491-496. https://pdfs.semanticscholar.org/82c8/e5d0cd4a7467f7f54ad823b2136b973eeb6e.pd

Kosko, B. (1988). Bidirectional associative memories. IEEE Transactions on Systems, Man, and Cybernetics, 18(1), 49-60. https://doi.org/10.1109/21.87054.

Li, X. L., Shao, Z. J. & Qian, J. X. (2002). An Optimizing Method Based on Autonomous Animats: Fish-swarm Algorithm. Systems Engineering - Theory & Practice, 22(11), 32-38. https://doi.org/10.12011/1000-6788(2002)11-32

Lorenz, E. N. (1963). Deterministic Nonperiodic Flow. Journal of the Atmospheric Sciences, 20, 130–141. https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2

Ma, H., Fei, M., Simon, D., & Chen, Z. (2015). Biogeography-based optimization in noisy environments. Transactions of the Institute of Measurement and Control, 37(2), 190–204. https://doi.org/10.1177/0142331214537015

Mackey, M. C. & Glass, L. (1977). Oscillation and chaos in physiological control systems. Science, 197(4300), 287-289. https://doi.org/10.1126/science.267326

Marvuglia, A. & Messineo, A. (2012). Using Recurrent Artificial Neural Networks to Forecast Household Electricity Consumption. Energy Procedia, 14, 45-55.

Mehrabian, A. R., & Lucas, C. (2006). A novel numerical optimization algorithm inspired from weed colonization. Ecological Informatics, 1(4), 355-366. https://doi.org/10.1016/j.ecoinf.2006.07.003

Mirjalili, S., Mirjalili, S. & Lewis, A. (2014). Let a biogeography-based optimizer train your multi-layer perceptron. Information Sciences, 269, 188-209. https://doi.org/10.1016/j.ins.2014.01.038

Müller-Navarra, M., Lessmann, S. & Voß, S. (2015). Sales Forecasting with Partial Recurrent Neural Networks: Empirical Insights and Benchmarking Results. 48th Hawaii International Conference on System Sciences, Kauai, HI, 1108-1116. https://doi.org/10.1109/HICSS.2015.135

Nawi, N. M. & Khan, A. & Rehman, G., Syed, M., Chiroma, H. & Herawan, T. (2014). Weight Optimization in Recurrent Neural Networks with Hybrid Metaheuristic Cuckoo Search Techniques for Data Classification. Mathematical Problems in Engineering. https://doi.org/10.1155/2015/868375

Osman, I. H., & Laporte, G. (1996). Metaheuristics: A bibliography. Annals of Operations Research, 63, 511–623. https://doi.org/10.1007/BF02125421

Palafox, L. & Iba, H. (2012). On the use of Population Based Incremental Learning to do Reverse Engineering on Gene Regulatory Networks. IEEE Congress on Evolutionary Computation, Brisbane, QLD, 1-8. https://doi.org/10.1109/CEC.2012.6256580

Pan, W-S. (2012). A new fruit fly optimization algorithm: taking the financial distress model as an example. Knowledge-Based Systems, 26, 69-74. https://doi.org/10.1016/j.knosys.2011.07.001

Passino, K. M. (2002). Biomimicry of bacterial foraging for distributed optimization and control. IEEE Control Systems Magazine, 22(3), 52-67. https://doi.org/10.1109/MCS.2002.1004010

Pham, D. T. & Karaboga, D. (1999). Training Elman and Jordan networks for system identification using genetic algorithms. Artificial Intelligence in Engineering, 13(2), 107-117. https://doi.org/10.1016/S0954-1810(98)00013-2

Robinson, T. & Fallside, F. (1991). A recurrent error propagation network speech recognition system. Computer Speech & Language, 5(3), 259-274. https://doi.org/10.1016/0885-2308(91)90010-N

Rodan, A., Faris, H. & Alqatawna, J. (2016). Optimizing Feedforward Neural Networks Using Biogeography Based Optimization for E-Mail Spam Identification. International Journal of Communications, Network and System Sciences, 9, 19-28. https://doi.org/10.4236/ijcns.2016.91002

Senjyu, T., Yona, A., Urasaki, N. & Funabashi, T. (2006). Application of Recurrent Neural Network to Long-Term-Ahead Generating Power Forecasting for Wind Power Generator. IEEE PES Power Systems Conference and Exposition, Atlanta, GA, 1260-1265. https://doi.org/10.1109/PSCE.2006.296487

Shamsuddin, M. (2004). Lecture note advanced artificial intelligence: Number of hidden neurons [Unpublished Doctoral Thesis]. Universiti Teknologi Malaysia.

Sharmaa, R., Kumar, V., Gaur, P. & Mittal, A. P. (2016). An adaptive PID like controller using mix locally recurrent neural network for robotic manipulator with variable payload. ISA Transactions, 62, 258-267. https://doi.org/10.1016/j.isatra.2016.01.016

Simon, D. (2008). Biogeography-based optimization. 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, TX (702-713). https://doi.org/10.1109/TEVC.2008.919004

Simon, D., Ergezer, M. & Du, D. (2009). Population distributions in biogeography-based optimization algorithms with elitism. 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, TX, 991-996. https://doi.org/10.1109/ICSMC.2009.5346058

Simon, D., Rarick, R., Ergezer, M. & Du, D. (2011). Analytical and numerical comparisons of biogeography-based optimization and genetic algorithms. Information Sciences,181(7), 1224-1248. https://doi.org/10.1016/j.ins.2010.12.006

Storn, R., & Price, K. (1997). Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. Journal of Global Optimization, 11, 341–359. https://doi.org/10.1023/A:1008202821328

Venayagamoorthy, G. A., Welch, R. L. & Ruffing, S. M. (2009). Comparison of Feedforward and Feedback Neural Network. Architectures for Short Term Wind Speed Prediction. IJCNN'09: Proceedings of the 2009 international joint conference on Neural Networks pp. 3141–3146. https://doi.org/10.1109/IJCNN.2009.5179034

Wolberg, W. H., & Mangasarian, O. L. (1990). Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences of the United States of America, 87(23), 9193–9196. https://doi.org/10.1073/pnas.87.23.9193

Xiao, P., Venayagamoorthy, G. K. & Corzine, K. A. (2007). Combined Training of Recurrent Neural Networks with Particle Swarm Optimization and Backpropagation Algorithms for Impedance Identification. IEEE Swarm Intelligence Symposium, Honolulu, HI, 9-15. https://doi.org/10.1109/SIS.2007.368020

Yang, X-S. (2009). Firefly algorithms for multimodal optimization. International Symposium on Stochastic Algorithms, 169-178. https://doi.org/10.1007/978-3-642-04944-6_14

Yang, X. S. (2010). A New Metaheuristic Bat-Inspired Algorithm. In J. R. González, D.A. Pelta, C. Cruz, G. Terrazas, & N. Krasnogor (Eds.), Nature Inspired Cooperative Strategies for Optimization (NICSO 2010). Studies in Computational Intelligence, vol 284. Springer.

Yang, X-S. & Deb, S. (2009). Cuckoo search via Levy flights. World Congress on Nature & Biologically Inspired Computing (NaBIC), Coimbatore, 210-214. https://doi.org/10.1109/NABIC.2009.5393690

Yoo, D. G., & Kim, J. H. (2014). Meta-heuristic algorithms as tools for hydrological science. Geoscience Letters, 1, 4. https://doi.org/10.1186/2196-4092-1-4

Zhang, X., Kang, Q., Tu, Q., Cheng, J. & Wang, X. (2019). Efficient and merged biogeography-based optimization algorithm for global optimization problems. Soft Computing, 23, 4483–4502. https://doi.org/10.1007/s00500-018-3113-1

Zhipeng, Y. Minfang, P., Hao, H. & Xianfeng, L. (2012). Fault Locating of Grounding Grids Based on Ant colony Optimizing Elman Neural Network. 2012 Third International Conference on Digital Manufacturing & Automation, GuiLin, 406-409. https://doi.org/10.1109/ICDMA.2012.97

Downloads

Published

2020-07-03

How to Cite

DHAHRI, H. (2020). Biogeography-Based Optimization for Weight Optimization in Elman Neural Network Compared with Meta-Heuristics Methods. BRAIN. Broad Research in Artificial Intelligence and Neuroscience, 11(2), 82-103. https://doi.org/10.18662/brain/11.2/76

Publish your work at the Scientific Publishing House LUMEN

It easy with us: publish now your work, novel, research, proceeding at Lumen Scientific Publishing House

Send your manuscript right now