The Study of the Influence of Architecture on Effectiveness of Deep Neural Networks Training

• Authors: Kolbusz Janusz , Różycki Paweł , Bartczak Tomasz

Title variants

PL

Badania wpływu architektury na skuteczność uczenia głębokich sieci neuronowych

• Languages of publication: EN PL

Abstracts

EN

Paper presents impact of the neural network architecture on the training effectiveness and training time. Selected network architectures and training algorithm are described. Presented experimental results of research confirming the significant influence of architecture on the success of network training.

PL

W artykule przedstawiono wpływ architektury sieci neuronowej na skuteczność i czas uczenia sieci. Opisano wybrane architektury sieci, algorytm uczenia oraz zaprezentowano wyniki badań potwierdzających znaczący wpływ architektury na sukces uczenia sieci.

Keywords

EN

deep learning; ANN architectures; Bridged MLP; NBN

PL

uczenie głębokie; architektury sztucznych sieci neuronowych; Bridged MLP; NBN

• Publisher: Akademia Finansów i Biznesu Vistula w Warszawie
• Journal: Zeszyty Naukowe Uczelni Vistula
• Year: 2018
• Issue: 59(2) Informatyka
• Pages: 60-71

Contributors

author

Kolbusz Janusz

• Wyższa Szkoła Informatyki i Zarządzania w Rzeszowie

author

Różycki Paweł

• Wyższa Szkoła Informatyki i Zarządzania w Rzeszowie

author

Bartczak Tomasz

• Akademia Finansów i Biznesu Vistula

References

• Bengio Y. (2009), Learning deep architectures for AI, “Foundations and Trends in Machine Learning”, No. 2(1).
• Boughrara H. et al. (2016), Facial expression recognition based on a mlp neural network using constructive training algorithm, “Multimedia Tools and Applications”, No. 75.2.
• Fahlman S.E., Lebiere C. (1990), The cascade-correlation learning architecture, (in:) Touretzky D.S. (Ed.) Advances in Neural Information Processing Systems 2, Morgan Kaufmann, San Mateo.
• Hang S. et al. (2013), Error back propagation for sequence training of context-dependent deep networks for conversational speech transcription, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.
• Hunter D., Yu H., Pukish M.S., Kolbusz J., and Wilamowski B.M. (2012), Selection of Proper Neural Network Sizes and Architectures – A Comparative Study, “IEEE Trans. on Industrial Informatics”, Vol. 8.
• Kaytez F. et al. (2015), Forecasting electricity consumption: a comparison of regression analysis, neural networks and least squares support vector machines, “International Journal of Electrical Power & Energy Systems”, No. 67.
• Lalis J.T. et al. (2013), Ubiquitous Stopping Criterion for Backpropagation Learning in Multilayer Perceptron Neural Networks.
• Lang K.L., Witbrock M.J. (1988), Learning to Tell Two Spirals Apart Proceedings of the 1988 Connectionists Models Summer School, Morgan Kaufman.
• Paczek S., Adhikari B. (2014), Analysis of MultiLayer Neural Networks with Direct and Cross Forward Connection, “Fundamenta Informaticae”, Vol. 133, No. 2-3.
• Rumelhart D.E., Hinton G.E., Williams R.J. (1986), Learning representations by back-propagating errors, “Nature”, Vol. 323.
• Wilamowski B.M., Hunter D., Malinowski A. (2003), Solving parity-N problems with feedforward neural networks, Proc. 2003 IEEE IJCNN.
• Wilamowski B.M., Yu H. (2010a), Neural Network Learning Without Backpropagation, “IEEE Trans. on Neural Networks”, Vol. 21, No.11.
• Wilamowski B.M., Yu H. (2010b), Improved Computation for Levenberg Marquardt Training, “IEEE Trans. on Neural Networks”, Vol. 21, No. 6.
• Wilamowski B.M., Yu H. (2017), NNT – Neural Networks Trainer, http://www.eng.auburn.edu/ wilambm/nnt/ [access: 15.09.2017]
• Xie T., Yu H., Hewlett J., Rozycki P., Wilamowski B.M. (2012), Fast and Efficient Second-Order Method for Training Radial Basis Function Networks, “IEEE Trans. on Neural Networks and Learning Systems”, Vol. 23, No. 4.
• Yu H., Reiner P., Xie T., Bartczak T., Wilamowski B.M. (2014), An Incremental Design of Radial Basis Function Networks, “IEEE Transactions on Neural Networks and Learning Systems”, Vol. 25, No. 10.
• Yu H., Xie T., Hewlett J., Rozycki P., Wilamowski B.M. (2012), Fast and Efficient Second Order Method for Training Radial Basis Function Networks, “IEEE Transactions on Neural Networks”, Vol. 24, Iss. 4.