Full-text resources of CEJSH and other databases are now available in the new Library of Science.
Visit https://bibliotekanauki.pl

PL EN


2024 | 15 | 1 | 27-58

Article title

The economics of deep and machine learning-based algorithms for COVID-19 prediction, detection, and diagnosis shaping the organizational management of hospitals

Content

Title variants

Languages of publication

Abstracts

EN
Research background: Deep and machine learning-based algorithms can assist in COVID-19 image-based medical diagnosis and symptom tracing, optimize intensive care unit admission, and use clinical data to determine patient prioritization and mortality risk, being pivotal in qualitative care provision, reducing medical errors, and increasing patient survival rates, thus diminishing the massive healthcare system burden in relation to severe COVID-19 inpatient stay duration, while increasing operational costs throughout the organizational management of hospitals. Data-driven financial and scenario-based contingency planning, predictive modelling tools, and risk pooling mechanisms should be deployed for additional medical equipment and unforeseen healthcare demand expenses. Purpose of the article: We show that deep and machine learning-based and clinical decision making systems can optimize patient survival likelihood and treatment outcomes with regard to susceptible, infected, and recovered individuals, performing accurate analyses by data modeling based on vital and clinical signs, surveillance data, and infection-related biomarkers, and furthering hospital facility optimization in terms of intensive care unit bed allocation. Methods: The review software systems employed for article screening and quality evaluation were: AMSTAR, AXIS, DistillerSR, Eppi-Reviewer, MMAT, PICO Portal, Rayyan, ROBIS, and SRDR. Findings & value added: Deep and machine learning-based clinical decision support tools can forecast COVID-19 spread, confirmed cases, and infection and mortality rates for data-driven appropriate treatment and resource allocations in effective therapeutic and diagnosis protocol development, by determining suitable measures and regulations and by using symptoms and comorbidities, vital signs, clinical and laboratory data and medical records across intensive care units, impacting the healthcare financing infrastructure. As a result of heightened use of personal protective equipment, hospital pharmacy and medication, outpatient treatment, and medical supplies, revenue loss and financial vulnerability occur, also due to expenses related to hiring additional staff and to critical resource expenditures. Hospital costs for COVID-19 medical care, screening, treatment capacity expansion, and personal protective equipment can lead to further financial losses while affecting COVID-19 frontline hospital workers and patients.

Year

Volume

15

Issue

1

Pages

27-58

Physical description

Dates

published
2024

Contributors

  • University of Zilina
  • Toronto Metropolitan University, Canada
  • Cardiff Metropolitan University, United Kingdom
  • Spiru Haret University
author
  • Curtin University
  • University of Warmia and Mazury in Olsztyn
  • Spiru Haret University
  • University of Zilina
author
  • Matej Bel University
  • Spiru Haret University
  • Spiru Haret University
  • University of Zilina
  • University of Zilina
  • Spiru Haret University
  • Spiru Haret University
  • University of Zilina
author
  • University of Presov
  • Radiology Department, Fundeni Clinical Institute
  • Grigore Alexandrescu Children’s Emergency Hospital
  • Spiru Haret University

References

  • Abdul Salam, M., Taha, S., & Ramadan, M. (2021). COVID-19 detection using federated machine learning. PLOS ONE, 16(6), e0252573.
  • Aktar, S., Ahamad, M. M., Rashed-Al-Mahfuz, M., Azad, A. K. M., Uddin, S., Kamal, A. H. M., Alyami, S. A., Lin, P., Islam, S. M. S., Quinn, J. M., Eapen, V., & Moni, M. A. (2021). Machine learning approach to predicting COVID-19 disease severity based on clinical blood test data: Statistical analysis and model development. JMIR Medical Informatics, 9(4), e25884.
  • Akter, S., Shamrat, F. M. J. M., Chakraborty, S., Karim, A., & Azam, S. (2021). COVID-19 detection using deep learning algorithm on chest X-ray images. Biology, 10, 1174.
  • Alassafi, M. O., Jarrah, M., & Alotaibi, R. (2022). Time series predicting of COVID-19 based on deep learning. Neurocomputing, 468, 335‒344.
  • Alballa, N., & Al-Turaiki, I. (2021). Machine learning approaches in COVID-19 diagnosis, mortality, and severity risk prediction: A review. Informatics in Medicine Unlocked, 24, 100564.
  • Alakus, T. B., & Turkoglu, I. (2020). Comparison of deep learning approaches to predict COVID-19 infection. Chaos, Solitons & Fractals, 140, 110120. https://doi. org/10.1016/j.chaos. 2020.110120.
  • Alali, Y., Harrou, F., & Sun, Y. (2022). A proficient approach to forecast COVID-19 spread via optimized dynamic machine learning models. Scientific Reports, 12, 2467.
  • Alotaibi, A., Shiblee, M., & Alshahrani, A. (2021). Prediction of severity of COVID-19-infected patients using machine learning techniques. Computers, 10, 31.
  • Al Shehri, W., Almalki, J., Mehmood, R., Alsaif, K., Alshahrani, S. M., Jannah, N., & Alangari, S. (2022). A novel COVID-19 detection technique using deep learning based approaches. Sustainability, 14, 12222.
  • Andrade, E. C. d., Pinheiro, P. R., Barros, A. L. B. d. P., Nunes, L. C., Pinheiro, L. I. C. C., Pinheiro, P. G. C. D., & Holanda Filho, R. (2022). Towards machine learning algorithms in predicting the clinical evolution of patients diagnosed with COVID-19. Applied Sciences, 12, 8939.
  • Ardabili, S. F., Mosavi, A., Ghamisi, P., Ferdinand, F., Varkonyi-Koczy, A. R., Reuter, U., Rabczuk, T., & Atkinson, P. M. (2020). COVID-19 outbreak prediction with machine learning. Algorithms, 13, 249.
  • Arpaci, I., Huang, S., Al-Emran, M., Al-Kabi, M. N., & Peng, M. (2021). Predicting the COVID-19 infection with fourteen clinical features using machine learning classification algorithms. Multimedia Tools and Applications, 80, 11943–11957.
  • Arvind, V., Kim, J. S., Cho, B. H., Geng, E., & Cho, S. K. (2021). Development of a machine learning algorithm to predict intubation among hospitalized patients with COVID-19. Journal of Critical Care, 62, 25–30.
  • Aslani, S., & Jacob, J. (2023). Utilisation of deep learning for COVID-19 diagnosis. Clinical Radiology, 78(2), 150–157.
  • Assaf, D., Gutman, Y., Neuman, Y., Segal, G., Amit, S., Gefen-Halevi, S., Shilo, N., Epstein, A., Mor-Cohen, R., Biber, A., Rahav, G., Levy, I., & Tirosh, A. (2020). Utilization of machine-learning models to accurately predict the risk for critical COVID-19. Internal and Emergency Medicine, 15, 1435–1443.
  • Babukarthik, R. G., Adiga, V. A. K., Sambasivam, G., Chandramohan, D., & Amudhavel, J. (2020). Prediction of COVID-19 using Genetic Deep Learning Convolutional Neural Network (GDCNN). IEEE Access, 8, 177647–177666.
  • Burdick, H., Lam, C., Mataraso, S., Siefkas, A., Braden, G., Phillip Dellinger, R., McCoy, A., Vincent, J.-L., Green-Saxena, A., Barnes, G., Hoffman, J., Calvert, J., Pellegrini, E., & Das, R. (2020). Prediction of respiratory decompensation in Covid-19 patients using machine learning: The READY trial. Computers in Biology and Medicine, 124, 103949.
  • Cobre, A. F., Stremel, D. P., Noleto, G. R., Fachi, M. M., Surek, M., Wiens, A., Tonin, F. S., & Pontarolo, R. (2021). Diagnosis and prediction of COVID-19 severity: Can biochemical tests and machine learning be used as prognostic indicators? Computers in Biology and Medicine, 134, 104531.
  • Das, A. K., Mishra, S., & Saraswathy Gopalan, S. (2020). Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ, 8, e10083.
  • Desai, S. B., Pareek, A., & Lungren, M. P. (2020). Deep learning and its role in COVID-19 medical imaging. Intelligence-Based Medicine, 3–4, 100013.
  • Devaraj, J., Elavarasan, R. M., Pugazhendhi, R., Shafiullah, G. M., Ganesan, S., Jeysree, A. K., Khan, I. A., & Hossain, E. (2021). Forecasting of COVID-19 cases using deep learning models: Is it reliable and practically significant? Results in Physics, 21, 103817.
  • Ebinger, J., Wells, M., Ouyang, D., Davis, T., Kaufman, N., Cheng, S., & Chugh, S. (2021). A machine learning algorithm predicts duration of hospitalization in COVID-19 patients. Intelligence-Based Medicine, 5, 100035.
  • Elham, J., Amirhossein, A., Nader, T., Alireza, Z., Soroush, S., Hadi, E., Jamaldini, S. H., Daaee, A., Babajani, A., Kashi, M. A. S., Jamshidi, M., Rahi, S. J., & Mansouri, N. (2022). Using machine learning to predict mortality for COVID-19 patients on day 0 in the ICU. Frontiers in Digital Health, 3, 681608.
  • Elhazmi, A., Al-Omari, A., Sallam, H., Mufti, H. N., Rabie, A. A., Alshahrani, M., Mady, A., Alghamdi, A., Altalaq, A., Azzam, M. H., Sindi, A., Kharaba, A., Al-Aseri, Z. A., Almekhlafi, G. A., Tashkandi, W., Alajmi, S. A., Faqihi, F., Alharthy, A., Al-Tawfiq, J. A., Melibari, R. G., Al-Hazzani, W., & Arabi, Y. M. (2022). Machine learning decision tree algorithm role for predicting mortality in critically ill adult COVID-19 patients admitted to the ICU. Journal of Infection and Public Health, 15(7), 826–834.
  • Fang, C., Bai, S., Chen, Q., Zhou, Y., Xia, L., Qin, L., Gong, S., Xie, X., Zhou, C., Tu, D., Zhang, C., Liu, X., Chen, W., Bai, X., & Torr, P. H. S. (2021). Deep learning for predicting COVID-19 malignant progression. Medical Image Analysis, 72, 102096.
  • Fernandes, F. T., de Oliveira, T. A., Teixeira, C. E., de Moraes Batista, A. F., Costa, G. D., & Filho, A. D. P. C. (2021). A multipurpose machine learning approach to predict COVID-19 negative prognosis in São Paulo, Brazil. Scientific Reports, 11, 3343.
  • Goodman-Meza, D., Rudas, A., Chiang, J. N., Adamson, P. C., Ebinger, J., Sun, N., Botting, P., Fulcher, J. A., Saab, F. G., Brook, R., Eskin, E., An, U., Kordi, M., Jew, B., Balliu, B., Chen, Z., Hill, B. L., Rahmani, E., Halperin, E., & Manuel, V. (2020). A machine learning algorithm to increase COVID-19 inpatient diagnostic capacity. PLoS ONE, 15(9), e0239474.
  • Gothai, E., Thamilselvan, R., Rajalaxmi, R. R., Sadana, R. M., Ragavi, A., & Sakthivel, R. (2023). Prediction of COVID-19 growth and trend using machine learning approach. Materials Today: Proceedings, 81(2), 597–601.
  • Halasz, G., Sperti, M., Villani, M., Michelucci, U., Agostoni, P., Biagi, A., Rossi, L., Botti, A., Mari, C., Maccarini, M., Pura, F., Roveda, L., Nardecchia, A., Mottola, E., Nolli, M., Salvioni, E., Mapelli, M., Deriu, M. A., Piga, D., & Piepoli M. (2021). A machine learning approach for mortality prediction in COVID-19 pneumonia: Development and evaluation of the Piacenza Score. Journal of Medical Internet Research, 23(5), e29058.
  • Ikemura, K., Bellin, E., Yagi, Y., Billett, H., Saada, M., Simone, K., Stahl, L., Szymanski, J., Goldstein, D. Y., & Reyes Gil, M. (2021). Using automated machine learning to predict the mortality of patients with COVID-19: Prediction model development study. Journal of Medical Internet Research, 23(2), e23458.
  • Iwendi, C., Huescas, C. G. Y., Chakraborty, C., & Mohan, S. (2022). COVID-19 health analysis and prediction using machine learning algorithms for Mexico and Brazil patients. Journal of Experimental & Theoretical Artificial Intelligence.
  • Jamshidi, M., Lalbakhsh, A., Talla, J., Peroutka, Z., Hadjilooei, F., Lalbakhsh, P., Jamshidi, M., La Spada, L., Mirmozafari, M., Dehghani, M., Sabet, A., Roshani, S., Roshani, S., Bayat-Makou, N., Mohamadzade, B., Malek, Z., Jamshidi, A., Kiani, S., Hashemi-Dezaki, H., & Mohyuddin, W. (2020). Artificial intelligence and COVID-19: Deep learning approaches for diagnosis and treatment. IEEE Access, 8, 109581–109595.
  • Kallel, A., Rekik, M., & Khemakhem, M. (2022). Hybrid-based framework for COVID-19 prediction via federated machine learning models. Journal of Supercomputing, 78, 7078–7105.
  • Kar, S., Chawla, R., Haranath, S. P., Ramasubban, S., Ramakrishnan, N., Vaishya, R., Sibal, A., & Reddy, S. (2021). Multivariable mortality risk prediction using machine learning for COVID-19 patients at admission (AICOVID). Scientific Reports, 11, 12801.
  • Kwekha-Rashid, A. S., Abduljabbar, H. N., & Alhayani, B. (2023). Coronavirus disease (COVID-19) cases analysis using machine-learning applications. Applied Nanoscience, 13, 2013–2025.
  • Laatifi, M., Douzi, S., Bouklouz, A., Ezzine, H., Jaafari, J., Zaid, Y., El Ouahidi, B., & Naciri, M. (2022). Machine learning approaches in Covid-19 severity risk prediction in Morocco. Journal of Big Data, 9, 5.
  • Lalmuanawma, S., Hussain, J., & Chhakchhuak, L. (2020). Applications of machine learning and artificial intelligence for Covid-19 (SARS-CoV-2) pandemic: A review. Chaos, Solitons & Fractals, 139, 110059.
  • Li, X., Ge, P., Zhu, J., Li, H., Graham, J., Singer, A., Richman, P. S., & Duong, T. Q. (2020). Deep learning prediction of likelihood of ICU admission and mortality in COVID-19 patients using clinical variables. PeerJ, 8, e10337.
  • Majhi, R., Thangeda, R., Sugasi, R. P., & Kumar N. (2021). Analysis and prediction of COVID-19 trajectory: A machine learning approach. Journal of Public Affairs, 21(4), e2537.
  • Malki, Z., Atlam, E. S., Hassanien, A. E., Dagnew, G., Elhosseini, M. A., & Gad, I. (2020). Association between weather data and COVID-19 pandemic predicting mortality rate: Machine learning approaches. Chaos, Solitons & Fractals, 138, 110137.
  • Malki, Z., Atlam, E. S., Ewis, A., Dagnew, G., Ghoneim, O. A., Mohamed, A. A., Abdel-Daim, M. M., & Gad, I. (2021). The COVID-19 pandemic: Prediction study based on machine learning models. Environmental Science and Pollution Research, 28, 40496–40506.
  • Meraihi, Y., Gabis, A. B., Mirjalili, S., Ramdane-Cherif, A., & Alsaadi, F. E. (2022). Machine learning-based research for COVID-19 detection, diagnosis, and prediction: A survey. SN Computer Science, 3, 286.
  • Mohan, S., John, A., Abugabah, A., Adimoolam, M., Kumar Singh, S., Kashif Bashir, A., & Sanzogni, L. (2022). An approach to forecast impact of Covid-19 using supervised machine learning model. Software: Practice and Experience, 52(4), 824–840.
  • Moulaei, K., Shanbehzadeh, M., Mohammadi-Taghiabad, Z., & Kazemi-Arpanahi, H. (2022). Comparing machine learning algorithms for predicting COVID-19 mortality. BMC Medical Informatics and Decision Making, 22, 2.
  • Muhammad, L. J., Algehyne, E. A., Usman, S. S., Ahmad, A., Chakraborty, C., & Mohammed, I. A. (2021). Supervised machine learning models for prediction of COVID-19 infection using epidemiology dataset. SN Computer Science, 2, 11.
  • Naeem, M., Yu, J., Aamir, M., Khan, S. A., Adeleye, O., & Khan, Z. (2021). Comparative analysis of machine learning approaches to analyze and predict the COVID-19 outbreak. PeerJ Computer Science, 7, e746.
  • Nassif, A. B., Shahin, I., Bader, M., Hassan, A., & Werghi, N. (2022). COVID-19 detection systems using deep-learning algorithms based on speech and image data. Mathematics, 10, 564.
  • Nemati, M., Ansary, J., & Nemati, N. (2020). Machine-learning approaches in COVID-19 survival analysis and discharge-time likelihood prediction using clinical data. Patterns, 1(5), 100074.
  • Ning, W., Lei, S., Yang, J., Cao, Y., Jiang, P., Yang, Q., Zhang, J., Wang, X., Chen, F., Geng, Z., Xiong, L., Zhou, H., Guo, Y., Zeng, Y., Shi, H., Wang, L., Xue, Y., & Wang, Z. (2020). Open resource of clinical data from patients with pneumonia for the prediction of COVID-19 outcomes via deep learning. Nature Biomedical Engineering, 4, 1197–1207.
  • Pan, P., Li, Y., Xiao, Y., Han, B., Su, L., Su, M., Li, Y., Zhang, S., Jiang, D., Chen, X., Zhou, F., Ma, L., Bao, P., Xie, L. (2020). Prognostic assessment of COVID-19 in the intensive care unit by machine learning methods: Model development and validation. Journal of Medical Internet Research, 22(11), e23128.
  • Patel, D., Kher, V., Desai, B., Lei, X., Cen, S., Nanda, N., Gholamrezanezhad, A., Duddalwar, V., Varghese, B., & Oberai, A. A. (2021). Machine learning based predictors for COVID-19 disease severity. Scientific Reports, 11, 4673.
  • Pinter, G., Felde, I., Mosavi, A., Ghamisi, P., & Gloaguen, R. (2020). COVID-19 pandemic prediction for Hungary; A hybrid machine learning approach. Mathematics, 8, 890.
  • Pourhomayoun, M., & Shakibi, M. (2021). Predicting mortality risk in patients with COVID-19 using machine learning to help medical decision-making. Smart Health, 20, 100178.
  • Quiroz-Juárez, M. A., Torres-Gómez, A., Hoyo-Ulloa, I., León-Montiel, R. D. J., & U’Ren, A. B. (2021). Identification of high-risk COVID-19 patients using machine learning. PLOS ONE, 16(9), e0257234.
  • Rahimi, I., Chen, F., & Gandomi, A. H. (2023). A review on COVID-19 forecasting models. Neural Computing and Applications, 35, 23671–23681.
  • Rahimi, I., Gandomi, A. H., Asteris, P. G., & Chen, F. (2021). Analysis and prediction of COVID-19 using SIR, SEIQR, and machine learning models: Australia, Italy, and UK cases. Information, 12, 109.
  • Rahman, M. M., Islam, M. M., Manik, M. M. H., Islam, M. R., & Al-Rakhami, M. S. (2021). Machine learning approaches for tackling novel coronavirus (COVID-19) pandemic. SN Computer Science, 2, 384.
  • Rustam, F., Reshi, A. A., Mehmood, A., Ullah, S., On, B.-W., Aslam, A., & Choi, G. S. (2020). COVID-19 future forecasting using supervised machine learning models. IEEE Access, 8, 101489–101499.
  • Saba, T., Abunadi, I., Shahzad, M. N., & Khan, A. R. (2021). Machine learning techniques to detect and forecast the daily total COVID-19 infected and deaths cases under different lockdown types. Microscopy Research and Technique, 84(7), 1462–1474. https://doi.org/ 10.1002/jemt.23702.
  • Satu, M. S., Howlader, K. C., Mahmud, M., Kaiser, M. S., Shariful Islam, S. M., Quinn, J. M. W., Alyami, S. A., & Moni, M. A. (2021). Short-term prediction of COVID-19 cases using machine learning models. Applied Sciences, 11, 4266.
  • Sayed, S. A.-F., Elkorany, A. M., & Sayed Mohammad, S. (2021). Applying different machine learning techniques for prediction of COVID-19 severity. IEEE Access, 9, 135697–135707.
  • Shahin, O. R., Alshammari, H. H., Taloba, A. I., & El-Aziz, R. M. A. (2022). Machine learning approach for autonomous detection and classification of COVID-19 virus. Computers and Electrical Engineering, 101, 108055.
  • Solayman, S., Aumi, S. A., Mery, C. S., Mubassir, M., & Khan, R. (2023). Automatic COVID-19 prediction using explainable machine learning techniques. International Journal of Cognitive Computing in Engineering, 4, 36–46.
  • Su, X., Sun, Y., Liu, H., Lang, Q., Zhang, Y., Zhang, J., Wang, C., & Chen, Y. (2023). An innovative ensemble model based on deep learning for predicting COVID-19 infection. Scientific Reports, 13, 12322.
  • Subudhi, S., Verma, A., Patel, A. B., Hardin, C. C., Khandekar, M. J., Lee, H., McEvoy, D., Stylianopoulos, T., Munn, L. L., Dutta, S., & Jain, R. K. (2021). Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19. npj Digital Medicine, 4, 87.
  • Tiwari, S., Chanak, P., & Singh, S. K. (2022). A review of the machine learning algorithms for Covid-19 case analysis. IEEE Transactions on Artificial Intelligence, 4(1), 44–59.
  • Tuli, S., Tuli, S., Tuli, R., & Gill, S. S. (2020). Predicting the growth and trend of COVID-19 pandemic using machine learning and cloud computing. Internet of Things, 11, 100222.
  • Ustebay, S., Sarmis, A., Kaya, G. K., & Sujan, M. (2023). A comparison of machine learning algorithms in predicting COVID-19 prognostics. Internal and Emergency Medicine, 18, 229–239.
  • Vepa, A., Saleem, A., Rakhshan, K., Daneshkhah, A., Sedighi, T., Shohaimi, S., Omar, A., Salari, N., Chatrabgoun, O., Dharmaraj, D., Sami, J., Parekh, S., Ibrahim, M., Raza, M., Kapila, P., & Chakrabarti, P. (2021). Using machine learning algorithms to develop a clinical decision-making tool for COVID-19 inpatients. International Journal of Environmental Research and Public Health, 18, 6228.
  • Verma, H., Mandal, S., & Gupta, A. (2022). Temporal deep learning architecture for prediction of COVID-19 cases in India. Expert Systems with Applications, 195, 116611.
  • Villavicencio, C. N., Macrohon, J. J. E., Inbaraj, X. A., Jeng, J.-H., & Hsieh, J.-G. (2021). COVID-19 Prediction applyin supervised machine learning algorithms with comparative analysis using WEKA. Algorithms, 14, 201.
  • Xu, W., Sun, N.-N., Gao, H.-N., Chen, Z.-Y., Yang, Y., Ju, B., & Tang, L.-L. (2021). Risk factors analysis of COVID-19 patients with ARDS and prediction based on machine learning. Scientific Reports, 11, 2933.
  • Yeşilkanat, C. M. (2020). Spatio-temporal estimation of the daily cases of COVID-19 in worldwide using random forest machine learning algorithm. Chaos, Solitons & Fractals, 140, 110210.
  • Yu, L., Halalau, A., Dalal, B., Abbas, A. E., Ivascu, F., Amin, M., & Nair, G. B. (2021). Machine learning methods to predict mechanical ventilation and mortality in patients with COVID-19. PLOS ONE, 16(4), e0249285.
  • Zeroual, A., Harrou, F., Dairi, A., & Sun, Y. (2020). Deep learning methods for forecasting COVID-19 time-Series data: A Comparative study. Chaos, Solitons & Fractals, 140, 110121.
  • Zhang, X., Lu, S., Wang, S.-H., Yu, X., Wang, S.-J., Yao, L., Pan, Y., & Zhang, Y.-D. (2022). Diagnosis of COVID-19 pneumonia via a novel deep learning architecture. Journal of Computer Science and Technology, 37, 330–343.
  • Zivkovic, M., Bacanin, N., Venkatachalam, K., Nayyar, A., Djordjevic, A., Strumberger, I., & Al-Turjman, F. (2021). COVID-19 cases prediction by using hybrid machine learning and beetle antennae search approach. Sustainable Cities and Society, 66, 102669.
  • Zoabi, Y., Deri-Rozov, S., & Shomron, N. (2021). Machine learning-based prediction of COVID-19 diagnosis based on symptoms. npj Digital Medicine, 4, 3.

Document Type

Publication order reference

Identifiers

Biblioteka Nauki
39832736

YADDA identifier

bwmeta1.element.ojs-doi-10_24136_oc_2984
JavaScript is turned off in your web browser. Turn it on to take full advantage of this site, then refresh the page.