Machine learning (ML) techniques are extensively applied to practical maritime transportation issues. Due to the difficulty and high cost of collecting large volumes of data in the maritime industry, in many maritime studies, ML models are trained with small training datasets. The relative predictive performances of these trained ML models are then compared with each other and with the conventional model using the same test set. The ML model that performs the best out of the ML models and better than the conventional model on the test set is regarded as the most effective in terms of this prediction task. However, in scenarios with small datasets, this common process may lead to an unfair comparison between the ML and the conventional model. Therefore, we propose a novel process to fairly compare multiple ML models and the conventional model. We first select the best ML model in terms of predictive performance for the validation set. Then, we combine the training and the validation sets to retrain the best ML model and compare it with the conventional model on the same test set. Based on historical port state control (PSC) inspection data, we examine both the common process and the novel process in terms of their ability to fairly compare ML models and the conventional model. The results show that the novel process is more effective at fairly comparing the ML models with the conventional model on different test sets. Therefore, the novel process enables a fair assessment of ML models' ability to predict key performance indicators in the context of limited data availability in the maritime industry, such as predicting the ship fuel consumption and port traffic volume, thereby enhancing their reliability for real-world applications.
Citation: Xi Luo, Ran Yan, Shuaian Wang, Lu Zhen. A fair evaluation of the potential of machine learning in maritime transportation[J]. Electronic Research Archive, 2023, 31(8): 4753-4772. doi: 10.3934/era.2023243
Machine learning (ML) techniques are extensively applied to practical maritime transportation issues. Due to the difficulty and high cost of collecting large volumes of data in the maritime industry, in many maritime studies, ML models are trained with small training datasets. The relative predictive performances of these trained ML models are then compared with each other and with the conventional model using the same test set. The ML model that performs the best out of the ML models and better than the conventional model on the test set is regarded as the most effective in terms of this prediction task. However, in scenarios with small datasets, this common process may lead to an unfair comparison between the ML and the conventional model. Therefore, we propose a novel process to fairly compare multiple ML models and the conventional model. We first select the best ML model in terms of predictive performance for the validation set. Then, we combine the training and the validation sets to retrain the best ML model and compare it with the conventional model on the same test set. Based on historical port state control (PSC) inspection data, we examine both the common process and the novel process in terms of their ability to fairly compare ML models and the conventional model. The results show that the novel process is more effective at fairly comparing the ML models with the conventional model on different test sets. Therefore, the novel process enables a fair assessment of ML models' ability to predict key performance indicators in the context of limited data availability in the maritime industry, such as predicting the ship fuel consumption and port traffic volume, thereby enhancing their reliability for real-world applications.
| [1] |
M. I. Jordan, T. M. Mitchell, Machine learning: Trends, perspectives, and prospects, Science, 349 (2015), 255–260. https://doi.org/10.1126/science.aaa8415 doi: 10.1126/science.aaa8415
|
| [2] |
I. H. Sarker, Machine learning: Algorithms, real-world applications and research directions, SN Comput. Sci., 2 (2021), 1–21. https://doi.org/10.1007/s42979-021-00592-x doi: 10.1007/s42979-021-00592-x
|
| [3] |
Y. Zhang, C. Ling, A strategy to apply machine learning to small datasets in materials science, npj Comput. Mater., 4 (2018). https://doi.org/10.1038/s41524-018-0081-z doi: 10.1038/s41524-018-0081-z
|
| [4] |
N. Ghadami, M. Gheibi, Z. Kian, M. G. Faramarz, R. Naghedi, M. Eftekhari, et al., Implementation of solar energy in smart cities using an integration of artificial neural network, photovoltaic system and classical Delphi methods, Sustainable Cities Soc., 74 (2021), 103149. https://doi.org/10.1016/j.scs.2021.103149 doi: 10.1016/j.scs.2021.103149
|
| [5] |
R. Yan, S. Wang, H. N. Psaraftis, Data analytics for fuel consumption management in maritime transportation: Status and perspectives, Transp. Res. Part E Logist. Transp. Rev., 155 (2021), 102489. https://doi.org/10.1016/j.tre.2021.102489 doi: 10.1016/j.tre.2021.102489
|
| [6] |
R. Yan, S. Wang, L. Zhen, G. Laporte, Emerging approaches applied to maritime transport research: Past and future, Commun. Transp. Res., 1 (2021), 100011. https://doi.org/10.1016/j.commtr.2021.100011 doi: 10.1016/j.commtr.2021.100011
|
| [7] |
T. Uyanık, Ç. Karatuğ, Y. Arslanoğlu, Machine learning approach to ship fuel consumption: A case of container vessel, Transp. Res. Part D Transp. Environ., 84 (2020), 102389. https://doi.org/10.1016/j.trd.2020.102389 doi: 10.1016/j.trd.2020.102389
|
| [8] |
A. Mazaheri, J. Montewka, P. Kujala, Modeling the risk of ship grounding—a literature review from a risk management perspective, WMU J. Marit. Aff., 13 (2014), 269–297. https://doi.org/10.1007/s13437-013-0056-3 doi: 10.1007/s13437-013-0056-3
|
| [9] |
B. Wu, X. Yan, T. L. Yip, Y. Wang, A flexible decision-support solution for intervention measures of grounded ships in the Yangtze River, Ocean Eng., 141 (2017), 237–248. https://doi.org/10.1016/j.oceaneng.2017.06.021 doi: 10.1016/j.oceaneng.2017.06.021
|
| [10] |
R. Yan, S. Wang, C. Peng, Ship selection in port state control: Status and perspectives, Marit. Policy Manage., 49 (2022), 600–615. https://doi.org/10.1080/03088839.2021.1889067 doi: 10.1080/03088839.2021.1889067
|
| [11] |
Z. Yang, Z. Yang, J. Yin, Realising advanced risk-based port state control inspection using data-driven Bayesian networks, Transp. Res. Part A Policy Pract., 110 (2018), 38–56. https://doi.org/10.1016/j.tra.2018.01.033 doi: 10.1016/j.tra.2018.01.033
|
| [12] |
Y. Leonov, V. Nikolov, A wavelet and neural network model for the prediction of dry bulk shipping indices, Marit. Econ. Logist., 14 (2012), 319–333. https://doi.org/10.1057/mel.2012.10 doi: 10.1057/mel.2012.10
|
| [13] |
Z. Yang, E. E. Mehmed, Artificial neural networks in freight rate forecasting, Marit. Econ. Logist., 21 (2019), 390–414. https://doi.org/10.1057/s41278-019-00121-x doi: 10.1057/s41278-019-00121-x
|
| [14] |
Q. Bi, K. E. Goodman, J. Kaminsky, J. Lessler, What is machine learning? A primer for the epidemiologist, Am. J. Epidemiol., 188 (2019), 2222–2239. https://doi.org/10.1093/aje/kwz189 doi: 10.1093/aje/kwz189
|
| [15] |
F. A. Faber, A. Lindmaa, O. A. Von Lilienfeld, R. Armiento, Machine learning energies of 2 million elpasolite (ABC2D6) crystals, Phys. Rev. Lett., 117 (2016), 135502. https://doi.org/10.1103/PhysRevLett.117.135502 doi: 10.1103/PhysRevLett.117.135502
|
| [16] |
W. Ng, B. Minasny, W. D. S. Mendes, J. A. M. Demattê, The influence of training sample size on the accuracy of deep learning models for the prediction of soil properties with near-infrared spectroscopy data, Soil, 6 (2020), 565–578. https://doi.org/10.5194/soil-6-565-2020 doi: 10.5194/soil-6-565-2020
|
| [17] | C. Baur, S. Albarqouni, N. Navab, Semi-supervised deep learning for fully convolutional networks, in Medical Image Computing and Computer Assisted Intervention−MICCAI 2017, Springer, (2017), 311–319. https://doi.org/10.48550/arXiv.1703.06000 |
| [18] | N. Doulamis, A. Doulamis, Semi-supervised deep learning for object tracking and classification, in 2014 IEEE International Conference on Image Processing (ICIP), (2014), 848–852. https://doi.org/10.1109/ICIP.2014.7025170 |
| [19] |
H. Wu, S. Prasad, Semi-supervised deep learning using pseudo labels for hyperspectral image classification, IEEE Trans. Image Process., 27 (2017), 1259–1270. https://doi.org/10.1109/TIP.2017.2772836 doi: 10.1109/TIP.2017.2772836
|
| [20] |
J. P. Petersen, O. Winther, D. J. Jacobsen, A machine-learning approach to predict main energy consumption under realistic operational conditions, Ship Technol. Res., 59 (2012), 64–72. https://doi.org/10.1179/str.2012.59.1.007 doi: 10.1179/str.2012.59.1.007
|
| [21] |
D. Ronen, The effect of oil price on the optimal speed of ships, J. Oper. Res. Soc., 33 (1982), 1035–1040. https://doi.org/10.1057/jors.1982.215 doi: 10.1057/jors.1982.215
|
| [22] |
S. C. Ryder, D. Chappell, Optimal speed and ship size for the liner trades, Marit. Policy Manage., 7 (1980), 55–57. https://doi.org/10.1080/03088838000000053 doi: 10.1080/03088838000000053
|
| [23] |
S. Wang, Q. Meng, Sailing speed optimization for container ships in a liner shipping network, Transp. Res. Part E Logist. Transp. Rev., 48 (2012), 701–714. https://doi.org/10.1016/j.tre.2011.12.003 doi: 10.1016/j.tre.2011.12.003
|
| [24] |
C. Gkerekos, I. Lazakis, G. Theotokatos, Machine learning models for predicting ship main engine fuel oil consumption: A comparative study, Ocean Eng., 188 (2019), 106282. https://doi.org/10.1016/j.oceaneng.2019.106282 doi: 10.1016/j.oceaneng.2019.106282
|
| [25] |
T. Uyanık, Y. Yalman, Ö. Kalenderli, Y. Arslanoğlu, Y. Terriche, C. L. Su, et al., Data-driven approach for estimating power and fuel consumption of ship: A case of container vessel, Mathematics, 10 (2022), 4167. https://doi.org/10.3390/math10224167 doi: 10.3390/math10224167
|
| [26] |
X. Li, Y. Du, Y. Chen, S. Nguyen, W. Zhang, A. Schönborn, et al., Data fusion and machine learning for ship fuel efficiency modeling: Part I–Voyage report data and meteorological data, Commun. Transp. Res., 2 (2022), 100074. https://doi.org/10.1016/j.commtr.2022.100074 doi: 10.1016/j.commtr.2022.100074
|
| [27] |
Y. Du, Y. Chen, X. Li, A. Schönborn, Z. Sun, Data fusion and machine learning for ship fuel efficiency modeling: Part Ⅱ–Voyage report data, AIS data and meteorological data, Commun. Transp. Res., 2 (2022), 100073. https://doi.org/10.1016/j.commtr.2022.100073 doi: 10.1016/j.commtr.2022.100073
|
| [28] |
S. Wang, R. Yan, X. Qu, Development of a non-parametric classifier: Effective identification, algorithm, and applications in port state control for maritime transportation, Transp. Res. Part B Methodol., 128 (2019), 129–157. https://doi.org/10.1016/j.trb.2019.07.017 doi: 10.1016/j.trb.2019.07.017
|
| [29] |
R. Yan, S. Wang, K. Fagerholt, A semi-"smart predict then optimize" (semi-SPO) method for efficient ship inspection, Transp. Res. Part B Methodol., 142 (2020), 100–125. https://doi.org/10.1016/j.trb.2020.09.014 doi: 10.1016/j.trb.2020.09.014
|
| [30] |
S. Wu, X. Chen, C. Shi, J. Fu, Y. Yan, S. Wang, Ship detention prediction via feature selection scheme and support vector machine (SVM), Marit. Policy Manage., 49 (2022), 140–153. https://doi.org/10.1080/03088839.2021.1875141 doi: 10.1080/03088839.2021.1875141
|
| [31] | WRS, World Shipping Register, 2023. Available from: https://world-ships.com/. |
| [32] |
W. Yi, S. Wu, L. Zhen, G. Chawynski, Bi-level programming subsidy design for promoting sustainable prefabricated product logistics, Cleaner Logist. Supply Chain, 1 (2021), 100005. https://doi.org/10.1016/j.clscn.2021.100005 doi: 10.1016/j.clscn.2021.100005
|
| [33] |
W. Yi, L. Zhen, Y. Jin, Stackelberg game analysis of government subsidy on sustainable off-site construction and low-carbon logistics, Cleaner Logist. Supply Chain, 2 (2021), 100013. https://doi.org/10.1016/j.clscn.2021.100013 doi: 10.1016/j.clscn.2021.100013
|
| [34] |
X. Bai, L. Cheng, Ç. Iris, Data-driven financial and operational risk management: Empirical evidence from the global tramp shipping industry, Transp. Res. Part E Logist. Transp. Rev., 158 (2022), 102617. https://doi.org/10.1016/j.tre.2022.102617 doi: 10.1016/j.tre.2022.102617
|
| [35] |
X. Chen, S. Wu, Y. Liu, W. Wu, S. Wang, A patrol routing problem for maritime crime-fighting, Transp. Res. Part E Logist. Transp. Rev., 168 (2022), 102940. https://doi.org/10.1016/j.tre.2022.102940 doi: 10.1016/j.tre.2022.102940
|
| [36] |
Z. Song, W. Tang, R. Zhao, G. Zhang, Implications of government subsidies on shipping companies' shore power usage strategies in port, Transp. Res. Part E Logist. Transp. Rev., 165 (2022), 102840. https://doi.org/10.1016/j.tre.2022.102840 doi: 10.1016/j.tre.2022.102840
|
| [37] |
Z. Tan, X. Zeng, S. Shao, J. Chen, H. Wang, Scrubber installation and green fuel for inland river ships with non-identical streamflow, Transp. Res. Part E Logist. Transp. Rev., 161 (2022), 102677. https://doi.org/10.1016/j.tre.2022.102677 doi: 10.1016/j.tre.2022.102677
|
| [38] |
Z. Tan, M. Zhang, S. Shao, J. Liang, D. Sheng, Evasion strategy for a coastal cargo ship with unpunctual arrival penalty under sulfur emission regulation, Transp. Res. Part E Logist. Transp. Rev., 164 (2022), 102818. https://doi.org/10.1016/j.tre.2022.102818 doi: 10.1016/j.tre.2022.102818
|
| [39] |
L. Zhen, W. Wang, S. Lin, Analytical comparison on two incentive policies for shore power equipped ships in berthing activities, Transp. Res. Part E Logist. Transp. Rev., 161 (2022), 102686. https://doi.org/10.1016/j.tre.2022.102686 doi: 10.1016/j.tre.2022.102686
|
| [40] |
P. Cariou, M. Q. Mejia Jr, F. C. Wolff, An econometric analysis of deficiencies noted in port state control inspections, Marit. Policy Manage., 34 (2007), 243–258. https://doi.org/10.1080/03088830701343047 doi: 10.1080/03088830701343047
|
| [41] |
P. Cariou, M. Q. Mejia, F. C. Wolff, Evidence on target factors used for port state control inspections, Mar. Policy, 33 (2009), 847–859. https://doi.org/10.1016/j.marpol.2009.03.004 doi: 10.1016/j.marpol.2009.03.004
|
| [42] |
Ş. Şanlıer, Analysis of port state control inspection data: The Black Sea Region, Mar. Policy, 112 (2020), 103757. https://doi.org/10.1016/j.marpol.2019.103757 doi: 10.1016/j.marpol.2019.103757
|
| [43] | Tokyo MoU, Black–Grey–White lists, 2017. Available from: https://www.tokyo-mou.org/doc/Flag%20performance%20list%202020.pdf. |
| [44] | Tokyo MoU, Information sheet of the new inspection regime (NIR), 2014. Available from: https://www.tokyo-mou.org/doc/NIR-information%20sheet-r.pdf. |
| [45] | Paris MoU, Criteria for responsibility assessment of recognized organizations (RO), 2013. Available from: https://www.parismou.org/criteria-ro-responsibility-assessment. |
| [46] | C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), Springer-Verlag, 2006. |
| [47] | T. Hastie, R. Tibshirani, J. H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York, 2009. |
| [48] | M. H. Hassoun, Fundamentals of Artificial Neural Networks, MIT press, Cambridge, 1995. |
| [49] | K. L. Priddy, P. E. Keller, Artificial Neural Networks: An Introduction, Society of Photo-Optical Instrument Engineers (SPIE), Bellingham, 2005. |
| [50] | D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint, (2017), arXiv: 1412.6980. https://doi.org/10.48550/arXiv.1412.6980 |
| [51] |
A. Dadashi, M. A. Dulebenets, M. M. Golias, A. Sheikholeslami, A novel continuous berth scheduling model at multiple marine container terminals with tidal considerations, Marit. Bus. Rev., 2 (2017), 142–157. https://doi.org/10.1108/MABR-02-2017-0010 doi: 10.1108/MABR-02-2017-0010
|
| [52] |
M. A. Dulebenets, A novel memetic algorithm with a deterministic parameter control for efficient berth scheduling at marine container terminals, Marit. Bus. Rev., 2 (2017), 302–330. https://doi.org/10.1108/MABR-04-2017-0012 doi: 10.1108/MABR-04-2017-0012
|
| [53] |
M. Kavoosi, M. A. Dulebenets, O. Abioye, J. Pasha, O. Theophilus, H. Wang, et al., Berth scheduling at marine container terminals: A universal island-based metaheuristic approach, Marit. Bus. Rev., 5 (2019), 30–66. https://doi.org/10.1108/MABR-08-2019-0032 doi: 10.1108/MABR-08-2019-0032
|
| [54] |
M. Kavoosi, M. A. Dulebenets, O. F. Abioye, J. Pasha, H. Wang, H. Chi, An augmented self-adaptive parameter control in evolutionary computation: A case study for the berth scheduling problem, Adv. Eng. Inform., 42 (2019), 100972. https://doi.org/10.1016/j.aei.2019.100972 doi: 10.1016/j.aei.2019.100972
|
| [55] |
M. A. Dulebenets, An Adaptive Island Evolutionary Algorithm for the berth scheduling problem, Memet. Comput., 12 (2020), 51–72. https://doi.org/10.1007/s12293-019-00292-3 doi: 10.1007/s12293-019-00292-3
|
| [56] |
D. Kizilay, D. T. Eliiyi, A comprehensive review of quay crane scheduling, yard operations and integrations thereof in container terminals, Flexible Serv. Manuf. J., 33 (2021), 1–42. https://doi.org/10.1007/s10696-020-09385-5 doi: 10.1007/s10696-020-09385-5
|
| [57] |
B. G. Zweers, S. Bhulai, R. D. van der Mei, Planning hinterland container transportation in congested deep-sea terminals, Flexible Serv. Manuf. J., 33 (2021), 583–622. https://doi.org/10.1007/s10696-020-09387-3 doi: 10.1007/s10696-020-09387-3
|
| [58] |
S. Tang, S. Xu, J. Gao, M. Ma, P. Liao, Effect of service priority on the integrated continuous berth allocation and quay crane assignment problem after port congestion, J. Mar. Sci. Eng., 10 (2022), 1259. https://doi.org/10.3390/jmse10091259 doi: 10.3390/jmse10091259
|
| [59] |
L. Guo, J. Zheng, H. Du, J. Du, Z. Zhu, The berth assignment and allocation problem considering cooperative liner carriers, Transp. Res. Part E Logist. Transp. Rev., 164 (2022), 102793. https://doi.org/10.1016/j.tre.2022.102793 doi: 10.1016/j.tre.2022.102793
|
| [60] |
L. Kolley, N. Rückert, M. Kastner, C. Jahn, K. Fischer, Robust berth scheduling using machine learning for vessel arrival time prediction, Flexible Serv. Manuf. J., 35 (2023), 29–69. https://doi.org/10.1007/s10696-022-09462-x doi: 10.1007/s10696-022-09462-x
|
| [61] |
J. He, N. Yan, J. Zhang, T. Wang, Battery electric buses charging schedule optimization considering time-of-use electricity price, J. Intell. Connected Veh., 5 (2022), 138–145. https://doi.org/10.1108/JICV-03-2022-0006 doi: 10.1108/JICV-03-2022-0006
|
| [62] | X. Qu, Y. Liu, Y. Chen, Y. Bie, Urban electric bus operation management: Review and outlook, J. Automot. Saf. Energy, 3 (2022), 407–420. |
| [63] | C. Sun, B. Liu, F. Sun, Review of energy-saving planning and control technology for new energy vehicles, J. Automot. Saf. Energy, 4 (2022), 593–616. |
| [64] | H. Wang, M. Ouyang, J. Li, F. Yang, Hydrogen fuel cell vehicle technology roadmap and progress in China, J. Automot. Saf. Energy, 2 (2022), 211–224. |
| [65] |
L. Xu, S. Jin, B. Li, J. Wu, Traffic signal coordination control for arterials with dedicated CAV lanes, J. Intell. Connected Veh., 5 (2022), 72–87. https://doi.org/10.1108/JICV-08-2021-0015 doi: 10.1108/JICV-08-2021-0015
|
Figures(5) / Tables(6)
Xi Luo, Ran Yan, Shuaian Wang, Lu Zhen. A fair evaluation of the potential of machine learning in maritime transportation[J]. Electronic Research Archive, 2023, 31(8): 4753-4772. doi: 10.3934/era.2023243
DownLoad: