The internet of things (IoT) and deep learning are emerging technologies in diverse research fields, including the provision of IT services in medical domains. In the COVID-19 era, intelligent medication behavior monitoring systems for stable patient monitoring are further required, because many patients cannot easily visit hospitals. Several previous studies made use of wearable devices to detect medication behaviors of patients. However, the wearable devices cause inconvenience while equipping the devices. In addition, they suffer from inconsistency problems due to errors of measured values. We devise a medication behavior monitoring system that uses the IoT and deep learning to avoid sensing errors and improve user experiences by effectively detecting various activities of patients. Based on the real-time operation of our proposed IoT device, the proposed solution processes captured images of patents via OpenPose to check medication situations. The proposed system identifies medication status on time by using a human activity recognition scheme and provides various notifications to patients' mobile devices. To support reliable communication between our system and doctors, we employ MQTT protocol with periodic data transmissions. Thus, the measured information of patient's medication status is transmitted to the doctors so that they can periodically perform remote treatments. Experimental results show that all medication behaviors are accurately detected and notified to the doctor efficiently, improving the accuracy of monitoring the patient's medication behavior.
Citation: Hyeji Roh, Seulgi Shin, Jinseo Han, Sangsoon Lim. A deep learning-based medication behavior monitoring system[J]. Mathematical Biosciences and Engineering, 2021, 18(2): 1513-1528. doi: 10.3934/mbe.2021078
The internet of things (IoT) and deep learning are emerging technologies in diverse research fields, including the provision of IT services in medical domains. In the COVID-19 era, intelligent medication behavior monitoring systems for stable patient monitoring are further required, because many patients cannot easily visit hospitals. Several previous studies made use of wearable devices to detect medication behaviors of patients. However, the wearable devices cause inconvenience while equipping the devices. In addition, they suffer from inconsistency problems due to errors of measured values. We devise a medication behavior monitoring system that uses the IoT and deep learning to avoid sensing errors and improve user experiences by effectively detecting various activities of patients. Based on the real-time operation of our proposed IoT device, the proposed solution processes captured images of patents via OpenPose to check medication situations. The proposed system identifies medication status on time by using a human activity recognition scheme and provides various notifications to patients' mobile devices. To support reliable communication between our system and doctors, we employ MQTT protocol with periodic data transmissions. Thus, the measured information of patient's medication status is transmitted to the doctors so that they can periodically perform remote treatments. Experimental results show that all medication behaviors are accurately detected and notified to the doctor efficiently, improving the accuracy of monitoring the patient's medication behavior.
| [1] | K. P. Kibiwott, Y. Zhao, J. Kogo, F Zhang, Verifiable fully outsourced attribute-based signcryption system for IoT, Math. Biosci. Eng., 16 (2019), 3561-3594. |
| [2] |
C. C. Chang, C. T. Li, Algebraic secret sharing using privacy homomorphisms for IoT-based healthcare systems, Math. Biosci. Eng., 16 (2019), 3367-3381. doi: 10.3934/mbe.2019168
|
| [3] | M. Aldeer, M. Javanmard, R. P. Martin, A Review of Medication Adherence Monitoring Technologies, Appl. Syst. Innovation, 2 (2018), 14. |
| [4] | C. E. Koop, R. Monsher, L. Kun, J. Geiling, E. Grigg, S. Long, et al., Future delivery of health care: Cybercare, IEEE Eng. Med. Biol. Mag., 27 (2008), 29-38. |
| [5] | R. Mutegeki, D. S. Han, A CNN-LSTM approach to human activity recognition, 2020 International Conference on Artificial Intelligence in Information and Communication, 2020. |
| [6] |
M. C. Sokol, K. A. McGuigan, R. R. Verbrugge, R. S. Epstein, Impact of medication adherence on hospitalization risk and healthcare cost, Med. Care, 43 (2005), 521-530. doi: 10.1097/01.mlr.0000163641.86870.af
|
| [7] | C. Zachariadis, T. H. Velivassaki, T. Zahariadis, K. Railis, H. C. Leligou, Matisse: A Smart Hospital Ecosystem, 2018 21st Euromicro Conference on Digital System Design (DSD), 2018. |
| [8] | V. Sanchez, R. Ro, L. Kent, J. Navas, O. Diyan, C. Roussel, et al., ScanAlert: Electronic Medication Monitor and Reminder to Improve Medical Adherence, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), 2019. |
| [9] | M. Ervasti, M. Isomursu, I. I. Leibar, Touch- and audio-based medication management service concept for vision impaired older people, IEEE International Conference on RFID-Technologies and Applications, 2011. |
| [10] |
S. Koch, Home telehealth-Current state and future trends, Int. J. Med. Inf., 75 (2006), 565-576. doi: 10.1016/j.ijmedinf.2005.09.002
|
| [11] | T. Edoh, Smart medicine transportation and medication monitoring system in EPharmacyNet, 2017 International Rural and Elderly Health Informatics Conference (IREHI), 2017. |
| [12] | F. Alshammari, K. Tearo, R. Orji, K. Hawkey, D. Reilly, MAR: A Study of the Impact of Positive and Negative Reinforcement on Medication Adherence Reminders, 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH), 2020. |
| [13] | A. Prakash, J. M. Beer, T. Deyle, C. Smarr, T. L. Chen, T. L. Mitzner, et al., Older Adults' Medication Management in the Home: How can Robots Help?, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2013. |
| [14] | M. Aldeer, R. P. Martin, Medication adherence monitoring using modern technology, 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), 2017. |
| [15] |
H. Yan, H. Huo, Y. Xu, M. Gidlund, Wireless sensor network based e-health system-implementation and experimental results, IEEE Trans. Consum. Electron., 56 (2010), 2288-2295. doi: 10.1109/TCE.2010.5681102
|
| [16] |
V. G. Koutkias, I. Chouvarda, A. Triantafyllidis, A. Malousi, G. D. Giaglis, N. Maglaveras, A Personalized Framework for Medication Treatment Management in Chronic Care, IEEE Trans. Inf. Technol. Biomed., 14 (2010), 464-472. doi: 10.1109/TITB.2009.2036367
|
| [17] |
J. Yu, Z. Chen, S. Kamata, J. Yang, Accurate system for automatic pill recognition using imprint information, IET Image Process., 9 (2015), 1039-1047. doi: 10.1049/iet-ipr.2014.1007
|
| [18] | S. Ling, A. Pastor, J. Li, Z. Che, J. Wang, J. Kim, et al., Few-Shot Pill Recognition, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. |
| [19] | J. Ma, A. Ovalle, D. M. Woodbridge, Medhere: A Smartwatch-based Medication Adherence Monitoring System using Machine Learning and Distributed Computing, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018. |
| [20] | A. Cheon, S. Y. Jung, C. Prather, M. Sarmiento, K. Wong, D. M. Woodbridge, A Machine Learning Approach to Detecting Low Medication State with Wearable Technologies, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2020. |
| [21] | J. E. Pedi Reddy, A. Chavan, AI-IoT based Smart Pill Expert System, 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI), 2020. |
| [22] |
W. Chang, L. Chen, C. Hsu, J. Chen, T. Yang, C. Lin, MedGlasses: A Wearable Smart-Glasses-Based Drug Pill Recognition System Using Deep Learning for Visually Impaired Chronic Patients, IEEE Access, 8 (2020), 17013-17024. doi: 10.1109/ACCESS.2020.2967400
|
| [23] | Y. Wang, J. Ribera, C. Liu, S. Yarlagadda, F. Zhu, Pill Recognition Using Minimal Labeled Data, 2017 IEEE Third International Conference on Multimedia Big Data (BigMM), 2017. |
| [24] | Y. Chen, K. Zhong, J. Zhang, Q. Sun, X. Zhao, LSTM Networks for Mobile Human Activity Recognition, International Conference on Artificial Intelligence: Technologies and Applications (ICAITA 2016), 2016. |
Figures(11) / Tables(3)
Hyeji Roh, Seulgi Shin, Jinseo Han, Sangsoon Lim. A deep learning-based medication behavior monitoring system[J]. Mathematical Biosciences and Engineering, 2021, 18(2): 1513-1528. doi: 10.3934/mbe.2021078
DownLoad: