Medical image management and analysis system based on web for fungal keratitis images

1.   Introduction

With the fast development of computer technology, medical imaging technology has been continuously improved, and the corresponding medical imaging equipment has been gradually updated ^[1]. These technologies and equipment have provided great help for the diagnosis and treatment of diseases. The number of medical images generated by the hospital is extremely large. How to manage it is also an important task. At the same time, all these medical images require doctors to diagnose ^[2]. For diagnosis, doctors in the imaging department are extremely scarce compared to this fast-growing huge imaging data. Doctors may miss some diseases due to fatigue or lack of experience ^[3]. At present, medical image management systems at home and abroad are rarely combined with computer-aided diagnosis systems. These two systems are independent, which reduces the work efficiency of the hospital.

Fungal keratitis is a disease of the ophthalmology, which is caused by trauma to the patient's eye and infection by external bacteria ^[4,5]. If not treated in time, it may lead to permanent blindness in the patient, and the blindness rate is second only to cataract. Only early diagnosis can better take treatment to reduce the rate of blindness ^[6,7]. There are many ways to check for fungal keratitis ^[8,9]. Traditional auxiliary tests include a smear on the cornea and a smear test and culture method under the microscope. Confocal Microscopy (CM) is a biopsy technique that can perform a live cornea. A non-invasive rapid test is a powerful tool for the diagnosis of fungal keratitis ^[9]. As shown in Figure 1, the corneal images taken by CM (model: Heidelberg HRT-3) are divided into normal and fungal keratitis groups. It is observed that the normal group has a clear background and fungal keratitis group. The background is messy, with hyphae or spores visible.

Figure 1.  The corneal images were taken by confocal microscopy. a. Normal corneal confocal microscopy image. b. Corneal confocal microscopy image of fungal keratitis.

DownLoad: Full-Size Img PowerPoint

Now more and more classification methods are used to classify medical diseases ^[10,11]. The medical image management and analysis system designed in this paper not only meets the management needs of hospital image data but also realizes the automatic diagnosis of confocal microscope images of fungal keratitis. This paper proposes a web-based medical image management and analysis system. Taking the image of fungal keratitis as an example, this article first selects the AlexNet ^[12], ZFNet ^[13], VGG16 ^[14] network as the classification network for classification, and then selects the optimal classification model. Based on the optimal classification model, this paper designs a web-based medical image management and analysis system. Through this system, medical user information can be systematically managed and intelligent diagnosis can be performed, thereby greatly improving the efficiency of doctors' diagnosis and treatment.

The rest of the paper is organized as follows: Section 2 introduces the basic structure of the system, Section 3 introduces the design of the system, and finally, Section 4 summarizes the paper and points out the possible work in the future.

2.   System introduction

2.1.   Requirements analysis

To implement the system, the system needs to implement the process shown in Figure 2. The main users of the system are medical staff and back-office management personnel. An analysis of these two roles shows that the user use case diagram of the system is shown in Figure 3. The functions of medical staff will meet the whole process of patients from visiting a doctor to printing graphic reports. The functions of background management staff make sure that the system runs stably.

Figure 2.  System work flow chart.

DownLoad: Full-Size Img PowerPoint

Figure 3.  Schematic diagram of system user instructions.

DownLoad: Full-Size Img PowerPoint

2.2.   Division of system function modules

The requirements analysis can be used to obtain which function the system needs to implement, so the functional requirements can be divided into functional modules. For medical staff, there are mainly patient appointment registration module, patient examination registration module, report module, computer-aided diagnosis module; for back-office management personnel, there are mainly user management module, department management module, imaging equipment management module, Log management module. The system functional architecture diagram is shown in Figure 4.

Figure 4.  System functional structure diagram.

DownLoad: Full-Size Img PowerPoint

3.   Design of system

3.1.   System overview

The medical image management and analysis system based on the B/S model adopts a layered design idea. The system is developed by the MVC architecture ^[15]. Figure 5 is the overall architecture diagram of the system, which is mainly divided into user layer, application layer, and data layer. The role of the user layer of this system is divided into background management personnel and medical staff. The application layer is the logical processing part. The main business is appointment registration, inspection registration, report, computer-aided diagnosis, and system management. In computer-aided diagnosis, the Keras deep learning platform interface is called and the diagnosis result is returned. Permission control and logging ensure the stable operation of the application layer and provide a certain guarantee for the system. The database provides data support for the above services, including the locations of the forms and images stored in the logical application layer. The front end of this system is developed using a combination of LayUI and JavaScript, and the back end is developed using the currently popular combination of SpringBoot ^[16] + MybatisPlus ^[17] + Mysql ^[18].

Figure 5.  Overall system architecture diagram.

DownLoad: Full-Size Img PowerPoint

3.2.   Design of data model and database

Design of data model. We elaborated on the system requirements analysis and the overall technical architecture of the system, and we need to design the data model before our formal development. As a method of data model design, the Entity-Relationship Diagram (E-R) Diagram can help us to view the data model more intuitively. The E-R diagram of this system is shown in Figure 6.

Figure 6.  The E-R diagram of this system.

DownLoad: Full-Size Img PowerPoint

Design of database. According to the E-R diagram of the system, we designed the database table and its table structure fields. The following table lists the main tables of the system.

The user information table (user_info). The user information table (Table 1) records the personal information of all users who can use the system, including administrators and medical staff. The differences between them are caused by different roles. At the same time, it also records the password and account when the user logs in. The password is encrypted with the MD5 algorithm to prevent the account from being stolen.

Table 1.  The user information table.

Field name	Date type	Length	IS NULL	Comment
user_id	bigint	20	N	Primary key
username	varchar	45	N	Login name
password	varchar	45	N	Login password
name	varchar	45	Y	Actual name
birthday	datetime	0	Y	Date of birth
sex	varchar	2	Y	Sex
email	varchar	45	N	Email
phone	varchar	45	Y	Phone
role_id	varchar	255	N	Role
dept_id	varchar	255	N	Department
status	varchar	45	N	Status, available by default
create_time	datetime	0	Y	Create time
create_user	varchar	255	Y	Create user
update_user	varchar	255	Y	Update user
update_time	datetime	0	Y	Update time

---

 | Show Table

DownLoad: CSV

The department information table (dept_info). The department information table (Table 2) records the information of different departments in the hospital, including name and introduction.

Table 2.  The department information table.

Field name	Date type	Length	IS NULL	Comment
dept_id	bigint	20	N	Primary key
name	varchar	45	N	Name
description	varchar	255	Y	Description
create_time	datetime	0	Y	Create time
create_user	varchar	255	Y	Create user
update_user	varchar	255	Y	Update user
update_time	datetime	0	Y	Update time

---

 | Show Table

DownLoad: CSV

The appointment registration table (preregis_patient). The appointment registration table (Table 3) records the information of the appointment examination before the patient enters the examination, including the basic personal information of the patient and the information that needs the appointment examination.

Table 3.  The appointment registration table.

Field name	Date type	Length	IS NULL	Comment
id	bigint	20	N	Appointment registration number
id_card	varchar	255	Y	Patient ID card number
name	varchar	10	Y	Patient name
age	int	10	Y	Patient age
sex	varchar	2	Y	Patient sex
phone	varchar	255	Y	Patient phone
status	int	2	Y	Patient sign-in status
plan_positon	varchar	255	Y	Site for appointment
plan_type	varchar	255	Y	Appointment check type
plan_time	datetime	0	Y	Schedule an appointment
plan_office	bigint	255	N	Foreign key, department's id
comment	varchar	255	Y	Comment
create_time	datetime	0	Y	Create time
create_user	varchar	255	Y	Create user
update_user	varchar	255	Y	Update user
update_time	datetime	0	Y	Update time

---

 | Show Table

DownLoad: CSV

The examination registration table (check_record). The examination registration table (Table 4) records the relevant information of the patients who come to the corresponding department for examination after the appointment of examination. In addition, it also includes the report ID and image path. The examination registration and the report are one-to-one, and there is only one report corresponding to it.

Table 4.  The examination registration table.

Field name	Date type	Length	IS NULL	Comment
id	bigint	20	N	Primary key
patient_id	bigint	2	N	Appointment registration number
check_type	varchar	255	Y	Type of inspection
check_positon	varchar	255	Y	Examined area
check_dep	bigint	20	N	Foreign key, department's id
picture_url	varchar	255	Y	Image path
report_id	bigint	20	N	Foreign key, report's id
status	int	2	Y	Status
create_time	datetime	0	Y	Create time
create_user	varchar	255	Y	Create user
update_user	varchar	255	Y	Update user
update_time	datetime	0	Y	Update time

---

 | Show Table

DownLoad: CSV

The report table (diagnose_report). The report table (Table 5) records the doctor's description and diagnosis results of the image diagnosis, and one report corresponds to one examination record.

Table 5.  The report table.

Field name	Date type	Length	IS NULL	Comment
id	bigint	20	N	Primary key
doctor_id	bigint	20	Y	Doctor's id
pic_expression	varchar	255	Y	Image representation
diagnose_advice	varchar	255	Y	Diagnose result
diagnose_time	datetime	0	Y	Diagnose time

---

 | Show Table

DownLoad: CSV

The fungal keratitis diagnosis record table (fungal_keratitis_diagnose). The fungal keratitis diagnosis record table (Table 6) records the relevant information of the diagnosis result obtained by the doctor calling the system to realize a good deep learning network, and records the diagnosis record every time the diagnosis is called.

Table 6.  The fungal keratitis diagnosis record table.

Field name	Date type	Length		IS NULL	Comment
id	bigint		20	N	Primary key
patient_id	bigint		20	Y	Appointment registration number
picture_url	varchar		255	Y	Image path
result	varchar		255	Y	Diagnose result
diagnose_time	datetime		0	Y	Diagnose time
diagnose_user	bigint		20	N	Doctor's id

---

 | Show Table

DownLoad: CSV

3.3.   Diagnosis of fungal keratitis based on deep learning

The images are divided into the normal group and the fungal keratitis patient group. The example diagrams are shown in Figure 1. A total of 1870 images, including 876 in the normal group and 994 in the fungal keratitis group. In order to increase the generalization ability of the model, this data set is divided according to the ratio of the training set: test set to 7:3. The specific sample distribution is shown in Table 7. In all subsequent experiments in this paper, if it is not re-declared, it means that the proportion is used for the experiment during the training.

Table 7.  Data set distribution.

Dataset	Training set	Test set	Total
Fungal keratitis	696	298	994
Normal	614	262	876
Total	1310	560	1870

---

 | Show Table

DownLoad: CSV

AlexNet, VGG16, and ZFNet are used in this experiment. The initial learning rate is set to 0.0001. The optimizer is trained by Adam to monitor the accuracy on the valid set, which is randomly selected from the training set during each training., the ratio of the training set: valid set to 5:1. If the accuracy on the valid set does not increase, the training ends when the iteration achieves 50, and the model with the highest accuracy is saved separately. Finally, the test set is put into the saved model for testing, and these models are compared using several evaluation indicators of accuracy, sensitivity, specificity, and Area Under Curve (AUC). These three networks have obtained very good results during the training process. The evaluation indicators of each network are shown in Table 8, and their ROC curves are shown in Figure 7.

Table 8.  Performance comparison table of each network.

Model	Accuracy	Sensitivity	Specificity	AUC
AlexNet	0.9875	0.9933	0.9810	0.9954
ZFNet	0.9911	0.9866	0.9962	0.9996
VGG16	0.9929	0.9933	0.9924	0.9997

---

 | Show Table

DownLoad: CSV

Figure 7.  ROC curve and AUC value of each network model.

DownLoad: Full-Size Img PowerPoint

It can be seen in Table 8 that in the data classification of this experiment, each index of VGG16 is relatively good, with the highest accuracy, sensitivity and AUC, followed by the relatively low specificity, next to ZFNet. From AUC value synthesis, the best performance of the experimental results is VGG16, followed by ZFNet, and finally AlexNet.

In this paper, based on the three basic learners, two integrated methods are adopted: the relative majority voting method and the weighted average method:

(a) The relative majority voting method.

The output result is the category with the highest number of votes. If there is a category with the same number of votes, select one randomly for output.

(b) The weighted-average method.

Suppose $n$ base classifiers have been obtained $\left\{{h}_{1}, {h}_{2}, L, {h}_{n}\right\}$, Each classifier has a weight $w$. Therefore, the output y of the integrated model can be obtained by the weighted average value of the classification results of each base classifier. The formula is as follows:

where ${w_i}$ is the weight of the ${h_i}$.

The value of d is set according to the accuracy ranking of each base classifier on the test set, the lowest setting is 1, the highest setting is n, and the sum of the weights of each base classifier is 1.

The three models (AlexNet, ZFNet, VGG16) obtained before are integrated by the above two methods. The experimental results are shown in Table 9.

Table 9.  Experimental results of integrated models.

Method	Accuracy	Sensitivity	Specificity
The relative majority voting method	0.9946	0.9933	0.9962
The weighted average method	0.9964	0.9966	0.9962

---

 | Show Table

DownLoad: CSV

The experimental results show that, compared with the single convolutional neural network, the performance of both the relative majority voting method and the weighted average method is improved, and the weighted average method outperforms, which is 0.3% higher than the VGG16 with the highest accuracy in Table 8, and other indicators are improved. Therefore, the integration method of multi convolution neural network in this paper is effective.

3.4.   Detail design of system

Inspection registration module. Input the appointment registration number and relevant inspection information generated after the appointment into the form, and finally upload the inspection image to the system, so that the doctor reading the film can view the image for diagnosis. All inspection registration information will be stored in the database, which can be queried in the inspection registration list. In the examination registration list, the doctor can query the specific information of a patient and can read the film for diagnosis. On the diagnosis page, the doctor can enlarge and rotate the selected key area of the patient's image, and finally, get the image performance information of the image and the diagnosis result saved in the system. After diagnosis, the diagnosis result is saved in the system and a diagnosis report is generated for medical staff to print. The report is shown as Figure 8.

Figure 8.  Print page of the report.

DownLoad: Full-Size Img PowerPoint

Intelligent diagnosis module for fungal keratitis. Based on the previous section, the deep learning method has been used to diagnose and classify the CM images of fungal keratitis. A good model has been obtained, so the intelligent diagnosis module of fungal keratitis has been developed in the computer-assisted diagnosis module. The doctor uploads the patient's examination image to the system, and it can return the output of the model, that is, normal or abnormal. It can help the doctor to provide reference value in the manual diagnosis of inspection registration. At the same time, all the results diagnosed by the model are recorded in the intelligent diagnosis record of fungal keratitis, as shown in Figure 9.

Figure 9.  Fungal keratitis diagnosis record page.

DownLoad: Full-Size Img PowerPoint

4.   Summary and outlook

In this paper, a web-based medical image management and analysis system is designed and implemented to improve the hospital's work efficiency and help to manage medical images. At the same time, deep learning technology can be used to automatically diagnose confocal microscope images of fungal keratitis online. The implemented medical image management and analysis system can run stably after testing, but there is still room for expansion. The computer-aided diagnosis module of the system can only automatically diagnose confocal microscopy images of fungal keratitis online at present because there are not many types of medical image data. When the system is officially used, with the gradual increase of users, the system will become more and more perfect. This is because this system has a self-learning function. When different pathological images are used in this system, the system will store these images and use them in the deep learning training process.

Acknowledgments

This work was supported in part by the NSFC No.91846205, the Key Research and Development Plan of Shandong Province under Grant 2017CXGC1503 and Grant 2018GSF118228, the Major Fundamental Research of Natural Science Foundation of Shandong Province under Grant ZR2019ZD05, the Intelligent perception and computing innovation platform of the Shenzhen Institute of Information Technology (No. SZIIT2019KJ021) and the Intelligent perception and computing innovation platform of the Shenzhen Institute of Information Technology (No. SZIIT2019KJ021).

Conflict of interest

No potential conflict of interest was reported by the authors.