Extracting relational triples from unstructured medical texts can provide a basis for the construction of large-scale medical knowledge graphs. The cascade binary pointer tagging network (CBPTN) shows excellent performance in the joint entity and relation extraction, so we try to explore its effectiveness in the joint entity and relation extraction of Chinese medical texts. In this paper, we propose two models based on the CBPTN: CBPTN with conditional layer normalization (Cas-CLN) and biaffine transformation-based CBPTN with multi-head selection (BTCAMS). Cas-CLN uses the CBPTN to decode the head entity and relation-tail entity successively and utilizes conditional layer normalization to enhance the connection between the two steps. BTCAMS detects all possible entities in a sentence by using the CBPTN and then determines the relation between each entity pair through biaffine transformation. We test the performance of the two models on two Chinese medical datasets: CMeIE and CEMRDS. The experimental results prove the effectiveness of the two models. Compared with the baseline CasREL, the F1 value of Cas-CLN and BTCAMS on the test data of CMeIE improved by 1.01 and 2.13%;
on the test data of CEMRDS, the F1 value improved by 1.99 and 0.68%.
Citation: Hongyang Chang, Hongying Zan, Tongfeng Guan, Kunli Zhang, Zhifang Sui. Application of cascade binary pointer tagging in joint entity and relation extraction of Chinese medical text[J]. Mathematical Biosciences and Engineering, 2022, 19(10): 10656-10672. doi: 10.3934/mbe.2022498
Extracting relational triples from unstructured medical texts can provide a basis for the construction of large-scale medical knowledge graphs. The cascade binary pointer tagging network (CBPTN) shows excellent performance in the joint entity and relation extraction, so we try to explore its effectiveness in the joint entity and relation extraction of Chinese medical texts. In this paper, we propose two models based on the CBPTN: CBPTN with conditional layer normalization (Cas-CLN) and biaffine transformation-based CBPTN with multi-head selection (BTCAMS). Cas-CLN uses the CBPTN to decode the head entity and relation-tail entity successively and utilizes conditional layer normalization to enhance the connection between the two steps. BTCAMS detects all possible entities in a sentence by using the CBPTN and then determines the relation between each entity pair through biaffine transformation. We test the performance of the two models on two Chinese medical datasets: CMeIE and CEMRDS. The experimental results prove the effectiveness of the two models. Compared with the baseline CasREL, the F1 value of Cas-CLN and BTCAMS on the test data of CMeIE improved by 1.01 and 2.13%;
on the test data of CEMRDS, the F1 value improved by 1.99 and 0.68%.
| [1] | T. Guan, H. Zan, X. Zhou, H. Xu, K. Zhang, CMeIE: Construction and evaluation of Chinese medical information extraction dataset, in Natural Language Processing and Chinese Computing (eds. X. Zhu, M. Zhang, Y. Hong, R. He), Springer International Publishing, Cham, (2020), 270–282. https://doi.org/10.1007/978-3-030-60450-9_22 |
| [2] | D. Zelenko, C. Aone, A. Richardella, Kernel methods for relation extraction, J. Mach. Learn. Res., 3 (2003), 1083–1106. Available from: http://www.jmlr.org/papers/volume3/zelenko03a/zelenko03a.pdf. |
| [3] | G. Zhou, J. Su, J. Zhang, M. Zhang, Exploring various knowledge in relation extraction, in Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05), Association for Computational Linguistics, Ann Arbor, Michigan, (2005), 427–434. https://doi.org/10.3115/1219840.1219893 |
| [4] | M. Mintz, S. Bills, R. Snow, D. Jurafsky, Distant supervision for relation extraction without labeled data, in Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Association for Computational Linguistics, Suntec, Singapore, (2009), 1003–1011. https://doi.org/10.1016/B978-0-12-374144-8.00264-2 |
| [5] | X. Yu, W. Lam, Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach, in Coling 2010: Posters, Coling 2010 Organizing Committee, Beijing, China, (2010), 1399–1407. Available from: https://aclanthology.org/C10-2160. |
| [6] | M. Miwa, Y. Sasaki, Modeling joint entity and relation extraction with table representation, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Doha, Qatar, (2014), 1858–1869. https://doi.org/10.3115/v1/D14-1200 |
| [7] | D. Zeng, K. Liu, S. Lai, G. Zhou, J. Zhao, Relation classification via convolutional deep neural network, in Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin City University and Association for Computational Linguistics, Dublin, Ireland, (2014), 2335–2344. Available from: https://aclanthology.org/C14-1220. |
| [8] | M. Miwa, M. Bansal, End-to-end relation extraction using LSTMs on sequences and tree structures, in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Berlin, Germany, (2016), 1105–1116. https://doi.org/10.18653/v1/P16-1105 |
| [9] | S. Zheng, F. Wang, H. Bao, Y. Hao, P. Zhou, B. Xu, Joint extraction of entities and relations based on a novel tagging scheme, in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Vancouver, Canada, (2017), 1227–1236. https://doi.org/10.18653/v1/P17-1113 |
| [10] | X. Zeng, D. Zeng, S. He, K. Liu, J. Zhao, Extracting relational facts by an end-to-end neural model with copy mechanism, in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Melbourne, Australia, (2018), 506–514. https://doi.org/10.18653/v1/P18-1047 |
| [11] | T. J. Fu, P. H. Li, W. Y. Ma, GraphRel: Modeling text as relational graphs for joint entity and relation extraction, in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, (2019), 1409–1418. https://doi.org/10.18653/v1/P19-1136 |
| [12] | X. Zeng, S. He, D. Zeng, K. Liu, S. Liu, J. Zhao, Learning the extraction order of multiple relational facts in a sentence with reinforcement learning, in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computational Linguistics, Hong Kong, China, (2019), 367–377. https://doi.org/10.4337/9781786433787.00038 |
| [13] | D. Zeng, H. Zhang, Q. Liu, CopyMTL: Copy mechanism for joint extraction of entities and relations with multi-task learning, in Proceedings of the AAAI Conference on Artificial Intelligence, 34 (2020), 9507–9514. https://doi.org/10.1609/aaai.v34i05.6495 |
| [14] | Z. Wei, J. Su, Y. Wang, Y. Tian, Y. Chang, A novel cascade binary tagging framework for relational triple extraction, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, (2020), 1476–1488. https://doi.org/10.18653/v1/2020.acl-main.136 |
| [15] | Y. Wang, B. Yu, Y. Zhang, T. Liu, H. Zhu, L. Sun, TPLinker: Single-stage joint extraction of entities and relations through token pair linking, in Proceedings of the 28th International Conference on Computational Linguistics, International Committee on Computational Linguistics, Barcelona, Spain (Online), (2020), 1572–1582. https://doi.org/10.18653/v1/2020.coling-main.138 |
| [16] |
K. Zhao, H. Xu, Y. Cheng, X. Li, K. Gao, Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction, Knowl. Based Syst., 219 (2021), 106888. https://doi.org/10.1016/j.knosys.2021.106888 doi: 10.1016/j.knosys.2021.106888
|
| [17] |
Ö. Uzuner, B. R. South, S. Shen, S. L. DuVall, 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text, J. Am. Med. Inf. Assoc., 18 (2011), 552–556. https://doi.org/10.1136/amiajnl-2011-000203 doi: 10.1136/amiajnl-2011-000203
|
| [18] |
W. Sun, A. Rumshisky, Ö. Uzuner, Evaluating temporal relations in clinical text: 2012 i2b2 Challenge, J. Am. Med. Inf. Assoc., 20 (2013), 806–813. https://doi.org/10.1136/amiajnl-2013-001628 doi: 10.1136/amiajnl-2013-001628
|
| [19] |
A. Stubbs, Ö. Uzuner, Annotating risk factors for heart disease in clinical narratives for diabetic patients, J. Biomed. Inf., 58 (2015), S78–S91. https://doi.org/10.1016/j.jbi.2015.05.009 doi: 10.1016/j.jbi.2015.05.009
|
| [20] |
C. H. Wei, Y. Peng, R. Leaman, A. P. Davis, C. J. Mattingly, J. Li, et al., Assessing the state of the art in biomedical relation extraction: Overview of the BioCreative V chemical-disease relation (CDR) task, Database, 2016 (2016), 1–8. https://doi.org/10.1093/database/baw032 doi: 10.1093/database/baw032
|
| [21] | Y. L. Yang, P. T. Lai, R. T. H. Tsai, A hybrid system for temporal relation extraction from discharge summaries, in Technologies and Applications of Artificial Intelligence, Springer International Publishing, Cham, (2014), 379–386. https://doi.org/10.1007/978-3-319-13987-6_35 |
| [22] | S. Sahu, A. Anand, K. Oruganty, M. Gattu, Relation extraction from clinical texts using domain invariant convolutional neural network, in Proceedings of the 15th Workshop on Biomedical Natural Language Processing, Association for Computational Linguistics, Berlin, Germany, (2016), 206–215. https://doi.org/10.18653/v1/W16-2928 |
| [23] | H. Zhou, H. Deng, L. Chen, Y. Yang, C. Jia, D. Huang, Exploiting syntactic and semantics information for chemical–disease relation extraction, Database, 2016 (2016), https://doi.org/10.1093/database/baw048 |
| [24] | D. Q. Nguyen, K. Verspoor, Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings, in Proceedings of the BioNLP 2018 workshop, Association for Computational Linguistics, Melbourne, Australia, (2018), 129–136. https://doi.org/10.18653/v1/W18-2314 |
| [25] | V. R. Chikka, K. Karlapalem, A hybrid deep learning approach for medical relation extraction, preprint, arXiv: 1806.11189. |
| [26] | S. Ramamoorthy, S. Murugan, An attentive sequence model for adverse drug event extraction from biomedical text, preprint, arXiv: 1801.00625. |
| [27] | H. Li, Q. Chen, B. Tang, X. Wang, Chemical-induced disease extraction via convolutional neural networks with attention, in 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE Computer Society, Los Alamitos, CA, USA, (2017), 1276–1279. https://doi.org/10.1109/BIBM.2017.8217843 |
| [28] | H. Zhou, C. Lang, Z. Liu, S. Ning, Y. Lin, L. Du, Knowledge-guided convolutional networks for chemical-disease relation extraction, BMC Bioinf., 20 (2019). https://doi.org/10.1186/s12859-019-2873-7 |
| [29] | A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O. Yakhnenko, Translating embeddings for modeling multi-relational data, in Advances in Neural Information Processing Systems, Curran Associates, Inc., 26 (2013), 1–9. Available from: https://proceedings.neurips.cc/paper/2013/file/1cecc7a77928ca8133fa24680a88d2f9-Paper.pdf. |
| [30] | J. Devlin, M. W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, (2019), 4171–4186. https://doi.org/10.18653/v1/N19-1423 |
| [31] | Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, et al., RoBERTa: A robustly optimized BERT pretraining approach, preprint, arXiv: 1907.11692. |
| [32] | Y. Sun, S. Wang, Y. Li, S. Feng, X. Chen, H. Zhang, et al., Ernie: Enhanced representation through knowledge integration, preprint, arXiv: 1904.09223. |
| [33] | J. Su, Conditional text generation based on conditional layer normalization, Available from: https://kexue.fm/archives/7124. |
| [34] | T. Dozat, C. D. Manning, Deep biaffine attention for neural dependency parsing, preprint, arXiv: 1611.01734. |
| [35] | J. Yu, B. Bohnet, M. Poesio, Named entity recognition as dependency parsing, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, (2020), 6470–6476. https://doi.org/10.18653/v1/2020.acl-main.577 |
| [36] | Y. Liu, Research on Automatic Extraction of Chinese Named Entities and Entity relations, MSE thesis, Zhengzhou University, 2019. |
Figures(5) / Tables(2)
Hongyang Chang, Hongying Zan, Tongfeng Guan, Kunli Zhang, Zhifang Sui. Application of cascade binary pointer tagging in joint entity and relation extraction of Chinese medical text[J]. Mathematical Biosciences and Engineering, 2022, 19(10): 10656-10672. doi: 10.3934/mbe.2022498
DownLoad: