Sentence coherence is an essential foundation for discourse coherence in natural language processing, as it plays a vital role in enhancing language expression, text readability, and improving the quality of written documents. With the development of e-government, automatic generation of official documents can significantly reduce the writing burden of government agencies. To ensure that the automatically generated official documents are coherent, we propose a sentence coherence evaluation model integrating repetitive words features, which introduces repetitive words features with neural network-based approach for the first time. Experiments were conducted on official documents dataset and THUCNews public dataset, our method has achieved an averaged 3.8% improvement in accuracy indicator compared to past research, reaching a 96.2% accuracy rate. This result is significantly better than the previous best method, proving the superiority of our approach in solving this problem.
Citation: Yunmei Shi, Yuanhua Li, Ning Li. Sentence coherence evaluation based on neural network and textual features for official documents[J]. Electronic Research Archive, 2023, 31(6): 3609-3624. doi: 10.3934/era.2023183
Related Papers:
[1]
Ruizhi Yang, Dan Jin, Wenlong Wang .
A diffusive predator-prey model with generalist predator and time delay. AIMS Mathematics, 2022, 7(3): 4574-4591.
doi: 10.3934/math.2022255
[2]
Yingyan Zhao, Changjin Xu, Yiya Xu, Jinting Lin, Yicheng Pang, Zixin Liu, Jianwei Shen .
Mathematical exploration on control of bifurcation for a 3D predator-prey model with delay. AIMS Mathematics, 2024, 9(11): 29883-29915.
doi: 10.3934/math.20241445
[3]
Sahabuddin Sarwardi, Hasanur Mollah, Aeshah A. Raezah, Fahad Al Basir .
Direction and stability of Hopf bifurcation in an eco-epidemic model with disease in prey and predator gestation delay using Crowley-Martin functional response. AIMS Mathematics, 2024, 9(10): 27930-27954.
doi: 10.3934/math.20241356
[4]
Xin-You Meng, Fan-Li Meng .
Bifurcation analysis of a special delayed predator-prey model with herd behavior and prey harvesting. AIMS Mathematics, 2021, 6(6): 5695-5719.
doi: 10.3934/math.2021336
[5]
Heping Jiang .
Complex dynamics induced by harvesting rate and delay in a diffusive Leslie-Gower predator-prey model. AIMS Mathematics, 2023, 8(9): 20718-20730.
doi: 10.3934/math.20231056
[6]
Qinghui Liu, Xin Zhang .
Chaos detection in predator-prey dynamics with delayed interactions and Ivlev-type functional response. AIMS Mathematics, 2024, 9(9): 24555-24575.
doi: 10.3934/math.20241196
[7]
Hairong Li, Yanling Tian, Ting Huang, Pinghua Yang .
Hopf bifurcation and hybrid control of a delayed diffusive semi-ratio-dependent predator-prey model. AIMS Mathematics, 2024, 9(10): 29608-29632.
doi: 10.3934/math.20241434
[8]
Fatao Wang, Ruizhi Yang, Yining Xie, Jing Zhao .
Hopf bifurcation in a delayed reaction diffusion predator-prey model with weak Allee effect on prey and fear effect on predator. AIMS Mathematics, 2023, 8(8): 17719-17743.
doi: 10.3934/math.2023905
[9]
Eric M. Takyi, Charles Ohanian, Margaret Cathcart, Nihal Kumar .
Sex-biased predation and predator intraspecific competition effects in a prey mating system. AIMS Mathematics, 2024, 9(1): 2435-2453.
doi: 10.3934/math.2024120
[10]
Liye Wang, Wenlong Wang, Ruizhi Yang .
Stability switch and Hopf bifurcations for a diffusive plankton system with nonlocal competition and toxic effect. AIMS Mathematics, 2023, 8(4): 9716-9739.
doi: 10.3934/math.2023490
Abstract
Sentence coherence is an essential foundation for discourse coherence in natural language processing, as it plays a vital role in enhancing language expression, text readability, and improving the quality of written documents. With the development of e-government, automatic generation of official documents can significantly reduce the writing burden of government agencies. To ensure that the automatically generated official documents are coherent, we propose a sentence coherence evaluation model integrating repetitive words features, which introduces repetitive words features with neural network-based approach for the first time. Experiments were conducted on official documents dataset and THUCNews public dataset, our method has achieved an averaged 3.8% improvement in accuracy indicator compared to past research, reaching a 96.2% accuracy rate. This result is significantly better than the previous best method, proving the superiority of our approach in solving this problem.
1.
Introduction
In survey sampling, it is well known fact that suitable use of the auxiliary information may improves the precision of an estimator for the unknown population parameters. The auxiliary information can be used either at the design stage or at estimation stage to increase the accuracy of the population parameter estimators. Several authors presented modified different type of estimators for estimating the finite population mean including [4,9,21,22,23,24,25,26,27].
The problem of estimation of finite population mean or total in two-stage sampling scheme using the auxiliary information has been well established. The two stage sampling scheme is an improvement over the cluster sampling, when it is not possible or easy to calculate all the units from the selected clusters. One of the main characteristic could be the budget, and it becomes too difficult to collect information from all the units within the selected clusters. To overcome this, one way is to select clusters, called first stage unit (fsus) and from the given population of interest, select a subsample from the selected clusters called the second stage units (ssu). This also benefits to increase the size of the first stage samples which consist of clusters, and assume to be heterogeneous groups. If there is no variation within clusters then might not be possible to collect information from all the units within selected clusters. In many situations, it is not possible to obtain the complete list of ultimate sampling units in large scale sample surveys, while a list of primary units of clusters may be available. In such situations, we select a random sample of first stage units or primary units using certain probability sampling schemes i.e simple random sampling (with or without replacement), systematic sampling and probability proportional to size (PPS), and then we can perform sub-sampling in selected clusters (first stage units). This approach is called two-stage sampling scheme.
Two-stage has a great varaity of applications, which go far beyond the immediate scop of sample survey. Whenever any process involves in chemical, physical, or biological tests that can be performed on a small amount of materail, it is likely to be drawn as a subsample from a larger amount that is itself a sample.
In large scale survey sampling, it is usual to adopt multistage sampling to estimate the population mean or total of the study variable y. [13] proposed a general class of estimators of a finite population mean using multi-auxiliary information under two stage sampling scheme. [1] proposed an alternative class of estimators in two stage sampling with two auxiliary variables. [10] proposed estimators for finite population mean under two-stage sampling using multivariate auxiliary information. [12] suggested a detailed note on ratio estimates in multi-stage sampling. [6] given some stratagies in two stage sampling using auxiliary information. [3] suggested a class of predictive estimaotrs in two stage sampling using auxiliary information. [8] gave a generalized method of estimation for two stage sampling using two auxiliary variables. [5] suggested chain ratio estimators in two stage sampling. For certain related work, we refer some latest articles, i.e., [14,15,16,17,18,19,20].
In this article, we propose an improved generalized class of estimators using two auxiliary variables under two-stage sampling scheme. The biases and mean sqaure errors of the proposed generalized class of estimators are derived up to first order of approximation. Based on the numerical results, the proposed class of estimators are more efficient than their existing counterparts.
2.
Symbols and notation
Consider a finite population U = {U1,U2,...,UN} is divided into N first-stage units (fsus) clusters in the population. Let N be the total number of first stage unit in population, n be the number of first stage units selected in the sample, Mi be the number of second stage units (ssus) belongs to the ith first stage units (fsus), (i = 1, 2, …, N), and mi be the number of fsus selected from the ith fsu in the sample of n fsus, (i = 1, 2, …, n).
Let yij, xij and zij be values of the study variable y and the auxiliary variables (xandz) respectively, for the jth ssus Ui=(j=1,2,...,Mi), in the ith fsus. The population mean of the study variable y and the auxiliary variables (x,z) are given by:
In order to obtain the biases and mean sqaured errors, we consider the following relative error terms:
e0=¯y∗−¯Y¯Y,e1=¯x∗−¯X¯X,e2=¯z∗−¯Z¯Z,
E(e20)=λC2by+1nN∑ni=1u2iθiC2iy=Vy,
E(e21)=λC2bx+1nN∑ni=1u2iθiC2ix=Vx,
E(e22)=λC2bz+1nN∑ni=1u2iθiC2iz=Vz,
E(e0e1)=λCbyx+1nN∑ni=1u2iθiCiyx=Vyx,
E(e0e2)=λCbyz+1nN∑ni=1u2iθiCiyz=Vyz,
E(e1e2)=λCbxz+1nN∑ni=1u2iθiCixz=Vxz,
Cby=Sby¯Y,Cbx=Sbx¯X,Cbz=Sbz¯Z,
Cbyx=Sbyx¯Y¯X,Cbyz=Sbyz¯Y¯Z,Cbxz=Sbxz¯X¯Z,
Ciyx=Siyx¯Y¯X,Ciyz=Sbyz¯Y¯Z,Cixz=Sixz¯X¯Z,
Ciy=Siy¯Y,Cix=Six¯X,Ciz=Siz¯Z,
where,
θi=(1mi−1Mi),λ=(1n−1N).
3.
Existing estimators
In this section, we consider several estimators of the finite population mean under two-stage sampling that are available in the sampling literature, the properties of all estimators considered here are obtained up-to the first order of approximation.
(ⅰ) The usual mean estimator ¯y∗=¯y∗0 and its variance under two-stage sampling are given by:
¯y∗0=1n∑ni=1ui¯yi,
(1)
and
V(¯y∗0)=¯Y2Vy=MSE(¯y∗0).
(2)
(ⅱ) The usual ratio estimator under two-stage sampling, is given by:
¯y∗R=¯y∗(¯X¯x∗),
(3)
where ¯X is the known population mean of x.
The bias and MSE of ¯y∗R to first order of approximation, are given by:
Bias(¯y∗R)=¯Y[Vx−Vyx],
(4)
and
MSE(¯y∗R)=¯Y2[Vy+Vx−2Vyx].
(5)
(ⅲ) [2] Exponential ratio type estimator under two-stage sampling, is given by:
¯y∗E=¯y∗exp(¯X−¯x∗¯X+¯x∗).
(6)
The bias and MSE of ¯y∗E to first order of approximation, are given by:
Bias(¯y∗E)=¯Y[38Vx−12Vyx],
(7)
and
MSE(¯y∗E)=¯Y2[Vy+14Vx−Vyx].
(8)
(ⅳ) The traditional difference estimator under two-stage sampling is given by:
¯y∗D=¯y∗+d(¯X−¯x∗),
(9)
where d is the constant.
The minimum variance of ¯y∗D, is given by:
V(¯y∗Dmin)=¯Y2Vy(1−ρ∗2)=MSE(¯y∗D),
(10)
where ρ∗=Vyx√Vy√Vx.
The optimum value of d is dopt=¯YVyx¯XVx.
(ⅴ) [7] Difference type estimator under two-stage sampling, is given by:
¯y∗Rao=d0¯y∗+d1(¯X−¯x∗),
(11)
where d0 and d1 are constants.
The bias and minimum MSE of ¯y∗Rao to first order of approximation, is given by:
The principal advantage of our proposed improved generalized class of estimators under two-stage sampling is that it is more flexible, efficient, than the existing estimators. The mean square errors based on two data sets are minimum and percentage relative efficiency is more than hundred as compared to the existing estimators considered here. We identified 11 estimators as members of the proposed class of estimators by substituting the different values of wi(i=1,2,3), δ and γ. On the lines of [2,7], we propose the following generalized improved class of estimators under two stage sampling for estimation of finite population mean using two auxiliary varaible as given by:
where wi(i=1,2,3) are constants, whose values are to be determined; δ and γ are constants i.e., (0≤δ, γ≤1) and can be used to construct the different estimators.
Using (25), solving ¯y∗G in terms of errors, we have
Solving (27), the minimum MSE of ¯y∗G to first order of approximation are given by:
MSE(¯y∗G)min=¯Y2[1−Ω24Ω1],
(28)
where
Ω1=ABC−AI2−BH2−CG2+2GHI+BC−I2,
and
Ω2=ABF2+ACE2−2AEFI+BCD2−2BDFH−2CDEG−D2I2+2DEHI
+2DFGI−E2H2+2EFGH−F2G2+4BCD+BF2−4BFH+CE2
−4CEG−4DI2−2EFI+4EHI+4FGI+4BC+4I2.
The optimum values of wi(i=1,2,3) are given by:
w1opt=Ω32Ω1,w2opt=¯YΩ42¯XΩ1, and w3opt=¯YΩ52¯ZΩ1,
where
Ω3=BCD−BFH−CEG−DI2+EHI+FGI+2GI+2BC−2I2,
Ω4=ACE−AFI−CDG+DHI−EH2+FGH+CE−2CG−FI+2HI,
Ω5=ABF−AEI−BDH+DGI+EGH−FG2+BF−2BH−EI+2GI.
From (28), we produce the following two estimators called ¯y∗G1 and ¯y∗G2. Put (δ=0,γ=1) and (δ=1,γ=0) in (25), we get the following two estimators respectively:
Population 1. [Source: [11], Model Assisted Survey Sampling]
There are 124 countries (second stage units) divided into 7 continents (first stage units) according to locations. Continent 7th consists of only one country therefore, we placed 7th continent in 6th continent.
We considered:
y = 1983 import (in millions U.S dollars),
x = 1983 export (in millions U.S dollars),
z = 1982 gross national product (in tens of millions of U.S dollars).
The data are divided into 6 clusters, having N=6, and n=3. Also ∑Ni=1Mi=124, ¯M=20.67. In Table 2, we show cluster sizes, and population means of the study variable (y) and the auxiliary variables (x,z). Tables 3 and 4 give some results.
Population 2. [Source: [11], Model Assisted Survey Sampling]
Similarly we considered the data as mentioned in Population 1,
y = 1983 import (in millions U.S dollars),
x = 1981 military expenditure (in tens of millions U.S dollars),
z = 1980 population (in millions).
The data are divided into 6 clusters having N=6, n=3, ∑Ni=1Mi=124,¯M=20.67.
In Table 5, we show cluster sizes, and means of the study variable (y) and the auxiliary variables (x,z). Tables 6 and 7 give some computation results.
The results based on Tables 2–7 are given in Tables 8 and 9 having biasses, mean square errors, and percentage relative efficiencies of the poposed and exisitng estimators w.r.t ¯y∗0.Tables 8 and 9 show that the proposed estimators perform well as compared to the existing estimators considered here.
Table 8.
Biases of different estimators in both data sets.
The following expression is used to obtain the Percent Relative Efficiency (PRE), i.e.,
PRE=MSE(¯y∗0)MSE(¯y∗i)×100,
where i=0,R,E,D,Rao,DR,DE,DD,DD(R),G1,G2.
6.
Discussion
As mentioned above, we used two real data sets to obtain the biases, MSEs or variances and PREs of all estimators under two-stage sampling scheme when using two auxiliary variables. In Tables 2–4 and Tables 5–7, we present the summary statistic of both population. From Tables 8 and 9, we observed that the proposed class of estimators ¯y∗G1 and ¯y∗G2 are more precise than the existing estimators ¯y∗0, ¯y∗R, ¯y∗E, ¯y∗D, ¯y∗Rao, ¯y∗DR, ¯y∗DE, ¯y∗DD, ¯y∗DD(R) in terms of MSEs and PREs. It is clear that the proposed improved generalized class of estimators, i.e., performs better than the estimators. As we increase the sample size the mean square error values decreases, and percentage relative efficiency give best results, which are the expected results.
7.
Conclusions
In this manuscript, we proposed a generalized class of estimators using two auxiliary variables under two-stage sampling for estimating the finite population mean. In addition, some well-known estimators of population mean like traditional unbiased estimator, usual ratio, exponential ratio type, traditional difference type, Rao difference type, difference-in- ratio type, difference-in-exponential ratio type, difference-in-difference, difference-difference ratio type estimator are created to be members of our suggested improved generalized class of estimators. Expression for the biases and mean squared error have been generated up to the first order of approximation. We identified 11 estimators as members of the proposed class of estimators by substituting the different values of wi(i=1,2,3), δ and γ. Both generalized class of estimators ¯y∗G1 and ¯y∗G2 perform better as compared to all other considered estimators, although ¯y∗G2 is the best. In Population 2, the performance of ratio estimator (¯y∗R) is weak. The gain in Population 1 is more as compared to Population 2.
Acknowledgments
The authors are thankful to the Editor-in-Chief and two anonymous referees for their careful reading of the paper and valuable comments which leads to a significant improvement in article.
Conflict of interest
The authors declare no conflict of interest.
References
[1]
S. Prabhu, K. Akhila, S. Sanriya, A hybrid approach towards automated essay evaluation based on BERT and feature engineering, in 2022 IEEE 7th International Conference for Convergence in Technology (I2CT), IEEE, Vadodara, India, (2022), 1–4. https://doi.org/10.1109/I2CT54291.2022.9824999
[2]
S. Jeon, M. Strube, Centering-based neural coherence modeling with hierarchical discourse segments, in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), ACL, Online, (2020), 7458–7472. https://doi.org/10.18653/v1/2020.emnlp-main.604
[3]
X. Tan, L. Zhang, D. Xiong, G. Zhou, Hierarchical modeling of global context for document-level neural machine translation, in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), ACL, Hong Kong, China, (2019), 1576–1585. https://doi.org/10.18653/v1/D19-1168
[4]
Q. Zhou, N. Yang, F. Wei, S. Huang, M. Zhou, T. Zhao, Neural document summarization by jointly learning to score and select sentences, in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), ACL, Melbourne, Australia, (2018), 654–663. https://doi.org/10.18653/v1/p18-1061
[5]
Y. Diao, H. Lin, L. Yang, X. Fan, Y. Chu, D. Wu, et al., CRHASum: extractive text summarization with contextualized-representation hierarchical-attention summarization network, Neural Comput. Appl., 32 (2020), 11491–11503. https://doi.org/10.1007/s00521-019-04638-3 doi: 10.1007/s00521-019-04638-3
[6]
P. Yang, L. Li, F. Luo, T. Liu, X. Sun, Enhancing topic-to-essay generation with external commonsense knowledge, in Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), ACL, Florence, Italy, (2019), 2002–2012. https://doi.org/10.18653/v1/p19-1193
[7]
X. L. Li, J. Thickstun, I. Gulrajani, P. Liang, T. B. Hashimoto, Diffusion-LM improves controllable text generation, arXiv preprint, (2022), arXiv: 2205.14217. https://doi.org/10.48550/arXiv.2205.14217
[8]
D. Parveen, H. M. Ramsl, M. Strube, Topical coherence for graph-based extractive summarization, in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), ACL, Lisbon, Portugal, (2015), 1949–1954. https://doi.org/10.18653/v1/d15-1226
[9]
L. Logeswaran, H. Lee, D. R. Radev, Sentence ordering and coherence modeling using recurrent neural networks, in Proceedings ofthe 8th AAAI Symposium on Educational Advances in ArtificialIntelligence, AAAI, Palo Alto, USA, 32 (2018), 5285–5292. https://doi.org/10.1609/aaai.v32i1.11997
[10]
Y. Liu, M. Lapata, Learning structured text representations, Trans. Assoc. Comput. Ling., 6 (2018), 63–75. https://doi.org/10.1162/tacl_a_00005 doi: 10.1162/tacl_a_00005
[11]
R. Barzilay, M. Lapata, Modeling local coherence: an entity-based approach, in 43rd Annual Meeting of the Association for Computational Linguistics (ACL), ACL, Michigan, USA, (2005), 141–148. https://doi.org/10.3115/1219840.1219858
[12]
A. Louis, A. Nenkova, A coherence model based on syntactic patterns, in Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), ACL, Jeju Island, Korea, (2012), 1157–1168.
[13]
D. Parveen, M. Mesgar, M. Strube, Generating coherent summaries of scientific articles using coherence patterns, in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), ACL, Austin, USA, (2016), 772–783. https://doi.org/10.18653/v1/d16-1074
[14]
J. Li, E. H. Hovy, A model of coherence based on distributed sentence representation, in Proceedings ofthe 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), ACL, Doha, Qatar, (2014), 2039–2048. https://doi.org/10.3115/v1/d14-1218
[15]
L. Mou, R. Men, G. Li, Y. Xu, L. Zhang, R. Yan, et al., Recognizing entailment and contradiction by tree-based convolution, arXiv preprint, (2016), arXiv: 1512.08422. https://doi.org/10.48550/arXiv.1512.08422
[16]
K. Luan, X. Du, C. Sun, B. Liu, X. Wang, Sentence ordering based on attention mechanism, J. Chin. Inf. Technol., 32 (2018), 123–130. https://doi.org/10.3969/j.issn.1003-0077.2018.01.016 doi: 10.3969/j.issn.1003-0077.2018.01.016
[17]
P. Xu, H. Saghir, J. S. Kang, T. Long, A. J. Bose, Y. Cao, et al., A cross-domain transferable neural coherence model, in Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), ACL, Florence, Italy, (2019), 678–687. https://doi.org/10.18653/v1/p19-1067
[18]
F. Xu, S. Du, M. Li, M. Wang, An entity-driven recursive neural network model for Chinese discourse coherence modeling, Int. J. Artif. Intell. Appl., 8 (2017), 1–9. https://doi.org/10.5121/ijaia.2017.8201 doi: 10.5121/ijaia.2017.8201
[19]
S. Du, F. Xu, M. Wang, An entity-driven bidirectional LSTM model for discourse coherence in Chinese, J. Chin. Inf. Technol., 31 (2017), 67–74. https://doi.org/10.3969/j.issn.1003-0077.2017.06.010 doi: 10.3969/j.issn.1003-0077.2017.06.010
[20]
K. Liu, H. Wang, Research on automatic summarization coherence based on discourse rhetoric structure in Chinese, J. Chin. Inf. Technol., 33 (2019), 77–84. https://doi.org/10.3969/j.issn.1003-0077.2019.01.009 doi: 10.3969/j.issn.1003-0077.2019.01.009
[21]
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, Q. V. Le, XLNet: generalized autoregressive pretraining for language understanding, in Annual Conference on Neural Information Processing Systems (NeurIPS), (2019), 5754–5764.
[22]
J. Devlin, M. W. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), ACL, Minneapolis, USA, (2019), 4171–4186. https://doi.org/10.18653/v1/n19-1423
[23]
W. Zhao, M. Strube, S. Eger, Discoscore: evaluating text generation with bert and discourse coherence, arXiv preprint, (2022), arXiv: 2201.11176. https://doi.org/10.48550/arXiv.2201.11176
[24]
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, Y. Artzi, BERTScore: evaluating text generation with BERT, arXiv preprint, (2019), arXiv: 1904.09675. https://doi.org/10.48550/arXiv.1904.09675
Yunmei Shi, Yuanhua Li, Ning Li. Sentence coherence evaluation based on neural network and textual features for official documents[J]. Electronic Research Archive, 2023, 31(6): 3609-3624. doi: 10.3934/era.2023183
Yunmei Shi, Yuanhua Li, Ning Li. Sentence coherence evaluation based on neural network and textual features for official documents[J]. Electronic Research Archive, 2023, 31(6): 3609-3624. doi: 10.3934/era.2023183