Permutation Entropy and Bubble Entropy: Possible interactions and synergies between order and sorting relations

David Cuesta-Frau; Borja Vargas; David Cuesta-Frau; Borja Vargas

doi:10.3934/mbe.2020086

Mathematical Biosciences and Engineering

2020, Volume 17, Issue 2: 1637-1658. doi: 10.3934/mbe.2020086

Previous Article Next Article

Research article Special Issues

Permutation Entropy and Bubble Entropy: Possible interactions and synergies between order and sorting relations

David Cuesta-Frau ^{1
,
,},
Borja Vargas ²

1.
Technological Institute of Informatics(ITI), Universitat Politècnica de València, Campus Alcoi, Plaza Ferrándiz y Carbonell, 2, 03801, Alcoi, Spain
2.
Department of Internal Medicine, Móstoles Teaching Hospital, Móstoles, 28935, Madrid, Spain

Received: 21 September 2019 Accepted: 25 November 2019 Published: 10 December 2019

Despite its widely demonstrated usefulness, there is still room for improvement in the basic Permutation Entropy (PE) algorithm, as several subsequent studies have proposed in the recent years. For example, some improved PE variants try to address possible PE weaknesses, such as its only focus on ordinal information, and not on amplitude, or the possible detrimental impact of equal values in subsequences due to motif ambiguity. Other evolved PE methods try to reduce the influence of input parameters. A good representative of this last point is the Bubble Entropy (BE) method. BE is based on sorting relations instead of ordinal patterns, and its promising capabilities have not been extensively assessed yet. The objective of the present study was to comparatively assess the classification performance of this new method, and study and exploit the possible synergies between PE and BE. The claimed superior performance of BE over PE was first evaluated by conducting a series of time series classification tests over a varied and diverse experimental set. The results of this assessment apparently suggested that there is a complementary relationship between PE and BE, instead of a superior/inferior relationship. A second set of experiments using PE and BE simultaneously as the input features of a clustering algorithm, demonstrated that with a proper algorithm configuration, classification accuracy and robustness can benefit from both measures.
- Permutation Entropy,
- Bubble Entropy,
- signal classification,
- clustering
Citation: David Cuesta-Frau, Borja Vargas. Permutation Entropy and Bubble Entropy: Possible interactions and synergies between order and sorting relations[J]. Mathematical Biosciences and Engineering, 2020, 17(2): 1637-1658. doi: 10.3934/mbe.2020086

Related Papers:

Abstract

Despite its widely demonstrated usefulness, there is still room for improvement in the basic Permutation Entropy (PE) algorithm, as several subsequent studies have proposed in the recent years. For example, some improved PE variants try to address possible PE weaknesses, such as its only focus on ordinal information, and not on amplitude, or the possible detrimental impact of equal values in subsequences due to motif ambiguity. Other evolved PE methods try to reduce the influence of input parameters. A good representative of this last point is the Bubble Entropy (BE) method. BE is based on sorting relations instead of ordinal patterns, and its promising capabilities have not been extensively assessed yet. The objective of the present study was to comparatively assess the classification performance of this new method, and study and exploit the possible synergies between PE and BE. The claimed superior performance of BE over PE was first evaluated by conducting a series of time series classification tests over a varied and diverse experimental set. The results of this assessment apparently suggested that there is a complementary relationship between PE and BE, instead of a superior/inferior relationship. A second set of experiments using PE and BE simultaneously as the input features of a clustering algorithm, demonstrated that with a proper algorithm configuration, classification accuracy and robustness can benefit from both measures.

References

[1]	C. Bandt and B. Pompe, Permutation entropy: A natural complexity measure for time series, Phys. Rev. Lett., 88 (2002), 174102.
[2]	M. Zanin, L. Zunino, O. A. Rosso and D. Papo, Permutation entropy and its main biomedical and econophysics applications: A review, Entropy, 14 (2012), 1553-1577.
[3]	D. Cuesta-Frau, P. Miró-Martínez, S. Oltra-Crespo, J. Jordán-Núñez, B. Vargas, P. González, et al., Model selection for body temperature signal classification using both amplitude and ordinalitybased entropy measures, Entropy, 20, (2018).
[4]	D. Cuesta-Frau, P. Miró-Martínez, S. Oltra-Crespo, J. Jordán-Núñez, B. Vargas and L. Vigil, Classification of glucose records from patients at diabetes risk using a combined permutation entropy algorithm, Comput. Meth. Program. Biomed., 165 (2018), 197-204.
[5]	D. Mateos, J. Diaz and P. Lamberti, Permutation entropy applied to the characterization of the clinical evolution of epileptic patients under pharmacological treatment, Entropy, 16 (2014), 5668-5676.
[6]	N. Nicolaou and J. Georgiou, The use of permutation entropy to characterize sleep electroencephalograms., Clin. EEG Neurosci., 421 (2011), 24-28.
[7]	A. Martínez-Rodrigo, B. García-Martínez, L. Zunino, R. Alcaraz and A. Fernández-Caballero, Multi-lag analysis of symbolic entropies on eeg recordings for distress recognition, Front. Neuroinform., 13 (2019), 40.
[8]	D. Li, X. Li, Z. Liang, L. J. Voss and J. W. Sleigh, Multiscale permutation entropy analysis of EEG recordings during sevoflurane anesthesia, J. Neural Eng., 7 (2010), 046010.
[9]	C. C. Naranjo, L. M. Sanchez-Rodriguez, M. B. Martínez, M. E. Báez and A. M. García, Permutation entropy analysis of heart rate variability for the assessment of cardiovascular autonomic neuropathy in type 1 diabetes mellitus, Comput. Biol. Med., 86 (2017), 90-97.
[10]	A. G. Ravelo-Garcia, J. L. Navarro-Mesa, U. Casanova-Blancas, S. González, P. Quintana, I. Guerra-Moreno, et al., Application of the permutation entropy over the heart rate variability for the improvement of electrocardiogram-based sleep breathing pause detection, Entropy, 17 (2015), 914-927.
[11]	C. Bian, C. Qin, Q. D. Y. Ma and Q. Shen, Modified Permutation-entropy analysis of heartbeat dynamics, Phys. Rev. E, 85 (2012), 021906.
[12]	M. Henry and G. Judge, Permutation entropy and information recovery in nonlinear dynamic economic time series, Econometrics, 7 (2019).
[13]	H. Danylchuk, N. Chebanova, N. Reznik and Y. Vitkovskyi, Modeling of investment attractiveness of countries using entropy analysis of regional stock markets, Global J. Environ. Sci. Manag., 5 (2019), 227-235.
[14]	F. Siokis, Credit market jitters in the course of the financial crisis: A permutation entropy approach in measuring informational efficiency in financial assets, Phys. A Statist. Mechan. Appl., 499 (2018).
[15]	A. F. Bariviera, L. Zunino, M. B. Guercio, L. Martinez and O. Rosso, Efficiency and credit ratings: A permutation-information-theory analysis, J. Statist. Mechan. Theory Exper., 2013 (2013), P08007.
[16]	A. F. Bariviera, M. B. Guercio, L. Martinez and O. Rosso, A permutation information theory tour through different interest rate maturities: the libor case, Philos. Transact. Royal Soc. A Math. Phys. Eng. Sci., 373 (2015).
[17]	J. Cánovas, G. García-Clemente and M. Muñoz-Guillermo, Comparing permutation entropy functions to detect structural changes in time series, Phys. A Statist. Mechan. Appl., 507 (2018), 153-174.
[18]	J. Zhang, Y. Zhao, M. Liu and L. Kong, Bearings fault diagnosis based on adaptive local iterative filtering-multiscale permutation entropy and multinomial logistic model with group-lasso, Advan. Mechan. Eng., 11 (2019), 1687814019836311.
[19]	J. Huang, X. Wang, D. Wang, Z. Wang and X. Hua, Analysis of weak fault in hydraulic system based on multi-scale permutation entropy of fault-sensitive intrinsic mode function and deep belief network, Entropy, 21 (2019).
[20]	B. Fadlallah, B. Chen, A. Keil and J. Príncipe, Weighted-permutation entropy: A complexity measure for time series incorporating amplitude information, Phys. Rev. E, 87 (2013), 022911.
[21]	J. Garland, T. R. Jones, M. Neuder, V. Morris, J. W. C. White and E. Bradley, Anomaly detection in paleoclimate records using permutation entropy, Entropy, 20 (2018).
[22]	B. Deng, L. Cai, S. Li, R. Wang, H. Yu, Y. Chen, et al., Multivariate multi-scale weighted permutation entropy analysis of eeg complexity for alzheimer's disease, Cogn. Neurodyn., 11 (2017), 217-231. doi: 10.1007/s11571-016-9418-9
[23]	L. Xiao-Feng and W. Yue, Fine-grained permutation entropy as a measure of natural complexity for time series, Chinese Phys. B, 18 (2009), 2690.
[24]	D. Cuesta-Frau, Permutation entropy: Influence of amplitude information on time series classification performance, Math. Biosci. Eng., 5 (2019), 1-16.
[25]	F. Traversaro, M. Risk, O. Rosso and F. Redelico, An empirical evaluation of alternative methods of estimation for Permutation Entropy in time series with tied values, arXiv e-prints, arXiv:1707.01517 (2017).
[26]	D. Cuesta-Frau, M. Varela-Entrecanales, A. Molina-Picó and B. Vargas, Patterns with equal values in permutation entropy: Do they really matter for biosignal classification?, Complexity, 2018 (2018), 1-15.
[27]	Y. Zou, R. V. Donner, N. Marwan, J. F. Donges and J. Kurths, Complex network approaches to nonlinear time series analysis, Phys. Rep., 787 (2019), 1-97, Complex network approaches to nonlinear time series analysis.
[28]	M. McCullough, M. Small, T. Stemler and H. Iu, Time lagged ordinal partition networks for capturing dynamics of continuous dynamical systems, Chaos Interdisciplin. J. Nonlinear Sci., 25 (2015).
[29]	D. Cuesta-Frau, A. Molina-Picó, B. Vargas and P. González, Permutation entropy: Enhancing discriminating power by using relative frequencies vector of ordinal patterns instead of their shannon entropy, Entropy, 21 (2019).
[30]	H. Azami and J. Escudero, Amplitude-aware permutation entropy: Illustration in spike detection and signal segmentation, Comput. Meth. Program. Biomed., 128 (2016), 40-51.
[31]	Z. Chen, L. Yaan, H. Liang and J. Yu, Improved permutation entropy for measuring complexity of time series under noisy condition, Complexity, 2019 (2019), 1-12.
[32]	G. Manis, M. Aktaruzzaman and R. Sassi, Bubble entropy: An entropy almost free of parameters, IEEE Transact. Biomed. Eng., 64 (2017), 2711-2718.
[33]	M. Riedl, A. Müller and N. Wessel, Practical considerations of permutation entropy, European Phys. J. Special Topics, 222 (2013), 249-262.
[34]	L. Zunino, F. Olivares, F. Scholkmann and O. A. Rosso, Permutation entropy based time series analysis: Equalities in the input signal can lead to false conclusions, Phys. Lett. A, 381 (2017), 1883-1892.
[35]	L. Citi, G. Guffanti and L. Mainardi, Rank-based multi-scale entropy analysis of heart rate variability, in Computing in Cardiology 2014, 2014, 597-600.
[36]	A. M. Unakafov and K. Keller, Conditional entropy of ordinal patterns, Phys. D Nonlinear Phenom., 269 (2014), 94-102.
[37]	Z. Liang, Y. Wang, X. Sun, D. Li, L. Voss, J. Sleigh, et al., Eeg entropy measures in anesthesia, Front. Comput. Neurosci., 9 (2015), 16.
[38]	D. E. Lake, J. S. Richman, M. P. Griffin and J. R. Moorman, Sample entropy analysis of neonatal heart rate variability, Am. J. Physiology-Regulatory Integrat. Comparat. Physiol., 283 (2002), R789-R797, PMID: 12185014.
[39]	S. M. Pincus, Approximate entropy as a measure of system complexity., Proceed. Nat. Acad. Sci., 88 (1991), 2297-2301.
[40]	T. Fawcett, An introduction to ROC analysis, Patt. Recogn. Lett., 27 (2006), 861-874, ROC Analysis in Pattern Recognition.
[41]	I. Unal, Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach, Comput. Math. Methods Med., 2017 (2017), 14.
[42]	A. Tharwat, Classification assessment methods, Appl. Comput. Inform., (2018).
[43]	A. M. Zoubir and D. R. Iskander, Bootstrap Techniques for Signal Processing, Cambridge University Press, 2004.
[44]	D. Kalpić, N. Hlupić and M. Lovrić, Student's t-Tests, 1559-1563, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011.
[45]	C.-Y. J. Peng, K. L. Lee and G. M. Ingersoll, An introduction to logistic regression analysis and reporting, J. Educat. Res., 96 (2002), 3-14.
[46]	C. M. Bishop, Neural Networks Patt. Recogn., Oxford University Press, Inc., New York, NY, USA, 1995.
[47]	A. K. Jain, M. N. Murty and P. J. Flynn, Data clustering: A review, ACM Comput. Surv., 31 (1999), 264-323.
[48]	J. Rodríguez-Sotelo, D. Peluffo-Ordoñez, D. Cuesta-Frau and G. Castellanos-Domínguez, Unsupervised feature relevance analysis applied to improve ecg heartbeat clustering, Comput. Meth. Program. Biomed., 108 (2012), 250-261.
[49]	D. Cuesta-Frau, J. C. Pérez-Cortés and G. Andreu-García, Clustering of electrocardiograph signals in computer-aided Holter analysis, Comput. Meth. Program. Biomed., 72 (2003), 179-196.
[50]	C. Murthy and N. Chowdhury, In search of optimal clusters using genetic algorithms, Patt. Recogn. Lett., 17 (1996), 825-832.
[51]	J. Sander, M. Ester, H.-P. Kriegel and X. Xu, Density-based clustering in spatial databases: The algorithm gdbscan and its applications, Data Min. Knowl. Discov., 2 (1998), 169-194.
[52]	J. Wu, Advances in K-means Clustering: A Data Mining Thinking, Springer Publishing Company, Incorporated, 2012.
[53]	S. Panda, S. Sahu, P. Jena and S. Chattopadhyay, Comparing fuzzy-c means and k-means clustering techniques: A comprehensive study, in Advances in Computer Science, Engineering & Applications (eds. D. C. Wyld, J. Zizka and D. Nagamalai), Springer Berlin Heidelberg, Berlin, Heidelberg, 2012, 451-460.
[54]	A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, et al., PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals, Circulation, 101 (2000), 215-220.
[55]	J. Jordán-Núnez, P. Miró-Martínez, B. Vargas, M. Varela-Entrecanales and D. Cuesta-Frau, Statistical models for fever forecasting based on advanced body temperature monitoring, J. Crit. Care, 37 (2017), 136-140.
[56]	D. Cuesta-Frau, M. Varela, P. Miró-Martínez, P. Galdos, D. Abásolo, R. Hornero, et al,, Predicting survival in critical patients by use of body temperature regularity measurement based on approximate entropy, Med. Biol. Eng. Comput., 45 (2007), 671-678.
[57]	C. Rodriguez de Castro, L. Vigil, B. Vargas, E. Garcia Delgado, R. Garcia-Carretero, J. RuizGaliana, et al., Glucose time series complexity as a predictor of type 2 diabetes, Diabetes Metab. Res. Rev., 30 (2016).
[58]	R. G. Andrzejak, K. Lehnertz, F. Mormann, C. Rieke, P. David and C. E. Elger, Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state, Phys. Rev. E, 64 (2001), 061907.
[59]	D. Cuesta-Frau, J. P. Murillo-Escobar, D. A. Orrego and E. Delgado-Trejos, Embedded dimension and time series length. practical influence on permutation entropy and its applications, Entropy, 21 (2019).
[60]	N. Iyengar, C. K. Peng, R. Morin, A. L. Goldberger and L. A. Lipsitz, Age-related alterations in the fractal scaling of cardiac interbeat interval dynamics, Am. J. Physiology-Regulatory Integrat. Comparat. Physiol., 271 (1996), R1078-R1084, PMID: 8898003.

Reader Comments

Your name:*

Email:*
© 2020 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)