A survey of methods for encrypted network traffic fingerprinting

Sunghyun Yu; Yoojae Won; Sunghyun Yu; Yoojae Won

doi:10.3934/mbe.2023101

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 2: 2183-2202. doi: 10.3934/mbe.2023101

Previous Article Next Article

Survey Special Issues

A survey of methods for encrypted network traffic fingerprinting

Sunghyun Yu ,
Yoojae Won ^,

Department of Computer Science and Engineering, Chungnam National University, Daejeon 34134, Korea

Received: 13 September 2022 Revised: 03 November 2022 Accepted: 09 November 2022 Published: 16 November 2022

Privacy protection in computer communication is gaining attention because plaintext transmission without encryption can be eavesdropped on and intercepted. Accordingly, the use of encrypted communication protocols is on the rise, along with the number of cyberattacks exploiting them. Decryption is essential for preventing attacks, but it risks privacy infringement and incurs additional costs. Network fingerprinting techniques are among the best alternatives, but existing techniques are based on information from the TCP/IP stack. They are expected to be less effective because cloud-based and software-defined networks have ambiguous boundaries, and network configurations not dependent on existing IP address schemes increase. Herein, we investigate and analyze the Transport Layer Security (TLS) fingerprinting technique, a technology that can analyze and classify encrypted traffic without decryption while addressing the problems of existing network fingerprinting techniques. Background knowledge and analysis information for each TLS fingerprinting technique is presented herein. We discuss the pros and cons of two groups of techniques, fingerprint collection and artificial intelligence (AI)-based. Regarding fingerprint collection techniques, separate discussions on handshake messages ClientHello/ServerHello, statistics of handshake state transitions, and client responses are provided. For AI-based techniques, discussions on statistical, time series, and graph techniques according to feature engineering are presented. In addition, we discuss hybrid and miscellaneous techniques that combine fingerprint collection with AI techniques. Based on these discussions, we identify the need for a step-by-step analysis and control study of cryptographic traffic to effectively use each technique and present a blueprint.
- fingerprinting,
- TLS,
- encrypted network traffic,
- privacy-preserving,
- cybersecurity,
- classification,
- identification
Citation: Sunghyun Yu, Yoojae Won. A survey of methods for encrypted network traffic fingerprinting[J]. Mathematical Biosciences and Engineering, 2023, 20(2): 2183-2202. doi: 10.3934/mbe.2023101

Related Papers:

Abstract

Privacy protection in computer communication is gaining attention because plaintext transmission without encryption can be eavesdropped on and intercepted. Accordingly, the use of encrypted communication protocols is on the rise, along with the number of cyberattacks exploiting them. Decryption is essential for preventing attacks, but it risks privacy infringement and incurs additional costs. Network fingerprinting techniques are among the best alternatives, but existing techniques are based on information from the TCP/IP stack. They are expected to be less effective because cloud-based and software-defined networks have ambiguous boundaries, and network configurations not dependent on existing IP address schemes increase. Herein, we investigate and analyze the Transport Layer Security (TLS) fingerprinting technique, a technology that can analyze and classify encrypted traffic without decryption while addressing the problems of existing network fingerprinting techniques. Background knowledge and analysis information for each TLS fingerprinting technique is presented herein. We discuss the pros and cons of two groups of techniques, fingerprint collection and artificial intelligence (AI)-based. Regarding fingerprint collection techniques, separate discussions on handshake messages ClientHello/ServerHello, statistics of handshake state transitions, and client responses are provided. For AI-based techniques, discussions on statistical, time series, and graph techniques according to feature engineering are presented. In addition, we discuss hybrid and miscellaneous techniques that combine fingerprint collection with AI techniques. Based on these discussions, we identify the need for a step-by-step analysis and control study of cryptographic traffic to effectively use each technique and present a blueprint.

References

[1]	T. W. Kim, A. E. Azzaoui, B. Koh, J. Kim, J. H. Park, A secret sharing-based distributed cloud system for privacy protection, Hum. Centric Comput. Inf. Sci., 12 (2022). https://doi.org/10.22967/HCIS.2022.12.020 doi: 10.22967/HCIS.2022.12.020
[2]	C. Blundo, C. De Maio, M. Parente, L. Siniscalchi, Targeted advertising that protects the privacy of social networks users, Hum. Centric Comput. Inf. Sci., 11 (2021), 18. https://doi.org/10.22967/HCIS.2021.11.018 doi: 10.22967/HCIS.2021.11.018
[3]	C. Jia, C. Jia, L. Kong, W. Lin, L. Qi, Privacy-aware retrieval of electronic medical records by fuzzy keyword search, Hum. Centric Comput. Inf. Sci., 12 (2022). https://doi.org/10.22967/HCIS.2022.12.041 doi: 10.22967/HCIS.2022.12.041
[4]	L. Orans, A. Hils, J. D'Hoinne, E. Ahlm, Gartner, Predicts 2017: Network and Gateway Security, 2016.
[5]	Let's encrypt stats. Available from: https://letsencrypt.org/stats/
[6]	Cisco encrypted traffic analytics white paper. Available from: https://www.cisco.com/c/en/us/solutions/collateral/enterprise-networks/enterprise-network-security/nb-09-encrytd-traf-anlytcs-wp-cte-en.pdf
[7]	F. Veysset, O. Courtay, O. Heen, New tool and technique for remote operating system fingerprinting, Intranode Softw. Technol., 4 (2002).
[8]	L. Bernaille, R. Teixeira, I. Akodkenou, A. Soule, K. Salamatian, Traffic classification on the fly, in: ACM Sigcomm Comput. Commun. Rev., 36 (2006), 23–26. https://doi.org/10.1145/1129582.1129589
[9]	P. Velan, M. Čermák, P. Čeleda, M. Drašar, A survey of methods for encrypted traffic classification and analysis, Int. J. Netw. Manag., 25 (2015), 355–374. https://doi.org/10.1002/nem.1901 doi: 10.1002/nem.1901
[10]	F. Pacheco, E. Exposito, M. Gineste, C. Baudoin, J. Aguilar, Towards the deployment of machine learning solutions in network traffic classification: A systematic survey, IEEE Commun. Surv. Tutor., 23 (2018), 1988–2014. https://doi.org/10.1109/COMST.2018.2883147 doi: 10.1109/COMST.2018.2883147
[11]	C. Oh, J. Ha, H. Roh, A survey on TLS-encrypted malware network traffic analysis applicable to security operations centers, Appl. Sci., 12 (2021), 155. https://doi.org/10.3390/app12010155 doi: 10.3390/app12010155
[12]	E. Papadogiannaki, S. Ioannidis, A survey on encrypted network traffic analysis applications, techniques, and countermeasures, ACM Comput. Surv., 54 (2021), 1–35. https://doi.org/10.1145/3457904 doi: 10.1145/3457904
[13]	H. Gao, W. Huang, T. Liu, Y. Yin, Y. Li, PPO₂: Location privacy-oriented task offloading to edge computing using reinforcement learning for intelligent autonomous transport systems, IEEE Trans. Intell. Transport. Syst., (2022), 1–14. https://doi.org/10.1109/TITS.2022.3169421 doi: 10.1109/TITS.2022.3169421
[14]	Z. Zhang, Y. Li, H. Dong, H. Gao, Y. Jin, W. Wang, Spectral-based directed graph network for malware detection, spectral-based directed graph network for malware detection, IEEE Trans. Netw. Sci. Eng., 8 (2021), 957–970. https://doi.org/10.1109/TNSE.2020.3024557 doi: 10.1109/TNSE.2020.3024557
[15]	H. Gao, B. Qiu, R. J. Duran Barroso, W. Hussain, Y. Xu, X. Wang, TSMAE: A novel anomaly detection approach for Internet of things time series data using memory-augmented autoencoder, IEEE Trans. Netw. Sci. Eng., (2022), 1–1. https://doi.org/10.1109/TNSE.2022.3163144 doi: 10.1109/TNSE.2022.3163144
[16]	P. Li, X. Wang, H. Gao, X. Xu, M. Iqbal, K. Dahal, A dynamic and scalable user-centric route planning algorithm based on polychromatic sets theory, IEEE Trans. Intell. Transp. Syst., 23 (2022), 2762–2772. https://doi.org/10.1109/TITS.2021.3085026 doi: 10.1109/TITS.2021.3085026
[17]	A. Freier, P. Karlton, P. Kocher, The secure sockets layer (SSL) protocol version 3.0, The Internet Engineering Task Force, IETF. https://www.rfc-editor.org/rfc/rfc6101.html
[18]	K. Moriarty, S. Farrell, Deprecating, TLS 1.0 and TLS 1.1, Internet Engineering Task Force, IETF. https://www.hjp.at/doc/rfc/rfc8996.html
[19]	R. L. Rivest, A. Shamir, L. Adleman, A method for obtaining digital signatures and public-key cryptosystems, Commun. ACM., 21 (1978), 120–126. https://doi.org/10.1145/359340.359342 doi: 10.1145/359340.359342
[20]	D. Johnson, A. Menezes, S. Vanstone, The elliptic curve digital signature algorithm (ECDSA), Int. J. Inf. Sec., 1 (2001), 36–63. https://doi.org/10.1007/s102070100002 doi: 10.1007/s102070100002
[21]	S. Josefsson, I. Liusvaara, Edwards-curve digital signature algorithm (EdDSA), Internet Engineering Task Force, IETF, (2017), No. rfc8032.
[22]	J. Bradley, B. Campbell, T. Lodderstedt, N. Sakimura, OAuth 2.0 mutual-TLS client authentication and certificate-bound access tokens, Internet Engineering Task Force, IETF, (2020), Rep. rfc8705.
[23]	K. L. Chung, Markov chains, Springer-Verlag, 1967. https://doi.org/10.1007/978-3-642-49686-8
[24]	D. Bertsekas, J. N. Tsitsiklis, Introduction to probability, Athena Scientific, 2008.
[25]	S. R. Eddy, What is a hidden Markov model?, Nat. Biotechnol., 22 (2004), 1315–1316. https://doi.org/10.1038/nbt1004-1315 doi: 10.1038/nbt1004-1315
[26]	L. Gong, X. Gong, Y. Liang, B. Zhang, A. Y. Yang, Life prediction of hydraulic concrete based on grey residual markov model, J. Inf. Process. Syst., 18 (2022), 457–469. https://doi.org/10.3745/JIPS.04.0247 doi: 10.3745/JIPS.04.0247
[27]	S. Dick, Artificial intelligence, Harv. Data Sci. Rev., 1 (2019). https://doi.org/10.1162/99608f92.92fe150c doi: 10.1162/99608f92.92fe150c
[28]	H. Yoon, S. Jeong, Electric power demand prediction using deep learning model with temperature data, KIPS transactions on software and data engineering, 11 (2022), 307–314. https://doi.org/10.3745/KTSDE.2022.11.7.307 doi: 10.3745/KTSDE.2022.11.7.307
[29]	H. Wang, Z. Lei, X. Zhang, B. Zhou, J. Peng, Machine learning basics, Deep Learn, (2016), 98–164.
[30]	M. I. Jordan, T. M. Mitchell, Machine learning: Trends, perspectives, and prospects, Science, 349 (2015), 255–260. https://doi.org/10.1126/science.aaa8415 doi: 10.1126/science.aaa8415
[31]	I. Ristic, HTTP client fingerprinting using SSL handshake analysis, 2009. Available from: http://www.ssllabs.com/projects/client-fingerprinting
[32]	M. Majkowski, SSL fingerprinting for p0f, 2012. Available from: https://idea.popcount.org/2012-06-17-ssl-fingerprinting-for-p0f
[33]	L. Brotherston, GitHub, FingerprinTLS, 2015. Available from: http://github.com/LeeBrotherston/tlsfingerprinting
[34]	J. Althouse, S. Engineering, J.A. Open Sourcing, 2017. Available from: http://engineering.salesforce.com/open-sourcing-ja3-92c9e53c3c41
[35]	TLS Fingerprinting addendum, Joy: A package for capturing and analyzing network Data features, 2019. Available from: https://github.com/cisco/joy
[36]	J. Althouse, T.L.S. Saleforce Engineering, Fingerprinting with JA3 and JA3S, 2019. Available from: http://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s
[37]	B. Anderson, D. McGrew, TLS beyond the browser: Combining end host and network data to understand application behavior, in: Proceedings of the Internet Measurement Conference, (2019), 379–392. https://doi.org/10.1145/3355369.3355601
[38]	B. Anderson, D. McGrew, Accurate TLS fingerprinting using destination context and knowledge bases, (2020), preprint. https://doi.org/10.48550/arXiv.2009.01939
[39]	M. Korczyński, A. Duda, Markov chain fingerprinting to classify encrypted traffic, in: IEEE Conference on Computer Communications, IEEE Publications Infocom, IEEE Publications, (2014), 781–789. https://doi.org/10.1109/INFOCOM.2014.6848005
[40]	C. Liu, Z. Cao, G. Xiong, G. Gou, S. M. Yiu, L. He, MaMPF: Encrypted traffic classification based on multi-attribute markov probability fingerprints, in: 26th International Symposium on Quality of Service (IWQoS), IEEE Publications/ACM, IEEE Publications, (2018), 1–10. https://doi.org/10.1109/IWQoS.2018.8624124
[41]	C. Liu, G. Xiong, G. Gou, S. M. Yiu, Z. Li, Z. Tian, Classifying encrypted traffic using adaptive fingerprints with multi-level attributes, World Wide Web, 24 (2021), 2071–2097. https://doi.org/10.1007/s11280-021-00940-0 doi: 10.1007/s11280-021-00940-0
[42]	D. Chao, A fingerprint enhancement and second-order Markov chain based malicious encrypted traffic identification scheme, in: Proceedings of the 2020 6th International Conference on Computing and Artificial Intelligence, (2020), 328–333. https://doi.org/10.1145/3404555.3404590
[43]	Y. Zhao, Y. N. Yang, K. Wu, Y. Hao, H. Su, Q. Zhao, A classification and identification technology of TLS encrypted traffic applications, in: I.E.E.E. IV International (Ed.) Conference on Big Data and Artificial Intelligence (BDAI), IEEE Publications, (2021), 160–164. https://doi.org/10.1109/BDAI52447.2021.9515274
[44]	B. Garn, D. E. Simos, S. Zauner, R. Kuhn, R. Kacker, Browser fingerprinting using combinatorial sequence testing, in: Proceedings of the 6th Annual Symposium on Hot Topics in the Science of Security, (2019), 1–9. https://doi.org/10.1145/3314058.3314062
[45]	B. Garn, S. Zauner, D. E. Simos, M. Leithner, R. Kuhn, R. Kacker, A Two-Step TLS-Based Browser fingerprinting approach using combinatorial sequences, Comput. Secur. J., 114 (2022), 102575. https://doi.org/10.1016/j.cose.2021.102575 doi: 10.1016/j.cose.2021.102575
[46]	R. Dubin, A. Dvir, O. Pele, O. Hadar, I know what you saw last minute—Encrypted http adaptive video streaming title classification, IEEE Trans. Inf. Forensics Secur., 12 (2017), 3039–3049. https://doi.org/10.1109/TIFS.2017.2730819 doi: 10.1109/TIFS.2017.2730819
[47]	L. Yang, S. Fu, Y. Luo, J. Shi, Markov probability fingerprints: A method for identifying encrypted video traffic, in: 16th International Conference on Mobility, Sensing and Networking (MSN), IEEE Publications, (2020), 283–290. https://doi.org/10.1109/MSN50589.2020.00055
[48]	K. Al-Naami, S. Chandra, A. Mustafa, L. Khan, Z. Lin, K. Hamlen, et al., Adaptive encrypted traffic fingerprinting with bi-directional dependence, in: Proceedings of the 32nd Annual Conference on Computer Security Applications, (2016), 177–188. https://doi.org/10.1145/2991079.2991123
[49]	A. Kanda, M. Hashimoto, Identification of TLS communications using randomness testing, in: 2021 IEEE Publications 45th Annual Computers, software, and Applications Conference (COMPSAC), 1099–1106. https://doi.org/10.1109/COMPSAC51774.2021.00150
[50]	L. E. Bassham Ⅲ, A. L. Rukhin, J. Soto, J. R. Nechvatal, M. E. Smid, E. B. Barker, et al., A statistical test suite for random and pseudorandom number generators for cryptographic applications, National Institute of Standards & Technology, 800–822, 2010.
[51]	K. Böttinger, D. Schuster, C. Eckert, Detecting fingerprinted data in TLS traffic, in: Proceedings of the 10th ACM Symposium on Information, Comput. Commun. Security, (2015), 633–638. https://doi.org/10.1145/2714576.2714595
[52]	Z. Zhang, C. Kang, G. Xiong, Z. Li, Deep forest with LRRS feature for fine-grained website fingerprinting with encrypted SSL/TLS, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, (2019), 851–860. https://doi.org/10.1145/3357384.3357993
[53]	J. Lu, G. Gou, M. Su, D. Song, C. Liu, C. Yang, et al., GAP-WF: Graph attention pooling network for fine-grained SSL/TLS Website fingerprinting, in: International Joint Conference on Neural Networks (IJCNN), IEEE, (2021), 1–8. https://doi.org/10.1109/IJCNN52387.2021.9533543
[54]	C. Richter, M. Finsterbusch, J. A. Müller, K. Hänßgen, Classification of TLS applications, in: Proceedings of the 9th International Conference on Internet Monitoring and Protection, ICIMP, (2014), 1–6.
[55]	B. Anderson, Classifying encrypted traffic with TLS-aware telemetry, in: CERT, FloCon2016, (2016).
[56]	B. Anderson, D. McGrew, Identifying encrypted malware traffic with contextual flow data, in: Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, (2016), 35–46. https://doi.org/10.1145/2996758.2996768
[57]	B. Anderson, S. Paul, D. McGrew, Deciphering malware's use of TLS (without decryption), J. Comput. Virol. Hacking Tech., 14 (2018), 195–211. https://doi.org/10.1007/s11416-017-0306-6 doi: 10.1007/s11416-017-0306-6

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)