Recognition of crowd behavior from mobile sensors with pattern analysis and graph clustering methods

Daniel Roggen; Martin Wirz; Gerhard Tröster; Dirk Helbing; Daniel Roggen; Martin Wirz; Gerhard Tröster; Dirk Helbing

doi:10.3934/nhm.2011.6.521

Networks and Heterogeneous Media

2011, Volume 6, Issue 3: 521-544. doi: 10.3934/nhm.2011.6.521

Previous Article Next Article

Recognition of crowd behavior from mobile sensors with pattern analysis and graph clustering methods

1.
Wearable Computing Laboratory, Gloriastrasse 35, ETH Zurich, CH-8092 Zurich
2.
CLU E11, Clausiusstrasse 50, ETH Zurich, CH-8092 Zurich

Received: 01 December 2010 Revised: 01 June 2011
Primary: 68T10, 91-04, 91-02, 91D30, 62H30; Secondary: 91C20, 93A15, 93A30.

Mobile on-body sensing has distinct advantages for the analysis and understanding of crowd dynamics: sensing is not geographically restricted to a specific instrumented area, mobile phones offer on-body sensing and they are already deployed on a large scale, and the rich sets of sensors they contain allows one to characterize the behavior of users through pattern recognition techniques.
In this paper we present a methodological framework for the machine recognition of crowd behavior from on-body sensors, such as those in mobile phones. The recognition of crowd behaviors opens the way to the acquisition of large-scale datasets for the analysis and understanding of crowd dynamics. It has also practical safety applications by providing improved crowd situational awareness in cases of emergency.
The framework comprises: behavioral recognition with the user's mobile device, pairwise analyses of the activity relatedness of two users, and graph clustering in order to uncover globally, which users participate in a given crowd behavior. We illustrate this framework for the identification of groups of persons walking, using empirically collected data.
We discuss the challenges and research avenues for theoretical and applied mathematics arising from the mobile sensing of crowd behaviors.
- Crowd dynamics,
- crowd behavior recognition,
- mobile sensing,
- machine learning
Citation: Daniel Roggen, Martin Wirz, Gerhard Tröster, Dirk Helbing. Recognition of crowd behavior from mobile sensors with pattern analysis and graph clustering methods[J]. Networks and Heterogeneous Media, 2011, 6(3): 521-544. doi: 10.3934/nhm.2011.6.521

Related Papers:

Abstract

Mobile on-body sensing has distinct advantages for the analysis and understanding of crowd dynamics: sensing is not geographically restricted to a specific instrumented area, mobile phones offer on-body sensing and they are already deployed on a large scale, and the rich sets of sensors they contain allows one to characterize the behavior of users through pattern recognition techniques.
In this paper we present a methodological framework for the machine recognition of crowd behavior from on-body sensors, such as those in mobile phones. The recognition of crowd behaviors opens the way to the acquisition of large-scale datasets for the analysis and understanding of crowd dynamics. It has also practical safety applications by providing improved crowd situational awareness in cases of emergency.
The framework comprises: behavioral recognition with the user's mobile device, pairwise analyses of the activity relatedness of two users, and graph clustering in order to uncover globally, which users participate in a given crowd behavior. We illustrate this framework for the identification of groups of persons walking, using empirically collected data.
We discuss the challenges and research avenues for theoretical and applied mathematics arising from the mobile sensing of crowd behaviors.

References

[1]	M. Bächlin, M. Plotnik, D. Roggen, I. Maidan, J. M. Hausdorff, N. Giladi and G. Tröster, Wearable assistant for parkinson's disease patients with the freezing of gait symptom, IEEE Transactions on Information Technology in Biomedicine, 14 (2010), 436-446. doi: 10.1109/TITB.2009.2036165
[2]	K. Bao and S. Intille, "Activity Recognition from User-Annotated Acceleration Data," Proc 2nd Int Conf Pervasive Computing, (2004), 1-17.
[3]	N. Bellomo and C. Dogbé, On the modelling crowd dynamics from scaling to hyperbolic macroscopic models, Mathematical Models and Methods in Applied Sciences, 18 (2008), 1317–-1345. doi: 10.1142/S0218202508003054
[4]	M. Benocci, C. Tacconi, E. Farella, L. Benini, L. Chiari and L. Vanzago, Accelerometer-based fall detection using optimized zigbee data streaming, Microelectronics Journal, 41 (2010), 703-710. doi: 10.1016/j.mejo.2010.06.014
[5]	C. Bettini, O. Brdiczka, K. Henricksen, J. Indulska, D. Nicklas, A. Ranganathan and D. Riboni, A survey of context modelling and reasoning techniques, Pervasive and Mobile Computing, 6 (2010), 161-180. doi: 10.1016/j.pmcj.2009.06.002
[6]	I. Borg and P Groenen, "Modern Multidimensional Scaling: Theory and Applications," Springer Series in Statistics, Springer-Verlag, New York, 1997.
[7]	M. Buchanan, The science of subtle signals, Strategy+Business, 48 (2007), 68-77.
[8]	I. Cohen and M. Goldszmidt, Properties and benefits of calibrated classifiers, Proc. Knowledge Discovery in Databases, 2004, 125-136.
[9]	V. Coscia and C. Canavesio, First-order macroscopic modelling of human crowd dynamics, Math. Mod. Meth. Appl. Sci., 18 (2008), 1217-1247. doi: 10.1142/S0218202508003017
[10]	N. Davies, D. P. Siewiorek and R. Sukthankar, Special issue: Activity-based computing, IEEE Pervasive Computing, 7 (2008), 20-21. doi: 10.1109/MPRV.2008.26
[11]	R. O. Duda, P. E. Hart and D. G. Stork, "Pattern Classification," Second edition, Wiley-Interscience, New York, 2001.
[12]	P. Eades, A heuristic for graph drawing, Congressus Numerantium, 42 (1984), 149-160.
[13]	N. Eagle, A. Pentland and D. Lazer, Inferring friendship network structure by using mobile phone data, Proc Natl Acad Sci U S A, 106 (2009), 15274-15278. doi: 10.1073/pnas.0900282106
[14]	T. Fawcett, "ROC graphs: Notes and practical considerations for researchers," Tech. report, HP Laboratories, 2004.
[15]	D. Figo, P. Diniz, D. Ferreira and J. Cardoso, Preprocessing techniques for context recognition from accelerometer data, Pervasive and Mobile Computing, 14 (2010), 645-662.
[16]	T. M. J Fruchterman and E. M. Reingold, Graph drawing by force-directed placement, Software - Practice and Experience, 21 (1991), 1129-1164. doi: 10.1002/spe.4380211102
[17]	M. González and A.-L. Barabási, Complex networks: From data to models, Nature Physics, 3 (2007), 224-225. doi: 10.1038/nphys581
[18]	D. Helbing, L. Buzna, A. Johansson and T. Werner, Self-organized pedestrian crowd dynamics: Experiments, simulations, and design solutions, Transportation Science, 39 (2005), 1-24. doi: 10.1287/trsc.1040.0108
[19]	D. Helbing, I. Farkas and T. Vicsek, Simulating dynamical features of escape panic, Nature, 407 (2000), 487-–490. doi: 10.1038/35035023
[20]	D. Helbing and A. Johansson, "Pedestrian, Crowd and Evacuation Dynamics," Meyers Encyclopedia of Complexity and Systems Science, Springer, Berlin, 2009.
[21]	D. Helbing and P. Molnár, Social force model for pedestrian dynamics, Physical Review E, 51 (1995), 4282-4286. doi: 10.1103/PhysRevE.51.4282
[22]	S. Hoogendoorn and P. Bovy, Simulation of pedestrian flows by optimal control and differential games, Opt. Cont. Appl. Meth., 24 (2003), 153-172. doi: 10.1002/oca.727
[23]	A. Johansson, D. Helbing and H. Z. Al-Abideen, From crowd dynamics to crowd safety: A video-based analysis, Advances in Complex Systems, 11 (2008), 497-527. doi: 10.1142/S0219525908001854
[24]	A. Johansson, D. Helbing and P. S. Shukla, Specification of the social force pedestrian model by evolutionary adjustment to video tracking data, Advances in Complex Systems, 10 (2007), 271-288. doi: 10.1142/S0219525907001355
[25]	S. Kallio, J. Kela, P. Korpipää and J. Mäntyjärvi, User independent gesture interaction for small handheld devices, International Journal of Pattern Recognition and Artificial Intelligence, 20 (2006), 505-524.
[26]	A. Kesting, M. Treiber and D. Helbing, Connectivity statistics of store-and-forward intervehicle communication, IEEE Transactions on Intelligent Transportation Systems, 11 (2010), 172-181. doi: 10.1109/TITS.2009.2037924
[27]	J. Kleinberg, The convergence of social and technological networks, Communications of the ACM, 51 (2008), 66-72. doi: 10.1145/1400214.1400232
[28]	M. H. Ko, G. West, S. Venkatesh and M. Kumar, "Online Context Recognition in Multisensor Systems Using Dynamic Time Warping," Proc. Conf. Intelligent Sensors, Sensor Networks and Information Processing Conference, (2005), 283-288.
[29]	N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury and A. T. Campbell, A survey of mobile phone sensing, IEEE Communications Magazine, 48 (2010), 140-150.
[30]	D. Lazer, A. Pentland, L. Adamic, S. Aral, A.-L. Barabási, D. Brewer, N. Christakis, N. Contractor, J. Fowler, M. Gutmann, T. Jebara, G. King, M. Macy, D. Roy and M. Van Alstyne, Computational social science, Science, 323 (2009), 721-723. doi: 10.1126/science.1167742
[31]	S. Mann, Humanistic computing: "Wearcom" as a new framework and application for intelligent signal processing, Proceedings of the IEEE 86 (1998), 2123-2151. doi: 10.1109/5.726784
[32]	S. McKeever, J. Ye, L. Coyle and S. Dobson, "Using Dempster-Shafer Theory of Evidence for Situation Inference," Proceedings of the 4th European conference on Smart sensing and context, Berlin, Heidelberg, EuroSSC'09, Springer-Verlag, (2009), 149-162.
[33]	T. M. Mitchell, Mining our reality, Science, 326 (2009), 1644-1645. doi: 10.1126/science.1174459
[34]	M. Moussaïd, N. Perozo, S. Garnier, D. Helbing and G. Theraulaz, The walking behaviour of pedestrian social groups and its impact on crowd dynamics, PLoS One, 5 (2010), e10047. doi: 10.1371/journal.pone.0010047
[35]	J.-K. Onnela, J. Saramäki, J. Hyvönen, G. Szabó, M. A. de Menezes, K. Kaski, A.-L. Barabási and J. Kertész, Analysis of a large-scale weighted network of one-to-one human communication, New Journal of Physics, 9 (2007).
[36]	J. A. Paradiso, J. Gips, M. Laibowitz, S. Sadi, D. Merrill, R. Aylward, P. Maes and A. Pentland, Identifying and facilitating social interaction with a wearable wireless sensor network, Personal and Ubiquitous Computing, 4 (2010), 137-152. doi: 10.1007/s00779-009-0239-2
[37]	A. Pentland, "Honest Signals: How They Shape Our World," Bradford Books, 2008.
[38]	A. Pentland, T. Choudhury, N. Eagle and P. Singh, Human dynamics: Computation for organizations, Pattern Recognition Letters, 26 (2005), 503-511. doi: 10.1016/j.patrec.2004.08.012
[39]	H. Qian, Y. Mao, W. Xiang and Z. Wang, Recognition of human activities using SVM multi-class classifier, Pattern Recognition Letters, 31 (2010), 100-111. doi: 10.1016/j.patrec.2009.09.019
[40]	C. Randell and H. Muller, "Context Awareness by Analysing Accelerometer Data," ISWC 2000: Proc. of the 4th Int'l Symposium on Wearable Computers, October 2000, 175-176.
[41]	A. Ranganathan, J. Al-Muhtadi and R. H. Campbell, Reasoning about uncertain contexts in pervasive computing environments, Pervasive Computing, IEEE, 3 (2004), 62-70. doi: 10.1109/MPRV.2004.1316821
[42]	S. Saxena, F. Brémond, M. Thonnat and R. Ma, "Crowd Behavior Recognition for Video Surveillance," Advanced Concepts for Intelligent Vision Systems (Berlin), Lecture Notes in Computer Science, 5259, Springer, (2008), 970-981.
[43]	S. E. Schaeffer, Graph clustering, Computer Science Review, 1 (2007), 27-64.
[44]	S. H. Shin, M. S. Lee, C. G. Park and H. S. Hong, "Pedestrian Dead Reckoning System with Phone Location Awareness Algorithm," Proc. Position Location and Navigation Symposium (PLANS), IEEE Press, 2010, 97-101.
[45]	T. Starner, J. Weaver and A. Pentland, Real-time American sign language recognition using desk and wearable computer based video, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (1998), 1371-1375.
[46]	T. Stiefmeier, D. Roggen, G. Ogris, P. Lukowicz and G. Tröster, Wearable activity tracking in car manufacturing, IEEE Pervasive Computing, 7 (2008), 42-50. doi: 10.1109/MPRV.2008.40
[47]	J. A. Ward, P. Lukowicz, G. Tröster and T. Starner, Activity recognition of assembly tasks using body-worn microphones and accelerometers, IEEE Trans. Pattern Analysis and Machine Intelligence, 28 (2006), 1553-1567.
[48]	J. A. Ward, P. Lukowicz and H. Gellersen, Performance metrics for activity recognition, ACM Transactions on Information Systems and Technology, 2 (2011), 6:1-6:23.
[49]	M. Wirz, D. Roggen and G. Tröster, "Decentralized Detection of Group Formations from Wearable Acceleration Sensors," IEEE Int. Conf. on Computational Science and Engineering, IEEE Press, 2009, 952-959.
[50]	_____, "A Methodology Towards the Detection of Collective Behavior Patterns by Means of Body-Worn Sensors," UbiLarge workshop at the 8th Int. Conf. on Pervasive Computing, 2010.
[51]	_____, "User Acceptance Study of a Mobile System for Assistance During Emergency Situations at Large-Scale Events," 3rd International Conference on Human Centric Computing, 2010.
[52]	P. Zappi, C. Lombriser, E. Farella, D. Roggen, L. Benini and G. Tröster, "Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection," 5th European Conf. on Wireless Sensor Networks (ed. R. Verdone), Springer, (2008), 17-33.
[53]	B. Zhan, D. N. Monekosso, P. Remagnino, S. A. Velastin and L.-Q. Xu, Crowd analysis: A survey, Machine Vision and Applications, 19 (2008), 345-–357. doi: 10.1007/s00138-008-0132-4

Reader Comments

Your name:*

Email:*
© 2011 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)