Using cultural, historical, and epidemiological data to inform, calibrate, and verify model structures in agent-based simulations

Lisa Sattenspiel; Jessica Dimka; Carolyn Orbann; Lisa Sattenspiel; Jessica Dimka; Carolyn Orbann

doi:10.3934/mbe.2019152

Mathematical Biosciences and Engineering

2019, Volume 16, Issue 4: 3071-3093. doi: 10.3934/mbe.2019152

Previous Article Next Article

Research article Special Issues

Using cultural, historical, and epidemiological data to inform, calibrate, and verify model structures in agent-based simulations

1.
Department of Anthropology, University of Missouri, Columbia, MO, USA
2.
Department of Anthropology, University of Pittsburgh, Pittsburgh, PA, USA
3.
Department of Health Sciences, University of Missouri, Columbia, MO, USA

Received: 15 December 2018 Accepted: 03 April 2019 Published: 10 April 2019

Agent-based simulation models are excellent tools for addressing questions about the spread of infectious diseases in human populations because realistic, complex behaviors as well as random factors can readily be incorporated. Agent-based models are flexible and allow for a wide variety of behaviors, time-related variables, and geographies, making the calibration process an extremely important step in model development. Such calibration procedures, including verification and validation, may be complicated, however, and usually require incorporation of substantial empirical data and theoretical knowledge of the populations and processes under study. This paper describes steps taken to build and calibrate an agent-based model of epidemic spread in an early 20^th century fishing village in Newfoundland and Labrador, including a description of some of the detailed ethnographic and historical data available. We illustrate how these data were used to develop the structure of specific parts of the model. The resulting model, however, is designed to reflect a generic small community during the early 20^th century and the spread of a directly transmitted disease within such a community, not the specific place that provided the data. Following the description of model development, we present the results of a replication study used to confirm the model behaves as intended. This study is also used to identify the number of simulations necessary for high confidence in average model output. We also present selected results from extensive sensitivity analyses to assess the effect that variation in parameter values has on model outcomes. After careful calibration and verification, the model can be used to address specific practical questions of interest. We provide an illustrative example of this process.
- agent-based modeling,
- model calibration,
- model verification,
- sensitivity analysis,
- replication analysis,
- epidemic model
Citation: Lisa Sattenspiel, Jessica Dimka, Carolyn Orbann. Using cultural, historical, and epidemiological data to inform, calibrate, and verify model structures in agent-based simulations[J]. Mathematical Biosciences and Engineering, 2019, 16(4): 3071-3093. doi: 10.3934/mbe.2019152

Related Papers:

Abstract

Agent-based simulation models are excellent tools for addressing questions about the spread of infectious diseases in human populations because realistic, complex behaviors as well as random factors can readily be incorporated. Agent-based models are flexible and allow for a wide variety of behaviors, time-related variables, and geographies, making the calibration process an extremely important step in model development. Such calibration procedures, including verification and validation, may be complicated, however, and usually require incorporation of substantial empirical data and theoretical knowledge of the populations and processes under study. This paper describes steps taken to build and calibrate an agent-based model of epidemic spread in an early 20^th century fishing village in Newfoundland and Labrador, including a description of some of the detailed ethnographic and historical data available. We illustrate how these data were used to develop the structure of specific parts of the model. The resulting model, however, is designed to reflect a generic small community during the early 20^th century and the spread of a directly transmitted disease within such a community, not the specific place that provided the data. Following the description of model development, we present the results of a replication study used to confirm the model behaves as intended. This study is also used to identify the number of simulations necessary for high confidence in average model output. We also present selected results from extensive sensitivity analyses to assess the effect that variation in parameter values has on model outcomes. After careful calibration and verification, the model can be used to address specific practical questions of interest. We provide an illustrative example of this process.

References

[1]	D. Bernoulli, Essai d'une nouvelle analyse de la mortalité causée par la petite vérole, et des avantages de l'inoculation pour la prévenir, Mém. Math. Phys. Acad. Roy. Sci., Paris, (1760), 1–45.
[2]	R. Dunbar and M. Spoors, Social networks, support cliques, and kinship, Hum. Nat., 6 (1995), 273–290.
[3]	G. Bruno, P. Nicola, V. Alessandro, et al., Modeling users' activity on twitter networks: Validation of Dunbar's number, PLoS ONE, 6 (2011), e22656.
[4]	R. A. Hill and R. I. Dunbar, Social network size in humans, Hum. Nat., 14 (2003), 53–72.
[5]	F. Ball and P. Neal, A general model for stochastic SIR epidemics with two levels of mixing, Math. Biosci., 180 (2002), 73–102.
[6]	F. Ball and P. Neal, Network epidemic models with two levels of mixing, Math. Biosci., 212 (2008), 69–87.
[7]	F. Ball, D. Sirl and P. Trapman, Analysis of a stochastic SIR epidemic on a random network incorporating household structure, Math. Biosci., 224 (2010), 53–73.
[8]	B. Heath, R. Hill and F. Ciarallo, A survey of agent-based modeling practices (January 1998 to July 2008), J. Artif. Soc. Soc. Simul., 12 (2009), 9.
[9]	J. M. Epstein, Generative Social Science: Studies in Agent-Based Computational Modeling, Princeton University Press, (2006).
[10]	R. Boero and F. Squazzoni, Does empirical embeddedness matter? Methodological issues on agent-based models for analytical social science, J. Artif. Soc. Soc. Simul., 8 (2005), 6.
[11]	C. Graebner, How to relate models to reality? An epistemological framework for the validation and verification of computational models, J. Artif. Soc. Soc. Simul., 21 (2018), 8.
[12]	J. S. Lee, T. Filatova, A. Ligmann-Zielinska, et al., The complexities of agent-based modeling output analysis, J. Artif. Soc. Soc. Simul., 18 (2015), 4.
[13]	E. Borgonovo and E. Plischke, Sensitivity analysis: a review of recent advances, Eur. J. Oper. Res., 248 (2016), 869–887.
[14]	B. G. Marcot, P. H. Singleton and N. H. Schumaker, Analysis of sensitivity and uncertainty in an individual‐based model of a threatened wildlife species, Nat. Resour. Model., 28 (2015), 37–58.
[15]	E. O. Nsoesie, R. J. Beckman and M. V. Marathe, Sensitivity analysis of an individual-based model for simulation of influenza epidemics, PLoS ONE, 7 (2012), e45414.
[16]	F. Pianosi, K. Beven, J. Freer, et al., Sensitivity analysis of environmental models: A systematic review with practical workflow, Environ. Model. Softw., 79 (2016), 214–232.
[17]	G. ten Broeke, G. van Voorn and A. Ligtenberg, Which sensitivity analysis method should I use for my agent-based model?, J. Artif. Soc. Soc. Simul., 19 (2016), 5.
[18]	F. Ferretti, A. Saltelli and S. Tarantola, Trends in sensitivity analysis practice in the last decade, Sci. Total Environ., 568 (2016), 666–670.
[19]	A. Saltelli and P. Annoni, How to avoid a perfunctory sensitivity analysis, Environ. Model. Softw., 25 (2010), 1508–1517.
[20]	J. Dimka, C. Orbann and L. Sattenspiel, Applications of agent-based modeling techniques to studies of historical epidemics: The 1918 flu in Newfoundland and Labrador, J. Can. Hist. Assoc., 25 (2014), 265–296.
[21]	C. Orbann, J. Dimka, E. Miller, et al., Agent‐based modeling and the second epidemiologic transition, in Modern Environments and Human Health: Revisiting the Second Epidemiological Transition (ed. M. K. Zuckerman), Wiley-Blackwell, (2014), 105–122.
[22]	L. Sattenspiel, E. Miller, J. Dimka, et al., Epidemic models with and without mortality: When does it matter?, in Mathematical and Statistical Modeling for Emerging and Re-emerging Infectious Diseases (eds. G. Chowell and M. J. Hyman), Springer International Publishing, (2016), 313–327.
[23]	M. C. Bootsma and N. M. Ferguson, The effect of public health measures on the 1918 influenza pandemic in US cities, Proc. Natl. Acad. Sci. USA., 104 (2007), 7588–7593.
[24]	R. M. Eggo, S. Cauchemez and N. M. Ferguson, Spatial dynamics of the 1918 influenza pandemic in England, Wales and the United States, J. R. Soc. Interface, 8 (2011), 233–243.
[25]	C. E. Mills, J. M. Robins and M. Lipsitch, Transmissibility of 1918 pandemic influenza, Nature, 432 (2004), 904–906.
[26]	D. Davis, The family and social change in the Newfoundland outport, Culture, 3 (1983), 19–32.
[27]	M. M. Firestone, Brothers and Rivals: Patrilocality in Savage Cove, Institute of Social and Economic Research, Memorial University of Newfoundland, (1967).
[28]	T. F. Nemec, "I Fish with My Brother": The Structure and Behaviour of Agnatic-based Fishing Crews in a Newfoundland Irish Outport, Institute of Social and Economic Research, Memorial University of Newfoundland, (1970).
[29]	M. Porter, "She was skipper of the shore-crew:" Notes on the history of the sexual division of labour in Newfoundland, Labour-Travail, 15 (1985), 105–123.
[30]	S. A. Queen and R. W. Habenstein, The Family in Various Cultures, Lippincott, (1974).
[31]	Newfoundland Colonial Secretary's Office, Census of Newfoundland and Labrador 1921, Colonial Secretary's Office, (1923).
[32]	Newfoundland's Grand Banks, Genealogical and historical data for the province of Newfoundland and Labrador, (2013). Available from: http://ngb.chebucto.org.
[33]	Department of Hygiene, Japanese Ministry of Interior, Chapter 7, Section 2. Epidemic records and preventive methods of influenza in the United States of America, in Influenza (Ryukousei Kanbou), Ministry of Interior, (1922), 431–484.
[34]	P. Neal, A household SIR epidemic model incorporating time of day effects, J. Appl. Probab., 53 (2016), 489–501.
[35]	S. Towers and G. Chowell, Impact of weekday social contact patterns on the modeling of influenza transmission, and determination of the influenza latent period, J. Theor. Biol., 312 (2012), 87–95.
[36]	E. Colman, K. Spies and S. Bansal, The reachability of contagion in temporal contact networks: How disease latency can exploit the rhythm of human behavior, BMC Infect. Dis., 18 (2018), 219.

Reader Comments

Your name:*

Email:*
© 2019 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)