An accessible approach to density estimation neural networks with data preprocessing

Bosi Hou; Jonathan E. Rubin; Bosi Hou; Jonathan E. Rubin

doi:10.3934/mbe.2025116

Mathematical Biosciences and Engineering

2025, Volume 22, Issue 12: 3130-3153. doi: 10.3934/mbe.2025116

Previous Article Next Article

Research article

An accessible approach to density estimation neural networks with data preprocessing

Bosi Hou ¹,
Jonathan E. Rubin ^{2
,
,}

1.
Data Science Institute, Columbia University, New York, NY 10027, USA
2.
Department of Mathematics, University of Pittsburgh, 301 Thackeray Hall, Pittsburgh, PA 15260, USA

Received: 26 March 2025 Revised: 15 September 2025 Accepted: 17 September 2025 Published: 03 November 2025

Density estimation neural networks (DENNs) represent a form of artificial neural network designed to provide an efficient approach to the Bayesian estimation of a probability density on a model parameter space, conditioned on an empirical observation of the underlying system. Despite their efficiency and potential, DENNs remain underutilized for parameter estimation in mathematical modeling. In this work, we aim to boost the accessibility of the DENN approach by providing a user-friendly introduction and code that makes it easy for users to harness existing, cutting-edge DENN software. Furthermore, we insert an easily-implemented preliminary data simulation step that reduces the computational demands of the approach and empirically demonstrates that it maintains the accuracy of parameter estimation for a stochastic oscillator model.
- parameter estimation,
- Bayesian inference,
- normalizing flows,
- deep learning,
- predator-prey cycles
Citation: Bosi Hou, Jonathan E. Rubin. An accessible approach to density estimation neural networks with data preprocessing[J]. Mathematical Biosciences and Engineering, 2025, 22(12): 3130-3153. doi: 10.3934/mbe.2025116

Related Papers:

Abstract

Density estimation neural networks (DENNs) represent a form of artificial neural network designed to provide an efficient approach to the Bayesian estimation of a probability density on a model parameter space, conditioned on an empirical observation of the underlying system. Despite their efficiency and potential, DENNs remain underutilized for parameter estimation in mathematical modeling. In this work, we aim to boost the accessibility of the DENN approach by providing a user-friendly introduction and code that makes it easy for users to harness existing, cutting-edge DENN software. Furthermore, we insert an easily-implemented preliminary data simulation step that reduces the computational demands of the approach and empirically demonstrates that it maintains the accuracy of parameter estimation for a stochastic oscillator model.

References

[1]	A. F. M. Smith, G. O. Roberts, Bayesian computation via the gibbs sampler and related markov chain monte carlo methods, J. R. Stat. Soc. B, 55 (1993), 3–23. https://doi.org/10.1111/j.2517-6161.1993.tb01466.x doi: 10.1111/j.2517-6161.1993.tb01466.x
[2]	J. Rothfuss, F. Ferreira, S. Walther, M. Ulrich, Conditional density estimation with neural networks: Best practices and benchmarks, preprint, arXiv: 1903.00954.
[3]	C. M. Bishop, Mixture Density Networks, Aston University, WorkingPaper, 1994.
[4]	J. M. Marin, P. Pudlo, C. P. Robert, R. J. Ryder, Approximate bayesian computational methods, Stat. Comput., 22 (2012), 1167–1180. https://doi.org/10.1007/s11222-011-9288-2 doi: 10.1007/s11222-011-9288-2
[5]	G. Papamakarios, D. Sterratt, I. Murray, Sequential neural likelihood: Fast likelihood-free inference with autoregressive flows, in The 22nd International Conference on Artificial Intelligence and Statistics, 89 (2019), 837–848. https://doi.org/10.1007/s00104-018-0714-2
[6]	E. Trentin, Multivariate density estimation with deep neural mixture models, Neural Process. Lett., 55 (2023), 9139–9154. https://doi.org/10.1007/s11063-023-11196-2 doi: 10.1007/s11063-023-11196-2
[7]	G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, B. Lakshminarayanan, Normalizing flows for probabilistic modeling and inference, J. Mach. Learn. Res., 22 (2021), 1–64.
[8]	M. L. Rosenzweig, R. H. MacArthur, Graphical representation and stability conditions of predator-prey interactions, Am. Nat., 97 (1963), 209–223. https://doi.org/10.1086/282272 doi: 10.1086/282272
[9]	D. Swigon, S. R. Stanhope, S. Zenker, J. E. Rubin, On the importance of the jacobian determinant in parameter inference for random parameter and random measurement error models, SIAM/ASA J. Uncertainty Quantif., 7 (2019), 975–1006. https://doi.org/10.1137/17M1114405 doi: 10.1137/17M1114405
[10]	S. Kullback, R. Leibler, On information and sufficiency, Ann. Math. Stat., 22 (1951), 79–86. https://doi.org/10.1137/17M1114405 doi: 10.1137/17M1114405
[11]	D. Rezende, S. Mohamed, Variational inference with normalizing flows, in Proceedings of Machine Learning Research, (2015), 1530–1538.
[12]	I. Kobyzev, S. J. Prince, M. A. Brubaker, Normalizing flows: An introduction and review of current methods, IEEE Trans. Pattern Anal. Mach. Intell., 43 (2021), 3964–3979. https://doi.org/10.1109/TPAMI.2020.2992934 doi: 10.1109/TPAMI.2020.2992934
[13]	C. S. Holling, The functional response of predators to prey density and its role in mimicry and population regulation, Mem. Entomol. Soc. Can., 97 (1965), 5–60. https://doi.org/10.4039/entm9745fv doi: 10.4039/entm9745fv
[14]	J. E. Rubin, D. J. D. Earn, P. E. Greenwood, T. L. Parsons, T. L. Abbott, Irregular population cycles driven by environmental stochasticity and saddle crawlbys, Oikos, 2023 (2023), e09290. https://doi.org/10.1111/oik.09290 doi: 10.1111/oik.09290
[15]	J. C. Cox, J. E. J. Ingersoll, S. A. Ross, A theory of the term structure of interest rates, Econometrica, 53 (1985), 385–407.
[16]	E. Allen, Environmental variability and mean-reverting processes, Discrete Contin. Dyn. Syst. Ser. B, 21 (2016), 2073–2089. https://doi.org/10.3934/dcdsb.2016037 doi: 10.3934/dcdsb.2016037
[17]	F. A. Viana, A tutorial on latin hypercube design of experiments, Qual. Reliab. Eng. Int., 32 (2016), 1975–1985. https://doi.org/10.1002/qre.1924 doi: 10.1002/qre.1924
[18]	Y. Wen, Z. Li, Y. Xiang, D. Reker, Improving molecular machine learning through adaptive subsampling with active learning, Digital Discovery, 2 (2023), 1134–1142. https://doi.org/10.1039/d3dd00037k doi: 10.1039/d3dd00037k
[19]	A. Tejero-Cantero, J. Boelts, M. Deistler, J. M. Lueckmann, C. Durkan, P. J. Gonçalves, et al., sbi: A toolkit for simulation-based inference, J. Open Source Software, 5 (2020), 2505. https://doi.org/10.21105/joss.02505 doi: 10.21105/joss.02505
[20]	J. Boelts, J. M. Lueckmann, R. Gao, J. H. Macke, Flexible and efficient simulation-based inference for models of decision-making, eLife, 11 (2022), e77220. https://doi.org/10.7554/eLife.77220 doi: 10.7554/eLife.77220
[21]	D. Reynolds, Gaussian Mixture Models, in Encyclopedia of Biometrics (eds. S. Z. Li, A. K. Jain), Springer US, Boston, MA, (2015), 827–832. https://doi.org/10.1007/978-1-4899-7488-4_196
[22]	P. Zhao, L. Lai, Analysis of knn density estimation, IEEE Trans. Inf. Theory, 68 (2022), 7971–7982. https://doi.org/10.1109/TIT.2022.3195870 doi: 10.1109/TIT.2022.3195870
[23]	D. Peel, G. J. McLachlan, Robust mixture modelling using the t distribution, Stat. Comput., 10 (2000), 339–348. https://doi.org/10.1023/A:1008981510081 doi: 10.1023/A:1008981510081
[24]	C. Jin, Y. Zhang, S. Balakrishnan, M. J. Wainwright, M. I. Jordan, Local maxima in the likelihood of gaussian mixture models: Structural results and algorithmic consequences, preprint, arXiv: 1609.00978.
[25]	G. E. Uhlenbeck, L. S. Ornstein, On the theory of the brownian motion, Phys. Rev., 36 (1930), 823–841. https://doi.org/10.1103/PhysRev.36.823 doi: 10.1103/PhysRev.36.823
[26]	X. Zhou, W. Zhang, Z. Chen, S. Diao, T. Zhang, Efficient neural network training via forward and backward propagation sparsification, in Neural Information Processing Systems, 36 (2021), 15216–15229.
[27]	T. Ma, L. Zhang, F. Cao, Y. Ge, A special multigrid strategy on non-uniform grids for solving 3d convection–diffusion problems with boundary/interior layers, Symmetry, 13 (2021), 1123. https://doi.org/10.3390/sym13071123 doi: 10.3390/sym13071123
[28]	P. Kitanidis, Introduction to Geostatistics: Applications in Hydrogeology, Cambridge University Press, Cambridge, UK, 1997.
[29]	D. Singh, B. Singh, Investigating the impact of data normalization on classification performance, Appl. Soft Comput. J., 97 (2019), 105524. https://doi.org/10.1016/j.asoc.2019.105524 doi: 10.1016/j.asoc.2019.105524
[30]	M. M. Ahsan, M. A. P. Mahmud, P. K. Saha, K. D. Gupta, Z. Siddique, Effect of data scaling methods on machine learning algorithms and model performance, Technologies, 9 (2021), 52. https://doi.org/10.3390/technologies9030052 doi: 10.3390/technologies9030052

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)