High-dimensional Parameter Learning over General Graphical State Space Models: Beating the Curse of Dimensionality

Abstract: Disease transmission systems are highly nonlinear and stochastic and are imperfectly observable. However, conducting high-dimensional parameter learning for partially observed, nonlinear, and stochastic spatiotemporal processes is a methodological challenge and is an open problem so far. We propose the iterated block particle filter (IBPF) algorithm for learning high-dimensional parameters over graphical state space models with general state spaces, measures, transition densities, and graph structure. Theoretical performance guarantees are obtained on beating the curse of dimensionality (COD), algorithm convergence, and likelihood maximization. Experiments on a highly nonlinear and non-Gaussian spatiotemporal model for measles transmission reveal that the iterated ensemble Kalman filter algorithm (Li et al. (2020), Science) is ineffective and the iterated filtering algorithm (Ionides et al. (2015), PNAS) suffers from the COD, while our IBPF algorithm beats COD consistently across various experiments with different metrics.

Talk based on paper: “Iterated Block Particle Filter for High-dimensional Parameter Learning: Beating the Curse of Dimensionality”, Ning Ning and Edward Ionides, ArXiv: https://arxiv.org/abs/2110.10745, 2021.