Real-time forecasting of unfolding epidemics represents an extraordinary challenge for mathematical and computational models due to the technical challenges posed by parameter estimation and validation. For this reason, the reliability of mathematical and computational models to support public-health decisions during an epidemic is currently under debate. The 2009 A/H1N1 pandemic represented an unprecedented opportunity to validate and assess real-time model predictions.
In the recently published paper:
Real-time numerical forecast of global epidemic spreading: case study of 2009 A/H1N1pdm
Michele Tizzoni, Paolo Bajardi, Chiara Poletto, José J. Ramasco, Duygu Balcan, Bruno Gonçalves, Nicola Perra, Vittoria Colizza, Alessandro Vespignani.
BMC Medicine, 10:165 (2012).
we present a thorough validation and assessment of the results obtained using GLEAM during the course of the 2009 A/H1N1 pandemic.
In detail, we used GLEAM in real-time to generate stochastic simulations of epidemic spread worldwide, yielding the incidence and seeding events at a daily resolution for 220 countries. Using a Monte Carlo Maximum Likelihood analysis, the model provided an estimate of the seasonal transmission potential during the early phase of the H1N1 pandemic, and generated ensemble forecasts for the activity peaks in the northern hemisphere in the fall/winter wave. The forecasts were published in September 2009, well before the peak weeks of epidemic activity in the northern hemisphere.
Now, we have validated those results against the real-life surveillance data collected in 48 countries, and their robustness assessed by focusing on 1) the peak timing of the pandemic; 2) the level of spatial resolution allowed by the model; and 3) the clinical attack rate and the effectiveness of the vaccine. In addition, we studied the effect of data incompleteness on the prediction reliability.
Real-time predictions of the peak timing are found to be in good agreement with the empirical data, showing strong robustness to data that may not be accessible in real time (such as pre-exposure immunity and adherence to vaccination campaigns), but that affect the predictions for the attack rates. The timing and spatial unfolding of the pandemic are critically sensitive to the level of mobility data integrated into the model. For a detailed comparison between the surveillance data on peak incidence and our forecasts, take a look at this page.
Left: peak weeks of the epidemic activity in the baseline stochastic forecast for seven Canadian Provinces, ordered eastward from top to bottom. Right: map of Canada, where the Canadian Provinces under study have been highlighted in red.
Our results show that large-scale models can be used to provide valuable real-time forecasts of influenza spreading, but they require high-performance computing. The quality of the forecast depends on the level of data integration, thus stressing the need for high-quality data in population-based models, and of progressive updates of validated available empirical knowledge to inform these models.
Difference of the median peak weeks in the reference stochastic forecast, where the full mobility dataset is considered, and the scenario that includes only the 500 busiest airports, as a function of the median peak week in the reference forecast. Dots are color coded according to the corresponding airport’s climate zone.