Validation of methods for forecasting the frequency of non-vaccine serotypes after introduction or switch of a pneumococcal conjugate vaccine

Background To anticipate the impact of new pneumococcal vaccines and guide future updates, accurate forecasts of changes in non-vaccine serotypes (NVTs) are needed. We developed and evaluated three models that incorporated different assumptions about the way in which NVTs will increase and generated ensemble predictions for the frequency of NVTs in different post- pneumococcal conjugate vaccines (PCV) periods. Methods We analyzed age- and serotype-specific invasive pneumococcal disease (IPD) cases from the United States CDCs Active Bacterial Core surveillance during the pre-PCV (1998-1999), early post-PCV7 (2000-2004), late post-PCV7/pre-PCV13 (2005-2009), early post-PCV13 (2010-2014), and late post-PCV13 (2015-2019) periods. These data were augmented with IPD cases from several countries and combined with serotype-specific invasiveness to infer serotype-specific carriage prevalence. Three models (Ranking, Proportionate, NFDS-lite) generated independent predictions of post-PCV IPD frequencies, which were integrated using an accuracy-weighted ensemble. Model performance was evaluated using the normalized root mean square error (NRMSE). Results A total of 23,959 non-PCV7 and 15,580 non-PCV13 cases were analyzed. NVT cases increased from the pre-PCV7 to the late post-PCV7 and post-PCV13 periods. The accuracy of predictions across age groups and models was consistent and high during the post-PCV13 periods but varied during the post-PCV7 periods. The Proportionate model (NRMSE=0.70-3.95) outperformed the NFDS-lite (NRMSE=0.93-8.91) and Ranking (NRMSE=1.51-5.37) models during the early-post-PCV7 period, whereas the NFDS-lite model (NRMSE=1.55-9.82) was superior to the Proportionate (NRMSE=1.45-10.22) and Ranking (NRMSE=1.86-11.35) models during the late post-PCV7 period. The Ensemble model improved on these individual models. Conclusions The Ensemble model offers a tool for forecasting serotype patterns to inform pneumococcal vaccines impact and future pneumococcal vaccine formulation.