Selective reporting of outcomes and results in interrupted time series studies of health interventions: a methodological study

Objective: Selective reporting bias occurs when authors report outcomes or results based on the P value, magnitude or direction of the results. Selective reporting has not been examined in interrupted time series (ITS) studies. Therefore, we investigated (1) discrepancies in outcome reporting, (2) completeness of results reporting, and (3) evidence of selective reporting bias in ITS studies. Design: We systematically searched for published peer-reviewed protocols of ITS studies of health interventions in 22 databases, and corresponding results reports addressing the protocol’s primary ITS research questions. For each primary research question, we identified outcomes that were reported in the protocol and the corresponding results report(s), and all results reported for those outcomes. Main outcome measure: We defined a discrepancy as any outcome that was added, omitted, or had its primacy reclassified (e.g., from primary to secondary) in the results reports compared to the protocol. Each result was classified as fully reported if it was reported with both an effect estimate and a measure of precision (such as a confidence interval). Each result was also classified as favourable or not favourable to the interruption, based on its statistical significance and direction of effect. Results: Our search for ITS protocols returned 4,590 abstracts. After excluding ongoing studies, protocols without published results for the primary research question, and records excluded for other reasons, we identified 44 ITS protocols (published 2010-2022) with 46 corresponding results reports. Among outcomes assessed for discrepancies, 52% (202/388) had a discrepancy, affecting 74% (31/42) of studies. Non-reporting of outcomes was prevalent, with 24% (132/553) of outcomes defined in the protocol not reported in the results report, and 60% (25/42) of protocols having at least one outcome not reported in the result reports. Only 28% (56/202) of discrepancies were justified by authors in the results reports. The association between a result favouring the interruption (based on statistical significance and direction of effect) and the result being fully reported was uncertain (OR=1.06 [95% CI 0.74 to 1.53]). Conclusion: Non-reporting of outcomes and discrepancies in outcome reporting were prevalent. Pre-specifying outcomes in protocols and registries helps mitigate selective reporting. However, outcomes should be described in sufficient detail for readers to detect any changes.