Unpublished systematic reviews and financial support: a meta-epidemiological study

BMC Research Notes volume 10, Article number: 703 (2017) Visit for more information SRS Financials

PROSPERO, an international prospective register of systematic reviews, was launched in February 2011 to reduce publication bias of systematic reviews (SRs). A questionnaire survey of SR researchers conducted in 2005 indicated the existence of unpublished SRs and the potential influence of lack of funding as a reason for non-publication. Here, we investigated the publication status of registered SRs in the 1st year that PROSPERO launched and assessed the association between publication and the existence of funding or conflicts of interest (COIs).

We identified 326 SRs registered in PROSPERO from February 2011 through February 2012. Among them, 85 SRs (26%) remained unpublished at least 65 months after registration. We found 241 published reports, including four conference abstracts and one poster presentation. Median time to publication from protocol registration was 16.3 months. Funding for SRs was associated with publication [odds ratio (OR) = 2.10; 95% confidence interval (CI) = 1.26 to 3.50]. We found no significant association of author-reported COIs with publication (OR = 2.35; 95% CI = 0.67 to 8.20). Twenty SRs were not published despite the authors reporting completion of the reviews in PROSPERO.

Publication bias occurs when the publication of study results is influenced by the strength of the research findings. It includes two fundamental concepts: study findings and non-publication . Many studies have reported bias in the dissemination of research findings other than systemic reviews (SRs) [2,3,4]. PROSPERO, an international prospective register of SRs, was launched in February 2011 to reduce publication bias of SRs . Before PROSPERO, there was no specific international registration system for SRs; therefore, the issue of unpublished SRs could not be assessed directly. A questionnaire survey of SR researchers conducted in 2005 indicated the existence of unpublished SRs and the potential influence of lack of funding as a reason for non-publication . As of June 2017, PROSPERO contains over 23,000 entries. Here, we investigated the publication status of SR protocols registered in PROSPERO and assessed the relationship of financial support for these SRs with their publication.

The association between publication and the existence of funding or conflicts of interest (COIs) was investigated using multivariable logistic regression analysis. Adjusted variables were funding and COIs because a previous study indicated the potential influence of lack of funding as a reason for non-publication . We also conducted a post hoc analysis for the SRs registered in PROSPERO from September 2011 to February 2012 (posterior half period). Statistical analysis was performed using StataSE version 13 (StataCorp, College Station, TX).

We first identified published SRs with published or updated status in PROSPERO records. One of them was only conference abstract publication. For 211 SRs with non-published status in PROSPREO, We investigated dissemination of the result in the publicly available space or published in any journals searched by Google or Google Scholar

The minimum and maximum time to publication were 0.87 and 56 months. The median was 16.3 months. The average was 19.9 months. *The time (in months) from protocol registration to publication of SRs was defined as the number of days from registration to publication divided by 30. The date of publication was selected for each review according to the following hierarchy: (1) the acceptance date, if available; (2) the online publication date, if available; (3) the earlier of the journal publication date or the date of author-reported “published” status in PROSPERO; (4) the earlier of the journal publication date of a conference abstract or the documented date of public poster dissemination detected by searching Google or Google Scholar. If only the publication month was reported, the midpoint of that month (day 15) was set as the publication day

This is the first reported direct assessment of SRs remaining unpublished after protocol registration in PROSPERO and the relationship between publication and funding source. We found a considerable proportion (26%) of unpublished SRs even at more than 65 months after protocol registration. Tricco et al. reported that the publication of SRs may be affected by whether the results are informative . We did not investigate the potential influence of the strength of research findings on publication (i.e., the effect of the clinical significance of SR results) because we could only access the published results retrospectively; however, these unpublished SRs might potentially impacted by the direction or strength of their findings.

Decullier et al. reported that funding was the determining factor for project initiation in clinical research, but once the project was initiated, funding had no significant influence on study completion or publication . Conversely, we found that funding for SRs was related to their publication. One likely reason for these conflicting findings is the difference in funding sources between SRs and clinical trials: most SRs have non-profit funding , whereas clinical trials tend to have for-profit funding.

Non-profit funding for registered SRs may mitigate the issue of non-publication after protocol registration. Although we did not find a significant association between author-reported COI and publication, COIs reported by authors are actually a mixture of financial- and academic-related issues. Further research is needed regarding the association of financial conflicts of interest with SR publication.

The low proportion of SRs with registered protocols is a possible limitation of this study. Page et al. reported that only 16% of SRs published in 2014 have publicly available protocols . Our sample included only SRs with protocols, which may not be representative of all SRs. However, the impact on assessment of non-publication of high-quality SRs was probably minimal, because the quality of reporting of published SRs in 2014 was still low despite the fact that reporting guidelines recommend protocol registration [10, 11].

Qualitative research by Tricco et al. revealed that the main reasons reported for non-publication were lack of time, overly broad SR scope, and few studies eligible for SRs as well as rejection . Because data regarding these factors are unavailable in PROSPERO records, we could not adjust for them in our analysis, which is a limitation of this study.

In addition, while this survey covered only the 1st year after PROSPERO was launched, the proportion of protocol-registered SRs appearing in high-impact journals is increasing . Future research should extend the search period to several years after launch in order to more fully investigate the characteristics of SR publications.

HT, YT, and YK: study concept and design, analysis and interpretation of data and critical revision of the manuscript for important intellectual content. HT: acquisition of data and drafting of the manuscript. All authors read and approve the final manuscript.

The authors declare that they have no competing interests.

Availability of data and materials

The authors only used publicly available data. The datasets analyzed during the current study available from Open Science Framework (osf.io/qmh4e).

Ethics approval and consent to participate

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Additi