Adoption of publication procedures to improve research integrity by alcohol and other drug journals

Background: The credibility crisis evident in many academic disciplines has led peer-reviewed journals to implement procedures designed to reduce use of flexible data analysis practices and selective reporting of results by authors. This exploratory study examined the adoption of six of these procedures by alcohol and other drug (AOD) journals. Methods: Thirty-four high-impact AOD journals were identified using the 2017 Clarivate Analytics’ Journal Citation Report. As an additional indicator of journal impact, the h-index of each journal was also identified using data from the Scimago Journal Ranking (SJR) webpage. The online instruction for authors of these journals were reviewed for any reference to the following six publication procedures: conflict of interest disclosure; reporting guidelines; clinical trial registration; registration of other study designs; data sharing; and registered reports. The webpages of the Center for Open Science and CONSORT were also reviewed for data pertaining to registered reports and reporting guidelines, respectively. Results: The average Scimago h-index of the 34 journals reviewed was 58.6 (range 12-167) and the average JCR impact factor 2.230 (range 0.683-5.953). The correlation between the two metrics was 0.67. The range of research integrity procedures adopted by the AOD journals was 0-5, with a mean of 2.62. Conflict of interest disclosure was required by all but one journal, whereas those procedures that required authors to pre-specify hypotheses and analytic methods prior to initiating data collection were only required by two. Five journals were listed as endorsers on the CONSORT webpage. These, and four other journals, recommended authors follow CONSORT guidelines when reporting randomized controlled trials. Eight of these nine journals also recommended use of at least one other set of guidelines when reporting results from research using non-randomized study designs (e.g., observational studies, quasi-experiments). Fifteen journals required pre-registration of clinical trials as a condition of publication, but only four required registration of other study designs. Twenty-five of the journals had a data sharing policy; in some cases, this was required by their publisher. None were judged to require data sharing as a condition of publication. Rather, in the majority of cases terms such as “requests” and “are encouraged” were used in describing the journal policy pertaining to this procedure. Conclusions: The publication procedures adopted by AOD journals are those of limited effectiveness due to their voluntary nature or lack of compliance and poor monitoring. More stringent requirements, such as pre-registration of hypotheses and registered reports, that lock researchers into specific hypotheses and analyses have not been widely adopted. Given problems with research reproducibility in disciplines that publish in AOD journals, this is an inadequate response by the field’s academic journals.