Low Reproducibility of Randomized Clinical Trials Methodology Related to Sampling: A Systematic Methodological Review

Document Type

Peer-Reviewed Article

Publication Date

2019

Abstract

Objectives: The reporting of sampling methods in Randomized Clinical Trials (RCTs) allows for research quality assessment, determination of sampling bias, and assures the presence of details necessary for reproducibility in future trials. The purpose of this study was to: (1) determine if sampling methodology was reproducible in RCTs related to musculoskeletal physical therapy (MSKPT) interventions to treat non-specific low back pain (NSLBP) and (2) establish if there was a relationship between sample reproducibility and established measures of research quality.

Methods: Data were collected through a systematic review by a professional librarian. The identified RCTs were assessed for methodological quality by two blinded individual reviewers. Data analysis was performed by a third, blinded researcher; additional comparisons were made based on Journal Impact Factor and PEDro score.

Results: Ninety-nine published peer-reviewed RCTs were identified that met inclusion criteria. Only 29% of the articles were judged to be reproducible based on the reported sampling methodology. There were meaningful correlations between two out of ten of the sampling reporting criteria and the judgement made if the sample was reported in significant detail to allow for replication. There was no relationship between sampling reporting criteria, Journal Impact Factors (JIFs), and Physiotherapy Evidence Database (PEDro) scores.

Discussion: The reporting of sampling methodology needs to be considered to ensure reproducibility and avoid sampling bias. Despite the proliferation of measures of research quality, the overall reporting quality of RCTs continues to be inadequate to allow widespread reproducibility of trials.

Level of Evidence: 1a

Comments

Published online March 7, 2019.

This article has a delayed release (embargo) and will be available in PMC on December 1, 2020.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6830291/

DOI

10.1080/10669817.2019.1587134

PMID

30935323

Publication

Journal of Manual & Manipulative Therapy

Volume

27

Issue

5

Publisher

Taylor & Francis

Pages

258-266

Link to Full Text


Share

COinS