Low Reproducibility of Randomized Clinical Trials Methodology Related to Sampling: A Systematic Methodological Review
Document Type
Peer-Reviewed Article
Publication Date
2019
Abstract
Objectives: The reporting of sampling methods in Randomized Clinical Trials (RCTs) allows for research quality assessment, determination of sampling bias, and assures the presence of details necessary for reproducibility in future trials. The purpose of this study was to: (1) determine if sampling methodology was reproducible in RCTs related to musculoskeletal physical therapy (MSKPT) interventions to treat non-specific low back pain (NSLBP) and (2) establish if there was a relationship between sample reproducibility and established measures of research quality.
Methods: Data were collected through a systematic review by a professional librarian. The identified RCTs were assessed for methodological quality by two blinded individual reviewers. Data analysis was performed by a third, blinded researcher; additional comparisons were made based on Journal Impact Factor and PEDro score.
Results: Ninety-nine published peer-reviewed RCTs were identified that met inclusion criteria. Only 29% of the articles were judged to be reproducible based on the reported sampling methodology. There were meaningful correlations between two out of ten of the sampling reporting criteria and the judgement made if the sample was reported in significant detail to allow for replication. There was no relationship between sampling reporting criteria, Journal Impact Factors (JIFs), and Physiotherapy Evidence Database (PEDro) scores.
Discussion: The reporting of sampling methodology needs to be considered to ensure reproducibility and avoid sampling bias. Despite the proliferation of measures of research quality, the overall reporting quality of RCTs continues to be inadequate to allow widespread reproducibility of trials.
Level of Evidence: 1a
DOI
10.1080/10669817.2019.1587134
PMID
30935323
Recommended Citation
Riley, S. P., Swanson, B. T., Brismée, J. M., Sawyer, S. F., & Dyer, E. (2019).Low reproducibility of randomized clinical trials methodology related to sampling: A systematic methodological review. Journal of Manual & Manipulative Therapy, 27(5), 258-266. Doi:10.1080/10669817.2019.1587134
Publication
Journal of Manual & Manipulative Therapy
Volume
27
Issue
5
Publisher
Taylor & Francis
Pages
258-266
Comments
Published online March 7, 2019.
This article has a delayed release (embargo) and will be available in PMC on December 1, 2020.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6830291/