Background The aim of the INSPECT-SR project is to develop a tool to identify problematic RCTs in systematic reviews. In Stage 1 of the project, a list of potential trustworthiness checks was created. The checks on this list must be evaluated to determine which should be included in the INSPECT-SR tool. Methods We attempted to apply 72 trustworthiness checks to RCTs in 50 Cochrane Reviews. For each, we recorded whether the check was passed, failed or possibly failed, or whether it was not feasible to complete the check. Following application of the checks, we recorded whether we had concerns about the authenticity of each RCT. We repeated each meta-analysis after removing RCTs flagged by each check, and again after removing RCTs where we had concerns about authenticity, to estimate the impact of trustworthiness assessment. Trustworthiness assessments were compared to Risk of Bias and GRADE assessments in the reviews. Results 95 RCTs were assessed. Following application of the checks, assessors had some or serious concerns about the authenticity of 25% and 6% of the RCTs, respectively. Removing RCTs with either some or serious concerns resulted in 22% of meta-analyses having no remaining RCTs. However, many checks proved difficult to understand or implement, which may have led to unwarranted scepticism in some instances. Furthermore, we restricted assessment to meta-analyses with no more than 5 RCTs (54% contained only 1 RCT), which will distort the impact on results. No relationship was identified between trustworthiness assessment and Risk of Bias or GRADE. Conclusions This study supports the case for routine trustworthiness assessment in systematic reviews, as problematic studies do not appear to be flagged by Risk of Bias assessment. The study produced evidence on the feasibility and impact of trustworthiness checks. These results will be used, in conjunction with those from a subsequent Delphi process, to determine which checks should be included in the INSPECT-SR tool.

Objectives: No published methods for research integrity review include both statistical techniques applied to groups of randomized trials and individual assessment of papers. We propose a method based on practical experience of investigating data integrity across the collected papers of an author or author group.

Study Design and Setting: We report our approach to investigating the collected papers of an author or author group suspected of academic misconduct.

Results: In the investigation of the work of an author or author group, we recommend a systematic search for the work of the involved authors in PubMed, Google Scholar, and the Retraction Watch database, as well as a search of trial registries for unpublished clinical trials. Summary information from studies should be tabulated to assess consistency between study registration, execution, and publication. Each paper should be investigated for unfeasible features of the governance, methodology, execution, results, and reporting of the study. Pairwise comparison of baseline and outcome tables between papers may reveal data duplication or unfeasibly large differences between baseline characteristics in similar studies. Assessment of baseline characteristics from multiple randomized trials using Carlisle's method can determine whether the data are consistent with a properly executed randomization process, as can checking whether reported baseline characteristics follow expected patterns for random variables such as Benford's law. If serious concerns are raised, a more thorough investigation should be performed by journals, publishers, and institutions.

Conclusion: These methods provide a systematic and reproducible way to assess the collected work of an author or group of authors.

Plain Language Summary: It is increasingly accepted that papers reporting on clinical studies may contain fraudulent or falsified data, often multiple papers by a single author or author group. Based on our experience assessing the research integrity of collections of papers by one author or author group, we present an approach to these investigations that combines published statistical methods with pragmatic assessment of study feasibility. This will help journals and publishers better identify groups of potentially untrustworthy studies.

"Consumed by Creed" is a research article that suggests a link between obsessive-compulsive symptoms and support for political violence. Analyzing the article and the data set provided by the authors, we found errors in their methodology and procedures, accompanied by a high degree of researcher freedom that could have led to the finding of false-positive results. Given the important political and psychological implications of the claimed findings and the weakness of the evidence presented, we recommend substantial caution in interpreting the results reported in the article, and we urge researchers to drastically improve current research practices in the field of radicalization to ensure that policy decisions are based on valid and reliable evidence.

Academic publishing is both an indication of scientific contribution and a currency for career advancement. This dual role gives rise to a normative scientific conflict: Does the structural incentive to publish constitute a conflict of interest (COI) that ought to be disclosed? In this paper, we address this conflict through an action research approach, engaging collaboratively and reflexively to answer four related questions: (1) What evidence suggests that researchers face a (financial) COI when publishing? (2) What are the benefits and drawbacks of explicitly acknowledging that publications function as academic currency? (3) How should such conflicts be disclosed? (4) Do mechanisms such as pre-registration and registered reports resolve these concerns? This paper contends that while researchers are clearly incentivised to publish, this interest need not necessarily constitute a conflict or be explicitly disclosed. Treating this issue as a normative scientific conflict does reveal the need for a shift in how researchers understand and navigate the subjective, self-interested dimensions of their work. We propose four key responses: (1) integrating discussions of COIs and biases more extensively into undergraduate science education, (2) promoting greater reflexivity in everyday research practice (e.g., through reflexivity journals, peer-led audit groups, and the reintegration of discussions on the historicity and cultural nature of research into scientific publications), (3) critically investigating institutional incentives and journal policies, and (4) proactively adopting methodological safeguards such as pre-registration. By addressing this conflict through action research, we demonstrate how normative tensions in science can be made productive — supporting both critical reflection and structural improvement.

Background and Objective: Randomized controlled trials (RCTs) inform health-care decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesize all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project is developing a tool to identify problematic RCTs in systematic reviews of health care-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion.

Methods: We assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorized these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list.

Results: Extensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool.

Conclusion: A comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool.

During the last decade, there has been a substantial acceleration in the open science movement. Most people would probably hope to have seen signs of positive change in that time, yet it seems that the process of improving the practice of science is moving at a glacial pace.