Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that enables research into pragmatic trials. It collects and distributes clean trial data, ratings and evaluations using PRECIS-2. This permits a variety of meta-epidemiological analyses that examine the effect of treatment across trials with different levels of pragmatism.
Background
Pragmatic trials are increasingly acknowledged as providing evidence from the real world for clinical decision-making. The term "pragmatic" however, is not used in a consistent manner and its definition and evaluation need further clarification. The purpose of pragmatic trials is to guide clinical practice and policy decisions, not to confirm the validity of a clinical or physiological hypothesis. A pragmatic trial should aim to be as close as possible to actual clinical practices that include recruiting participants, setting, design, delivery and execution of interventions, determining and analysis results, as well as primary analyses. This is a major distinction from explanatory trials (as described by Schwartz and Lellouch1) which are designed to provide more complete confirmation of a hypothesis.
Studies that are truly pragmatic should be careful not to blind patients or clinicians in order to cause bias in estimates of the effects of treatment. The pragmatic trials also include patients from different healthcare settings to ensure that the outcomes can be compared to the real world.
Additionally studies that are pragmatic should focus on outcomes that are vital to patients, such as quality of life or functional recovery. This is especially important for trials involving surgical procedures that are invasive or have potentially dangerous adverse events. The CRASH trial29, for example, focused on functional outcomes to compare a 2-page case-report with an electronic system for the monitoring of hospitalized patients with chronic heart failure. Similarly, the catheter trial28 utilized symptomatic catheter-associated urinary tract infections as the primary outcome.
In addition to these characteristics pragmatic trials should reduce the requirements for data collection and trial procedures to cut costs and time commitments. Additionally pragmatic trials should try to make their results as applicable to clinical practice as possible by making sure that their primary analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Read Even more that do not meet the criteria for pragmatism but contain features contrary to pragmatism have been published in journals of varying types and incorrectly labeled pragmatic. This can lead to misleading claims of pragmatism, and the term's use should be standardised. The development of a PRECIS-2 tool that provides an objective, standardized assessment of pragmatic features is a good start.
Methods
In a pragmatic research study, the goal is to inform policy or clinical decisions by showing how an intervention can be integrated into routine treatment in real-world contexts. Explanatory trials test hypotheses concerning the cause-effect relationship within idealised environments. In this way, pragmatic trials can have less internal validity than explanation studies and are more susceptible to biases in their design analysis, conduct, and design. Despite their limitations, pragmatic studies can be a valuable source of information to make decisions in the context of healthcare.
The PRECIS-2 tool evaluates an RCT on 9 domains, with scores ranging from 1 to 5 (very pragmatic). In this study, the recruitment, organisation, flexibility: delivery, flexible adherence and follow-up domains were awarded high scores, however the primary outcome and the method of missing data were not at the limit of practicality. This suggests that a trial can be designed with well-thought-out practical features, yet not damaging the quality.
However, it's difficult to determine the degree of pragmatism a trial is since pragmaticity is not a definite attribute; some aspects of a trial can be more pragmatic than others. The pragmatism of a trial can be affected by modifications to the protocol or the logistics during the trial. Koppenaal and colleagues discovered that 36% of 89 pragmatic studies were placebo-controlled, or conducted prior to licensing. The majority of them were single-center. They are not close to the norm and can only be called pragmatic if their sponsors agree that these trials are not blinded.
Another common aspect of pragmatic trials is that the researchers try to make their results more valuable by studying subgroups of the trial. However, this can lead to unbalanced results and lower statistical power, which increases the chance of not or misinterpreting the results of the primary outcome. In the case of the pragmatic studies that were included in this meta-analysis this was a major issue because the secondary outcomes were not adjusted to account for the differences in the baseline covariates.
In addition the pragmatic trials may present challenges in the collection and interpretation of safety data. This is due to the fact that adverse events are usually self-reported and are prone to reporting delays, inaccuracies or coding errors. It is important to improve the accuracy and quality of the outcomes in these trials.
Results
While the definition of pragmatism does not require that all clinical trials are 100% pragmatic, there are benefits of including pragmatic elements in trials. These include:
By incorporating routine patients, the results of the trial can be more quickly translated into clinical practice. However, pragmatic trials have disadvantages. The right type of heterogeneity, like, can help a study expand its findings to different settings or patients. However the wrong type of heterogeneity could reduce the sensitivity of an assay and thus lessen the power of a trial to detect even minor effects of treatment.
A number of studies have attempted to categorize pragmatic trials, with a variety of definitions and scoring systems. Schwartz and Lellouch1 developed a framework to distinguish between research studies that prove a clinical or physiological hypothesis, and pragmatic trials that help in the selection of appropriate treatments in the real-world clinical setting. The framework was comprised of nine domains, each scored on a scale ranging from 1-5, with 1 being more informative and 5 indicating more pragmatic. The domains included recruitment of intervention, setting up, delivery of intervention, flexible adhering to the program and primary analysis.
The original PRECIS tool3 was an adapted version of the PRECIS tool3 that was based on the same scale and domains. Koppenaal et. al10 devised an adaptation of this assessment, dubbed the Pragmascope which was more user-friendly to use for systematic reviews. They discovered that pragmatic reviews scored higher across all domains, however they scored lower in the primary analysis domain.
This difference in primary analysis domains can be due to the way in which most pragmatic trials analyze data. Certain explanatory trials however, do not. The overall score for systematic reviews that were pragmatic was lower when the domains of organization, flexible delivery, and following-up were combined.
It is important to remember that a pragmatic trial does not necessarily mean a poor quality trial, and indeed there is an increasing rate of clinical trials (as defined by MEDLINE search, but this is neither specific or sensitive) that use the term "pragmatic" in their title or abstract. The use of these words in abstracts and titles could indicate a greater understanding of the importance of pragmatism, but it isn't clear if this is manifested in the content of the articles.
Conclusions
In recent years, pragmatic trials are gaining popularity in research as the value of real world evidence is becoming increasingly acknowledged. They are randomized trials that evaluate real-world alternatives to experimental treatments in development. They are conducted with populations of patients more closely resembling those treated in regular care. This approach has the potential to overcome limitations of observational studies which include the biases that arise from relying on volunteers and the lack of accessibility and coding flexibility in national registry systems.
Pragmatic trials offer other advantages, such as the ability to leverage existing data sources, and a greater likelihood of detecting meaningful distinctions from traditional trials. However, pragmatic tests may have some limitations that limit their reliability and generalizability. The participation rates in certain trials could be lower than anticipated due to the healthy-volunteering effect, financial incentives or competition from other research studies. The necessity to recruit people in a timely fashion also reduces the size of the sample and impact of many pragmatic trials. Practical trials aren't always equipped with controls to ensure that observed variations aren't due to biases during the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs that were published between 2022 and 2022 that self-described as pragmatism. The PRECIS-2 tool was employed to assess pragmatism. It covers areas like eligibility criteria and flexibility in recruitment and adherence to intervention and follow-up. They discovered that 14 of the trials scored as highly or pragmatic sensible (i.e., scoring 5 or more) in one or more of these domains, and that the majority of them were single-center.
Studies that have high pragmatism scores tend to have broader criteria for eligibility than conventional RCTs. They also contain patients from a variety of hospitals. According to the authors, may make pragmatic trials more relevant and relevant to the daily clinical. However, they don't guarantee that a trial will be free of bias. In addition, the pragmatism that is present in trials is not a fixed attribute; a pragmatic trial that does not possess all the characteristics of a explanatory trial may yield reliable and relevant results.