Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial, open data platform and infrastructure that facilitates research on pragmatic trials. It collects and distributes cleaned trial data, ratings and evaluations using PRECIS-2. This allows for diverse meta-epidemiological analyses to compare treatment effect estimates across trials of different levels of pragmatism.
Background
Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. However, the usage of the term "pragmatic" is not uniform and its definition and evaluation requires further clarification. The purpose of pragmatic trials is to inform clinical practice and policy decisions, rather than confirm an hypothesis that is based on a clinical or physiological basis. A pragmatic trial should aim to be as close as it is to the real-world clinical practice which include the recruiting participants, setting up, implementation and delivery of interventions, determining and analysis results, as well as primary analysis. This is a key difference from explanatory trials (as described by Schwartz and Lellouch1), which are intended to provide a more complete confirmation of a hypothesis.
Truly pragmatic trials should not blind participants or clinicians. This can lead to bias in the estimations of the effect of treatment. The trials that are pragmatic should also try to enroll patients from a variety of health care settings so that their results can be applied to the real world.
Finally, pragmatic trials must focus on outcomes that matter to patients, like quality of life and functional recovery. This is especially important for trials that involve the use of invasive procedures or could have dangerous adverse effects. The CRASH trial29 compared a 2 page report with an electronic monitoring system for patients in hospitals with chronic cardiac failure. The catheter trial28 on the other hand utilized symptomatic catheter-related urinary tract infections as its primary outcome.
In addition to these features pragmatic trials should also reduce the requirements for data collection and trial procedures to cut down on costs and time commitments. Additionally these trials should strive to make their results as relevant to actual clinical practice as is possible. This can be achieved by ensuring that their analysis is based on an intention-to treat method (as defined in CONSORT extensions).
Despite these requirements, a number of RCTs with features that defy the concept of pragmatism have been mislabeled as pragmatic and published in journals of all types. This can result in misleading claims of pragmaticity and the usage of the term must be standardized. The creation of a PRECIS-2 tool that can provide a standardized objective evaluation of the pragmatic characteristics is a good start.
Methods
In a pragmatic study, the goal is to inform policy or clinical decisions by demonstrating how an intervention could be integrated into routine care in real-world situations. Explanatory trials test hypotheses regarding the cause-effect relation within idealized settings. In this way, pragmatic trials could have less internal validity than explanation studies and are more susceptible to biases in their design analysis, conduct, and design. Despite these limitations, pragmatic trials may contribute valuable information to decision-making in healthcare.
The PRECIS-2 tool evaluates the degree of pragmatism in an RCT by assessing it across 9 domains that range from 1 (very explicative) to 5 (very pragmatic). In this study, the recruitment, organisation, flexibility: delivery, flexible adherence and follow-up domains received high scores, however, the primary outcome and the method for missing data fell below the practical limit. our website suggests that a trial can be designed with well-thought-out practical features, but without damaging the quality.
It is difficult to determine the degree of pragmatism that is present in a study because pragmatism is not a possess a specific attribute. Certain aspects of a study can be more pragmatic than others. Additionally, logistical or protocol changes during an experiment can alter its pragmatism score. Additionally, 36% of the 89 pragmatic trials discovered by Koppenaal and colleagues were placebo-controlled, or conducted prior to approval and a majority of them were single-center. Therefore, they aren't quite as typical and can only be called pragmatic in the event that their sponsors are supportive of the lack of blinding in these trials.
A common feature of pragmatic studies is that researchers try to make their findings more meaningful by studying subgroups within the trial. However, this can lead to unbalanced results and lower statistical power, thereby increasing the risk of either not detecting or incorrectly detecting differences in the primary outcome. This was the case in the meta-analysis of pragmatic trials as secondary outcomes were not adjusted for covariates' differences at the time of baseline.
In addition, pragmatic studies can pose difficulties in the gathering and interpretation of safety data. It is because adverse events are usually self-reported, and are prone to errors, delays or coding differences. Therefore, it is crucial to enhance the quality of outcomes ascertainment in these trials, ideally by using national registry databases instead of relying on participants to report adverse events on the trial's own database.
Results
While the definition of pragmatism may not require that all trials are 100 percent pragmatic, there are some advantages of including pragmatic elements in clinical trials. These include:
Enhancing sensitivity to issues in the real world, reducing study size and cost, and enabling the trial results to be more quickly implemented into clinical practice (by including patients from routine care). However, pragmatic trials can also have drawbacks. The right amount of heterogeneity, for example could help a study extend its findings to different patients or settings. However the wrong kind of heterogeneity can reduce the sensitivity of an assay, and therefore reduce a trial's power to detect small treatment effects.
Numerous studies have attempted to classify pragmatic trials using various definitions and scoring systems. Schwartz and Lellouch1 have developed a framework that can discern between explanation-based studies that confirm a physiological or clinical hypothesis and pragmatic studies that guide the selection of appropriate therapies in real world clinical practice. Their framework comprised nine domains, each scored on a scale ranging from 1 to 5, with 1 indicating more explanatory and 5 suggesting more pragmatic. The domains covered recruitment and setting up, the delivery of intervention, flex compliance and primary analysis.
The initial PRECIS tool3 featured similar domains and a scale of 1 to 5. Koppenaal et al10 devised an adaptation to this assessment, dubbed the Pragmascope which was more user-friendly to use in systematic reviews. They found that pragmatic reviews scored higher on average in most domains, but scored lower in the primary analysis domain.
The difference in the main analysis domain could be explained by the fact that the majority of pragmatic trials analyze their data in the intention to treat method while some explanation trials do not. The overall score was lower for pragmatic systematic reviews when the domains of organisation, flexible delivery and follow-up were combined.
It is crucial to keep in mind that a pragmatic study does not mean a low-quality trial. In fact, there are increasing numbers of clinical trials that employ the word 'pragmatic,' either in their title or abstract (as defined by MEDLINE, but that is neither precise nor sensitive). The use of these terms in abstracts and titles could indicate a greater understanding of the importance of pragmatism, however, it is not clear if this is manifested in the content of the articles.
Conclusions
In recent times, pragmatic trials are increasing in popularity in research because the importance of real-world evidence is increasingly recognized. They are clinical trials randomized that evaluate real-world alternatives to care instead of experimental treatments in development. They include patient populations that more closely mirror the ones who are treated in routine medical care, they utilize comparisons that are commonplace in practice (e.g., existing drugs) and depend on participants' self-reports of outcomes. This method is able to overcome the limitations of observational research, like the biases that are associated with the reliance on volunteers as well as the insufficient availability and codes that vary in national registers.
Pragmatic trials also have advantages, including the ability to draw on existing data sources, and a greater chance of detecting significant distinctions from traditional trials. However, these tests could be prone to limitations that undermine their validity and generalizability. For instance, participation rates in some trials could be lower than expected due to the healthy-volunteer influence and financial incentives or competition for participants from other research studies (e.g. industry trials). Practical trials are often limited by the need to enroll participants quickly. Practical trials aren't always equipped with controls to ensure that observed variations aren't due to biases in the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs that were published between 2022 and 2022 that self-described as pragmatism. They assessed pragmatism using the PRECIS-2 tool, which includes the domains eligibility criteria as well as recruitment, flexibility in adherence to interventions and follow-up. They discovered 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or more) in at least one of these domains.
Trials that have high pragmatism scores tend to have broader criteria for eligibility than traditional RCTs. They also have populations from various hospitals. The authors claim that these characteristics could make pragmatic trials more effective and applicable to everyday clinical practice, however they do not guarantee that a trial using a pragmatic approach is free of bias. The pragmatism is not a fixed characteristic the test that does not possess all the characteristics of an explicative study could still yield valuable and valid results.