When evaluating the pros and cons of adding a new drug therapy to a drug plan, it is best practice to consider all the available scientific evidence and be mindful of the quality of that evidence.
Clinical trials provide essential insight into the clinical value of a therapy and are the cornerstone for evidence-based medicine, which focuses on the integration of best available research evidence with clinical expertise and patient values. The challenge is that not all clinical trials are created equal. That makes it essential for decision-makers to understand the characteristics of good clinical trial design so they can make well-informed choices for members, employers and payers.
“Anybody who is in the seat of making decisions around adding or rejecting drugs needs to have a solid appreciation for what is the evidence being brought to the table [and] how do you judge value,” says Dr. Paul Oh, Medical Director, Cardiovascular Prevention and Rehabilitation Program, Toronto Rehabilitation Institute, University Health Network, who also serves on the WSIB Ontario Drug Advisory Committee. “It’s fundamental to decision-making.”
As part of that process, “you have to be able to sift the good from the bad in terms of trial design to know how much weight to give each trial in your decision,” says Mark Jackson, Consultant Pharmacist at TELUS Health.
What is the clinical trial design?
The gold standard for a clinical trial design is a randomized, double-blind, controlled study (RCT). Randomized means participants are divided by chance into treatment groups; it is a simple and powerful method of limiting differences between participants in each treatment group, beyond the treatments under investigation. Double-blind means neither the participant nor the people running the study know which group each participant is in. Controlled means the new therapy is being compared against either a placebo or an existing therapy.
Both Dr. Oh and Jackson prefer “head-to-head” comparisons with existing therapies. As Jackson points out, all a placebo-controlled trial can tell you is, “the drug is better than nothing.” He adds, “The regulators like to see placebo-controlled trials, but from a payer perspectiveyou’re trying to make your decisions based on, is it better than the alternatives?”
A systematic review represents the pinnacle of evidence, because it identifies all available evidence and then compares and synthesizes study results to provide a more complete picture of the different treatment effects. A systematic review can incorporate various study types, including head-to-head RCTs, non-comparative clinical trials and observational studies.
Jackson emphasizes the significance of a trial’s population. The sample size must be large enough to make statistical comparisons, but beyond that he recommends asking, “Is it representative of the population that you’re covering?”
Private payers need to know their population demographics and assess whether the clinical evidence is based on a sample of patients that aligns with that population.
What are the treatments?
Dr. Oh looks closely at all the attributes of a treatment. “How does it compare to the current standards: dose and duration and frequency and routes of administration?” he asks. Important, too, is whether the new therapy can be easily implemented in a real-world environment.
Dr. Oh adds, “I’m much more excited when the new therapy is being compared against a very good standard therapy – the most commonly used, the one with the best evidence.” In a head-to-head study, he emphasizes, two therapies are compared within a homogeneous population using the same outcomes measured in the same way. “That’s how we can judge whether it’s the new therapy or the old therapy that is really making the difference, as opposed to many of these other variables that come into play when we start thinking about indirect comparisons [with] different populations, different co-interventions, different geographic locations and different ways of assessing the outcomes,” Dr. Oh says.
Are the results clinically significant?
Dr. Oh likes to see a well-defined minimal clinically important difference – the smallest change a patient would consider to be important and that would therefore lead to a change in the patient’s treatment. He also looks for clearly articulated outcomes that are measurable with validated tools. And, within the results themselves, Dr. Oh looks for consistency across subgroups and pays close attention to any differences based on sex, age, geographic region or comorbidities.
What is the safety profile?
Safety is as important as a new therapy’s efficacy. “I’ll want to see that there is a very good safety analysis that includes the entire population that was entered into this study,” continues Dr. Oh. “I’m looking for serious adverse events as well as patterns in the less serious adverse events… I want to know which safety events led to a discontinuation of therapy as well.”
The safety analysis helps to answer the question “does the benefit outweigh the risk?” says Jackson. It’s also important to tease out whether a serious adverse event was attributable to the drug, and not to other factors, he says.
A customized approach for rare diseases
The appraisal process for a clinical trial of a therapy for a rare disease – a condition that affects fewer than one in 2,000 Canadians2 – must take into account the disease’s low prevalence.
A meaningful clinical effect is of course important as well. “You want to make sure that the drug is going to be making a significant impact on the person’s quality of life and how they’re able to function and stay at work,” Jackson says.
Check your conclusions
You’ve examined a clinical trial. You’ve drawn your conclusions. Now, it’s worth seeing what expert evaluators say. “You can do your own critical appraisal of trials, but it’s often helpful to look at how other people have critically appraised the trials as well,” says Jackson. Ultimately, the decision about whether to add a new therapy to a formulary depends on more than the evidence from clinical trials – but understanding how to critically appraise clinical trials is a necessary first step. ■