Last week in an article was published on the Forbes website David Shaywitz asked the question: Will Real World Performance Replace RCTs As Healthcare’s Most Important Standard? The article first lays out the case that Randomized Controlled Trials (RCTs) are expensive to conduct and may not be reflective of how an intervention performs in real life due to how subjects are tracked during the trial. The article then makes the case that rather than RCTs, Real World Evidence (RWE) might allow us to bring newer interventions to market with appropriate approvals but with little cost.
RCTs, especially appropriately blinded ones, are the one of greatest tools designed by modern medicine. What are RCTs? They are studies, controlled experiments in fact, in which subjects are first chosen to be as alike to each other as possible by ensuring they all have the same condition for which a treatment is being tested with as few factors that can confound the results of the study. Then they are assigned to either a treatment arm of the study or to a control arm. The control arm intervention may be no treatment, treatment as usual or a sugar pill or sham treatment. Whether a subject ends up in treatment or control arm is left to chance (the ‘randomized’ part of name). The sugar pill or sham treatment ensures that the patient is ‘blinded’ to what she is receiving. This reduces the impact of the placebo effect, i.e. the improvement a patient experiences because she feels she is being helped. The researchers are ‘blinded’ by ensuring the people measuring the results don’t know which subjects are in treatment arm and which are in the control arm. This is to ensure that the person measuring the impact does noot attribute effects to a treatment because they know. This is the “double-blind” part in the double-blind RCTs — both the subject and the person measuring the effects of the treatment are ‘blind’ to which subject is getting the treatment vs. placebo. The whole purpose of the RCT is to ensure that different kinds of biases are reduced. Eventually if a treatment passes the RCT test, one can confidently say that it was not by chance alone.
Many a clinically intuitive, chemically similar or works-in-rats treatment has failed in an RCT and saved many humans from unnecessary suffering that comes from the burden of side-effects without adequate benefit. It is true that RCTs are expensive to conduct. It is also true that, because RCTs are extremely controlled experiments, their findings are sometimes not easily generalizable to the real world in which patients have multiple health conditions (not just the one being treated in the RCT) and not many mechanisms to ensure that they take their medicines. As a result, treatments often don’t work as well in the real world as do in RCT’s.
However, ditching the RCT means ditching the main tool we have to study interventions while minimizing bias. Shaywitz makes the case that in some instances, especially in case of digital health solutions that don’t have the risk of a variety of side-effects like medications, relying on RWE instead of RCT for approval might save cost. It’s true it would save the cost of the trials needed to get approval. But if the approval is received without minimizing bias in the way that an RCT does, we could end up tearing a hole in our collective pocket before concluding that these ‘solutions’ really don’t work.
Today we don’t have a systematic way of taking RWE into account after a treatment is approved and successfully launched. That’s what we need to fix without ditching the RCT.
