1. What are the primary factors to consider before delving into a Randomized Controlled Trial (RCT)?

Before you start, consider whether the RCT is likely to be worth reading! Any rheumatology RCT that is published in the New England Journal of Medicine, JAMA, or The Lancet is likely to be interesting, impactful, and well worth your time. There are typically only 1-2 of those published per month, so you should have no trouble keeping up with them. Smaller RCTs that are published in smaller journals, subset analyses from previously reported data, long-term extension studies, and any Phase 1 or Phase 2 trials are probably not worth your time. 

2. Explain the significance of randomization in the context of an RCT.

I do not mean to be dramatic here, but it is necessary: randomization is magical. I put randomization on par with antibiotics, clean drinking water, and vaccinations as the most important innovations in medicine. When you randomize people, you automatically fix many of the intractable problems that plague observational research.

The first and most important one is confounding. Observational studies that try to estimate the effect of medical interventions frequently fail to address differences in age, gender, comorbidities, or disease activity between groups who receive interventions. This is a problem! Randomization (given enough participants) automatically gives you two groups with roughly equal distributions of relevant variables and equilibrates the variables you cannot measure.

The second one is time-related biases. If I wanted to assess the impact of tocilizumab on patients with giant cell arteritis, I would estimate the risk of flares at 1 year among patients who received tocilizumab for one year. When should I start observing the control group? Because they never received tocilizumab, you wind up asking yourself “When does ‘never’ start?” It’s an unanswerable question and very difficult to fix outside of a randomized trial.

3. Define and differentiate between per-protocol and intention-to-treat analyses in the context of RCTs.

This is a difficult concept to understand, so let me explain by example. Let’s say we want to run an RCT of stem cell transplantation for systemic sclerosis. We would randomize people to receive stem cell transplantation or to receive standard of care. At the end of the trial, we have to decide what to do with patients who dropped out of the trial or crossed over between groups. In a “per protocol” analysis, we would analyze participants by whether or not they received stem cell transplantation. This seems reasonable, right?

Not so fast. Remember how I said that randomization is magical? A per-protocol analysis violates our magical randomization process because relevant confounders are allowed to sneak back into the ostensibly randomized trial. The participants who were randomized to stem cell transplantation and did not receive it probably had a good reason for that. Perhaps they were sicker? Or did they die during the conditioning regimen? Analysing patients by what actually happened feels reasonable, but in practice, it often biases the ultimate results of the study.

In an intention-to-treat analysis, patients are analysed by the group to which they were assigned. This often feels weird to people – why would we pretend patients received a treatment when they did not? In practice, though, it preserves the initial randomization and protects the scientific integrity of the analysis plan. I would also point out that it mirrors clinical practice better than you think. When we make decisions as clinicians, we do not know whether patients will actually receive our therapies. All we can do is recommend them or not recommend them. In a sense, clinical practice itself is “intention to treat.”

4. How would you distinguish between statistical significance and clinical significance in the context of RCT findings?

Statistical significance and clinical significance could not be more different! Statistical significance refers to a lot of the things I discussed above. Did we enrol enough patients and was the difference between the groups sufficient to reject our null hypothesis? This is a useful question; we want to know if our trial accomplished what it set out to do.

It is also not the important question! The important question is whether the intervention in question is likely to matter to my patients. Large RCTs – and often large observational studies – may be capable of finding a statistically significant result that is not clinically significant. Similarly, small RCTs may observe a clinically significant difference between groups, but because they enrolled too few patients, they may not have met our thresholds for statistical significance. I have created a handy graphic to explain!

In short, though, ask yourself (1) was this study “statistically significant” and then always ask yourself (2) would this difference in outcomes matter to my patients?