I thought it would be useful to explain exactly what is a randomized controlled trial (also called randomized clinical trial; RCT) and what are the different “levels of evidence” in science-based medicine research. For people who aren’t or haven’t before been involved in research, these terms can be confusing and the exact definitions and procedures can be unclear.
RCTs are the “gold standard” of medical research and there is some controversy about this – sometimes legitimate concern and sometimes due to a lack of understanding of the intent and scope of an RCT. So why do we use the RCT as the highest standard of evidence and why are other forms of evidence not as acceptable?
Here are some common terms used in the context of research and what they mean.
Random – This means that each person has an equal chance of being assigned to either the treatment group, the placebo group, or the control group. This helps reduce bias that would happen by putting, for example, healthier people in the treatment group which might give a false positive result.
Control – A control group lessens the effect of non-treatment variables such as researcher bias, environmental changes, and biological variation. In other worse, a control helps isolate the effect of the treatment by accounting for other variables. Controls can also be used to compare the new treatment with the current most common treatment. For example, if there is a certain drug to treat Condition X, but researchers are developing a new formulation, the control group would get the traditional drug and the treatment group would get the new drug. Sometimes designs with a control group will switch groups halfway through so that everyone gets the traditional treatment eventually, in case the new treatment doesn’t work (more on ethics below).
Placebo – A placebo group is a type of control group that accounts for the feelings of improvement participants would have simply by being in the study, seeing researchers, and being given anything – all of which can make people “feel better”. People can easily convince themselves that they feel an effect, even if they are given a sugar pill with no active ingredient. A placebo group allows the researchers to account for that effect in the group receiving real treatment. If there’s no difference, it’s likely that the treatment doesn’t work.
Anecdote – A.k.a.: hearsay. This is a personal report and doesn’t constitute good evidence because 1) people can be really bad at giving objective reports and 2) while an observation may be “true” it’s not necessarily an accurate picture of what happened. For example “my mother ate a lot of real butter all her life and never had a cholesterol problem, therefore butter isn’t bad for you”. While the account may be true in that the person did live a long time, it ignores key details (like her lipolipid count compared to the average person) and makes a generalization based on only one data point.
What is an RCT?
There are many kinds of RCT, but the best kind is the blinded, placebo-controlled RCT. These are important in drug trials to ensure that the drug is having the intended effect, and to accurately attribute side effects, independent of potential bias from the participants and/or the researchers.
If the participant knows they are getting a drug they are less likely to be objective in their report of its effects. If they think they have an equal chance of getting a drug or a placebo, they have less expectation of an effect and can be more objective. This allows researchers to better determine if the effect was real or perceived. However, people are still subject to the effects of just participating in a study and thinking they might have gotten the treatment. Enter – the placebo control.
It’s important to compare a new treatment to other groups (placebo, current treatment, etc.) rather than in isolation to make sure that the effect is objective. Avoiding a comparison means that any observed difference may be recorded as a treatment effect, when it might have been a placebo effect. Placebos account for the expectation of an effect and general participation in the clinical trial (i.e., you feel better because you’re doing anything).
This is why blinded RCTs are the highest level of evidence – they eliminate bias wherever possible. Whereas an anecdote relies on a personal experience that may not be accurate and can’t be independently verified, so it can’t be treated as generally true; an RCT relies on comparisons that eliminate bias as much as possible and give the most objective results.
It’s ironic, then, that much of the opposition to RCTs is lack of generalizability and failure to take the individual into account, when anecdotes are by definition not generalizable because they are based on one (or few) individual(s). Yet naturopaths, homeopaths, etc. use anecdotes as evidence for the general application of such treatments as acupuncture, homeopathy, etc.
Of course, though, any study is only worth its salt if it is a well-designed study and the researchers do appropriate statistical analyses of the results. The generalization of an RCT largely depends on its design, not the characteristics of RCTs in general. Also, some treatments can’t be blinded – for example, an invasive treatment (i.e., surgery) or a behaviour (i.e., therapy, exercise, diet, etc.). So there are some legitimate limitations to the RCT, but generally it’s the best tool we have, especially for drug treatments.
Levels of Evidence
In evidence-based medicine (EBM) it is helpful to separate studies based on their independence from bias in order to evaluate a consensus in the most objective way possible. Ideally (i.e., when done properly), the RCT is the most removed from bias as, when blinded, both the patients and the researchers are unaware of who is getting the real treatment and who is getting placebo. Even stronger evidence is a systematic review of RCTs which collects the findings of several different studies and looks at the overall trends.
In contrast, the conclusions of lower level evidence such as patient testimonials, case reports, and expert opinion are suspect because of the placebo effect, differing opinions among “experts, and observation bias. Although these kinds of studies are necessary and helpful first steps in most research, the results must carry to better-designed studies for replication of the findings in order to make sure that the results are objective.
EBM can be distinguished slightly from science-based medicine (SBM). People following EBM can sometimes not see the forest for the trees. EBM can be misused to promote certain ideals by focusing on the results of a single or a few studies, that may be poorly designed, despite a body of literature that does not support those findings (or directly opposes them).
SBM, on the other hand, looks at data in context of the literature and in terms of historical scientific importance – for example, how long have studies gone on this topic without finding promising results? Where EBM might recommend “further study“, SBM might recommend moving on to something more promising.
Some RCTs are not ethical because they would require a potentially effective treatment to be withheld from one of the groups. For example, although the HPV vaccine can be tested for safety in case-control trials, etc. during development, it’s unethical to treat one group of people with the vaccine and the other group with a placebo and then expose each to HPV. That would be an unacceptable risk to the participant’s health.
Another example is AIDS. When developing new drugs for AIDS, if the new drugs are demonstrably effective in preliminary case-control studies, it’s unethical to then systematically withhold or give that treatment to determine overall effectiveness. Researchers can look at the people who have gotten the treatment so far and compare to those didn’t get the treatment, but that is not random. However, as alluded to above, researchers can compare to a treatment that is already available, depending on the study design.
So in some cases, demanding a placebo-controlled RCT for final conclusions on a treatment demonstrates a misunderstanding of research ethics. Researchers can gather sufficient information with a number of well-designed non-RTC studies. If those studies consistently demonstrate that the treatment is effective, it is reasonable to proceed. If, however, those studies consistently show failure, it is not appropriate to continue promoting the treatment and eventually it becomes a waste of resources to continue studying the same treatments when nothing has changed between now and the other countless studies that have demonstrated no effect.
This illustrates another side of ethics – diverting research funds to apparent lost causes (such as homeopathy) when money could be spent on more necessary or promising treatments (such as treatments for cancer, AIDS, diabetes, heart disease, stroke, etc). Is this ethical?
Although there are some limitations to the RCT, it is the best tool we have to ensure objectivity. Objectivity is important so that people’s health isn’t at risk due to a researcher’s personal beliefs. Sometimes opposition of the RCT is based on those political or personal beliefs – such as when well-designed RCTs fail to support acupuncture or homeopathy. However, the RCT isn’t out to get alt-med – for example, RCTs looking at use of the Nettie pot recently showed effectiveness and appropriate safety for short-term use.
The RCT merely eliminates the bias that is present in anecdotal and/or unblinded, uncontrolled studies from well-meaning researchers that truly believe their treatments will work. When an entire body of well-designed research doesn’t demonstrate promising results, it’s time to move on. We can’t blame the RCT when the findings don’t fit our wishes, as the purpose of it is to eliminate the influence of those wishes.
Consensus isn’t decided by one RCT or one anecdote. Rather a body of literature contributes to the understanding of a particular treatment. It’s important to look at research studies in context and interpret their results with a clear understanding of the bias that may be present in a particular study.