The Journey of Research - Levels of Evidence

by Kim Hugel - part 2 in Kim's series of articles on Pharmacist-Initiated Research

Part 1 - can be found here.

Part 2 - can be found here.


What kind of information are we getting?

I think all of us were haunted by the “levels of evidence pyramid” in our pharmacy school days.  As you move up the pyramid, the quality of evidence increases. However, in practice, finding evidence that would qualify as being “ideal research” is difficult.  I know that, in pediatrics, we are usually not fortunate enough to have prospective, randomized, double blind studies for every clinical question so it becomes important to realize the differences in the types of studies that can be done.   For completeness sake, I’ll give a brief definition of each type of study.

(Photo: via Langara)

Systemic Reviews

Systematic reviews are summaries of evidence pertaining to a specific clinical question.  They collect results from case control studies, cohort studies, animal studies and randomized controlled studies to give an overall trend. Essentially, a meta-analysis is a type of systemic review that combines the results of all available trails as if they were one large study.  This increases sample size, which can be especially useful for disease states that are not common leading to many studies with small sample sizes.

Randomized Controlled Trials

Generally, randomized controlled trials are thought of as the gold standard of clinical trials. As the name implies, study subjects are randomly assigned to either receive an intervention or not (be a control).  The outcome of interest is then assessed in both the intervention and control groups.  These studies are designed to eliminate some sources of bias but they may not always be feasible and can be difficult and time consuming to complete.

Cohort Studies

Cohort and case control studies are both observational in nature, meaning subjects are not randomized to exposure or non-exposure.  I always mixed up these types of studies, so I've included some schematics trying to explain them.

Prospective Cohort Study Versus Retrospective Cohort Study:

In cohort studies, a group that has been exposed to an intervention of interest is compared to a group that has not been exposed to that intervention.  The two groups are then assessed for an outcome of interest.  Cohort studies can be prospective or retrospective in nature. In prospective studies, you identify patients with and without the risk factor of interest and follow them to see if they develop the outcome of interest in the future (i.e. 5 years down the road). In retrospective cohort studies, you look at your study population, identify those with and without the risk factor of interest and see if they have the outcome of interest presently.  For example, in a study with the hypothesis that smoking causes lung cancer, groups of smokers and non smokers would be identified.  If it was a prospective study, they would be followed for a period of time to see if they developed lung cancer.  If it was a retrospective study, there may be smokers and non smokers that had lung cancer up to that given point in time.  If there were an increased number of patients with lung cancer in the smokers group, it would be suggested that smoking caused lung cancer.

Case Control Studies:

Case control studies compare two groups of patients- one group of patients (the cases) have an outcome of interest while the other group of patients (the controls) do not have the outcome of interest.  The groups are compared to see if they have exposure to a suspected intervention (like an agent of harm).  In our smoking and lung cancer study, cases would be the subjects with lung cancer and controls would be patients without lung cancer.  There would be people that smoked in both groups but, if there were a larger proportion of patients that smoked in the group with lung cancer, it could be suggested, but not proven, that smoking causes lung cancer.

Case Reports and Case Series

Case reports and series are reports of a single patient or very small groups of patients that something interesting happens to.   There are no control groups and these generally tend to be descriptive in nature.  Although these can help identify areas that require further study and trends, they aren’t generally thought to be especially robust sources of information, due to the lack of control groups and difficulty in drawing cause and effect conclusions.

Editorials and Expert Opinion

As shown in the pyramid, these tend to be considered the bottom of the barrel in terms of robustness of information. Although expert opinion is generally well informed (these are, after all, experts in whatever field they are voicing their opinion about), these tend to be based on anecdotal evidence and are especially susceptible to bias from whoever is writing them.

Animal Studies

Although this type of study is not seen in the pyramid, I included it because it was especially relevant in our case.  Animal studies are necessary to be done prior to a drug going on to human clinical study and can give insight to possible interactions or adverse effects when a medication is used in humans.  However, it is important to realize that results from this type of study aren’t always relevant to clinical practice.  The studies are done in a different species, often at doses that are significantly higher than equivalent doses in practice and are done in stringent lab environments.

Our Research Topic

So in writing this blog entry, I realized that it was going to be ridiculously long if I continued with the appraisal part.  So instead, I’ll give you a summary of the information that was found to answer our PICO question and save the resultant appraisal for the next blog entry (I know, you are probably on the edge of your seat by now!)

There appeared to be case reports and animal studies that supported an interaction between methotrexate and beta-lactam antibiotics.  Patients ranged in age from 8-50 years of age and concurrent antibiotics included amoxicillin, piperacillin-tazobactam, ceftazadime, oxacillin and mezlocillin.  Methotrexate doses ranged from 50mg once weekly to single doses of 12g/m2 (so doses and ages of patients were similar to what we use and see in our population).  Effects of the concurrent administration ranged from elevated serum methotrexate levels to increased GI toxicity and even one reported death in a patient who experienced acute renal failure, aplastic anemia and mucositis following the administration of oxacillin with his methotrexate dose.

Animal studies in rabbits and monkeys showed that penicillin and piperacillin both decreased elimination of methotrexate while the cephalosporin ceftriaxone did not appear to affect eliminiation.

In our case, it was clear that there was room for more research in this area, as we were unable to find anything more than case reports and animal studies.  Stay tuned for next entry with tricks to appraisal of information!



Kim is a Hematology/Oncology/Transplant Pharmacist at the Alberta Children's Hospital in Calgary, AB. She is also a valuable member of the CAPhO Communications Committee.