Keyboard controls: Space bar - toggle play/pause; Right and Left Arrow - seek the video forwards and back; Up and Down Arrow - increase and decrease the volume; M key - toggle mute/unmute; F key - toggle fullscreen off and on.
Kristina Elliott from the National Library of Medicine discusses the role of ClinicalTrials.gov in making the results of clinical research transparent to the public. This presentation was created for the 2018 Medical Library Association (MLA) Annual meeting.
Transcript
If you’re a librarian, you know how to use information as a tool. You know how to conduct a review, where to look, and how to judge the reliability of what you find. You also know the limitations, and you address them. But what about when those limitations are artificial? Do you know when information exists that you just don’t have access to? We’re going to talk about using transparency as a tool to find information that has never before been made public. Information about clinical research that informs all health decision making and affects the everyday health of all individuals. To do that, we’re going to understand how clinical trials are conducted.
Trials are scientific experiments. They’re run by groups of researchers overseen by a single principle investigator who understands the data and science of the study. Trials are run according to a protocol. That’s a prespecified documented research plan. This is exactly what data are going to be collected and why.
Clinical trials require careful data analysis. It means that researchers have to understand why they’re collecting what they’re collecting and how they’re going to analyze it in order to answer a specific question. The goal of a trial is to assess health outcomes through predetermined measures.
Clinical trials involve human beings, subjects who agree to give up their right to choose an option about their health care. By doing this they put themselves at risk, and they count on oversight by institutional review boards and ethics committees who judge the science and the ethics of a study.
Clinical trials involve different interventions assigned to those subjects based on that protocol. These can include drugs and devices, but also behavioral interventions, diet, sounds, images, anything that differs from standard care.
Clinical trials are big expensive enterprises. They’re funded by different organizations like the National Institutes of Health, industry, and many, many others.
So why conduct clinical trials? Well, the point is to obtain new and generalizable knowledge that will contribute to the evidence base and inform future medical decision making.
So how is this supposed to work? Well, those participants go through the trial run by researchers in that principle investigator. At the end of a trial results are published in journals and those journals are read by different people who make decisions about those results. These can include systematic reviews, metanalyses, even practice guidelines. All this goes into making informed health care decisions by a doctor. Those decisions improve the health outcomes of their patients.
Now this model rests on one assumption: That you can only receive evidence-based care if the results of trials are reflected in medical journals. However, this doesn’t always happen. One problem is the file drawer problem. This happens when some clinical trials get published and some just get locked away. This can happen for a lot of reasons. Sometimes it’s a business decision. Sometimes the researchers decide that the results of their trial aren’t very interesting. We know that this happens a lot. This is one study where researchers looked at trials registered in ClinicalTrials.gov to see the publication rate after completion. They found that at a median of 51 months, a third of NIH-funded trials remained unpublished.
Another problem is cherry picking. This happens when some trial results are published and some are just locked away. Again, this can happen for a variety of reasons. This is an example where researchers [indiscernible] in their publication. Here researchers were looking at Celecoxib against two comparators to find the rate of alter complications. They published the six-month data that showed Celecoxib was better than its comparators. However, the problem with that was in their pre-specified research plan they said that they would be looking at the 12-month data, which it turns out they had. At 12 months, Celecoxib was no better than its comparators. The problem was that nobody knew the researchers had the 12-month data. That included the journal publishers. This happens a lot. This is one study that compared results published in ClinicalTrials.gov against the published literature. They found that trial results were more completely reported at ClinicalTrials.gov than in the published article. So, let’s revisit that assumption. How about you can only receive evidence-based care if the results of trials are accurately and completely reflected in medical journals?
We know there are three key issues. One is that not all trials get published. Another is that publications don’t always include all prespecified outcome measures. Another is that sometimes unacknowledged changes are made to the trial protocol that affects the interpretation of those results. So, let’s revisit that assumption. How about saying you can only receive evidence-based care if the design and results of those trials are accurately and completely made public from start to finish? That means that you can look and see what those researchers originally designed the trial to do. You can see any changes made throughout the life of the study. That means that you can better understand what those results actually mean. That’s what transparency looks like. When ClinicalTrials.gov launched in 2000, that level of transparency had never before existed.
ClinicalTrials.gov launched because of a 1997 law—the Food and Drug Administration Modernization Act or FDAMA. That required the NIH to launch a website that would allow the public to find clinical trials. After ClinicalTrials.gov launched, one policy in 2005, the International Committee of Medical Journal Editors or ICMJE caused a significant increase in registration on the site. This policy required researchers to register their trial before enrollment of the first participants in order to publish in ICMJE journals. Two years later the Food and Drug Administration’s Amendment Act or FDAAA required certain FDA regulated drug and device trials to both register and report their results on ClinicalTrials.gov. In 2017 that law was codified as regulation. At the same time, NIH launched a policy that required all NIH funded clinical trials of all different interventions to register and report their results on ClinicalTrials.gov.
These policies vary in scope and the type of trial that’s affected. But they’ve all done a lot to increase registration on the ClinicalTrials.gov site. This is a graph showing the change in registration over time. You can see in 2005 that the ICMJE policy significantly bumped up the number of registrations. In 2007, FDAAA increased even more, and since then the site has grown. Today ClinicalTrials.gov is the world’s largest database of clinical trials.
So, if you go to ClinicalTrials.gov what exactly are you looking at when you access a record? So, a ClinicalTrials.gov record is composed of two parts—the registration section and the results section. At registration, information is submitted as a trial begins. This information summarizes the information from the trial protocol—that prespecified document that tells us exactly what’s being studied. It also includes information about recruitment—so the eligibility criteria and locations where the study is being conducted. This means that someone interested in the study knows what’s required.
Results are submitted after a trial is completed. ClinicalTrials.gov includes summary-level results not raw data or individual participant level data, and also not manuscripts from publications. These include baseline data and all prespecified outcome measures. Most of the information end result is similar to what would be published in a journal article.
Information on ClinicalTrials.gov is updated throughout the life of a study. Records need to be updated at least once a year and other changes that are important to know about quickly must be made more often. So how does this happen? So, a sponsor or principle investigator needs to submit the data to ClinicalTrials.gov. ClinicalTrials.gov staff review the records based on quality control criteria. They are looking for apparent errors, deficiencies, and inconsistencies in the record. They’re not looking for accuracy, safety, or scientific validity; that’s something that the sponsors and investigators are responsible for ensuring. Every time a record is updated, it goes through that quality control criteria review, and then changes are also archived with every record update. Changes are available in the ClinicalTrials.gov records for every record. That means that you can go back and see exactly what changed in a record, when, and you can compare different versions of a record. And records on ClinicalTrials.gov are never deleted or removed.
So, we saw that registration has increased on the site over time, but so has the results database since its launch in 2008. That means that ClinicalTrials.gov today has a significant breadth and depth of information. ClinicalTrials.gov doesn’t only include records from the United States; it includes studies from all over the world. ClinicalTrials.gov doesn’t only have drug and device studies, we have studies of all different types of interventions. So, you can’t make assumptions about what study is available on ClinicalTrials.gov and what isn’t. ClinicalTrials.gov also includes observational studies and even results of observational studies.
One way to think about how trials are done is that they produce a lot of different types of information. So that includes the protocol and statistical analysis plan, but there are also things like the individual participant dataset and other study documents that tell you exactly what happened throughout a study and individual results from that study. Until ClinicalTrials.gov existed there was no one place to go to find all this information. The ClinicalTrials.gov record includes a lot of information about the protocol, includes results; it even includes the full study protocol—its statistical analysis plan—and informed consent form. But ClinicalTrials.gov also links out to other sources of information like publications, participant datasets, funding details. There’s a lot of information about clinical trials that has always been hard to get to. They’ve required a complicated search and sometimes that information is jut impossible to find. ClinicalTrials.gov works as a platform that holds all this information in one place.
So ClinicalTrials.gov has some obvious uses like finding trials to volunteer for or tracking the progress of a trial over time and finding results at the end. But ClinicalTrials.gov also works to discover the whole clinical research landscape. It’s a place to compliment a literature review, a source of grey literature and unpublished trials that have previously been impossible to find. It supports planning a new protocol—you can see what questions have been asked before. For an IRB it’s important to see if the study that’s being proposed has already been conducted. You can even look at sources of funding and see if there are resource gaps or even overlaps.
These are some quotes from organizations that encourage users to go to ClinicalTrials.gov to find sources of information that can’t be found in a literature search. This means that ClinicalTrials.gov is a compliment to any literature review. If you’re only looking at the published literature, you’re going to be missing important information about a trial.
And then some questions can only be answered using ClinicalTrials.gov. For example, if you want to understand if your institution is following those laws and policies we looked at, you would need to look at ClinicalTrials.gov. You can compare any published trial against its ClinicalTrials.gov results record and see if they’re reporting all results.
You can also be an advocate for your patient community. You can see if people are researching treatments that are important or relevant to them. These are just a few examples of novel, interesting research that has been conducted using ClinicalTrials.gov data.
So, we know that ClinicalTrials.gov often has a more complete record of trial results than what’s available in the published literature. That’s not always a bad thing. So, this is an example of a group of researchers who specifically tell readers in their published article to go to ClinicalTrials.gov for all their secondary outcome measure data. Sometimes it’s just not going to fit in a published article, but this means that ClinicalTrials.gov is one place to go to find all that information. This is what it looked like in the journal article, so you would be able to go straight to that ClinicalTrials.gov link and see those relevant results. This is something we’re seeing more and more of and are going to continue to see in the future. This means that for librarians looking for all sources of information, you need to go to ClinicalTrials.gov to get it.
We hope that this has tested some of your assumptions, and maybe you’ve thought of some questions that can be answered using ClinicalTrials.gov. That means that we want to hear from you. You can contact us at register@clinical trials.gov. Good luck.
Last Reviewed: August 10, 2021