The Next Normal: Supporting Biomedical Discovery, Clinical Practice, and Self-Care

As we start year three of the COVID-19 pandemic, it’s time for NLM to take stock of the parts of our past that will support the next normal and what we might need to change as we continue to fulfill our mission to acquire, collect, preserve, and disseminate biomedical literature to the world.

Today, I invite you to join me in considering the assumptions and presumptions we made about how scientists, clinicians, librarians and patients are using critical NLM resources and how we might need to update those assumptions to meet future needs. I will give you a hint… it’s not all bad—in fact, I find it quite exciting!

Let’s highlight some of our assumptions about how people are using our services, at least from my perspective. We anticipated the need for access to medical literature across the Network of the National Library of Medicine and created DOCLINE, an interlibrary loan request routing system that quickly and efficiently links participating libraries’ journal holdings. We also anticipated that we were preparing the literature and our genomic databases for humans to read and peruse. Now we’re finding that more than half of the accesses to NLM resources are generated and driven by computers through application programming interfaces. Even our MedlinePlus resource for patients now connects tailored electronic responses through MedlinePlus Connect to computer-generated queries originating in electronic health records.

Perhaps, and most importantly, we realize that while sometimes the information we present is actually read by a living person, other times the information we provide—for example, about clinical trials (ClinicalTrials.gov) or genotype and phenotype data (dbGaP)—is actually processed by computers! Increasingly, we provide direct access to the raw, machine-readable versions of our resources so those versions can be entered into specialized analysis programs, which allow natural-language processing programs to find studies with similar findings or machine-learning models to determine the similarities between two gene sequences. For example, NLM makes it possible for advocacy groups to download study information from all ClinicalTrials.gov records so anyone can use their own programs to point out trials that may be of interest to their constituents or to compare summaries of research results for related studies.

Machine learning and artificial intelligence have progressed to the point that they perform reasonably well in connecting similar articles—to this end, our LitCovid open-resource literature hub has served as an electronic companion to the human curation of coronavirus literature. NLM’s LitCovid is more efficient and has a sophisticated search function to create pathways that are more relevant and are more likely to curate articles that fulfill the needs of our users. Most importantly, innovations such as LitCovid help our users manage the vast and ever-growing collection of biomedical literature, now numbering more than 34 million citations in NLM’s PubMed, the most heavily used biomedical literature citation database.

Partnerships are a critical asset to bring biomedical knowledge into the hands (and eyes) of those who need it. Over the last decade, NLM moved toward a new model for managing citation data in PubMed. We released the PubMed Data Management system that allows publishers to quickly update or correct nearly all elements of their citations and that accelerates the delivery of correct and complete citation data to PubMed users.

As part of the MEDLINE 2022 Initiative, NLM transitioned to automated Medical Subject Headings (MeSH) indexing of MEDLINE citations in PubMed. Automated MeSH indexing significantly decreases the time for indexed citations to appear in PubMed without sacrificing the quality MEDLINE is known to provide. Our human indexers can focus their expertise on curation efforts to validate assigned MeSH terms, thereby continuously improving the automated indexing algorithm and enhancing discoverability of gene and chemical information in the future.

We’re already preparing for the next normal—what do you think it will be like?

I envision making our vast resources increasingly available to those who need them and forging stronger partnerships that improve users’ ability to acquire and understand knowledge. Imagine a service, designed and run by patients, that could pull and synthesize the latest information about a disease, recommendations for managing a clinical issue, or help a young investigator better pinpoint areas ripe for new interrogation! The next normal will make the best use of human judgment and creativity by selecting and organizing relevant data to create a story that forms the foundation of new inquiry or the basis of new clinical care. Come along and help us co-create the next normal!

ClinicalTrials.gov Modernization Effort: Beta Releases Now Available

Guest post by Anna M. Fine, PharmD, MS, acting director of ClinicalTrials.gov at the National Library of Medicine, National Institutes of Health.

Earlier this year, we provided an update on NLM’s efforts to modernize ClinicalTrials.gov, the world’s largest publicly accessible database of privately and publicly funded clinical trials. NLM released a request for information, hosted public webinars, and adopted a user-centered design approach intended to help ensure that modernization is responsive to user needs.

These activities, together with input from the NLM Board of Regents Public Service Working Group on ClinicalTrials.gov Modernization, supported the development of beta versions of a new ClinicalTrials.gov website and components of the information submission system, also known as the Protocol Registration and Results System (PRS). The beta releases feature a modern look and feel and provide updated technology to support users.

NLM is pleased to announce the public availability of these beta releases, which present the first of many new features to come.

ClinicalTrials.gov Website and PRS Goals and Implementation
The goals of the beta releases of the ClinicalTrials.gov website and components of the PRS are to:

  • Introduce users to the new technology platforms and evaluate their real-world performance,
  • Provide foundational features that will be expanded over time,
  • Provide PRS users with improved functionality to manage their record portfolios and workflows, and
  • Collect user input to inform future development activities.
Figure 1: The beta ClinicalTrials.gov homepage is now available by clicking on the banner link on Home – ClinicalTrials.gov or directly via https://beta.clinicaltrials.gov/.

The beta sites will function in parallel to the current systems to allow for usability testing and iterative improvements and to enable features and functions to be added without disrupting the current user experience. Users will gradually be transitioned to the new sites, or site components, as they are completed. Communication about changes and their related timelines will occur early and often to allow an easy transition to the new user experience.

Highlights of the Beta PRS Release
Study sponsors and investigators play a critical role in providing the information that makes ClinicalTrials.gov useful to data researchers, patients and their advocates, and the broader public. To better support the individuals who provide this information via the PRS, the modernization effort will make the submission of high-quality study information more intuitive, which in turn, will also benefit the end users of the data. These changes are intended to help PRS users meet existing reporting requirements; they do not alter those requirements.

The first beta PRS release will be available early 2022 and focuses on new workflow and portfolio management features such as customizable display and multiple download options to help data submitters save time. A recorded demonstration highlighting the features of the upcoming release is now available.

Highlights of the Beta ClinicalTrials.gov Release
The modernization effort has implemented a responsive site design that better supports mobile device users and added information written in plain language. The current release focuses on basic features, with more features to be added over time based on user feedback.

The first beta ClinicalTrials.gov release includes:

  • Redesigned home page and search feature,
  • Search results page with updated filter functionality,
  • New study record design with improved record navigation and study location information displayed using a web-based mapping application, and
  • Informational content presented in plain language such as Background and Learn About Studies webpages.

Again, this release is foundational, and more features will be added in the future. Importantly, we are developing advanced features to better meet the needs of data researchers and similar users, who use clinical trial information in their work.

Measuring Success, Next Steps, and Future Releases
User feedback received from website surveys, analytics, and continued engagement will help us assess the success of the first beta releases. Following validation of the technology’s performance and implementation of revisions made in response to user feedback, subsequent releases will address additional priorities identified by our user engagement efforts that are expected to help meet other patient and data researcher needs. Specific details of each component and release will be communicated to stakeholders as information becomes available.

For more information about activities and achievements during the modernization effort’s first two years, see the September 2021 NLM report summarizing the progress through August 2021, or view the October 2021 webinar for an update on the latest progress. To learn about related events and upcoming site enhancements and features, visit the ClinicalTrials.gov Modernization webpage. We appreciate your support as we seek to improve the features and functionality of this critical NLM resource. If you would like to receive email updates, I encourage you to sign up for Hot Off the PRS.

Anna M. Fine has worked for the federal government for 16 years. She leads the ClinicalTrials.gov program including the modernization initiative and research, policy, and outreach activities.

Progress Towards a Modernized ClinicalTrials.gov

Guest post by Rebecca Williams, PharmD, MPH, acting director of ClinicalTrials.gov at the National Library of Medicine, National Institutes of Health.

In 2019, NLM introduced a multi-year effort to modernize ClinicalTrials.gov, the world’s largest publicly accessible database of privately and publicly funded clinical trials. This effort was launched with a commitment to engage with and serve the millions of users who rely on this essential resource — with a focus on delivering an improved user experience on an updated platform that will accommodate growth and enhance efficiency.

In keeping with that promise, NLM has embarked on several stakeholder activities as part of the roadmap for modernization that we want to highlight in this post. We will also continue to share opportunities for involvement and invite you to join us for an upcoming webinar on February 18, 2021 at 3 pm ET to learn more about our modernization efforts.

Starting Out

Early in the process, our modernization team reached out to stakeholders through a request for information (RFI) to solicit input on topics around website functionality, information submission processes, and use of data standards. We received nearly 270 responses, which were summarized and discussed during a virtual interactive public meeting held in April 2020, and attended by nearly 400 participants. This robust feedback from the stakeholder community, along with input gathered from leaders and stakeholders across the NIH, provided the foundation to identify high-impact user-driven goals and set priorities. Currently, to advance the crucial next step of setting modernization goals and priorities, the team is working closely with NLM’s Board of Regents Public Service Working Group on ClinicalTrials.gov Modernization. Members in this group represent a range of stakeholder perspectives and provide input to ensure the continued integrity and utility of ClinicalTrials.gov.

Serving Stakeholders

Feedback received in response to last year’s RFI revealed several themes for modernizing ClinicalTrials.gov including ways to improve the management of search results, study records, and plain language information for the website, as well as enhancements to support structured and unstructured data, the quality control review process, and workflow management for information submission.

Three key external stakeholder groups were identified and consist of diverse individuals that share common interests and goals, but also have unique circumstances and needs. Feedback received from our engagement efforts, as well as the working group, reinforce the importance of being able to serve all stakeholders, including:

  1. Data providers, who are responsible for submitting, updating, and managing clinical trial registration and results information through the ClinicalTrials.gov submission system, also known as the Protocol Registration and Results System (PRS). The clinical trial information provided by this stakeholder group is listed on ClinicalTrials.gov for use by others.

2.Patients and their advocates, who find and use information about clinical trials for themselves or others through ClinicalTrials.gov. 

3. Data researchers, who use clinical trial information to study the clinical research enterprise, such as detecting trends in research and gaps in medical knowledge, identifying trials for use in systematic reviews and meta-analyses, and validating reported outcome measures and study design through ClinicalTrials.gov or its application programming interface, or API.

We aim to ensure that:

  1. Clinical trial information is current, complete, and reliable.
  2. All users can easily find and use information about clinical trials.
  3. Clinical trial information, resources, and tools provide value to the research ecosystem.

Looking Ahead

NLM is employing a user-centered design approach to modernization. This includes user feedback loops and prototypes to elicit user input on new features in an iterative manner. A new version of the ClinicalTrials.gov website will be released first, in parallel to the current website, to obtain maximum user input. Work on the website began with extensive planning and research and has since shifted into implementation that includes building the technical foundation in a cloud-based platform; designing key elements for the homepage, search results, and study records; and testing approaches for the continuous improvement of search results. We expect to release the new website for broad user testing by Fall 2021. Work related to the PRS is in the planning and research stage, with prototype development underway.

As ClinicalTrials.gov and the PRS undergo “construction,” be assured that one of the core principles guiding NLM’s ClinicalTrials.gov modernization effort is to minimize disruption to our users while continuing to deliver improved services to maximize the value of this resource to the people who rely on it.

We are also developing continuity approaches for the PRS, management portal, and website to ensure a smooth transition to updated features. We will continue to provide stakeholders with updates and a timeline of activities as they become available.

Stay Informed and Involved

NLM is committed to keeping you informed throughout this modernization effort. We invite you to join us for an upcoming webinar on February 18, 2021 at 3 pm ET where you will receive additional updates on our modernization efforts and learn about future opportunities for you and your communities to be involved. To register, please visit the ClinicalTrials.gov webinar registration page.

Rebecca Williams, PharmD, MPH, oversees the ClinicalTrials.gov program. Her research interests involve improving the quality of reporting of clinical research and evaluating the clinical research enterprise.

Celebrating 20 Years of ClinicalTrials.gov and Looking to the Future

Guest post by Rebecca Williams, PharmD, MPH, acting director of ClinicalTrials.gov at the National Library of Medicine, National Institutes of Health.

As ClinicalTrials.gov celebrates its 20th anniversary on February 29, 2020, we’re asking for your input on how it can best continue to serve your needs for many more years to come.

ClinicalTrials.gov is the world’s largest public clinical research registry and results database, giving patients, families, health care providers, researchers, and others easy access to information on clinical studies relating to a wide range of diseases and conditions. This online resource, which has more than 145,000 unique visitors every day, is operated by NLM and makes available information provided directly by the sponsors and investigators conducting the research.

NLM has launched an effort to modernize ClinicalTrials.gov to deliver an improved user experience on an updated platform that will accommodate growth and enhance efficiency. Creating a roadmap for modernization requires feedback from a wide array of stakeholders on how to continue serving, balancing, and prioritizing their varied information needs. These stakeholders include sponsors and investigators who submit clinical trial information to the site, academic institutions, nonprofit and advocacy organizations, government agencies, and the public, all of whom can access and use the information that ClinicalTrials.gov contains free of charge.

To obtain timely, detailed, and actionable input, we have issued a Request for Information (RFI) to solicit comments on the following topics: website functionality, information submission processes, and use of data standards.

Recognizing that ClinicalTrials.gov supports a network of stakeholders who contribute to, and rely on, clinical research, our aim is to understand how the system can better support this network and to identify opportunities for improving its compatibility with existing clinical trial management tools and processes. It is important to note that this RFI focuses on the functionality of ClinicalTrials.gov and is not intended to modify existing legal and policy requirements for clinical trial registration and results submission.

Over its 20-year history, ClinicalTrials.gov has helped shape the way in which clinical trial information is made transparent and discoverable to the public (see figure 1). In 2000, sponsors and investigators began submitting structured summaries of clinical trial protocols for the public to view. Over time, new policies and laws reinforced this practice, and ClinicalTrials.gov now contains over 320,000 study listings, with 56,000 studies currently seeking participants.

In 2008, ClinicalTrials.gov added its results database for sponsors and investigators to share summary results after trial completion. There are now over 40,000 results summaries posted on ClinicalTrials.gov, providing the public with timely access to information that may not be available in the peer-reviewed literature.

Figure 1: Total number of posted study records per year on ClinicalTrials.gov and timeline of major events from 1997 to 2019

Sharing information throughout the life cycle of a clinical trial (see figure 2) supports conduct of a landscape analysis prior to conducting new research and advances important public health goals, including supporting people who are looking to participate in clinical research, tracking the progress of clinical trials, allowing for the evaluation of the integrity of reported research, and providing more complete clinical trial information to help inform patient care. The modernization effort currently underway builds on the solid foundation established during the last 20 years.

Figure 2: Role of ClinicalTrials.gov in supporting use and sharing of information throughout the study life cycle

Share Your Feedback!

Responses to the RFI must be received by March 14, 2020. We expect a wide range of comments and are taking steps to manage and share the feedback. We will summarize the responses during a public meeting on April 30 on the main campus of the National Institutes of Health in Bethesda, Maryland, that will also be accessible by webcast. Details on the meeting will be available soon. In addition, we are engaging the NLM Board of Regents to provide input as we develop a roadmap for modernization, including establishing priorities and identifying the roles that various stakeholders might play in modernizing ClinicalTrials.gov.

Want to Learn More?

To learn more about the RFI and how to share your feedback, please join us for a webinar on January 22. We look forward to working with you to learn more about — and consider how to meet — your needs as we embark on this multiyear modernization effort.

Photo of Rebecca Williams, PharmD, MPH

Rebecca Williams, PharmD, MPH, oversees the ClinicalTrials.gov program. Her research interests involve improving the quality of reporting of clinical research and evaluating the clinical research enterprise.

Engaging Users to Support the Modernization of ClinicalTrials.gov

Guest post by Rebecca Williams, PharmD, MPH, acting director of ClinicalTrials.gov at the National Library of Medicine, National Institutes of Health.

ClinicalTrials.gov is the largest public clinical research registry and results database in the world – providing patients, health care providers, and researchers with information on more than 300,000 clinical studies of a wide range of diseases and conditions. More than 145,000 unique visitors use the public website daily to find and learn about clinical studies, resulting in an average of 215 million pageviews each month.

Recognizing the value of ClinicalTrials.gov to millions of users, the Board of Regents of the National Library of Medicine (NLM) described in the 2017-2027 strategic plan the importance of ensuring the long-term sustainability of this resource. NLM is committed to this goal and aims to modernize ClinicalTrials.gov to deliver a modern user experience on a flexible, extensible, scalable, and sustainable platform that will accommodate growth and enhance efficiency.

We are undertaking this effort to make ClinicalTrials.gov an even more valuable resource with a renewed commitment to engage with and serve the people who rely on it.

These users include the sponsors and investigators who submit clinical trial information for inclusion on the site through the submission portal. They also include patients, health care providers, and researchers who access listed information on ClinicalTrials.gov, whether directly or indirectly through other sites and services that use the ClinicalTrials.gov application programming interface.

Over the past several years, we have conducted testing with users and have already made some improvements in response to this feedback. With modernization, we will continue to support key functions identified by users of ClinicalTrials.gov while also seeking ways to make it an even more valuable resource.

To continue the modernization process, we are now seeking broader engagement with users to further help us determine how to evolve ClinicalTrials.gov. We are spending this summer looking inward by engaging our fellow National Institutes of Health Institutes and Centers to understand how ClinicalTrials.gov could better help in fulfilling NIH’s goals of clinical trial stewardship and transparency

This fall, we plan to expand our reach outward and are proposing to establish a working group of the NLM Board of Regents to focus on the modernization of ClinicalTrials.gov. This working group will provide a transparent forum for communicating and receiving input about efforts to enrich and modernize ClinicalTrials.gov. We want to ensure that we understand and consider changing needs while simultaneously maximizing the value of the growing amount of available information and preserving the integrity of ClinicalTrials.gov as a trusted resource.

We’ve already taken some steps to be more proactive in communicating with our users. We just launched “Hot Off the PRS!” (sign up to receive email announcements), a new informational bulletin for users of the ClinicalTrials.gov Protocol Registration and Results System (PRS). These updates provide timely announcements about new PRS features, relevant regulations (42 CFR Part 11) and policies, and information about other offerings such as the PRS Guided Tutorials (BETA), a new training resource with step-by-step instructions for submitting results information.

We’re excited about how greater user engagement will enrich and modernize ClinicalTrials.gov, improving its value for everyone throughout the clinical research lifecycle.

Please let us know what else we can do to make ClinicalTrials.gov the best it can be.

Photo of Rebecca Williams, PharmD, MPH

Rebecca Williams, PharmD, MPH, oversees the technical, scientific, policy, regulatory and outreach activities related to the operation of ClinicalTrials.gov. Her research interests relate to improving the quality of reporting of clinical research and evaluating the clinical research enterprise.

%d bloggers like this: