Traveling a Bridge2AI in a Quest for High-Quality, FAIR Data Sets

This blog was authored by NIH staff who serve on the Bridge to Artificial Intelligence (Bridge2AI) Working Group.

In April 2021, we introduced NIH Common Fund’s Bridge to Artificial Intelligence (Bridge2AI) program to tap the potential of artificial intelligence (AI) for revolutionizing biomedical discovery, increasing our understanding of human health, and improving the practice of medicine. In the past year, Bridge2AI researchers have been creating guidance and standards for the development of ethically sourced, state-of-the-art, AI-ready data sets to help solve some of the most pressing challenges in human health such as uncovering how genetic, behavioral, and environmental factors influence health and wellness. The program will also support the training required to enable the broader biomedical and behavioral research community to leverage AI technologies.

The NIH initiative will support diverse teams and tools to ensure that data sets adhere to FAIR (Findable, Accessible, Interoperable, and Reusable) data principles. Beyond ensuring compliance to FAIR principles, Bridge2AI will develop and disseminate best practices that promote a culture of diversity and continuous ethical inquiry into how data are collected.

The Bridge2AI program will support innovative data-generation projects nationwide to collect complex AI-ready data in four biomedical areas:

Clinical Care Informatics—Intensive care units treat patients with urgent medical conditions such as sepsis and cardiac arrest. This data generation project will collect, integrate, annotate, and share high-resolution physiological data from adult and pediatric critical care patients from 14 health systems that can then be used by AI technologies to identify approaches to improve recovery from acute illness.

Functional GenomicsWithin each cell in the human body lies a wealth of information about health, disease, and the impact of environmental factors. This project will generate richly detailed proteomic, genomic, and cellular imaging data to help predict disease mechanisms and associated gene pathways and networks for a variety of health outcomes.

Precision Public Health—The human voice is as unique as a fingerprint and has been found to contain acoustic signatures of human health and disease. This project will collect large-scale multimodal data sets containing voice, genomic, and clinical data, which AI technologies can use to help improve screening for and the diagnosis and treatment of a variety of developmental, neurological, and mental health conditions.

Return to Health—Much can be learned by uncovering how individuals move from a less healthy to a healthier state, a process called salutogenesis. This project will collect data from a diverse population with varying stages of type 2 diabetes to help improve our understanding of chronic disease progression and recovery. To learn more about Bridge2AI and salutogenesis, please view Bridging Our Way to Health Restoration by Helene M. Langevin, MD, director of the National Center for Complementary and Integrative Health.

To support these data generation projects, the Bridge2AI program includes a BRIDGE Center with a range of expertise to support interdisciplinary team science. The center will facilitate development of cross-cutting products such as standards harmonization, ethical AI best practices, and workforce development opportunities for the research community.

One of the goals of Bridge2AI is to foster a culture that will identify, assess, and address ethical issues as an integral part of creating AI-ready data sets. Ethical considerations include informed consent, data privacy, bias in data, and its impact on fairness and trustworthiness of AI applications, equity, and justice, and inclusion and transparency in design.

Every component of the Bridge2AI program includes a plan for incorporating diverse perspectives at every step. The BRIDGE Center will serve as a hub for supporting ethical and trustworthy AI development across Bridge2AI with the goal of providing tools, best practices, and resources to address cross-cutting biomedical challenges.

Learn more about Bridge2AI in the press release and video. Find the latest news by visiting the Bridge2AI website and following the @NIH_CommonFund on Twitter.

Top Row (left to right):
Patricia Flatley Brennan, RN, PhD, Director, National Library of Medicine
Michael F. Chiang, MD, Director, National Eye Institute
Eric D. Green, MD, PhD, Director, National Human Genome Research Institute

Bottom Row (left to right):
Helene M. Langevin, MD, Director, National Center for Complementary and Integrative Health
Bruce J. Tromberg, PhD, Director, National Institute of Biomedical Imaging and Bioengineering

RADx-UP Program Addresses Data Gaps in Underrepresented Communities

Guest post by Richard J. Hodes, MD, Director, National Institute on Aging, and Eliseo Pérez-Stable, MD, Director, National Institute on Minority Health and Health Disparities, NIH.

A few months into the COVID-19 pandemic, we shared how NIH was working to speed innovation in the development, commercialization, and implementation of technologies for COVID-19 through NIH’s Rapid Acceleration of Diagnostics (RADx) initiative.

Two years later, one of the RADx programs—RADx Underserved Populations (RADx-UP)—reflects on lessons learned that have broken the mold of standard research paradigms to address health disparities.

Use of Common Data Elements

RADx-UP has presented unique challenges in terms of data collection, privacy concerns, measurement standardization, principles of data-sharing, and the opportunity to reexamine community-engaged research. Establishment of Common Data Elements (CDEs)—standardized, precisely defined questions paired with a set of allowable responses used systematically across different sites, studies, or clinical trials to ensure that the whole is greater than the sum of its parts—are not commonly used in community-engaged research. Use of CDEs enables data harmonization, aggregation, and analysis of related data across study sites as well as the ability to investigate relationships among data in unrelated data sets. CDEs can also lend statistical power to analyses of data for small subpopulations typically underrepresented in research.

RADx-UP is a community-engaged research program that builds on years of developing partnerships between communities and scientists. RADx-UP has funded 127 research projects with sites in every state and six U.S. territories as well as a RADx-UP Coordination and Data Collection Center (CDCC). RADx-UP assesses the needs and barriers related to COVID-19 testing and increase access to COVID-19 testing in underserved and vulnerable populations experiencing the highest rates of disparities in morbidity and mortality.

The COVID-19 pandemic necessitated establishing RADx-UP and its associated CDEs with unprecedented speed relying heavily on data elements derived from those already defined in the NIH-based PhenX Toolkit and Disaster Research Response (DR2) resources. The short time frame for this process did not allow for as extensive collaboration and input from RADx-UP investigators and community partners that would have been ideal. Additionally, many researchers, especially community partners engaged in RADx-UP projects, were not familiar with CDE data collection practices. As a result, CDE questionnaires had to be modified as studies progressed to better suit the needs of the consortium and investigators new to CDE collection had to be familiarized with these processes quickly. NIH program officers, NIH RADx-UP and CDCC leadership and engagement impact teams (EITs)—staff liaisons provided by the CDCC that link RADx-UP research teams to testing, data, and community-engagement resources—helped research teams implement and adjust CDE collection, ensured alignment across consortium research teams, and assisted with other data-related issues that arose.

All RADx programs are required to collect a standardized set of CDEs, including sociodemographic, medical history, and health status elements with the intent to provide researchers rapid access to data for secondary research analyses in the RADx Data Hub, the central repository for RADx data. However, implementation of CDEs in the context of underserved communities in the rapidly evolving COVID-19 pandemic presented complex issues for consideration.

Some of these issues included data privacy, the risk of re-identification of underserved and undocumented populations, and data collection burden on participants as well as researchers. The privacy of health data is protected under federal law. The RADx-UP program instituted measures to ensure program participants’ data remain protected and de-identified using a token-based hashing algorithm methodology that allows researchers to share individual-level participant data without exposing personally identifiable information. To address data collection and respondent burden concerns, projects modified questions to allow some flexibility in expanding response options more appropriate to some underserved communities. The CDCC also developed COLECTIV, a digital interface for projects to directly enter data into the data repository and included gateway questions to relieve respondent burden.

Respect for Tribal Data Sovereignty

RADx-UP leadership and investigators recognized that additional considerations for tribal sovereignty, practices, and policies needed to be addressed for projects that include American Indian and Alaska Native (AI/AN) participants. Through consultations with the NIH Tribal Advisory Committee and the broader AI/AN community and meetings with an informal RADx-UP AI/AN project working group established by the CDCC, NIH realized that deposition of tribal data into the RADx Data Hub would not meet the cultural, governance, or sovereignty needs of AI/AN RADx research data. In response, NIH hopes to establish a RADx Tribal Data Repository (TDR) responsible for the collection, protection, and sharing of data collected in AI/AN communities with respect for the practices and policies of Tribal data sovereignty. Applications for the repository have been solicited and NIH hopes to make an award for the TDR sometime in FY23.

Rapid Data Sharing

One of the largest hurdles the RADx-UP program has faced is implementing rapid sharing of research data for secondary analyses and to inform decision-making and public health practices related to the COVID-19 pandemic. RADx-UP research teams are expected to share their data on a timely cadence before data collection ends. This is a far more stringent practice relative to the current standard NIH data-sharing policy that requires data to be shared at the time of acceptance for publication of the main findings from the final data set. NIH and CDCC staff have worked together with the RADx research community to highlight the importance of and compliance with rapid data-sharing. Within the first six months, a total of 69 Phase 1 projects began transmitting CDE data to the RADx-UP CDCC. The COVID-19 pandemic posed a tremendous challenge, and NIH responded by collaborating with vulnerable and underserved communities. This collaboration has opened an unprecedented opportunity to build on a now established foundation for future research to address gaps in understanding the broader social, cultural, and structural factors that influence disparities in morbidity and mortality from COVID-19 and other diseases. Data collection and sharing efforts of the RADx-UP initiative comprise a significant contribution. Collaboration among the NIH, research investigators, and communities impacted by COVID-19 has been the catalyst. To learn more about RADx-UP, please visit a recent journal article available on PubMed.


Dr. Hodes has served as NIA director since 1993, overseeing studies of the biological, clinical, behavioral, and social aspects of aging. He has devoted his tenure to the development of a strong, diverse, and balanced research program focused on the genetics and biology of aging, basic and clinical studies aimed at reducing disease and disability, and investigation of the behavioral and social aspects of aging. Ultimately, these efforts have one goal — improving the health and quality of life for older people and their families. As a leading researcher in the field of immunology, Dr. Hodes has published more than 250 peer-reviewed papers.

Dr. Pérez-Stable practiced primary care internal medicine for 37 years at the University of California, San Francisco before becoming the Director of NIMHD in 2015. His research interests have centered on improving the health of individuals from racial and ethnic minority communities through effective prevention interventions, understanding underlying causes of health disparities, and advancing patient-centered care for underserved populations. Recognized as a leader in Latino health care and disparities research, he spent 32 years leading research on smoking cessation and tobacco control in Latino populations in the United States and Latin America. Dr. Pérez-Stable has published more than 300 peer-reviewed papers.

The Importance of Listening—and Listening More—in Strategic Planning

I have spent much of the last year deeply involved in the creation of the very first NIH Digital Strategy Plan—an effort that requires listening across myriad NIH institutions and our stakeholders to glean what each finds important to our mission of seeking out and applying fundamental health care knowledge. I am proud to serve as co-chair of this effort with Andrea T. Norris, MBA, NIH’s Chief Information Officer and Director of NIH’s Center for Information Technology. Since October 2021, more than 40 staff members from across NIH have been working to develop this plan.

The purpose of the NIH Digital Strategy Plan is to identify 10 to 12 key capabilities needed to support NIH’s intramural research and extramural research program management, administration, and operations. Intramural research is NIH’s internal research program, and extramural research is research and training opportunities funded by NIH at universities, medical centers, and other institutions around the world. This helps create a vision for the future in which agency-wide commitments and institute-specific investments unite to form an information infrastructure vigorous enough to support the largest organization in the world that conducts and funds research.

In terms of information infrastructure, NIH is like other large, complex organizations where individual institute, center, office, and division investments complement enterprise-wide resources. We are creating a roadmap to make sure the organization has a robust, efficient, and secure information platform on which to conduct business. Developing the NIH Digital Strategy Plan is a lot like the parable of the blind men and the elephant, where different people can have different perceptions of the same thing. So how do we build a holistic strategic vision for the information technology capabilities needed for the future of NIH?

Actually, in much the same way as the Rajah in the parable gently nudges the six blind men towards truth: by instructing them to share their own vision on the journey of putting all the parts together!

Well, that’s what our committee has been doing. We’ve held over 25 listening and benchmarking sessions with stakeholders across NIH, as well as government and non-governmental enterprises that share a similar mission. We listened, and we asked questions, and then we listened some more.

Listening takes time, attention, and presence. As co-chairs of this effort, Ms. Norris and I have attended almost every session to convey our commitment, our interest, and our realization that the future of NIH depends strongly on the information technologies needed to support it. We’ve gleaned novel ideas and heard echoes of important themes. We’ve heard where there are areas of consensus and some frank differences. We’ve listened, learned, and discussed what we’ve heard. We’ve also listened for things that may not have been said.

In an enterprise this large, no single team can or should do all the listening. Our committee members also attended many of the sessions, and we used our conversations with them to verify or refine what we heard. Committee members also reached out to stakeholders around the country to hear how they viewed their future and what best practices they followed to make IT investments.

Listening works best when guided by questions, verifying what is being heard, and elaborating when needed. Listening is NOT list-making nor a process of determining which themes are raised more often or which ideas gain resonance. Listening is engagement, deepening one’s understanding of a complex environment by reaching deeply within and without.

In a few months, I’ll report on the results of our efforts to build the inaugural strategic plan for digital strategy at NIH. For now, I’ll continue to listen.

Using Large Datasets to Improve Health Outcomes

Guest post by Lyn Hardy, PhD, RN, Program Officer, Division of Extramural Programs, National Library of Medicine, National Institutes of Health.

Before the advent of algorithms to determine the best way to treat and prevent heart disease, a health care provider looking for best practices for their patients may not have had the resources to find that best method. Today, health care decision-making for individuals and their health care providers is made easier by predictive and preventive models, which were developed with the goal of guiding the decision-making process. One example is the Patient Level Prediction of Clinical Outcomes and Cost-Effectiveness project led by Columbia University Health Sciences.

These models are created using computer algorithms (a set of rules for problem-solving) based on data science methods that analyze large amounts of data. While computers can analyze facts within the data, they rely on human programming to define what pieces of data or what data types are important to include in the analysis to create a valid algorithm and model. The results are translated into information that health care providers can use to understand patterns and provide methods for predicting and preventing illness. If a health care provider is looking for ways to prevent heart disease, an accurate model might describe methods—like exercise, diet, and mindfulness practices—that can achieve that goal.

Algorithms and models have benefited the world by using special data science methods and techniques to understand patterns that guide clinical decisions, but identifying data used in their development still requires practitioners to be conscious of the results. Research has shown that algorithms and models can be misleading or biased if they do not account for population differences like gender, race, and age. These biases, also known as algorithmic fairness, can adversely affect the health of underserved populations by not giving individuals and health care providers information specific to and that directly addresses their diversity. An example of potential algorithm bias is creating an algorithm to treat hypertension without including variated treatments for women or considering life-related stress or the environment.

Researchers are focusing on methods to create fair and equitable algorithms and models to provide all populations with the best and most appropriate health care decisions. Researchers in our NLM Extramural Programs analyze this data through NLM funding opportunities that foster scientific inquiry so we better understand algorithmic effects on minority and marginalized populations. Some of those funding opportunities include NLM Research Grants in Biomedical Informatics and Data Science (R01 Clinical Trial Optional) and the NIH Research Project Grant (Parent R01 Clinical Trial Not Allowed).

NLM is interested in state-of-the-art methods and approaches to address problems using large health data sets and tools to analyze them. Specific areas of interest include:

  • Developing and testing computational or statistical approaches to apply to large or merged health data sets containing human and non-human data, with a focus on understanding and characterizing the gaps, errors, biases, and other limitations in the data or inferences based on the data.
  • Exploring approaches to correct these biases or compensate for missing data, including introducing debiasing techniques and policies or using synthetic data.
  • Testing new statistical algorithms or other computational approaches to strengthen research designs using specific types of biomedical and social/behavioral data.
  • Generating metadata that adequately characterizes the data, including its provenance, intended use, and processes by which it was collected and verified.
  • Improving approaches for integrating, mining, and analyzing health data in a way that preserves that data’s confidentiality, accuracy, completeness, and overall security.

These funding opportunities encourage inquiry into algorithmic fairness to improve health care for all individuals, especially those who are underserved. By using new research models that account for diverse populations, we will be able to provide data that will support the best treatment outcomes for everyone.

Dr. Hardy’s work and expertise focus on using health informatics to improve public health and health care decision-making. Dr. Hardy has held positions as a researcher and academician and is active in national informatics organizations. She has written and edited books on informatics and health care.

The Next Normal: Supporting Biomedical Discovery, Clinical Practice, and Self-Care

As we start year three of the COVID-19 pandemic, it’s time for NLM to take stock of the parts of our past that will support the next normal and what we might need to change as we continue to fulfill our mission to acquire, collect, preserve, and disseminate biomedical literature to the world.

Today, I invite you to join me in considering the assumptions and presumptions we made about how scientists, clinicians, librarians and patients are using critical NLM resources and how we might need to update those assumptions to meet future needs. I will give you a hint… it’s not all bad—in fact, I find it quite exciting!

Let’s highlight some of our assumptions about how people are using our services, at least from my perspective. We anticipated the need for access to medical literature across the Network of the National Library of Medicine and created DOCLINE, an interlibrary loan request routing system that quickly and efficiently links participating libraries’ journal holdings. We also anticipated that we were preparing the literature and our genomic databases for humans to read and peruse. Now we’re finding that more than half of the accesses to NLM resources are generated and driven by computers through application programming interfaces. Even our MedlinePlus resource for patients now connects tailored electronic responses through MedlinePlus Connect to computer-generated queries originating in electronic health records.

Perhaps, and most importantly, we realize that while sometimes the information we present is actually read by a living person, other times the information we provide—for example, about clinical trials (ClinicalTrials.gov) or genotype and phenotype data (dbGaP)—is actually processed by computers! Increasingly, we provide direct access to the raw, machine-readable versions of our resources so those versions can be entered into specialized analysis programs, which allow natural-language processing programs to find studies with similar findings or machine-learning models to determine the similarities between two gene sequences. For example, NLM makes it possible for advocacy groups to download study information from all ClinicalTrials.gov records so anyone can use their own programs to point out trials that may be of interest to their constituents or to compare summaries of research results for related studies.

Machine learning and artificial intelligence have progressed to the point that they perform reasonably well in connecting similar articles—to this end, our LitCovid open-resource literature hub has served as an electronic companion to the human curation of coronavirus literature. NLM’s LitCovid is more efficient and has a sophisticated search function to create pathways that are more relevant and are more likely to curate articles that fulfill the needs of our users. Most importantly, innovations such as LitCovid help our users manage the vast and ever-growing collection of biomedical literature, now numbering more than 34 million citations in NLM’s PubMed, the most heavily used biomedical literature citation database.

Partnerships are a critical asset to bring biomedical knowledge into the hands (and eyes) of those who need it. Over the last decade, NLM moved toward a new model for managing citation data in PubMed. We released the PubMed Data Management system that allows publishers to quickly update or correct nearly all elements of their citations and that accelerates the delivery of correct and complete citation data to PubMed users.

As part of the MEDLINE 2022 Initiative, NLM transitioned to automated Medical Subject Headings (MeSH) indexing of MEDLINE citations in PubMed. Automated MeSH indexing significantly decreases the time for indexed citations to appear in PubMed without sacrificing the quality MEDLINE is known to provide. Our human indexers can focus their expertise on curation efforts to validate assigned MeSH terms, thereby continuously improving the automated indexing algorithm and enhancing discoverability of gene and chemical information in the future.

We’re already preparing for the next normal—what do you think it will be like?

I envision making our vast resources increasingly available to those who need them and forging stronger partnerships that improve users’ ability to acquire and understand knowledge. Imagine a service, designed and run by patients, that could pull and synthesize the latest information about a disease, recommendations for managing a clinical issue, or help a young investigator better pinpoint areas ripe for new interrogation! The next normal will make the best use of human judgment and creativity by selecting and organizing relevant data to create a story that forms the foundation of new inquiry or the basis of new clinical care. Come along and help us co-create the next normal!

%d bloggers like this: