RADx-UP Program Addresses Data Gaps in Underrepresented Communities

Guest post by Richard J. Hodes, MD, Director, National Institute on Aging, and Eliseo Pérez-Stable, MD, Director, National Institute on Minority Health and Health Disparities, NIH.

A few months into the COVID-19 pandemic, we shared how NIH was working to speed innovation in the development, commercialization, and implementation of technologies for COVID-19 through NIH’s Rapid Acceleration of Diagnostics (RADx) initiative.

Two years later, one of the RADx programs—RADx Underserved Populations (RADx-UP)—reflects on lessons learned that have broken the mold of standard research paradigms to address health disparities.

Use of Common Data Elements

RADx-UP has presented unique challenges in terms of data collection, privacy concerns, measurement standardization, principles of data-sharing, and the opportunity to reexamine community-engaged research. Establishment of Common Data Elements (CDEs)—standardized, precisely defined questions paired with a set of allowable responses used systematically across different sites, studies, or clinical trials to ensure that the whole is greater than the sum of its parts—are not commonly used in community-engaged research. Use of CDEs enables data harmonization, aggregation, and analysis of related data across study sites as well as the ability to investigate relationships among data in unrelated data sets. CDEs can also lend statistical power to analyses of data for small subpopulations typically underrepresented in research.

RADx-UP is a community-engaged research program that builds on years of developing partnerships between communities and scientists. RADx-UP has funded 127 research projects with sites in every state and six U.S. territories as well as a RADx-UP Coordination and Data Collection Center (CDCC). RADx-UP assesses the needs and barriers related to COVID-19 testing and increase access to COVID-19 testing in underserved and vulnerable populations experiencing the highest rates of disparities in morbidity and mortality.

The COVID-19 pandemic necessitated establishing RADx-UP and its associated CDEs with unprecedented speed relying heavily on data elements derived from those already defined in the NIH-based PhenX Toolkit and Disaster Research Response (DR2) resources. The short time frame for this process did not allow for as extensive collaboration and input from RADx-UP investigators and community partners that would have been ideal. Additionally, many researchers, especially community partners engaged in RADx-UP projects, were not familiar with CDE data collection practices. As a result, CDE questionnaires had to be modified as studies progressed to better suit the needs of the consortium and investigators new to CDE collection had to be familiarized with these processes quickly. NIH program officers, NIH RADx-UP and CDCC leadership and engagement impact teams (EITs)—staff liaisons provided by the CDCC that link RADx-UP research teams to testing, data, and community-engagement resources—helped research teams implement and adjust CDE collection, ensured alignment across consortium research teams, and assisted with other data-related issues that arose.

All RADx programs are required to collect a standardized set of CDEs, including sociodemographic, medical history, and health status elements with the intent to provide researchers rapid access to data for secondary research analyses in the RADx Data Hub, the central repository for RADx data. However, implementation of CDEs in the context of underserved communities in the rapidly evolving COVID-19 pandemic presented complex issues for consideration.

Some of these issues included data privacy, the risk of re-identification of underserved and undocumented populations, and data collection burden on participants as well as researchers. The privacy of health data is protected under federal law. The RADx-UP program instituted measures to ensure program participants’ data remain protected and de-identified using a token-based hashing algorithm methodology that allows researchers to share individual-level participant data without exposing personally identifiable information. To address data collection and respondent burden concerns, projects modified questions to allow some flexibility in expanding response options more appropriate to some underserved communities. The CDCC also developed COLECTIV, a digital interface for projects to directly enter data into the data repository and included gateway questions to relieve respondent burden.

Respect for Tribal Data Sovereignty

RADx-UP leadership and investigators recognized that additional considerations for tribal sovereignty, practices, and policies needed to be addressed for projects that include American Indian and Alaska Native (AI/AN) participants. Through consultations with the NIH Tribal Advisory Committee and the broader AI/AN community and meetings with an informal RADx-UP AI/AN project working group established by the CDCC, NIH realized that deposition of tribal data into the RADx Data Hub would not meet the cultural, governance, or sovereignty needs of AI/AN RADx research data. In response, NIH hopes to establish a RADx Tribal Data Repository (TDR) responsible for the collection, protection, and sharing of data collected in AI/AN communities with respect for the practices and policies of Tribal data sovereignty. Applications for the repository have been solicited and NIH hopes to make an award for the TDR sometime in FY23.

Rapid Data Sharing

One of the largest hurdles the RADx-UP program has faced is implementing rapid sharing of research data for secondary analyses and to inform decision-making and public health practices related to the COVID-19 pandemic. RADx-UP research teams are expected to share their data on a timely cadence before data collection ends. This is a far more stringent practice relative to the current standard NIH data-sharing policy that requires data to be shared at the time of acceptance for publication of the main findings from the final data set. NIH and CDCC staff have worked together with the RADx research community to highlight the importance of and compliance with rapid data-sharing. Within the first six months, a total of 69 Phase 1 projects began transmitting CDE data to the RADx-UP CDCC. The COVID-19 pandemic posed a tremendous challenge, and NIH responded by collaborating with vulnerable and underserved communities. This collaboration has opened an unprecedented opportunity to build on a now established foundation for future research to address gaps in understanding the broader social, cultural, and structural factors that influence disparities in morbidity and mortality from COVID-19 and other diseases. Data collection and sharing efforts of the RADx-UP initiative comprise a significant contribution. Collaboration among the NIH, research investigators, and communities impacted by COVID-19 has been the catalyst. To learn more about RADx-UP, please visit a recent journal article available on PubMed.


Dr. Hodes has served as NIA director since 1993, overseeing studies of the biological, clinical, behavioral, and social aspects of aging. He has devoted his tenure to the development of a strong, diverse, and balanced research program focused on the genetics and biology of aging, basic and clinical studies aimed at reducing disease and disability, and investigation of the behavioral and social aspects of aging. Ultimately, these efforts have one goal — improving the health and quality of life for older people and their families. As a leading researcher in the field of immunology, Dr. Hodes has published more than 250 peer-reviewed papers.

Dr. Pérez-Stable practiced primary care internal medicine for 37 years at the University of California, San Francisco before becoming the Director of NIMHD in 2015. His research interests have centered on improving the health of individuals from racial and ethnic minority communities through effective prevention interventions, understanding underlying causes of health disparities, and advancing patient-centered care for underserved populations. Recognized as a leader in Latino health care and disparities research, he spent 32 years leading research on smoking cessation and tobacco control in Latino populations in the United States and Latin America. Dr. Pérez-Stable has published more than 300 peer-reviewed papers.

Meet the NLM Investigators: Dr. Demner-Fushman Knows the Answers to Your Questions!

Meet my close colleague, Dr. Dina Demner-Fushman! This brilliant researcher is the face behind what many of you have already accessed on NLM’s websites. Many of you will agree with me when I say that having one PhD is extremely impressive–but would you believe she has TWO?! In addition to her master’s degree, Dr. Demner-Fushman has PhDs in immunology and computer science.

Dr. Demner-Fushman and her team use advanced artificial intelligence (AI), natural language processing, and data mining techniques to answer consumers’ questions about a variety of health topics. Did you know that it was Dr. Demner-Fushman’s research that led to the developmental stages of the indexing initiative that produced the current iteration of the MEDLINE resource? This work helps all of us navigate a plethora of NLM resources.

Check out the infographic below to learn more about the innovative, important research happening in Dr. Demner-Fushman’s lab.

Infographic titled: Biomedical Question Answering. The title area features a picture of Dr. Demner-Fushman along with her title and accreditations (MD, Phd): Investigator, Computational Health. The first column of the graphic explores her short and long-term goals  for her projects. The center column describes the processes she uses to achieve these goals, and the last column depicts a simple graphic illustrating a Q and A service.

What makes your team unique? Tell us more about the people working in your lab.   

It is a diverse, multicultural team. Some were even born after I got my first IT job checking computers at Hunter College for Y2K compliance. The team is united by the task of enabling computers to understand health-related information needs and the socioeconomic and professional status of people who come to NLM seeking information. It is a group of exceptionally dedicated and talented people. Our diverse backgrounds make us see all possible aspects of addressing the informational and emotional needs of our users. 

What is your advice for young scientists or people interested in pursuing a career in research?  

  • Be proactive: Seek information and take advantage of training opportunities.  
  • Be brave: Admit you don’t know or don’t understand something. Most people will try to help.  
  • Be bold: Reach out to people who you would like to work with or to discuss your ideas.  
  • Be honest.  
  • Be patient: Research implies working hard, sometimes without immediate results. Even if research is your passion and fun, sometimes you have to do things that you might not enjoy or you might fear but still have to do, like giving talks or writing paper.

What do you enjoy about working at NLM?  

The community of dedicated people across all divisions, the mission, and the intellectual freedom.  

Where are you planning to travel to this year?  

I was just in Dublin, Ireland, in May for the 60th meeting of the Association for Computational Linguistics and co-chaired the BioNLP workshop for the 15th time. I loved Dublin when I visited shortly before the pandemic. I enjoyed revisiting a place I loved and discovering new things to love.

What are you reading right now?  

In the Garden of Beasts by Erik Larson. It provides an amazing view of pre-World War II Germany and political relations. I hope some lessons have been learned! 

You’ve read her words, now hear them for yourself. Follow our NLM YouTube page for more exciting content from the NLM staff that make it all possible. If you’d like to learn more about our Intramural Research Program (IRP), view job opportunities, and explore research highlights, I invite you to explore our recently redesigned NLM IRP webpage.

Transcript [Demner-Fushman]*: When people need information, what they really like is to ask a question and get a really good comprehensive answer, and to also know that the answer is true and correct.

When I started my independent clinician career, I had lots of questions, but I was sometimes not even sure if I was getting the right answer. “Question answering” is this system to understand the question, what the question is about, and why it is asked. When the answer is found, it’s usually not a single answer: It’s parts of the answer in different places. It’s multiple answers. So, all of that then needs to be condensed into one comprehensive answer with evidence of where the answer came from. So that’s the focus of my research.

On the surface, very similar questions asked by clinicians and by the public should be answered very differently. Different deep-learning systems are needed to find the answers to the same question asked by two different people.

The long-term goal is one entry point to all the NLM resources. It doesn’t matter who the person is and how they ask their question or look for information. We should be able to recognize what the person needs and provide it. There is no one—other than NLM—who is specifically dedicated to biomedical information retrieval and biomedical question answering. Although it seems industry is doing that kind of research as well, it is not their main focus, whereas we keep people focused on what really matters for health and advancing medicine.

*Transcript edited for clarity

Is Age Really Just a Number?

Last week I turned 69! Can you believe that??? This is so amazing to me—how could I be THAT OLD?? Two years ago (when I was just 67!), I shared that…

In midlife, I think I’m where I’m supposed to be, because I feel like I’m 39, think I look like I’m 49, believe I have a career worthy of someone who’s 59, and am approaching the wisdom of someone who’s 69.

So now that I am 69, I still believe all those things are true—particularly the wisdom part. I am wiser about the speed of change, the value of tempering my vision with a dose of realism, and the importance of understanding people clearly. I still feel youthful, look pretty good for a woman my age, and remain proud of my career.

But suppose I want to pick the number that really represents my age. Age is a very important descriptor of patients and research participants. Across all types of clinical research, one of the most common variables collected is a participant’s age. Age is an important indicator of many things human, from physical capabilities that determine their likely response to a treatment, to potential behavioral or mental health challenges. Knowing participants’ ages helps guide the interpretation of research results, allowing scientists and clinicians to determine the relevance of those results to specific groups of people or to better understand the clinical manifestation of a disease. And knowing the age of a participant provides evidence that our NIH studies appropriately engage people across their lifespan.

You might be surprised to know that there are many ways to represent age. For most of us, age is estimated by counting the number of years since our birth. However, for babies, it may be more important to know the number of days, weeks, or months since birth. Some studies compute age as the difference between the date of birth and the date that the data are collected. In fact, in the PhenX Toolkit, a web-based catalogue of expert-provided recommended measurement protocols, there are almost 200 different ways to measure age in a research study. Sometimes information about age is acquired through self-report of the participant, and other times the information is obtained from some existing document like a patient’s clinical record. The PhenX Toolkit is an enumeration of a wide range of measurement approaches and allows for broad coverage in a way that lets a researcher pick the measure that best represents the phenomena of interest to their study.

Over the past decade, NLM has supported the creation, identification, and distribution of Common Data Elements (CDEs). CDEs are specialized ways to measure concepts common to two or more research projects in a manner that is consistent across studies. Using a similar approach to measures similar concepts sounds like a no-brainer, right? It improves the rigor and reproducibility of research and allows data collected in different studies to be grouped together, adding power to the interpretation of research efforts. The COVID-19 pandemic illustrated the value of the common approaches to measuring research concepts by allowing us to track this deadly virus and its manifestations across time and people.

NLM established the NIH CDE Repository to serve as a one-stop location for research programs and for NIH Institutes and Centers to house CDEs and make them available to other researchers. Each record includes the definition of the variable as an indicator of the concept, a way to measure the variable (usually a question-and-answer pair with acceptable responses), and machine-readable codes where possible. Recently, the NIH CDE Repository began supporting an NIH governance process that indicates which of the proposed CDEs that have been received are described with sufficient rigor to be designated as NIH-endorsed. This endorsement helps potential users who are seeking good ways to measure complex concepts. NIH-endorsed CDEs support FAIR (findable, accessible, interoperable, and reusable) data sharing. Adherence to FAIR principles provides high-quality, “computation-ready” data with standardized vocabularies and readable metadata retrievable by identifiers that modernize the NIH data ecosystem. When data are collected consistently across studies using CDEs, it’s possible to integrate data from multiple studies, which can make it easier to get meaningful results. CDEs can also make it easier to reuse data for future research by improving the data quality.

So if I wanted to be “counted” according to the years-alive mode of assessing age, I guess I am 69. But if you really want to know something else, like how happy I am in my career or how I’m feeling, don’t be surprised if I give a different number!

Using Large Datasets to Improve Health Outcomes

Guest post by Lyn Hardy, PhD, RN, Program Officer, Division of Extramural Programs, National Library of Medicine, National Institutes of Health.

Before the advent of algorithms to determine the best way to treat and prevent heart disease, a health care provider looking for best practices for their patients may not have had the resources to find that best method. Today, health care decision-making for individuals and their health care providers is made easier by predictive and preventive models, which were developed with the goal of guiding the decision-making process. One example is the Patient Level Prediction of Clinical Outcomes and Cost-Effectiveness project led by Columbia University Health Sciences.

These models are created using computer algorithms (a set of rules for problem-solving) based on data science methods that analyze large amounts of data. While computers can analyze facts within the data, they rely on human programming to define what pieces of data or what data types are important to include in the analysis to create a valid algorithm and model. The results are translated into information that health care providers can use to understand patterns and provide methods for predicting and preventing illness. If a health care provider is looking for ways to prevent heart disease, an accurate model might describe methods—like exercise, diet, and mindfulness practices—that can achieve that goal.

Algorithms and models have benefited the world by using special data science methods and techniques to understand patterns that guide clinical decisions, but identifying data used in their development still requires practitioners to be conscious of the results. Research has shown that algorithms and models can be misleading or biased if they do not account for population differences like gender, race, and age. These biases, also known as algorithmic fairness, can adversely affect the health of underserved populations by not giving individuals and health care providers information specific to and that directly addresses their diversity. An example of potential algorithm bias is creating an algorithm to treat hypertension without including variated treatments for women or considering life-related stress or the environment.

Researchers are focusing on methods to create fair and equitable algorithms and models to provide all populations with the best and most appropriate health care decisions. Researchers in our NLM Extramural Programs analyze this data through NLM funding opportunities that foster scientific inquiry so we better understand algorithmic effects on minority and marginalized populations. Some of those funding opportunities include NLM Research Grants in Biomedical Informatics and Data Science (R01 Clinical Trial Optional) and the NIH Research Project Grant (Parent R01 Clinical Trial Not Allowed).

NLM is interested in state-of-the-art methods and approaches to address problems using large health data sets and tools to analyze them. Specific areas of interest include:

  • Developing and testing computational or statistical approaches to apply to large or merged health data sets containing human and non-human data, with a focus on understanding and characterizing the gaps, errors, biases, and other limitations in the data or inferences based on the data.
  • Exploring approaches to correct these biases or compensate for missing data, including introducing debiasing techniques and policies or using synthetic data.
  • Testing new statistical algorithms or other computational approaches to strengthen research designs using specific types of biomedical and social/behavioral data.
  • Generating metadata that adequately characterizes the data, including its provenance, intended use, and processes by which it was collected and verified.
  • Improving approaches for integrating, mining, and analyzing health data in a way that preserves that data’s confidentiality, accuracy, completeness, and overall security.

These funding opportunities encourage inquiry into algorithmic fairness to improve health care for all individuals, especially those who are underserved. By using new research models that account for diverse populations, we will be able to provide data that will support the best treatment outcomes for everyone.

Dr. Hardy’s work and expertise focus on using health informatics to improve public health and health care decision-making. Dr. Hardy has held positions as a researcher and academician and is active in national informatics organizations. She has written and edited books on informatics and health care.

The Next Normal: Supporting Biomedical Discovery, Clinical Practice, and Self-Care

As we start year three of the COVID-19 pandemic, it’s time for NLM to take stock of the parts of our past that will support the next normal and what we might need to change as we continue to fulfill our mission to acquire, collect, preserve, and disseminate biomedical literature to the world.

Today, I invite you to join me in considering the assumptions and presumptions we made about how scientists, clinicians, librarians and patients are using critical NLM resources and how we might need to update those assumptions to meet future needs. I will give you a hint… it’s not all bad—in fact, I find it quite exciting!

Let’s highlight some of our assumptions about how people are using our services, at least from my perspective. We anticipated the need for access to medical literature across the Network of the National Library of Medicine and created DOCLINE, an interlibrary loan request routing system that quickly and efficiently links participating libraries’ journal holdings. We also anticipated that we were preparing the literature and our genomic databases for humans to read and peruse. Now we’re finding that more than half of the accesses to NLM resources are generated and driven by computers through application programming interfaces. Even our MedlinePlus resource for patients now connects tailored electronic responses through MedlinePlus Connect to computer-generated queries originating in electronic health records.

Perhaps, and most importantly, we realize that while sometimes the information we present is actually read by a living person, other times the information we provide—for example, about clinical trials (ClinicalTrials.gov) or genotype and phenotype data (dbGaP)—is actually processed by computers! Increasingly, we provide direct access to the raw, machine-readable versions of our resources so those versions can be entered into specialized analysis programs, which allow natural-language processing programs to find studies with similar findings or machine-learning models to determine the similarities between two gene sequences. For example, NLM makes it possible for advocacy groups to download study information from all ClinicalTrials.gov records so anyone can use their own programs to point out trials that may be of interest to their constituents or to compare summaries of research results for related studies.

Machine learning and artificial intelligence have progressed to the point that they perform reasonably well in connecting similar articles—to this end, our LitCovid open-resource literature hub has served as an electronic companion to the human curation of coronavirus literature. NLM’s LitCovid is more efficient and has a sophisticated search function to create pathways that are more relevant and are more likely to curate articles that fulfill the needs of our users. Most importantly, innovations such as LitCovid help our users manage the vast and ever-growing collection of biomedical literature, now numbering more than 34 million citations in NLM’s PubMed, the most heavily used biomedical literature citation database.

Partnerships are a critical asset to bring biomedical knowledge into the hands (and eyes) of those who need it. Over the last decade, NLM moved toward a new model for managing citation data in PubMed. We released the PubMed Data Management system that allows publishers to quickly update or correct nearly all elements of their citations and that accelerates the delivery of correct and complete citation data to PubMed users.

As part of the MEDLINE 2022 Initiative, NLM transitioned to automated Medical Subject Headings (MeSH) indexing of MEDLINE citations in PubMed. Automated MeSH indexing significantly decreases the time for indexed citations to appear in PubMed without sacrificing the quality MEDLINE is known to provide. Our human indexers can focus their expertise on curation efforts to validate assigned MeSH terms, thereby continuously improving the automated indexing algorithm and enhancing discoverability of gene and chemical information in the future.

We’re already preparing for the next normal—what do you think it will be like?

I envision making our vast resources increasingly available to those who need them and forging stronger partnerships that improve users’ ability to acquire and understand knowledge. Imagine a service, designed and run by patients, that could pull and synthesize the latest information about a disease, recommendations for managing a clinical issue, or help a young investigator better pinpoint areas ripe for new interrogation! The next normal will make the best use of human judgment and creativity by selecting and organizing relevant data to create a story that forms the foundation of new inquiry or the basis of new clinical care. Come along and help us co-create the next normal!

%d bloggers like this: