The Importance of Listening—and Listening More—in Strategic Planning

I have spent much of the last year deeply involved in the creation of the very first NIH Digital Strategy Plan—an effort that requires listening across myriad NIH institutions and our stakeholders to glean what each finds important to our mission of seeking out and applying fundamental health care knowledge. I am proud to serve as co-chair of this effort with Andrea T. Norris, MBA, NIH’s Chief Information Officer and Director of NIH’s Center for Information Technology. Since October 2021, more than 40 staff members from across NIH have been working to develop this plan.

The purpose of the NIH Digital Strategy Plan is to identify 10 to 12 key capabilities needed to support NIH’s intramural research and extramural research program management, administration, and operations. Intramural research is NIH’s internal research program, and extramural research is research and training opportunities funded by NIH at universities, medical centers, and other institutions around the world. This helps create a vision for the future in which agency-wide commitments and institute-specific investments unite to form an information infrastructure vigorous enough to support the largest organization in the world that conducts and funds research.

In terms of information infrastructure, NIH is like other large, complex organizations where individual institute, center, office, and division investments complement enterprise-wide resources. We are creating a roadmap to make sure the organization has a robust, efficient, and secure information platform on which to conduct business. Developing the NIH Digital Strategy Plan is a lot like the parable of the blind men and the elephant, where different people can have different perceptions of the same thing. So how do we build a holistic strategic vision for the information technology capabilities needed for the future of NIH?

Actually, in much the same way as the Rajah in the parable gently nudges the six blind men towards truth: by instructing them to share their own vision on the journey of putting all the parts together!

Well, that’s what our committee has been doing. We’ve held over 25 listening and benchmarking sessions with stakeholders across NIH, as well as government and non-governmental enterprises that share a similar mission. We listened, and we asked questions, and then we listened some more.

Listening takes time, attention, and presence. As co-chairs of this effort, Ms. Norris and I have attended almost every session to convey our commitment, our interest, and our realization that the future of NIH depends strongly on the information technologies needed to support it. We’ve gleaned novel ideas and heard echoes of important themes. We’ve heard where there are areas of consensus and some frank differences. We’ve listened, learned, and discussed what we’ve heard. We’ve also listened for things that may not have been said.

In an enterprise this large, no single team can or should do all the listening. Our committee members also attended many of the sessions, and we used our conversations with them to verify or refine what we heard. Committee members also reached out to stakeholders around the country to hear how they viewed their future and what best practices they followed to make IT investments.

Listening works best when guided by questions, verifying what is being heard, and elaborating when needed. Listening is NOT list-making nor a process of determining which themes are raised more often or which ideas gain resonance. Listening is engagement, deepening one’s understanding of a complex environment by reaching deeply within and without.

In a few months, I’ll report on the results of our efforts to build the inaugural strategic plan for digital strategy at NIH. For now, I’ll continue to listen.

Meet the NLM Investigators: Dr. Demner-Fushman Knows the Answers to Your Questions!

Meet my close colleague, Dr. Dina Demner-Fushman! This brilliant researcher is the face behind what many of you have already accessed on NLM’s websites. Many of you will agree with me when I say that having one PhD is extremely impressive–but would you believe she has TWO?! In addition to her master’s degree, Dr. Demner-Fushman has PhDs in immunology and computer science.

Dr. Demner-Fushman and her team use advanced artificial intelligence (AI), natural language processing, and data mining techniques to answer consumers’ questions about a variety of health topics. Did you know that it was Dr. Demner-Fushman’s research that led to the developmental stages of the indexing initiative that produced the current iteration of the MEDLINE resource? This work helps all of us navigate a plethora of NLM resources.

Check out the infographic below to learn more about the innovative, important research happening in Dr. Demner-Fushman’s lab.

Infographic titled: Biomedical Question Answering. The title area features a picture of Dr. Demner-Fushman along with her title and accreditations (MD, Phd): Investigator, Computational Health. The first column of the graphic explores her short and long-term goals  for her projects. The center column describes the processes she uses to achieve these goals, and the last column depicts a simple graphic illustrating a Q and A service.

What makes your team unique? Tell us more about the people working in your lab.   

It is a diverse, multicultural team. Some were even born after I got my first IT job checking computers at Hunter College for Y2K compliance. The team is united by the task of enabling computers to understand health-related information needs and the socioeconomic and professional status of people who come to NLM seeking information. It is a group of exceptionally dedicated and talented people. Our diverse backgrounds make us see all possible aspects of addressing the informational and emotional needs of our users. 

What is your advice for young scientists or people interested in pursuing a career in research?  

  • Be proactive: Seek information and take advantage of training opportunities.  
  • Be brave: Admit you don’t know or don’t understand something. Most people will try to help.  
  • Be bold: Reach out to people who you would like to work with or to discuss your ideas.  
  • Be honest.  
  • Be patient: Research implies working hard, sometimes without immediate results. Even if research is your passion and fun, sometimes you have to do things that you might not enjoy or you might fear but still have to do, like giving talks or writing paper.

What do you enjoy about working at NLM?  

The community of dedicated people across all divisions, the mission, and the intellectual freedom.  

Where are you planning to travel to this year?  

I was just in Dublin, Ireland, in May for the 60th meeting of the Association for Computational Linguistics and co-chaired the BioNLP workshop for the 15th time. I loved Dublin when I visited shortly before the pandemic. I enjoyed revisiting a place I loved and discovering new things to love.

What are you reading right now?  

In the Garden of Beasts by Erik Larson. It provides an amazing view of pre-World War II Germany and political relations. I hope some lessons have been learned! 

You’ve read her words, now hear them for yourself. Follow our NLM YouTube page for more exciting content from the NLM staff that make it all possible. If you’d like to learn more about our Intramural Research Program (IRP), view job opportunities, and explore research highlights, I invite you to explore our recently redesigned NLM IRP webpage.

Transcript [Demner-Fushman]*: When people need information, what they really like is to ask a question and get a really good comprehensive answer, and to also know that the answer is true and correct.

When I started my independent clinician career, I had lots of questions, but I was sometimes not even sure if I was getting the right answer. “Question answering” is this system to understand the question, what the question is about, and why it is asked. When the answer is found, it’s usually not a single answer: It’s parts of the answer in different places. It’s multiple answers. So, all of that then needs to be condensed into one comprehensive answer with evidence of where the answer came from. So that’s the focus of my research.

On the surface, very similar questions asked by clinicians and by the public should be answered very differently. Different deep-learning systems are needed to find the answers to the same question asked by two different people.

The long-term goal is one entry point to all the NLM resources. It doesn’t matter who the person is and how they ask their question or look for information. We should be able to recognize what the person needs and provide it. There is no one—other than NLM—who is specifically dedicated to biomedical information retrieval and biomedical question answering. Although it seems industry is doing that kind of research as well, it is not their main focus, whereas we keep people focused on what really matters for health and advancing medicine.

*Transcript edited for clarity

Is Age Really Just a Number?

Last week I turned 69! Can you believe that??? This is so amazing to me—how could I be THAT OLD?? Two years ago (when I was just 67!), I shared that…

In midlife, I think I’m where I’m supposed to be, because I feel like I’m 39, think I look like I’m 49, believe I have a career worthy of someone who’s 59, and am approaching the wisdom of someone who’s 69.

So now that I am 69, I still believe all those things are true—particularly the wisdom part. I am wiser about the speed of change, the value of tempering my vision with a dose of realism, and the importance of understanding people clearly. I still feel youthful, look pretty good for a woman my age, and remain proud of my career.

But suppose I want to pick the number that really represents my age. Age is a very important descriptor of patients and research participants. Across all types of clinical research, one of the most common variables collected is a participant’s age. Age is an important indicator of many things human, from physical capabilities that determine their likely response to a treatment, to potential behavioral or mental health challenges. Knowing participants’ ages helps guide the interpretation of research results, allowing scientists and clinicians to determine the relevance of those results to specific groups of people or to better understand the clinical manifestation of a disease. And knowing the age of a participant provides evidence that our NIH studies appropriately engage people across their lifespan.

You might be surprised to know that there are many ways to represent age. For most of us, age is estimated by counting the number of years since our birth. However, for babies, it may be more important to know the number of days, weeks, or months since birth. Some studies compute age as the difference between the date of birth and the date that the data are collected. In fact, in the PhenX Toolkit, a web-based catalogue of expert-provided recommended measurement protocols, there are almost 200 different ways to measure age in a research study. Sometimes information about age is acquired through self-report of the participant, and other times the information is obtained from some existing document like a patient’s clinical record. The PhenX Toolkit is an enumeration of a wide range of measurement approaches and allows for broad coverage in a way that lets a researcher pick the measure that best represents the phenomena of interest to their study.

Over the past decade, NLM has supported the creation, identification, and distribution of Common Data Elements (CDEs). CDEs are specialized ways to measure concepts common to two or more research projects in a manner that is consistent across studies. Using a similar approach to measures similar concepts sounds like a no-brainer, right? It improves the rigor and reproducibility of research and allows data collected in different studies to be grouped together, adding power to the interpretation of research efforts. The COVID-19 pandemic illustrated the value of the common approaches to measuring research concepts by allowing us to track this deadly virus and its manifestations across time and people.

NLM established the NIH CDE Repository to serve as a one-stop location for research programs and for NIH Institutes and Centers to house CDEs and make them available to other researchers. Each record includes the definition of the variable as an indicator of the concept, a way to measure the variable (usually a question-and-answer pair with acceptable responses), and machine-readable codes where possible. Recently, the NIH CDE Repository began supporting an NIH governance process that indicates which of the proposed CDEs that have been received are described with sufficient rigor to be designated as NIH-endorsed. This endorsement helps potential users who are seeking good ways to measure complex concepts. NIH-endorsed CDEs support FAIR (findable, accessible, interoperable, and reusable) data sharing. Adherence to FAIR principles provides high-quality, “computation-ready” data with standardized vocabularies and readable metadata retrievable by identifiers that modernize the NIH data ecosystem. When data are collected consistently across studies using CDEs, it’s possible to integrate data from multiple studies, which can make it easier to get meaningful results. CDEs can also make it easier to reuse data for future research by improving the data quality.

So if I wanted to be “counted” according to the years-alive mode of assessing age, I guess I am 69. But if you really want to know something else, like how happy I am in my career or how I’m feeling, don’t be surprised if I give a different number!

Using Large Datasets to Improve Health Outcomes

Guest post by Lyn Hardy, PhD, RN, Program Officer, Division of Extramural Programs, National Library of Medicine, National Institutes of Health.

Before the advent of algorithms to determine the best way to treat and prevent heart disease, a health care provider looking for best practices for their patients may not have had the resources to find that best method. Today, health care decision-making for individuals and their health care providers is made easier by predictive and preventive models, which were developed with the goal of guiding the decision-making process. One example is the Patient Level Prediction of Clinical Outcomes and Cost-Effectiveness project led by Columbia University Health Sciences.

These models are created using computer algorithms (a set of rules for problem-solving) based on data science methods that analyze large amounts of data. While computers can analyze facts within the data, they rely on human programming to define what pieces of data or what data types are important to include in the analysis to create a valid algorithm and model. The results are translated into information that health care providers can use to understand patterns and provide methods for predicting and preventing illness. If a health care provider is looking for ways to prevent heart disease, an accurate model might describe methods—like exercise, diet, and mindfulness practices—that can achieve that goal.

Algorithms and models have benefited the world by using special data science methods and techniques to understand patterns that guide clinical decisions, but identifying data used in their development still requires practitioners to be conscious of the results. Research has shown that algorithms and models can be misleading or biased if they do not account for population differences like gender, race, and age. These biases, also known as algorithmic fairness, can adversely affect the health of underserved populations by not giving individuals and health care providers information specific to and that directly addresses their diversity. An example of potential algorithm bias is creating an algorithm to treat hypertension without including variated treatments for women or considering life-related stress or the environment.

Researchers are focusing on methods to create fair and equitable algorithms and models to provide all populations with the best and most appropriate health care decisions. Researchers in our NLM Extramural Programs analyze this data through NLM funding opportunities that foster scientific inquiry so we better understand algorithmic effects on minority and marginalized populations. Some of those funding opportunities include NLM Research Grants in Biomedical Informatics and Data Science (R01 Clinical Trial Optional) and the NIH Research Project Grant (Parent R01 Clinical Trial Not Allowed).

NLM is interested in state-of-the-art methods and approaches to address problems using large health data sets and tools to analyze them. Specific areas of interest include:

  • Developing and testing computational or statistical approaches to apply to large or merged health data sets containing human and non-human data, with a focus on understanding and characterizing the gaps, errors, biases, and other limitations in the data or inferences based on the data.
  • Exploring approaches to correct these biases or compensate for missing data, including introducing debiasing techniques and policies or using synthetic data.
  • Testing new statistical algorithms or other computational approaches to strengthen research designs using specific types of biomedical and social/behavioral data.
  • Generating metadata that adequately characterizes the data, including its provenance, intended use, and processes by which it was collected and verified.
  • Improving approaches for integrating, mining, and analyzing health data in a way that preserves that data’s confidentiality, accuracy, completeness, and overall security.

These funding opportunities encourage inquiry into algorithmic fairness to improve health care for all individuals, especially those who are underserved. By using new research models that account for diverse populations, we will be able to provide data that will support the best treatment outcomes for everyone.

Dr. Hardy’s work and expertise focus on using health informatics to improve public health and health care decision-making. Dr. Hardy has held positions as a researcher and academician and is active in national informatics organizations. She has written and edited books on informatics and health care.

How Being an ICU Nurse Prepared Me to be NLM Director

In mid-May, at their 2022 National Teaching Institute & Critical Care Exposition in Houston, Texas, I received a great honor from the American Association of Critical Care Nurses (AACN): the AACN Pioneering Spirit Award. I was delighted to receive this prestigious award, which recognizes significant contributions that influence progressive and critical care nursing worldwide and relate to AACN’s values of integrity, inclusion, transformation, leadership, and relationships. I was humbled to receive this award for my work during my tenure as NLM Director, and it’s in large part due to the work that so many NLM employees do every day.

This acknowledgement from AACN is deeply meaningful to me because critical care nursing has been a part of my professional identity for almost 50 years! In 1974, while I was still in nursing school, I was assigned to work as a nursing assistant in the critical care medical unit at Lankenau Medical Center outside Philadelphia. After graduating in 1975, I became part of the nursing team in the surgical intensive care unit (ICU) at the very same hospital.

These early experiences have touched every part of my career, including my role at NLM—the epicenter for biomedical informatics and computational health data science research and the largest biomedical library in the world.

Then: Learning from My Teachers and Colleagues

I learned from Kathy McCauley, cardiac-care nurse extraordinaire, about the importance of the scientific basis of nursing. Nurses’ deep knowledge of physiology, pharmacology, and anatomy enables the bedside critical care nurse to almost instantaneously recognize vital changes in a patient’s medical status and determine just the right interventions to rebalance fluid or improve oxygenation. My colleague and ICU nurse, Nora Kelly, modeled respect for patient dignity that, to this day, shapes my work to support patient self-management using effective computer technologies. Nora showed me that even in the midst of an often hectic, fast-paced ICU environment, there was always time to provide a patient with comfort, help a person into a more comfortable position, or complete basic hygiene and grooming around tubes and monitor wires.

Now: Serving as Your NLM Director

What stands out the most to me now are the lessons about the importance of in-the-moment information processing; interdisciplinary teamwork supported by nurses, physicians, respiratory therapists, pharmacists, social workers, and others; and personal accountability that shape my everyday life as the director of NLM. Delivering high-quality care under extreme levels of uncertainty and risk is the hallmark of critical care. I learned early on that time was of the essence—there was rarely an opportunity to pause and read an article or two as one pondered how to intervene in a physiology cascade that could lead to sudden death.

The insights from these experiences taught me that for information to truly support in-the-moment care, NLM needed to make its resources open and available in machine-readable formats. It is our job to use machine-learning algorithms to make available NLM’s vast repository of biomedical and scientific literature that drives contemporary drug management or clinical guidelines interpretation. NLM invests in research that helps ICU professionals quickly interpret patient charts so they can predict the likelihood of pulmonary embolism diagnosis or track a patient’s probable health outcome trajectory using observations noted in their electronic health record.

NLM in the ICU

ICU patients in hospitals around the country are all supported by the best interprofessional teams that understand the unique aspects of patient care, whether that’s to advance the patient’s progress towards wellness or to provide alternative end-of-life care focused entirely on comfort. Because of the diversity of caregivers and professionals across hospital ICUs, we must acquire, organize, and disseminate the literature to all biomedical professional groups when they need it most.

It is in this spirit that each division in NLM—including our Library Operations team managing our NLM Collection, our MEDLINE Literature Selection Technical Review Committee to impanel experts across many specializations, and our PubMed and PubMed Central with the tools to index and catalog records—accelerates the dissemination of knowledge from many disciplines. Clinicians are required to have deep expertise and stay abreast of new research within their specialty and to recognize potentially valuable literature from other disciplines. In support of this requirement, we organize over 34 million citations by clinical problem and physiological underpinning. That way, no matter what your specialty, each search identifies literature from a wide range of perspectives and refines our “relevance-based results return” according to those patterns most valued by our patrons, as described by NLM’s Best Match algorithm.

Patients often find themselves in the ICU from somewhere else in the health care system and are frequently discharged not to their homes, but to other less-intensive clinical care units. To understand their conditions and efficiently guide their care in a vast, complex, and time-sensitive setting, health care interprofessional teams should understand all ICU clinical information and events so they can translate and transmit that information to the responsible post-discharge teams. This information flow relies on health data standards so that events that occur in one place are well understood in the next. NLM plays an important role by forecasting how health care settings like ICUs will use health data standards to promote interoperability and by shaping the public policies that protect patient records. NLM shares its expertise in data science, health information technologies, and computer science with our fellow federal agencies and with the private sector to make sure patient records are accessible while remaining private and secure.

Connecting the Dots

I remember the enormous intimacy involved in my ICU nursing experience, often including myself and a patient, at times the patient’s family, and certainly every time the rest of the care team. But teamwork only works when each member holds sacred their responsibility to the patient and the care that they require. Personal accountability does not occur in a vacuum; rather, it is molded and shaped through conversations with colleagues, collaborative care-planning rounds, candid postmortem reviews, and quiet heart-to-hearts in the staff lounge. Even these efforts are touched by NLM, from providing literature and guidelines that lay out the various roles of professionals to furnishing our citations repository with the contact information of those authors whose work guides clinical thinking. In this way, NLM becomes a partner for personal accountability.

If only that fledgling ICU nurse from 50 years ago knew that her entire cultural and practical experience was preparing her to direct the most important health science library in the world! Because of who she was as that nurse and who we are as NLM, critical care remains a cornerstone of health care information and systems in best support of all patients. If you have ideas for how NLM can better support the critical care of YOUR patients, please let us know!

%d bloggers like this: