Asking the right questions and receiving the most useful answers

Guest post by Lawrence M. Fagan, Associate Director (retired), Biomedical Informatics Training Program, Stanford University.

As online resources proliferate, it becomes harder to figure out which resources—and which parts of those resources—will best answer patients’ questions about their medical care. Patients have access to multiple websites that summarize information about a particular disease, myriad patient communities, and many online research databases.

This resource overload problem isn’t new, however. As more and more data became available in the Intensive Care Units of the 1980s, it grew increasingly difficult to determine the most important measurement to track for optimal care.

In short, more information isn’t necessarily the best solution when it comes to answering patient questions.

After a career in informatics, I now moderate an online community for patients with a particular subtype of lymphoma. Many questions that arise in the group can easily be answered by reviewing existing online content. Health librarians are excellent resources to help guide patients to the correct resources and articles.  However, some queries are less straightforward than others, such as: “What is the one thing you wished you knew before the procedure?” Rather than asking for a recitation of the steps in the procedure, this question is asking what was unexpected—or what step would have benefited from more patient or caregiver preparation. Correspondingly, these types of questions are hard to organize, store, and retrieve from patient-oriented databases.

Sometimes, the community of patients can recognize patterns that escape the notice of medical providers. For example, a lymphoma patient may complain of repeated sinus infections. It’s worth noting that patients often turn to their primary care provider to treat their sinus infections, and those visits may lead to antibiotic prescriptions. In this scenario, group members have pointed out the potential link between treatment with the drug Rituximab and a decrease in the body’s immunoglobulin levels. This connection leads to suggestions to explore an alternative treatment for chronic sinus infections (in this special context) using immunoglobulin replacement therapy rather than antibiotics.

Specialized online communities can also provide help with detailed care issues, including the treatment of side effects for uncommonly used drugs with which local healthcare providers might not be familiar.

Online communities can also suggest researching databases to answer patient questions. helps locate experimental treatments for specific medical conditions. Some community discussions about trials go beyond what’s included in the database. For instance, group members may discuss the optimal order of clinical trials in a specific medical area, based on an analysis of the inclusion criteria for the various trials. In addition, there are ancillary questions about trial logistics that aren’t found in the database, such as, “I live in the San Francisco area—is it feasible to participate in Trial X at City of Hope in Southern California?” Setting up comprehensive links between the clinical databases and discussions in patient communities would help patients access the answers to their questions more efficiently.

The answers to these specialized questions are often found in the archives of online communities or in the memories of group participants. Yet, it is not easy to find the right community for a particular medical problem, and in my understanding there is no central repository of links to online communities. Moreover, while many community links can be found in MedlinePlus, static links to community websites often become stale, as sites may migrate locations over time. Some of the ACOR cancer communities, for example, have migrated to  As patients find a community of interest, it is important that they determine whether the conversations are ongoing and whether the participants are knowledgeable and supportive. The Mayo Clinic offers a short discussion detailing the pros and cons of support groups.

Researchers have examined patient and clinician information needs for more than a quarter century. These models, however, have only rarely been incorporated into information retrieval systems. One successful example (aimed at providers) is the use of “clinical queries” in PubMed, designed for searching the scientific literature. This brings us to a critical question: What would it take to reengineer the patient-oriented retrieval systems so that these focused queries drive most patient sites?

For now, we have communities of patients and dedicated professionals who are ready and willing to help point to the most useful answers.

Please note: The mention of any commercial or trade name is for information and does not imply endorsement on the part of the author or the National Library of Medicine.

Many thanks to Dave deBronkart, Janet Freeman-Daily, Robin Martinez, Tracie Tavel, and Roni Zeiger who reviewed earlier versions of this blog post.

Outdoor portrait of Lawrence M. Fagan.Lawrence Fagan, MD, PhD, retired in 2012 from his role as Associate Director of the Stanford University Biomedical Informatics Training Program. He is a Fellow of the American College of Medical Informatics. His current interests are in patient engagement, precision health, and preventing medical errors.

Reflections on a Reflection

NLM’s role within NIH

What is the role of the National Library of Medicine within NIH?

A rare, low-humidity August day got me thinking about that question as I walked toward NLM one morning.

Catching sight of NLM’s Lister Hill Center, I saw the bright blue sky and scattered white clouds reflected in the windows that comprise the tower’s face. I then became aware that, given our location at the southeastern corner of campus, almost all of NIH was behind me. And those windows that both reflect the morning sky and allow the building’s occupants to see out got me thinking about how the Library does the same, both reflecting NIH and looking out, forward, toward the future.

NLM reflects NIH priorities and discoveries, as we curate and characterize biomedical and health science knowledge. We are a repository of that knowledge and a tool for discovery. We acquire, index, and make available relevant literature; amass and organize genetic, molecular, chemical, and toxicological information and data; develop methods for analyzing large data sets; construct sophisticated search strategies; and create algorithms, software, and other tools that leverage our holdings, whether literature or data. We then train NIH scientists and science support staff how to use them to speed discovery, sharing our products along with our talents. We also collaborate on committees across NIH, bringing to bear our expertise in medical informatics, standards, and data science on the complex medical and public health problems the greater institution is tackling.

But like the wall of windows in the Lister Hill Center, the Library also provides a way to look out and see what’s around us. By aggregating literature into meaningful clusters, NLM documents what’s known and maps the research terrain, highlighting questions still unanswered and the paths that lead to new discoveries. By collecting and exposing medicine’s past, NLM  gives historical context to contemporary ideas. NLM’s connections to the literature across disciplines help NIH investigators situate biomedical knowledge in the context of policy, philosophy, psychology, and the arts. And our molecular databases are increasingly linked to others in repositories around the world, bringing together articles, charts, models, images, and even clinical records and facilitating the bench-to-bedside pathway of discovery.

Windows also invite contemplation and thoughts about the future, and NLM’s work supports that as well. The tools we build can sharpen an investigator’s ability to foresee the future, from prediction algorithms to natural language processing and pattern recognition. We also have our own thoughts and plans for the future, for an NLM that supports data-driven discovery, data-powered health, and patient-centered care, and we’ll continue to pursue those plans in partnership with NIH.

Like windows on a sunny day, NLM reflects back to NIH the biomedical knowledge it needs to achieve its mission. And, like windows, we provide NIH a way to see and assess its environment, its context, and to think about its future. Those thoughts, in turn, will create the research world of tomorrow—for NIH and for NLM users and stakeholders around the globe. As has been the case since our founding in 1836, NLM will be there, reflecting, guiding, inspiring.

An Oath Grounded in the Constitution

Tomorrow, September 12, is the two-year anniversary of my swearing in as the Library’s fourth director since it officially became the US National Library of Medicine.

What a day that was!

I delivered my first major address to my NLM colleagues, with my family, friends, and NIH Director Francis Collins in the front row. I wanted NLM staff to know how much I wanted to know and work with them, and I wanted Dr. Collins to recognize what a great operation we already were.

I still treasure having my siblings and my mother here. My brothers, all five to 15 years younger than I, were impressed with the place but a little surprised that their sister was selected for this position and held in such high regard. (Ya’ gotta love brothers, right?)

The swearing-in itself took less than three minutes (including photos), but as I reflect on the occasion, they were the most important three minutes of the afternoon.

All federal employees take an oath of office, a requirement stemming from Article VI of the Constitution. The original oath, spare in its simplicity, stated, “I do solemnly swear (or affirm) that I will support the Constitution of the United States.” Over the years, the oath grew to include more conditions, with some additions—such as the Civil-War era’s Ironclad Test Oath—being short-lived. But the heart of the oath, upholding the Constitution, has never wavered.

The Constitution, a complex and much-debated document, establishes the three branches that provide our government’s structure and ensure a separation of powers, i.e., the legislative, executive, and judicial branches. Much of the Constitution details elements of government I have little to do with (at least directly), but the Preamble—only 52 words long—packs a punch.

These opening words set out the principles that guide us and unite us:

We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defense, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America.

As I read and re-read these words, I realized these principles also establish the foundation of our work as federal employees. In fact, many of them underlie the very work we do here at NLM, and everything we accomplish ultimately points back to them.

The data and information we acquire, the products we develop, and the services we offer help patients, health professionals, researchers, and policy makers “promote the general welfare” of our citizens and “secure the blessings of liberty” through improved health and well-being.

We “establish justice” by ensuring fairness in our collections and in people’s access to them. Those collections must remain sufficiently broad and robust to provide an even-handed, impartial view of what health means in society. Our literature and data must be as balanced and objective as we can make them, limiting the intentional (or even unintentional) bias that privileges one perspective on health over another. Our terminologies, which literally label matters pertaining to medicine, health, and well-being, must expand beyond the biological definitions to include the social and behavioral domains. And our products and services must be equally accessible to all, which means that both our technology and our outreach efforts must make those products and services understandable and actionable to people of all levels of income, resources, and self-actualization.

And all of these actions, these authorities, these products and services we offer, come together—like the work of the federal government as a whole—to contribute to “a more perfect union,” one that ensures the benefits detailed in the Preamble “to ourselves and our posterity.”

Not surprisingly, I find that upholding the Constitution, as I swore to do two years ago, is woven so tightly into what I do that it’s inescapable. But reflecting on how I do it and what is means is powerful nonetheless. It brings perspective to the decisions we make, the investments we endorse, and the products and services we bring to society. And it reminds me, as I noted last year, why I do it—namely, for all of us, for “We the people.”

In the context of the work of the Library, what does it mean to you to support and defend the Constitution? I’d love to get your thoughts.

How much does it cost to keep data?

Study to forecast long-term costs

Guest post by Elizabeth Kittrie, NLM’s Senior Planning and Evaluation Officer.

As scientific research becomes more data-intensive, scientists and their institutions are increasingly faced with complex questions about which data to retain, for how long, and at what cost.

The decision to preserve and archive research data should not be posed as a yes or no question. Instead, we should ask, “For how many years should this subset of data be preserved or archived?” (By the way, “forever” is not an acceptable response.)

Answering questions about research data preservation and archiving is neither straightforward nor uniform. Certain types of research data may derive value from their unique qualities or because of the costs associated with the original data collection. Other types of research data are relatively easy to collect at low cost; yet once collected, they are rarely re-used.

To create a sustainable data ecosystem, as outlined in both the NLM Strategic Plan and the NIH Strategic Plan for Data Science, we need strategies to address fundamental questions like:

  • What is the future value of research data?
  • For how long must a dataset be preserved before it should be reviewed for long-term archiving?
  • What are the resources necessary to support persistent data storage?

We believe that economic approaches—including forecasting long-term costs, balancing economic considerations with non-monetary factors, and determining the return on public investment from data availability—can help us make preservation and archiving decisions.

Economic approaches…can help us make preservation and archiving decisions.

To that end, NLM has contracted with the National Academies of Sciences, Engineering, and Medicine (NASEM) for a study on forecasting the long-term costs for preserving, archiving, and promoting access to biomedical data. For this study, NASEM will appoint an ad hoc committee that will develop and demonstrate a framework for forecasting these costs and estimating potential benefits to research. In so doing, the committee will examine and evaluate the following:

  • Economic factors to be considered when examining the life-cycle cost for data sets (e.g., data acquisition, preservation, and dissemination);
  • Cost consequences for various practices in accessioning and de-accessioning data sets;
  • Economic factors to be considered in designating data sets as high value;
  • Assumptions built in to the data collection and/or modeling processes;
  • Anticipated technological disruptors and future developments in data science in a 5- to 10-year horizon; and
  • Critical factors for successful adoption of data forecasting approaches by research and program management staff.

The committee will provide a consensus report and two case studies illustrating the framework’s application to different biomedical contexts relevant to NLM’s data resources. Relevant life-cycle costs will be delineated, as will any assumptions underlying the models. To the extent practicable, NASEM will identify strategies to communicate results and gain acceptance of the applicability of these models.

As part of its information gathering, NASEM will host a two-day public workshop in late June 2019 to generate ideas and approaches for the committee to consider.  We will provide further details on the workshop and how you can participate in the coming months.

As a next step in advancing this study, we are supporting NASEM’s efforts to solicit names of committee members, as well as topics for the committee to consider.  If you have suggestions, please contact Michelle Schwalbe, Director of the Board on Mathematical Sciences and Analytics at NASEM.

casual headshot of Elizabeth KittrieElizabeth Kittrie is NLM’s Senior Planning and Evaluation Officer. She previously served as a Senior Advisor to the Associate Director for Data Science at the National Institutes of Health and as Senior Advisor to the Chief Technology Officer of the US Department of Health and Human Services. Prior to joining HHS, she served as the first Associate Director for the Department of Biomedical Informatics at Arizona State University.

When September Doesn’t Mean Back to School

At this point in my life, two years into my role at the National Library of Medicine, I greet Labor Day with a bit of nostalgia. I was an academic for 30 years, and spent most of the 25 years prior to that as a student, doing the school thing from kindergarten through grad school. You might say that my epigenome has been shaped by the academic calendar, with the waning summer sparking excitement over the promise of a new school year, new things to learn, new friends to make.

In contrast, September here brings the last month of the federal fiscal year, and let me tell you, it is busier than all get out! We have to finish grant awards. We must renew, close out, or refresh contracts, confirm (or move) project deadlines, and account for the year’s work. In addition, we are in the throes of preparing the Congressional Justification for our budget one year hence, so as we’re putting fiscal year 2018 (FY2018) to bed, we’re also starting to prepare the FY2020 budget—all while Congress debates the FY2019 appropriations. It’s a budgetary juggling act to keep all the plans and paperwork moving, and my colleagues who deal with grants, contracts, budget, and procurement will be thoroughly focused for the next four and a half weeks, making sure the balls don’t fall.

But amidst the fiscal frenzy, there’s still a hint of the old, familiar September and the promises that come with it of new things to learn and new friends to make.

This coming year, FY2019, we’ll be wrapping our arms fully around data science. Joyce Backus is leading an NLM-wide strategy to improve the data science knowledge and skills of our entire workforce. Our NCBI team is re-writing the book on secure data access, taking a modular approach to identity management and data access. The Tox team, in collaboration with other staff across NLM, is migrating our essential toxicology resources to a more modern and robust platform. And those are just a few examples.

As for the new people to meet—we’ve just launched a search for three new investigators for our intramural program; we’re growing our complement of contracts management staff; and we’re adding new program managers across Library Operations.

So in some ways, it is like my old Septembers only somewhere else, a different location but the same sense of newness, excitement, learning, and opportunity.

I hope all the students stepping into the classroom or on to campus this year find the same exhilaration greeting them. May you, your teachers, professors, and parents have a bright and successful academic year. And may the rest of us find the continued promise of fresh opportunities and new, innovative ways to serve science and society.