Calling on Librarians to Help Ensure the Credibility of Published Research Results

Guest post by Jennifer Marill, Kathryn Funk, and Jerry Sheehan.

The National Institutes of Health (NIH) took a simple but significant step Friday to protect the credibility of published findings from its funded research.

NIH Guide Notice OD-18-011 calls upon NIH stakeholders to help authors of scientific journal articles adhere to the principles of research integrity and publication ethics; identify journals that follow best practices promoted by professional scholarly publishing organizations; and avoid publishing in journals that do not have a clearly stated and rigorous peer review process. The notice identifies several resources authors can consult when considering publishing venues, including Think Check Submit, a publishing industry resource, and consumer information on predatory journals from the Federal Trade Commission.

Librarians have an especially important role to play in guiding researcher-authors to high-quality journals. Librarians regularly develop and apply rigorous collection criteria when selecting journals to include in their collections and make available to their constituents. Librarians promote high-quality journals of relevance to their local communities. As a result, librarians are extremely familiar with journal publishers and the journals their constituents use for research and publication.

The National Library of Medicine (NLM) is no exception. One of NLM’s important functions is to select journals for its collection. The journal guidelines from the NLM Collection Development Manual call for journals that demonstrate good editorial quality and elements that contribute to the objectivity, credibility, and scientific quality of its content. It expects journals and journal publishers to conform with guidelines and best practices promoted by professional scholarly publishing organizations, such as the recommendations of the International Committee of Medical Journal Editors and the joint statement of principles of the Committee on Publication Ethics, Directory of Open Access Journals, Open Access Scholarly Publishers Association and World Association of Medical Editors.

Criteria for accepting journals for MEDLINE or PubMed Central are even more selective, reflecting the considerable resources associated with indexing the literature and providing long-term preservation and public access to full-text literature. MEDLINE currently indexes some 5,600 journals; PubMed Central has about 2,000 journals that regularly submit their full content. PubMed Central is also the repository for the articles resulting from NIH-funded research.

For the most part, NIH-funded researchers do a good job of publishing in high-quality journals.  More than 815,000 journal articles reporting on NIH-funded research have been made publicly accessible in PubMed Central since the NIH Public Access policy became mandatory in 2008. More than 90 percent of these articles are published in journals currently indexed in MEDLINE. The remainder are distributed across thousands of journals, some 3,000 of which have only a single article in PubMed Central. While many are quality journals with sound editorial practices, effective peer review, and scientific merit, it can often be difficult for a researcher-author to evaluate these factors.

That’s where local librarians can be of great assistance. And many already are—helping researchers at their local institutions select publishing venues.

If you have a good practice in your library, let us know about it so we can all learn how best to protect the credibility of published research results.

Jennifer Marill serves as chief of NLM’s Technical Services Division and the Library’s collection development officer. Kathryn Funk is a program manager and librarian for PubMed Central. And Jerry Sheehan is the Library’s deputy director.

Mining for Treasure, Discovering MEDLINE

Reusing a noteworthy dataset to great result

Guest post by Joyce Backus and Kathel Dunn, both from NLM’s Division of Library Operations.

As shrinking budgets tighten belts at hospitals and academic institutions, medical libraries have come under scrutiny. In response, librarians have had to articulate the value they bring to the institution and to the customers—students, researchers, clinicians, or patients—they serve.

In 2011-2012, as such scrutiny swelled, Joanne Marshall and her team set out to study the very question these medical institutions faced: Do libraries add value? They collected 16,122 individual responses from health professionals at 118 hospitals served by 56 health libraries in the United States and Canada. The team sought to determine whether physicians, residents, and nurses perceived their libraries’ information resources as valuable and whether the information obtained impacted patient care.

The resulting article, “The Value of Library and Information Services in Patient Care,” published in 2013, gave medical librarians strong talking points, including the overall perceived value of libraries as time-savers that positively impact patient care.

Now the datasets from that study are being reused to great result.

Over the last year we teamed up with Joanne Marshall and Amber Wells, both from the University of North Carolina-Chapel Hill, to dive into the data.

Our goal: to understand the value and impact of MEDLINE in medical libraries.

We re-discovered (as has been written about before) the value of MEDLINE in changing patient care. We also found its preeminent role shines even more brightly in a dataset like this one that includes other sources. We saw the significance of MEDLINE as a single source of information but also as a source used in combination with full-text journals, books, drug databases, websites, and colleague consultations.

We were reminded, too, of the importance of the National Network of Libraries of Medicine (NNLM) to our work; the trust in the NNLM; each library’s connectedness to the other; and how the everyday web of relationships prompts cooperation and collaboration, including the successful implementation of the value of libraries study itself.

For us this re-discovery comes at a key time, when we’re examining NLM products and services as part of the strategic planning process. We are actively identifying methodologies and tools to elevate all our collections—from datasets to incunabula—and make them greater engines of discovery in service of health.

But what about your library’s resources?

The data mining challenge we gave ourselves is our guide for medical librarians everywhere: look at your data, what’s in front of you, and then others’ data. What can they tell you about what’s happening now, what will likely happen in the future, what’s being used, and how it’s being used?

If you don’t know where to start, check out the Medical Library Association’s Research Training Institute, recommended research skills, and mentoring program. In addition, the NNLM’s site on program evaluation includes tools for determining cost benefit and return on investment.

Librarians positively impact health care and health care research. Now it’s time to have that same impact on our own profession. The data are there. It’s time we see what they have to tell us.

More information

Value of Library and Information Services in Patient Care Study


Lindberg DA, Siegel ER, Rapp BA, Wallingford KT, Wilson SR. Use of MEDLINE by physicians for clinical problem solving. JAMA. 1993; 269: 3124-9.

Demner-Fushman D, Hauser SE, Humphrey SM, Ford GM, Jacobs JL, Thoma GR. MEDLINE as a source of just-in-time answers to clinical questions. AMIA Annual Symposium Proceedings. 2006:190-4.

Sneiderman CA, Demner-Fushman D, Fiszman M, Ide NC, Rindflesch TC. Knowledge-based methods to help clinicians find answers in MEDLINE. Journal of the American Medical Informatics Association. 2007 Nov-Dec; 14(6):772-80.

Joyce Backus serves as the associate director for Library Operations at NLM. Kathel Dunn is the NLM Associate Fellowship coordinator.

Photo credit (ammonite, top): William Warby [Wikimedia Commons (CC BY 2.0)]

Essential and Invisible

On the importance of being recognized

Recently I had the pleasure of attending the annual meeting of the South Central Chapter of the Medical Library Association. There I met librarians and other information management professionals who serve the South Central Region of the National Network of Libraries of Medicine, many of whom have been quite busy lately ensuring continuity of library operations through Hurricane Harvey and its aftermath.

impromptu group photo
Dr. Brennan visits with staff from the NNLM/South Central Region in Albuquerque, October 15, 2017.

I attended the meeting to give an update of how things are going back here in Bethesda, describe my vision for NLM’s role in data-powered health, and listen to attendees’ issues and questions.

As usual, when I get out and about, I learn so much, particularly how front-line librarians in hospitals and academic health science centers are increasingly becoming embedded in complex care processes and sophisticated biomedical research. I’m also hearing how often they feel simultaneously essential and invisible.

The librarians I met told stories of team members, months into a collaboration, finally recognizing the person retrieving key literature or managing data was in fact the university’s librarian; or young faculty members who didn’t realize the health science librarian could help locate funding opportunities; or physicians sharing patient-education materials without knowing the hospital library staff developed them.

“Essential and invisible” sums up the state of modern libraries and contemporary librarians—and, to some extent, the state of the National Library of Medicine itself.

As libraries becomes less of a place to go to, librarians are bringing biomedical information to the people they serve, through pop-up libraries, health literacy classes, or targeted emails that deliver expert, preformed searches to the inboxes of busy clinicians.

Similarly, NLM’s resources are becoming untethered from the Library and our website. The entire database is embedded in a patient advocacy search engine. MedlinePlus Connect integrates NLM’s patient education materials into electronic health records across the country. And nearly 60 APIs give seamless access to NLM’s carefully curated content, whether that’s peer-reviewed literature, clinical value sets, or standardized drug names.

These shared assets enrich users’ experiences, but do those users know who’s doing the enriching? Probably not.

Essential and invisible.

Of course, it’s great that federal investments into high-quality health information are put to good use—it’s why we exist—but it’s also important that users recognize in the resources they use the high-quality, trusted brand that is NLM. Brand recognition—and the professional expertise behind that brand—contribute to the credibility of the information provided.

So the next time you encounter something helpful for health or biomedical research, take a moment to find out—is it NLM inside?

The Sport that Made Me a Better Leader

Late last month, I dined with the NLM/Association of Academic Health Sciences Libraries (AAHSL) Leadership Fellows on the final day of their program and got more than just lunch. I ended up with an interesting realization about how a sport I enjoy has helped shape how I lead.

The curriculum for the year-long NLM/AAHSL fellowship program includes such topics as power and influence, managing a workforce, and diversity and inclusion. The program inevitably sparks self-reflection for both fellows and their mentors, and though I spent only a short time with the group, it got me thinking, too, about issues related to leadership, personal awareness, and growth. Since then, I’ve been musing over what has helped me be an effective leader at NLM.

Certainly, having a terrific staff and the support of NIH leadership makes the whole process easier. I’m fortunate to have both.

I’ve also received words of wisdom from experienced colleagues, gleaned key insights from books, and recalled valuable lessons from management courses I’ve attended over the years.

But, believe it or not, nothing has prepared me more for senior leadership than squash—the sport, not the food.

For those of you who don’t know the game, Wikipedia explains it this way:

A ball sport played by two or four players in a four-walled court with a small, hollow rubber ball. The players must alternate in striking the ball with their racket and hit the ball onto the playable surfaces of the four walls of the court.

The game’s high speed calls for quick movements and even faster thinking. You have to predict the ball’s angle of return based on the point of contact with the racket and the velocity of the hit. You have to plan your shot based on your opponent’s position and your own. And you have to do all this while avoiding the other player as you both navigate your way around a very small space. (The entire court is about 7 feet by 10 feet.)

The situational awareness and mental agility needed to pull all this off build important skills for management and those high-pressure, think-on-your-feet moments, but I think no skill is more important than timing.

Precision timing—striking the ball at just the right moment—is key to success in squash. Precision timing is also important for leadership.

One must determine how long to let a conversation proceed before weighing in, or how many emails in a chain should pass before making a statement. One must decide if one has entered an argument on its first round or if it is an old engagement rehashed many times. And importantly, one must be prompt with feedback, good and bad, both to reveal one’s values to the organization and to expose one’s preferences and pleasures.

But squash has taught me more than getting the timing right.

Power shots in squash send a 2” ball rocketing through the air. And when that ball hits you unexpectedly, on the arm, leg, or even face, you quickly realize that, even though it is very, very small and you are very, very big, it can really hurt. A lot.

Translating this to management, I’ve seen how a small, seemingly inconsequential statement can serve as the prelude to a major problem, or, on the flip side, how well-timed praise, delivered when the recipient is ready to hear it, can have profound positive impact.

Squash has also taught me to play nice with my partners and observe the rules of engagement on the court. Management situations bring their own rules of engagement, sometimes defined, sometimes not. The unwritten rules can be the trickiest, with rules sometimes changing based on where the engagement occurs. Understanding and respecting protocol, positions at the table, and the tenor of discourse as it varies from office to hall way to conference room are skills worth developing.

I’ve learned through squash that I can tussle with an opponent over controlling the T-zone (the prime spot from which it is easiest to reach most shots) and still enjoy a cool drink and camaraderie afterward. Managerially, this means that conflict is part of the game, and going toe-to-toe over important issues doesn’t—and shouldn’t—stand in the way of collegiality. (Another key take-away: It always helps to share a cool drink afterward!)

And ultimately, in squash as in management, there are always opportunities to improve.

What has influenced your leadership? What makes you a good manager? What makes your manager a good manager?

Addressing Health Disparities to the Benefit of All

Guest post by Lisa Lang, head of NLM’s National Information Center on Health Services Research and Health Care Technology

Singer-actress Selena Gomez shocked her fans this past September with the announcement that she had received a kidney transplant to combat organ damage caused by lupus.

Lupus, an autoimmune condition, strikes women much more than men, with minority women especially vulnerable. Not only is lupus two to three times more common in African American women than in Caucasian women, but recent studies funded by the CDC suggest that, like Ms. Gomez, Hispanic and non-Hispanic Asian women are more likely to have lupus-related kidney disease (lupus nephritis)—a potentially fatal complication.

Documenting such health disparities is crucial to understanding and addressing them. Significantly, the studies mentioned above are the first registries in the United States with sufficient Asians and Hispanics involved to measure the number of people diagnosed with lupus within these populations.

Investment in research examining potential solutions for health care disparities is essential.

In 2014, The Lancet featured a study that examined patterns, gaps, and directions of health disparity and equity research. Jointly conducted by the American Academy of Medical Colleges and AcademyHealth, a non-profit dedicated to enhancing and promoting health services research and a long-time NLM partner, the study examined changes in US investments in health equities and disparities research over time. Using abstracts in the NLM database HSRProj (Health Services Research Projects in Progress), the researchers found an overall shift in disparities-focused projects. From 2007 to 2011, health services research studies seeking to document specific disparities gave way to studies examining how best to alleviate such disparities. In fact, over half of the disparities-focused health services research funded in 2011 “aimed to reduce or eliminate a documented inequity.” The researchers also found significant differences in the attention given to particular conditions, groups, and outcomes. An update by AcademyHealth (publication forthcoming) found these differences continue in more recently funded HSR projects.

A more nuanced appreciation of affected groups is also critical to addressing health disparities. For example, the designation “Hispanic” is an over-simplification, an umbrella construct that obscures potentially important cultural, environmental, and even genetic differences we must acknowledge and appreciate if we are to maximize the benefits promised by personalized medicine. Reviews such as “Hispanic health in the USA: a scoping review of the literature” and “Controversies and evidence for cardiovascular disease in the diverse Hispanic population” highlight questions and conditions that would be informed by richer, more granular, data.

Lupus is one such condition. Research into this disease’s prevalence and impact among Hispanics is underway, but more attention may be warranted. There are almost 100 active clinical studies in the US targeting lupus currently listed in and, of these, 15 address lupus nephritis. And while about 5% of ongoing or recently completed projects in the HSRProj database explicitly focus on Hispanic populations, only one, funded by the Patient-Centered Outcomes Research Institute, specifically addresses lupus. (You can see this study’s baseline measures and results on

Perhaps a celebrity like Ms. Gomez publicly discussing her experience with lupus will spark more attention from both researchers and the public seeking to contribute to knowledge and cures.

After all, we are all both fundamentally unique and alike. Reducing—or better yet, eliminating—health disparities benefits us all.

Guest blogger Lisa Lang is Assistant Director for Health Services Research Information and also Head of NLM’s National Information Center on Health Services Research and Health Care Technology (NICHSR).

Photo credit (The Scales of Justice, top): Darius Norvilas [Flickr (CC BY-NC 2.0)] | altered background