July 24, 2014

Data-Driven Libraries: Moving From Outputs to Outcomes

An Increasing focus on impact reshapes how leaders evaluate and design service

By Rebecca Miller, with Francine Fialkoff & Michael Kelley

Something special happens when 150 public library directors, deputy directors, and trustees gather in one room to talk about what their communities get from their libraries. The ideas and examples whip back and forth, as do equally intense questions about roadblocks to all sorts of challenges and strategies for overcoming them. The headiness of this to and fro was palpable at LJ’s third Director’s Summit, held at Columbus Metropolitan Library (CML), OH, December 5–6, 2011. (For more perspective on the discoveries from this event see Francine Fialkoff’s editorial “Moving to Outcomes,” p. 8.)

These library leaders came to discuss the urgent need to reshape how library service is evaluated in order to articulate outcomes better to stakeholders and share success stories with the community. Pure data such as gate counts, computer uses, and more aren’t as satisfying to those who hold the purse strings as are measurements that articulate impact. “It’s pretty simple,” said Jeanne Goodrich, executive director, Las Vegas–Clark County Library District, NV. “Why we do what we do drives what we do. An outcome answers the question, ‘So what?’”

“It’s not good enough for our small business and our entrepreneurial groups to think, ‘Yeah, the library’s a good place because it helps my kids with their homework,’” said Steven Potter, director of libraries, Mid-Continent Public Library, Independence, MO. “People have to understand that the library is important to them directly. When they understand that, they begin to understand the true value, the outcomes, we are trying to achieve.”

ljx120101webDataDrive1 Data Driven Libraries: Moving From Outputs to Outcomes

HEADY EXCHANGES (1st row): Columbus Metropolitan Library (CML) was the site of the LJ Directors’ Summit, hosted by (ctr. photo) Patrick Losinski (ctr.), here with CivicTechnologies Mark Futterman (l.) and keynoter Lee Rainie from the Pew Center; (r.) Charlotte Meckenburg Library’s Michele Gorman (with mic) (2d, l.–r.): Garry Golden, futurist; CML’s Alison Circle (and LJ blogger) with Cuyahoga Cty., OH, director Sari Feldman and Tracy Strobel; and panelists/directors Steven Potter, Mid-Continent PL, MO, Jeanne Goodrich, Las Vegas-Clark Cty., NV, and Bill Ptacek, King Cty., WA (3d, l.–r.): Innovative Interface’s Neil Block; OCLC’s Cathy De Rosa; Rachel Rubin, Bexley PL, OH, with Cleveland PL director Felton Thomas; a breakout session (4th row, l.–r.): directors Brian Hare, Liberty Center PL, OH; Greta Southard, Boone Cty. PL, KY; Jim Cooper, Salt Lake County; and Jos Holman, Tippecanoe Cty. PL, IN. Photos by Dane Khy

Squishy vs. hard

The trick, many agreed, is that outcomes are “squishy” while outputs are hard—and libraries are accustomed to hard proof of what they deliver. “A lot of us are still stuck in the idea that we want a number, we want to show that something increased five percent over the year before, we want a precise number that, ideally, we don’t have to gather but a computer can spit out,” added Potter. Those outputs will continue to get counted, too, especially since they are still required by the states and the Institute of Museum and Library Services, but the real story around effectiveness has to be developed.

The approach to “counting” squishy outcomes, and perhaps a coping method for the anxiety of needing to express impact anew, may be some form of the “Logic Model,” so named by Bill Ptacek, director of the King County Library System, Issaquah, WA, LJ’s 2011 Library of the Year. The model goes something like this, he said: “KCLS will do this , which will result in these things , so our patrons will blank , as measured by blank .” This formulation for getting at the story of the library’s impact is, he said, “the only way you can survive in this; otherwise you are going to drive yourself batty trying to prove everything.”

Ptacek recommended building on existing research from other institutions to connect the dots regarding the impact on the overall community, not just library users. And, all agreed, libraries must respond to local needs, which may differ from neighborhood to neighborhood, such as adult and family literacy, fun in hard times, managing diabetes, and dealing with foreclosures.

ljx120101webDataDrive2 Data Driven Libraries: Moving From Outputs to Outcomes

PARTNERS (l.–r.): Sponsor Gale Cengage’s Ron Stefanski with CML’s Losinski (l.); panelists Clay Marsh, M.D., Center for Critical Care and Respiratory Medicine, Ohio State Univ. Medical Ctr.; David Harrison, Columbus State Community Coll.; and Brad Mitchell, Ohio Appalachian Collaborative at Battelle for Kids; OrangeBoy’s Sandy Swanson (standing with mic). Photos by Dane Khy

The dotted line

Drawing a dotted line that leads beyond library walls to connect to other institutions helps measure a library’s outcome, such as working with local schools to track reading readiness after a summer reading program, or using GIS to identify communities at risk and gearing programming toward them.

CML’s Alison Circle, for example, explained how the library thought of an outcome as “a condition in the city we are trying to change.” In one instance, they plotted Franklin County schools on a map and identified six neighborhoods where they saw a chance to support at-risk parents. “They are their children’s first teachers,” she noted. Every parent gets a Ready To Read packet distributed widely, including via churches, food banks, laundromats, and pediatric clinics.

The Las Vegas–Clark County library, noted Danielle Milam, director of development, is living with a 30 percent budget cut. “We know we’ll never get to that place we were before,” she said. “We need the tools to help us reset our game.” One such tool she identified is the Community Connect software from CIVICTechnologies, a summit sponsor, which uses library patron data, U.S. census demographic data, and market segmentation data. The software, she said, “helps the library understand where patrons live, what transactions they are making, where they make those transactions, how they are behaving, what their lifestyles are.”

“It answers the questions, ‘Who are we serving? Who are we not serving? Who do we need to serve? and Are our service strategies matching the population?’” Milam said. The library uses this information to “understand in ways we couldn’t find out at the Town Hall [meeting] or in focus groups. It goes way beyond the concentric circles we used to make for our service areas.”

Inspiration beyond libraries

Looking outside of libraries for perspective can be helpful, as a group of nonlibrarians presenting at the summit illustrated. Lee Rainie, director of the Pew Internet & American Life Project and a former journalist, pointed out that “no matter how good your data are, you still need stories.” He challenged the participants to ask themselves a series of questions: “What’s the franchise? What’s your mobile play? What’s the gift economy play? How do we manage it?” He posited, for one, that libraries could position themselves to be the center of civic know-how—especially important in an election year.

From his perch, David Harrison, president, Columbus State Community College, sees a need to look at outcomes such as how a community college “can become a pathway to a BA to the workforce.” To do that, he said, he needs to work with high schools and expand partnerships to develop “deep alignment with postsecondary partners and K-12 partners.”

Brad Mitchell, senior director, Ohio Appalachian Collaborative at Battelle for Kids, took the concept of outcomes a step further. In his work he asks how he can “create rural capital and measures that reverse the brain drain, accelerate brain gain, and amplify wealth gain.” To do so, he thinks in terms of articulating success with regard to cultural changes that support educational attainment by helping kids make stronger choices. So, inputs have progressed from outputs to outcomes and on to mind-sets.

The world of medicine offers lots of great examples of ways to connect powerfully to have impact. Clay Marsh, director of the Center for Critical Care and Respiratory Medicine, Ohio State University Medical Center, charged the audience to look for disruptive solutions. For example, he said, “we have a lot of sick care but not much preventative care.” And, referring to Atul Gawande’s book The Checklist Manifesto, he pointed out the value of simple but groundbreaking controls in a process that can make significant improvements. “All endeavors need checklists,” he argued, “not just medicine.”

Groundbreaking ideas come from around the globe, as illustrated by Garry Golden, lead futurist at FutureThink. He described a number of initiatives that could stimulate out-of-the-box thinking on how to impact library users in fresh ways. In several examples, the Khan Academy, a series of online video tutorials, pulls learning out of formal environments, and the Uni Project, a unit of shelving and seating that can be set up in public spaces, provides a library experience in unusual settings. He also posited that baby boomers could soon become the largest student population. On data, he provocatively argued that we “need descriptive data, but we also need data that is predictive and prescriptive. All service innovation will be oriented toward prescriptive data.”

Golden and CML director Patrick Losinski will further explore the ideas discussed at the summit in an LJ webcast February 24 (www.libraryjournal.com/directorssummit).


Rebecca Miller is Editor-in-Chief, School Library Journal. Francine Fialkoff is Editor-in-Chief and Michael Kelley is Executive Editor, Features & News, LJ

This article was published in Library Journal. Subscribe today and save up to 35% off the regular subscription rate.

Share

Comments

  1. This was a great event with very good (and much needed) information. I was very pleased to be able to be part of a reactor panel this year. I was a big believer in outcomes before attending. But this event really encouraged me to come back and do even more. Most of all, I can’t wait for the next Director’s Summit.