One of the most effective ways to test and evaluate a new program or service is to conduct a pilot project, but how do you scale up from there? How do you translate the small successes into sustainable, permanent additions to your library?
The example of the Clinton-Macomb Public Library (CMPL), near Detroit, can shed some light on the subject. In 2015, CMPL was selected as one of some 60 communities to participate in the White House’s ConnectED Library Challenge, the goal of which was to connect every public school student in town to library resources, particularly ebooks.
Jennifer Koerber led the “Coding as a Practical Library Program” session during Library Journal‘s Spring 16′ coding online course. Featuring up-to-the minute content and expertise on launching a coding program in your library and promoting digital literacy in your community, the next Coding Program Workshop starts February 28, 2017.
CMPL interpreted the challenge ambitiously, deciding that its goal was not just to provide electronic resource access to every K-12 student in the service area but also to get students to use the library’s resources. To start with, CMPL worked with a single school district in the first pilot, issuing virtual library cards to the 10,000 students who didn’t have a physical card; within nine months, 1,000 of those students had upgraded to a full-service card. Extending to a second school district added another 10,700 students, and a third increased the total number by 3,000 more. Students and teachers were using online resources. For example, visits to Biography in Context rose from 27 to 803 in one year. Yet CMPL doesn’t plan to stop there. According to Larry Neal, CMPL director, “We’ve made it a goal to expand the program to our entire cooperative of 22 libraries, serving 150,000 students (ten percent of Michigan’s total).”
To those looking to grow from a pilot to a solution at scale, Neal offers a short list of advice:
- “Always focus on the needs of the end user,” which helps when obstacles and setbacks occur and keeps you solution-oriented.
- “A pilot project should be a testing lab that is flexible and organic and offers the opportunity to learn, experiment, adapt, succeed, and fail.”
- “Keep solid documentation about the decisions made from the start and the reasoning behind each one. Review and update as the project grows; there may be opportunities to improve on the first implementation, or try a different approach to compare.”
- “Think about sustainability and scale from the start, taking views of the project from ‘being in the weeds’ to ‘30,000 feet,’ but don’t overwhelm with complexity.”
- “Don’t be afraid of success. If you are doing something great, the momentum will build, as will the opportunities for additional partnerships and collaborations.”
Planning pilot projects
Whether your program will serve 150,000 people or 50, a successful execution starts with a solid pilot: focused, supportable, immediate, and measurable. Without trying to change everything at once, create a pilot that tests the new service you’re offering—“the real value of your pilot study is what it tells you about your method,” says Nick Grove, digital outreach librarian for Idaho’s Meridian Library District’s (MLD) unBound technology branch and a 2016 LJ Mover & Shaker. He also stresses preplanning: “Our project went from conception to implementation in eight months, in part because we had been working on various ideas for a couple of years but had no means to act on them.” When a 2102 statewide initiative encouraged teen Maker spaces in libraries, MLD staff knew they didn’t have a dedicated space to use. “Instead, we pulled equipment out of storage for programs and outreach,” says Grove, whose weekly Make It program grew from ten to 30 teens/tweens. Grove used that success and the lessons learned about the types of technology and assistance the community wanted to pitch a whole separate technology library branch to the library board and the city’s urban renewal agency. Opened in October 2015, unBound’s door counts have grown by 75 percent over the first year and are still climbing.
Carnegie Library of Pittsburgh (CLP) children’s services coordinator Mary Beth Parks believes that a sound project plan is the key to success. “The advice I have for libraries beginning a pilot project is to create a solid foundation,” she says. Such a plan would include the following components:
- How does the project relate to the library’s strategic plan?
- How does the project meet users’ needs?
- Is the project community driven?
- What data supports the need?
- What are the goals, audience, objectives, outcomes, actions, evaluation?
- What partners will be working with you?
In 2013, CLP children’s librarians began a Children’s Technology Pilot that reached 11,500 youngsters and caregivers over eight months at five pilot locations, plus pilot-related activities elsewhere. This substantial success led to dedicated iPads for children’s programming, an expanded “All Hands on Tech” technology skills series at all 19 CLP facilities, and support for adult and teen librarians in their own technology/Maker space efforts.
Choose your rollout
There are three general ways to implement a new service: simultaneous adoption systemwide; a phased approach by location or by department; or a parallel approach, using both old and new services at the same time. Pilot projects are often part of a phased approach, starting small and then gradually adding service areas until the program is fully live.
At the Fresno City Public Library (FCPL), CA, Michelle Gordon and the WithOut Walls (WoW!) mobile services department wanted to improve on bookmobile service to seniors by taking the library’s resources inside the senior facilities. Using mobile circulation technology and a dedicated collection culled from extras at other FCPL locations, the Senior Pop-Up Library began with a pilot of six existing bookmobile locations. It scaled up initially to ten sites (adding several stops original to the program) and again to 15. Over a year, the program more than doubled in size with the same three staff members who began it, using what was learned from the early iterations to streamline the addition of new sites. Since 2015, the Senior Pop-Up Library has doubled the number of programs to seniors countywide with quadruple the attendance over the previous year; despite its resources being available to residents for only a few days each month, its monthly circulation exceeds a 24-7 self-service kiosk’s numbers.
Gordon’s advice for scaling up is to focus on “flexibility, statistics, and support. It’s all about being flexible—we had never done anything like it before, so whenever we had an issue…we had to figure out how to solve problems every time a new one occurred.” She and her team kept excellent statistics (“it’s the only way to learn if the program is working, reaching the right people, and is worth the library’s time and resources”) and had the full support of the administration. Adds Gordon, “The pop-ups are very rewarding and have become a staple service of FCPL.”
Another advantage to the pilot/phased approach is the opportunity to accommodate the varying resources of different locations or departments. In 2008, the Public Library of Cincinnati and Hamilton County (PLCHC) began a slow transition to a “roving reference”–style mobile public service model. After spending several years changing infrastructure and culture to support the new model, PLCHC started removing reference desks and getting staff into the stacks in late 2013. Over the past three years, the library has converted two of six main departments and six of 40 branches to mobile-first public service (five newly built/renovated branches were planned that way from the get-go). Paula Brehm-Heeger, strategic initiatives director for PLCHC, says that multiple pilots and a phased rollout was the only path to success, given the technology and building changes needed. “It would have been very difficult (and not feasible from a budgeting perspective) to implement this new service model simultaneously in all locations,” she says. In addition, “changes were incorporated as the model was [phased in]…and the original branches are being retrofitted to integrate those updates.”
CMPL’s Neal worked with extremely different logistical situations for the virtual library card project. “[One school system worked with] four autonomous libraries on three different automation systems, ranging in size from a single full-time staff member to 60 FTEs,” he says. “We had to have a lot of up-front discussions to figure out how to make things as easy as possible…and to offer consistent services despite the great differences in library resources.”
As you’re scaling up, don’t rush: think in terms of percentages of change over time, and don’t fixate on achieving big numbers. Fresno’s stats aren’t huge, but they validated an approach to outreach that’s being expanded beyond the initial service population. Meridian’s Grove turned ten teens attending the first Make It programs into an entirely new branch and service model. The Children’s Tech Pilot in Pittsburgh distributed a few iPads to five locations; technology and programming are now in place systemwide. Small well-run pilots can have enormous impact, especially when the message is spread widely.
The more staff you can include in your pilot—whether through direct participation or speaking with participating staff—the more successful your apparatus is likely to be. Nearly all the projects discussed here found ways to include staff not directly involved in the pilot.
Pittsburgh’s Parks suggests that staff “participating in the project can discuss their experiences with staff at other locations. They can also participate in future [project] trainings and information sessions.”
Paula Brehm-Heeger at PLCHC extends this point: “[My] main advice is to communicate as much as possible about the project to staff not involved in the pilot. While we communicated the details of this model to staff, many were still unsure of how it worked and were misinformed. Shadowing helped staff better understand the service model… [and] reduced some of the fear that comes with change.” As they scaled up, “we prioritized communication from staff members directly involved in developing and working with the model.”
Meridian’s Grove attributes unBound’s success to involved staff. “[Staff] had gained experience [with the technology] through the pilot. If we had tried to switch staff members to someone not involved in the pilot, I don’t think it would have been as successful,” he contends.
Depending on the size of your project and the partners involved, you might need to extend this approach beyond the library’s employees. Says Neal of CMPL, “One bigger challenge was implementing our school outreach librarian team…. Our relationship with the schools has always been cordial but distant,” but the ConnectED partnership was a chance to improve. The results have been overwhelming. “We started out by saying ‘yes’ to everything and are now working to organize what we offer better to keep things manageable and sustainable and yet meet the needs and requests of educators.”
Change as you go
Successful pilots can lead to great implementations, and they can also lead to new opportunities that reach beyond the original scope of the project. When the Fresno Senior Pop-Up Library pilot began, WoW! staff only had an overloaded Prius wagon to transport materials. A presentation on the success of the pilot—already at capacity—convinced the library board to buy a customized cargo van. “The new van allows us to offer the Pop-Up Library at any event…so while we’ve expanded the number of senior facilities that we visit, we have also expanded pop-up services in general,” says Gordon.
During the planning stage for unBound, Grove saw change everywhere. “Be aware of current trends that have changed between the time of the pilot and the scaling up…. [We also] kept in mind that our target audience had changed (from teens to adults) and…our location had changed [from existing library space to a new branch]. Not ideal conditions for scaling up, but we made the best of it.” He sums up his experience using an online meme: “ ‘If the plan doesn’t work, change the plan but never the goal.’ Try to anticipate what things will look like if they fail, proceed as planned, or exceed expectations. Be bold and flexible but also reasonable and consistent.”
Know your limits
As Neal says, “We try to remain nimble and iterate on the fly, but the larger the program scales, the more challenge there will be.”
As a result, one way to address the challenges of scaling up is to know and enforce limits on what your library can do.
As Gordon expanded the Senior Pop-Up Library in Fresno, she discovered the need to say no. After they chose requirements for Pop-Up stops (sites approximately three miles from a library or with a majority of skilled nursing/assisted living residents), some of the bookmobile stops they’d hoped to replace with Pop-Up stops didn’t meet the criteria. They backed out of serving those locations, leaving them to the more effective bookmobile or branch services already in place.
Courtney Saldana, a 2016 LJ Mover & Shaker and creator of the Skills for Teen Parenting (STeP) program at the Ontario City Library, CA, asked a related question: “What are you not willing to give up? Any scaling up process is going to have hiccups, but if you can hold onto the passion that inspired you, you can be the inspiration to get others on board.” For her, having a teen librarian running STeP was essential; as the California State Library expanded the program, Saldana has found that to be true, with the most successful program run by Ady Huertas, former teen librarian, at the San Diego Public Library.
Sometimes, there’s a need to say no at the very beginning, and you must be able to walk away from the work you’ve already done. In 2014, Salt Lake City Public Library (SLCPL) was approached by a group of community members interested in funding a two-year pilot of 24-7 service at the Main Library. Deborah Ehrman, then acting director SLCPL, explains, “We were considering keeping two floors of our six-floor building open during the overnight hours to reduce the cost of staffing the full library…. The goal was to make full library services available to everyone, regardless of the time they were able to travel to our location.”
A two-year pilot would allow the library to observe seasonal trends; give ample time to make adjustments and measure their effectiveness; and have sufficient data to determine whether the service should continue. “We worked with a consultant to perform a needs assessment—a difficult task since no other public libraries were offering 24-7 service for us to examine,” says Ehrman. Leaders met with key stakeholders and conducted a community survey to predict how many people they thought would use the building and the services they would want access to—feedback was evenly split between those in favor and those opposed to the proposed service. “We used the predicted use numbers to develop a staffing and security plan and budget for the service. Ultimately, the library Board of Directors determined the pilot was cost prohibitive and discontinued the inquiry.”
From first plans to final rollout, scaling up a pilot program can be daunting. Solid planning, staff buy-in and involvement, accepting change, and knowing your limits give you the best chances for success.
Pilot Project Planning
Planning, iterating, and scaling up a successful pilot are challenges that are not unique to libraries. Below, find practical resources for further hands-on learning from beyond the field.
The entire pilot project process from a Six Sigma study guide
Tips and recommendations for successfully pilot testing a program from the Office of Adolescent Health of the Department of Health and Human Services
Running and evaluating a records management pilot, from the U.S. National Archives; can be easily used as a template for other library pilot programs
How to evaluate the success of a program
Nine steps to developing a scaling up strategy from the World Health Organization
Lessons learned from start-up pilots