June 23, 2017

LJ Index 2015: Understanding Star Status Shifts

A natural assumption upon learning that a library won Stars for the first time, won more or fewer Stars, or lost Star status is that that library’s per capita statistics for visits, circulation, public Internet terminal use, or program attendance must have changed dramatically. However, there are three sets of factors that can affect a library’s Star status, and two of them can apply even when there is no significant change in a library’s own statistics.

Peers join, peers leave

Star Library ratings are based on scores on the LJ Index of Public Library Service. To be eligible to receive an LJ Index score, every library must meet three conditions: 1) have a legal service area population of at least 1,000 people, 2) spend at least $10,000 annually in total operating expenditures, and 3) report all of the current LJ Index statistics. Each year, there are libraries that did not meet these three criteria the previous year but now do. Likewise, there are libraries that met the three criteria previously but no longer do. In addition, each year, some libraries move up and down among spending peer groups, changing the basis of comparison for all of the libraries in both the new and former groups. Those changes affect the composition of the spending peer group to which each library’s data is compared.

Peer stats change

A second explanation for how a library’s Star status can change—even if its own numbers don’t—is changes in the data for other libraries to which it is being compared. The LJ Index scores libraries based on how their data compare to the averages on the same statistics for spending peers. If one’s peer libraries report higher or lower numbers than they did the previous year, the impact on a library’s Star status can be substantial.

Change Begins at home

The third explanation for how a library’s Star status can change is the one we first mentioned: when its own reported statistics change dramatically. Each year, libraries can change their service areas, introduce service improvements, and improve how they measure services.

Outliers & anomalies

It is also possible, however, that the Star Library fortunes of some libraries—and their peers—can be affected in problematic ways. A truism among the federal Public Library Survey’s State Data Coordinators is that statistical inconsistencies often result from key staffing changes—anyone from the director to the line staff member who actually counts something.

Another circumstance that introduces a risk of anomalous data is the introduction of new data elements or new ways of counting them. Each year, for most of the statistics in most of the nine spending peer groups, there are “outliers”—usually reporting statistics that are incredibly high compared to those of the next few libraries.

These statistics are part of IMLS’s final Public Library Survey database, thus they were vetted by IMLS’s contractor (for this data set, the U.S. Census Bureau) and a state library agency and confirmed (when questioned) by a local library. Nonetheless, one can be forgiven for looking at at least one of the four per capita statistics that earned a particular library Star status and thinking “this doesn’t pass the giggle test.” Perhaps an outlandish-looking statistic in fact does have a legitimate explanation; perhaps it doesn’t. Either way, such reports affect the Star fortunes of not only the reporting library but of all libraries in their spending peer group.

» Next page: “Find Your Library”

This article was published in Library Journal. Subscribe today and save up to 35% off the regular subscription rate.

Keith Curry Lance & Ray Lyons About Keith Curry Lance & Ray Lyons

Keith Curry Lance (keithlance@comcast.net) is an independent consultant based in suburban Denver. He also consults with the Colorado-based RSL Research Group. In both capacities, he conducts research on libraries of all types for state library agencies, state library associations, and other library-related organizations. For more information, visit http://www.KeithCurryLance.com.
Ray Lyons (raylyons@gmail.com) is an independent consultant and statistical programmer in Cleveland. His articles on library statistics and assessment have also appeared in Public Library Quarterly, Public Libraries, and Evidence Based Library and Information Practice. He blogs on library statistics and assessment at libperformance.com.

Share