May 23, 2017

LJ Index 2013: The Case for New Outputs

Since at least 1987, public libraries have been collecting and using three per capita output measures: circulation, visits, and program attendance. Since 1989, data on public library circulation and visits has been collected by state library agencies and compiled nationally. Through 2007, this was done by the National Center for Education Statistics (NCES) and, since then, by the federal Institute of Museum and Library Services (IMLS). Beginning in 2001, public libraries began reporting uses of electronic resources, and, in 2007, that data element evolved to become uses of public Internet computers. Finally, in 2005, libraries started to report total program attendance. These four per capita statistics comprise the output measures underlying the LJ Index of Public Library Service, on which the annual Star Library ratings are based.

While several new data elements have been added since 2007, there have been no new output measures. We are glad to report that the state library agencies recently voted to add circulation of electronic materials, starting with the 2013 data collection. In all likelihood, electronic circulation per capita will join the four current LJ Index statistics, starting with the Index scores and Star ratings to be based on 2013 data (i.e., the 2015 article).

So what’s missing?

We believe two output data elements are conspicuously absent from the federal data set: visits to library websites and usage of Wi-Fi access provided by public libraries.

New outputs correlate with LJ Index

To test this belief, we analyzed data for our own state, Colorado, as it already collects both of these newer output measures. Two types of analysis were conducted. First, we carried out a bivariate correlation analysis to examine the relationships between each of our proposed new national output data elements—library website visits (operationalized as library homepage hits, in Colorado) and Wi-Fi access usage—and each of the four current LJ Index outputs—circulation, visits, program attendance, and public Internet computer use. (See Table 1. Note: A correlation coefficient of .00 indicates that there is no relationship, positive or negative, between two variables. A correlation coefficient of 1.00 indicates a perfect relationship, usually only achieved by correlating a variable with itself, as illustrated by the diagonal cells in this table. The statistical significance statistic, .00, indicates that there is less than one chance in 100 that the correlation is spurious.)

What these correlation analysis results indicate is that:

  • • Wi-Fi access usage is highly correlated with three of the four LJ Index variables—circulation (.73), visits (.73), and public Internet computer use (.79)—as well as with library homepage hits (i.e., library website visits, .80).
  • • Library homepage hits (i.e., library website visits) is correlated moderately with two of the four LJ Index variables—program attendance (.65) and circulation (.52).

That these two new electronic output measures are very strongly related to each other as well as being related to half or more of the current LJ Index measures suggests to us that they are likely candidates to join the LJ Index of Public Library Service when national data on them are available.

An expanded LJ Index

To test this belief, we conducted a factor analysis to consider the viability of the two new electronic output measures as additions to a single index alongside the four output measures for which data are already available nationally. (See Table 4. Factor loadings indicate the extent to which individual variables are related to the overall factor they represent collectively.) What these factor analysis results indicate is that:

  • • Both library homepage hits (i.e., library website visits, .84) and Wi-Fi access usage per capita (.78) contribute strongly to a single factor based on public library outputs.
  • • The four LJ Index per capita measures continue to be closely enough related to one another collectively to contribute strongly to a single such factor (based on factor loadings ranging from .78 to .92).

Wi-Fi access and other output measures

Some might wonder why it is so important to begin collecting and compiling national data on Wi-Fi access usage. After all, many public libraries still do not provide it, and many that do might experience great difficulty in capturing data about it.

So, why bother? Because it represents a watershed in the relationship among public libraries, their users, and technology. Prior to a few years ago, most users of public libraries accessed electronic resources in one of two ways—either by visiting a public library and using a public computer or by visiting a library virtually via a computer at home, at school, or at work. However, the near-ubiquity of personal notebook computers, netbooks, tablet computers, and smartphones has released many users from the constraints of limited access to desktop computers in their libraries. Today, an increasing number of users bring their own electronic devices into the library—or, indeed, only as far as the library parking lot. Coupled with remote access to library resources, Wi-Fi access means that, for many—if not most—users, public Internet computers in public libraries are no longer the only game in town.

It is difficult to overestimate how much the revolution in personal, handheld computing is impacting libraries, also creating a new level of information consumption across most age groups that may be benefiting public libraries. This possibility may explain what we saw when comparing Colorado libraries with and without Wi-Fi access usage.

Although only 70 of Colorado’s 111 public library jurisdictions—63.1 percent—reported data for Wi-Fi access usage in 2011, the state provided enough data to consider what a critical service Wi-Fi access in particular is becoming to many public libraries. (See Table 3.) When the 50 libraries reporting Wi-Fi access usage greater than zero were compared with the 20 that reported zero, their averages for per capita circulation, visits, and public Internet computer use differed dramatically. In fact, public libraries with Wi-Fi access reported twofold the per capita circulation (14.0 vs. 7.6), visits (12.9 vs. 6.0), and public Internet computer use (3.0 vs. 1.3) as libraries without Wi-Fi access.

Doable and worth doing

Colorado and a few other states have begun to collect data on library website visits and Wi-Fi access usage. While response rates need to be improved, sufficient numbers of libraries are collecting this data to demonstrate that it is doable. Developing new measures usually requires some trial and error that does not happen until we “just do it,” to borrow a phrase from Nike. Remember, it took a few years for electronic resources use to evolve into public Internet computer use. If no one at state or national levels asks for the data, few at the local level will find any reason to overcome whatever obstacles they face in reporting it.

Some might argue that these measures are premature, because enough libraries still confront difficulties reporting them. About a quarter century of collaborative federal-state experience collecting and disseminating public library data calls this position into question. Whenever the need for new data elements arose, libraries had no incentive to overcome new data collection challenges until they were asked to collect and record a new data item. Only sincere efforts to collect data generate solutions to the data collection challenges some libraries encounter. No data element and no data on it are ever going to be perfect—certainly not from the word go and quite possibly not ever. Sometimes, for the first few years, new data may even be so questionably valid or unreliable that it should not be reported.

Still, hope springs that the federal and state partners in the annual data collection effort will begin to take steps to collect conspicuously missing output data, such as the two data elements examined here, and—if its quality is not acceptable—to analyze whatever data is reported to help to determine the steps necessary to meet acceptable levels. This is precisely what is going on in Colorado and in the other states where public library leaders have had the foresight to keep moving forward, trying to make progress on these pressing data issues.

Available data on library website visits and Wi-Fi access usage from Colorado is sufficiently strongly related to the existing LJ Index measures to indicate that, if national data on them were available, they would most likely pass muster statistically to join the LJ Index. These additions would make it a more comprehensive and realistic reflection of what today’s public libraries actually do. In perilous times for public libraries, we need all the data we can get, to provide the most complete picture possible of all that libraries do for the public.

» Next page: “Spotlights: New and Improved Stars”

This article was published in Library Journal. Subscribe today and save up to 35% off the regular subscription rate.

Keith Curry Lance & Ray Lyons About Keith Curry Lance & Ray Lyons

Keith Curry Lance ( is an independent consultant based in suburban Denver. He also consults with the Colorado-based RSL Research Group. In both capacities, he conducts research on libraries of all types for state library agencies, state library associations, and other library-related organizations. For more information, visit
Ray Lyons ( is an independent consultant and statistical programmer in Cleveland. His articles on library statistics and assessment have also appeared in Public Library Quarterly, Public Libraries, and Evidence Based Library and Information Practice. He blogs on library statistics and assessment at

Claims of “fake news” have vaulted once-dry information literacy into the spotlight. To seize the teachable moment, this online course will offer up-to-date tools and effective tactics to enable patrons to critically assess sources, facts, and context. Competitive team discounts are available. Please call 646-380-0773 to inquire about our discounted team rates.