One can only wonder at what passes for “research” at some library schools these days. If you’re curious, check out the “Dominican Study” on Public Library Summer Reading Programs: Close the Reading Gap.
Three professors at Dominican (and a whole bunch of people across the country) spent three years and almost half a million dollars (almost $300,000 from an IMLS grant—your tax dollars at work!) to show that kids who like to read have better reading scores than those who don’t.
The most significant thing we should notice about the study is that it appears on Dominican’s website rather than in a peer-reviewed journal. Peer review would help this study, so I’m providing some.
What the study seems to show, if you read only the executive summary, is that public library summer reading programs have a positive effect on children’s reading ability.
The study even claims it has shown this, based on a final study of 149 students. Consider these three quotes from the study, though.
This Dominican University study provides a rigorous evaluation of the impact of public library summer reading programs on summer reading loss through the examination of third-grade students from large and small communities in rural, urban, and suburban areas, paying particular attention to those students from low-income families.
Another lesson learned from this study was the severe attrition that resulted. We began the study with a participant pool of more than 800 rising third grade students at 11 sites. We received 367 (45%) signed consent forms from parents. At the end of the study we were able to match only 149 (40%) students to the outcomes of interest. In a single setting we could ensure more control and effort could be devoted to participant recruitment and instrument completion.
While not definitive in addressing the additive effect of summer library reading programs, this study has been helpful in demonstrating the need for more rigorously controlled research
In these three quotes we can trace a path from hyperbole to honesty. It “provides a rigorous evaluation,” we’re told near the beginning. Rigorous, indeed! We see from the middle quote that the study was never going to be especially large and that by the end it studied 149 students, and this from eleven participating schools across seven states. What exactly could a study with this tiny, uncontrolled, self-selected sample actually show us? Is this really statistically valid at all? I don’t think so.
And finally we’re told that three years and half a million dollars doesn’t tell us anything at all about summer reading programs. Thanks for wasting my tax dollars!
Even if we were to accept the validity of this study (which we shouldn’t), the claims that public library summer reading programs improve student reading ability are still dubious. The literature search clearly shows that it’s low-income students who need the most help and have the least access to books. The executive summary calls for more summer reading programs in low-income areas. The study claims that summer reading programs themselves are what make a difference in reading achievement.
However, the study doesn’t address other possible and just as likely interpretations of its own claims, especially these:
Students who participated in the public library summer reading program … were at a higher socioeconomic level than the group of students who did not participate.
Families of students who participated in the public library summer reading program had more books in their homes than those families of students not participating.
Students enrolled in the public library summer reading program reported that they like to read books, like to go to the library, and picked their own books to read.
Am I the only one seeing a pattern here? Students from higher socioeconomic levels and with more books in their home (thus with presumably richer and better educated parents) read better than students with poorer and less educated parents and no access to books. What a surprise!
Oh, and students who go to the library for reading programs like to read books and go to the library. My goodness, this is indeed intriguing!
And they like to pick their own books to read. Hmmm. You know which children like to pick their own books to read? Children who read a lot and have actual reading tastes.
We also shouldn’t forget that the students participating had parents who were more likely to sign and return consent forms. That in itself could be a significant factor, as it provides evidence of organization and school participation on the parents’ part, which itself probably affects student learning.
Thus, it at least seems possible that the summer reading program had no effect whatsoever on reading scores. The kids participating were self-selected from the group of children who already read more and had more money and access to books, so of course they have higher reading scores. They participated in summer reading programs because they like to read already. However, the “study” doesn’t address this inconvenient alternate interpretation of the scanty evidence.
The study recommends making reading material more available to poor students. That would be great, but we can make that recommendation without spending three years and half a million dollars.
The study also recommends: “Recognizing that public libraries play a significant role in helping to close the achievement gap in school performance.”
However, it doesn’t provide us with the evidence necessary to support that recognition. At its best, it shows that without public libraries, those poorer students who have no access to books at home cannot possibly keep up their summer reading without access to public libraries, and thus will continue to fall behind in their reading ability. But we already knew that!
It does NOT show that the students who most need summer reading programs are the ones participating in them. On the contrary, it suggests that the ones who need them least are the ones who participate the most. Again, hardly surprising. It shows only that kids who read more read better, and we didn’t need a study to prove that.
What’s sad is that this “study” is already being touted as a defense of summer reading programs. Someone posted it to LIS News and commented that “It’s a compelling, evidence-based study that all librarians should memorize the key points from!” Guess what? No, it isn’t!
More importantly, this isn’t the way to defend summer reading programs. We already know that children who read more read better. We already know that children whose parents have more money and education have more access to books and perform better in school than children with poorer and less educated parents.
The argument to defend summer reading programs can’t be more shoddy social “scientific” studies. It’s a political argument, perhaps ultimately a moral argument.
IF we want poor children to succeed in school, thus becoming successful human beings and engaged citizens, then we MUST offer them access to reading materials and we have to do our best to convince their uneducated parents of the benefits of participating in programs designed to get children to love reading.
If, on the other hand, we as a society don’t care about increasing gaps in education, income, and quality of life between the rich and the poor, and we don’t care whether everyone should have at least the opportunity to succeed, then no amount of studies like this one (or, hopefully, statistically valid and rigorously analyzed studies) will make any difference at all. We know kids who read more read better and kids without access to books can’t read them.
So, my peer review evaluation is that a) this study isn’t scientifically valid; b) that if it were scientifically valid it still doesn’t support the claims and recommendations it makes; c) that even if it did support the claims it makes, it doesn’t provide an argument for supporting summer reading programs, because that rationale would have to be political, not statistical. If this is the best we can do, it’s no wonder libraries are in trouble.