Annoyed Librarian
Search LibraryJournal.com ....
Subscribe to LJ
Inside Annoyed Librarian

On “Research” and Summer Reading Programs

One can only wonder at what passes for “research” at some library schools these days. If you’re curious, check out the “Dominican Study” on Public Library Summer Reading Programs: Close the Reading Gap.

Three professors at Dominican (and a whole bunch of people across the country) spent three years and almost half a million dollars (almost $300,000 from an IMLS grant—your tax dollars at work!) to show that kids who like to read have better reading scores than those who don’t.

The most significant thing we should notice about the study is that it appears on Dominican’s website rather than in a peer-reviewed journal. Peer review would help this study, so I’m providing some.

What the study seems to show, if you read only the executive summary, is that public library summer reading programs have a positive effect on children’s reading ability.

The study even claims it has shown this, based on a final study of 149 students. Consider these three quotes from the study, though.

This Dominican University study provides a rigorous evaluation of the impact of public library summer reading programs on summer reading loss through the examination of third-grade students from large and small communities in rural, urban, and suburban areas, paying particular attention to those students from low-income families.

Another lesson learned from this study was the severe attrition that resulted. We began the study with a participant pool of more than 800 rising third grade students at 11 sites. We received 367 (45%) signed consent forms from parents. At the end of the study we were able to match only 149 (40%) students to the outcomes of interest. In a single setting we could ensure more control and effort could be devoted to participant recruitment and instrument completion.

While not definitive in addressing the additive effect of summer library reading programs, this study has been helpful in demonstrating the need for more rigorously controlled research
studies.

In these three quotes we can trace a path from hyperbole to honesty. It “provides a rigorous evaluation,” we’re told near the beginning. Rigorous, indeed! We see from the middle quote that the study was never going to be especially large and that by the end it studied 149 students, and this from eleven participating schools across seven states. What exactly could a study with this tiny, uncontrolled, self-selected sample actually show us? Is this really statistically valid at all? I don’t think so.

And finally we’re told that three years and half a million dollars doesn’t tell us anything at all about summer reading programs. Thanks for wasting my tax dollars!

Even if we were to accept the validity of this study (which we shouldn’t), the claims that public library summer reading programs improve student reading ability are still dubious. The literature search clearly shows that it’s low-income students who need the most help and have the least access to books. The executive summary calls for more summer reading programs in low-income areas. The study claims that summer reading programs themselves are what make a difference in reading achievement.

However, the study doesn’t address other possible and just as likely interpretations of its own claims, especially these:

Students who participated in the public library summer reading program … were at a higher socioeconomic level than the group of students who did not participate.

Families of students who participated in the public library summer reading program had more books in their homes than those families of students not participating.

Students enrolled in the public library summer reading program reported that they like to read books, like to go to the library, and picked their own books to read.

Am I the only one seeing a pattern here? Students from higher socioeconomic levels and with more books in their home (thus with presumably richer and better educated parents) read better than students with poorer and less educated parents and no access to books. What a surprise!

Oh, and students who go to the library for reading programs like to read books and go to the library. My goodness, this is indeed intriguing!

And they like to pick their own books to read. Hmmm. You know which children like to pick their own books to read? Children who read a lot and have actual reading tastes.

We also shouldn’t forget that the students participating had parents who were more likely to sign and return consent forms. That in itself could be a significant factor, as it provides evidence of organization and school participation on the parents’ part, which itself probably affects student learning.

Thus, it at least seems possible that the summer reading program had no effect whatsoever on reading scores. The kids participating were self-selected from the group of children who already read more and had more money and access to books, so of course they have higher reading scores. They participated in summer reading programs because they like to read already. However, the “study” doesn’t address this inconvenient alternate interpretation of the scanty evidence.

The study recommends making reading material more available to poor students. That would be great, but we can make that recommendation without spending three years and half a million dollars.

The study also recommends: “Recognizing that public libraries play a significant role in helping to close the achievement gap in school performance.”

However, it doesn’t provide us with the evidence necessary to support that recognition. At its best, it shows that without public libraries, those poorer students who have no access to books at home cannot possibly keep up their summer reading without access to public libraries, and thus will continue to fall behind in their reading ability. But we already knew that!

It does NOT show that the students who most need summer reading programs are the ones participating in them. On the contrary, it suggests that the ones who need them least are the ones who participate the most. Again, hardly surprising. It shows only that kids who read more read better, and we didn’t need a study to prove that.

What’s sad is that this “study” is already being touted as a defense of summer reading programs. Someone posted it to LIS News and commented that “It’s a compelling, evidence-based study that all librarians should memorize the key points from!” Guess what? No, it isn’t!

More importantly, this isn’t the way to defend summer reading programs. We already know that children who read more read better. We already know that children whose parents have more money and education have more access to books and perform better in school than children with poorer and less educated parents.

The argument to defend summer reading programs can’t be more shoddy social “scientific” studies. It’s a political argument, perhaps ultimately a moral argument.

IF we want poor children to succeed in school, thus becoming successful human beings and engaged citizens, then we MUST offer them access to reading materials and we have to do our best to convince their uneducated parents of the benefits of participating in programs designed to get children to love reading.

If, on the other hand, we as a society don’t care about increasing gaps in education, income, and quality of life between the rich and the poor, and we don’t care whether everyone should have at least the opportunity to succeed, then no amount of studies like this one (or, hopefully, statistically valid and rigorously analyzed studies) will make any difference at all. We know kids who read more read better and kids without access to books can’t read them.

So, my peer review evaluation is that a) this study isn’t scientifically valid; b) that if it were scientifically valid it still doesn’t support the claims and recommendations it makes;  c) that even if it did support the claims it makes, it doesn’t provide an argument for supporting summer reading programs, because that rationale would have to be political, not statistical. If this is the best we can do, it’s no wonder libraries are in trouble.

PrintFriendlyEmailTwitterLinkedInGoogle+FacebookShare

Comments

  1. Michael says:

    When I was an LIS student, my mind was blown when I discovered that there were professors who do this sort of “research” full time. How can any reasonably intelligent person devote their entire career to so much nonsense? Especially when you work at a university where real and valuable research is actually happening.

  2. unstricken says:

    this “study” is ludicrous. i’m sorry to say that my worthless MLIS is from Dominican (for reasons of geographic necessity), and this study typifies the junior-college ability-level of most of the profs I encountered in the program. thank you, AL, for giving it a semblance of peer review–about as much as it deserves.

  3. Real Librarian says:

    Sadly, if you look at library closings/shorting hours/reducing staff/etc you see that it is being done in lower income neighborhoods. More affluent neighborhoods would be up in arms about the library closing. The soccer moms wouldn’t have a place to send junior while she has and affair with the coach.

    Working class people are too busy just trying to keep their head above water working two or three jobs just to make the rent. They don’t have time to go to meetings or influence local pols.

    The rich get richer and the poor increase in number.

    Welcome to the third world.

  4. Michael says:

    I find it hilarious that someone who just whines and complains about others—and hides behind a psuedonym to hide their identity—has the freaking nerve to criticize people who at least are willing to publicly acknowledge and “publish” their work. At least they’re attempting to quantify something. What do you do except complain about others with your holier-than-thou attitude? How about you publish your name so you can also receive some real-world peer review? You’re a coward. Grow up.

    [AL: I find it hilarious that, as usual, I focus on ideas while my opponents focus on personalities. I also find it amusing that you're not bothering to defend the study, but instead are engaging in an ad hominem attack. Very brave and adult of you!]

  5. nerdylibrarian says:

    While I’m less than impressed by the account of this study, I do think more research needs to be done in the public and special library sectors.

    Business academics do research in areas outside of their academic setting, why don’t libray academics?

    While I was an LIS student with an interest in working in public or special libraries, I found it very difficult to write lit reviews covering any type of library other than a university library because all the research was being done in academic libraries by tenure track librarians who needed to publish.

    When I tried to investigate a topic in a public library setting, I would be lucky to come up with 3 professional-”we tried this and we think it worked”- articles and a theory paper by Kuhlthau (or similar).

    I think it’s very easy for library academics to stick to the academic library setting and forget that there are libraries out there that cater to different audiences but are just as legitimate and just as likely to benefit from research as academic libraries are.

  6. Peter says:

    So it’s 103 pages long – looks impressive. The appendices start on page 53. The Executive Summary starts on page 6. There are lots of title pages for sections and full pages given to quotes in very large type. I counted only 41 pages of actual text. With reaaaaaally wide margins. Nevermind the poor methodology, I don’t think that I would have had the nerve to hand something like this in when I was in library school. It’s like they used every student trick to make the work look more substantial than it really is.

  7. NotMarianTheLibrarian says:

    Gosh … I could have sworn someone came to the same conclusions in the early 20th century!

    Having worked for a large urban public library in the South, what I saw was this: branches in wealthier parts of town did booming business with high materials circulation counts, high gate counts, etc. The branches in the poorer parts of town had none of that … except gate count in summer. Poor people needed the AC the branch provided.

    I had the misfortune to be a librarian who could “float.” Those weeks spent in the poor parts of town were God-awful – nine hours flowed like molasses. As a library user my entire life I wondered why the heck our tax dollars were being wasted in branches that got little or no use. Getting out of that system and that job was one of the best days of my life.

  8. anonymous says:

    I just wanted to point out that a study that has no findings or has negative findings is NOT scientifically invalid – that is the way science works.

    Not that I disagree with your basic point that this particular study was a waste of taxpayer dollars. It was too small a sample and appears that they self-selected the results.

    Actually, finding out that summer reading programs do anything or not is a good idea. Just don’t let librarians, or anyone else in education, conduct the study.

  9. Real Librarian says:

    Peer reviewed?

    So some cat loving, bun-wearing, spinster reading the paper and giving it her stamp of approval would make this “research” better?

    And I thought the library profession didn’t have a sense of humor.

  10. liberry says:

    Thank you AL for pointing out the sloppy methodology here. It is similar to the studies trying to correlate the presence of a school library media specialist to increased reading scores. I find it tragic that these studies were done so poorly they hurt more than helped.

  11. k.sol says:

    anonymous — while it’s not invalid to have a study with no results, it’s pretty questionable to make policy recommendations based on results that were not shown. I think summer reading is a great thing. I just don’t think this study proves it.

  12. special librarian says:

    Why only school or academic library?what about special libraries ?

  13. Real Librarian says:

    “Why only school or academic library?what about special libraries ?”

    Special librarians are the ones who arrive at conferences in the short buses.

  14. Patty B says:

    My thoughts after reading this are curiosity about the cynical tone. I’m mystified by why you’re put off as much as you are by this so-called study.

    Here’s a quote: “Even if we were to accept the validity of this study (which we shouldn’t), the claims that public library summer reading programs improve student reading ability are still dubious.”

    It’s one thing to criticize the study — something I halfheartedly agree with. But it’s another to go so far as to almost claim that summer reading programs do NOT improve student reading. It is well supported by plenty of other studies — granted, not by this study, but it’s not a farfetched concept. You seem angry.

  15. anonymous says:

    AL and the researchers both make the same fundamental mistake — trying to compare achievement scores between low SES and high SES readers and correlating the variance with a particular reading intervention. The real story is that summer reading programs may very well give high SES students additional advantage, and result in higher achievement scores than high SES students might otherwise achieve. This in itself would be valuable; we need to recognize that while we want to do what we can to pull up achievement scores for low achievers, we also want to provide every advantage to high achievers to assure they reach their potential. Think of summer reading programs not as equalizers but as part of an overall approach to assure that the gifted and talented are nurtured and not abandoned on the mantle of no child left behind.

  16. Ann Perrigo says:

    I would like to see a definitive study with the thesis: Children who read during the summer months are more likely to maintain their reading level than children who do not read during the summer. In my mind summer reading programs are a means to an end–keep those kids reading and they will be more prepared for school in the fall. Would this research have greater potential? Why did it need to be tied to summer reading programs?

  17. sally sue says:

    The study had good intentions. The biggest thing that surprised me is that it showed that the kids participating in summer reading programs have books in the home and a higher social-economic level than those not participating. Summer reading is supposed to target low income-low home literacy kids. I wish we were doing that, but that wasn’t true, at least not for the kids in this study.