I had a realization a few days ago, about my reading habits and my approach to intellectual work.
I was reading The Bursting of the Academic Library Bubble, by Steven Bell, in Library Journal. That article contains a link to the blog post DIY U: Is There a Bubble in the Higher Education Market?, by Michael Feldstein. Actually I read both of these pieces on my iPhone, in the Read It Later app. Actually I first saw the Bell piece in my RSS reader and saved it to Read It Later; then when I read it in RiL, also saved the Feldstein piece, because RiL allows you to click on a link and save the page to, well, read later. I’m not sure if any of that matters, if the entrance of the material into my personal information space made any difference here. But I thought I’d mention that for context.
Anyway, I realized something as I was reading the Feldstein piece. What I realized, first, was that it was worthy of being a journal publication, in a venue like, for example, the The Journal of Higher Education. But, notably, it is not a journal pub, it’s a blog post. Then I very quickly realized that I didn’t care that it’s not a journal pub. Journal pubs go through peer review; that’s what supposedly guarantees the quality of the work. But it was a good piece of writing, and I realized that I wouldn’t feel any different about its quality if it were a journal pub. I don’t know Michael Feldstein from Adam, I don’t know what he’s done previously, and I don’t know if his methodology is sound. But his blog post was linked to by Steven Bell, who, while I don’t always agree with him, I know his writing and I respect his opinions and I’ve found his methodology to be sound. In other words, I trust Feldstein because I trust Bell. A link from Bell is, for me, as good as having gone through the peer review process. Indeed, a link from Bell is having gone through the peer review process, though the review was only done by one peer, and the criteria were probably different from the criteria that would be used in reviewing a manuscript for a journal. That, combined with my own internal gauge of the quality of writing, made a blog post as good as a journal pub, for me.
So there were two things going on here. First, trust. I trust Feldstein because I trust Bell. If I didn’t know who Bell was, I probably wouldn’t trust his writing or the writing of those he links to. So there’s the issue of developing trusted sources. Second, my personal knowledge. I know what decent scholarship looks like, because I read a lot of it. Plus, even if I don’t have the Econ background to fully interpret Feldstein’s data and analyses, I have enough Econ to make enough sense of it and at least get a grip on its face validity.
I’m not really prepared to make any kind of sweeping general statements, based on this experience. I haven’t thought it through, for one thing, and anyway one data point does not a trend make. But look… I tell my undergrads that peer review is a crutch: it’s a social mechanism that ensures quality scholarship so that you can trust what you’re reading, even if you don’t have the background to interpret it yourself. Peer review breaks down sometimes, of course. But so does any trust system. I guess if I were to make a sweeping general statement, it would be this: What is peer review for? It’s a vetting process. These days, though, there are all kinds of cracks showing in the peer review process, and all kinds of modifications — not to say alternatives — being tried. So perhaps we just need to acknowledge that that vetting process can happen in ways other than peer review, and figure out how to build social mechanisms that have the status of peer review and comparable trust mechanisms built into them. Not a very satisfying sweeping general statement, I suppose.