Assessing Our Assessment
Users are at the center of Library 2.0, and assessment is the key to finding out what they want. If users are going to do all the things predicted by the Library 2.0 world view - participate in the creation and use of interactive, collaborative library services - it follows that we need to know what is useful to them.
My working definition of assessment is the gathering of user feedback or data and taking direct follow-up action, if warranted, as a result. Emphasis mine!
At the University at Albany Libraries, how are we doing on this score? Here is a selected list of the kinds of assessment we've been doing, with brief commentary.
- OPAC: In the 20-odd years of its existence, we have just begun usability studies.
- Public Web site: In the ten years of its existence, we've done two online surveys and a couple of surveys of grad students in the Information Science program. A handful of small follow-up actions were taken a few years ago.
- UNL 205: Instructors have been distributing course surveys. Honest question: how has this driven change? I suspect that it has, but I'm out of the loop.
- LibQUAL+: We have run the survey twice. The results of the second survey aren't available, at least on our Web site. The results of the first survey especially highlighted needs of graduate students, and there has been follow-up.
- Focus groups: We did some focus groups a few years ago. I'm unaware of the follow-up activities.
- Miscellaneous student discussions or surveys on specific topics, e.g., a libray coffee shop. Our Dean has also been meeting with student groups.
- Statistics: We've been gathering a hefty amount of stats annually for reporting purposes. We have numbers galore about the use of our various resources and services, and have been running a Web analytics program since 2000. This is something of a grey area because we don't consistently use these numbers for assessment but rather for reporting.
- Campus-wide surveys have put us in a positive light.
What does this list tell us? We should be commended for what we do, definitely. All of this reflects good-faith efforts. I suspect that these activities are not dissimilar to what many of our peer libraries do.
However - in my view - we don't yet have a culture of assessment that drives the nature and kind of services that we offer. Many of these activities are sporadic, or after-the-fact activities, and we don't routinely use them as guidelines for change. Collaborating with users as we bring up or change services is not a fundamental part of our way of doing things. I'll ask just one question to prove my point: how were users integral to the planning process for building the Science Library?
Please note that this is not a denial of our responsiveness to the user feedback reaching us on a day-to-day basis through informal means. We take user comments seriously. We all know that.
In a follow-up post, I'll attempt to come up with a few ideas that reflect a Library 2.0 way of assessment.

Comments
You are right to ask why we do things in academic libraries without really trying to find out whether or not they are things our users really need or want in the first place. Good examples include library blogs and virtual reference/IM. I'm not saying these are bad technologies, but it depends on why and how they were implemented in the first place. Some of the things you've done, like LibQual, are more evaluation than user needs assessment. In an ADDIE process they are the bookends of the design process. I would suggest you further explore the whole area of design thinking. We are exploring this in greater detail through our work in the Blended Librarians Community (http://blendedlibrarian.org). I've also written some posts about this in the past at ACRLog. See:
http://acrlblog.org/2006/03/29/the-more-we-know-the-better-we-can-do/
http://acrlblog.org/2006/02/27/ipods-and-pencils-it%e2%80%99s-the-user-experience-age-and-we%e2%80%99re-not-ready/
http://acrlblog.org/2006/02/20/the-ratcheting-up-of-technology/
Good luck with your blog.
Posted by: steven bell | September 19, 2006 03:02 PM
Evaluation, e.g., LibQUAL+, as an assessment tool may be one step removed from the goal of user needs assessment, but the former can serve as a door opening to the latter, as it has for us in identifying grad students as a group in need of outreach.
The University of Rochester work is fascinating, and I'm hoping that other academic libraries can take away lessons from this work. This kind of innovative project inspires a thought I've often had that every library can't engage in every activitiy that would make it "great." How can the rest of us gain the most benefit from what the few are doing? The question haunts me.
Posted by: Laura Cohen | September 20, 2006 08:05 AM