Library Assessment Conference 08 – Overview

This overview ended up being a synthesis of what I learned. The conference Web site includes presenter’s PowerPoints where available: http://libraryassessment.org/

Assessment defined
Since WMU is currently defining assessment as mainly about student learning outcomes, it makes it hard for libraries to come up with appropriate assessments, as we usually do not teach semester long courses. If that was all there was to assessment, then three quarters of this conference would have been irrelevant and even LibQual would not be considered an assessment tool. (One contentious commentator called it just an evaluation tool.) Assessment is a complex concept, and the goal is a Culture of Assessment, where people automatically assess to improve what they do. One person said that instead of having a Culture of Complaint, turn it into a Culture of Assessment. Even better, someone called it a Culture of Improvement. I understand assessment in libraries as a way of gathering data on our collections, services and space, and the needs and activities of our users to improve the learning, teaching and research experience of the whole Western Michigan University community.

Organizational culture

• Assessment goes hand in hand with strategic planning
• “No action without a plan, no plan without data” Rick Luce
• Most research libraries have an assessment person, sometimes combined with marketing or communications, sometimes just a designated 15% spent on assessment
• Most research libraries have an assessment committee, as do we, though it has not been very active and has focused mostly on LibQual
• Employee satisfaction and input
• Employees are users too and good judges of quality
• Employee satisfaction important in good service
• ClimateQual, a tool developed by UofMD to measure staff perceptions about their libraries
• Interview or survey individual employees
• Focus groups and retreats
• Gather data in a timely manner and pass it to appropriate people for implementation
• One place for data – so all can access – Penn has - DataFarm
• Important to present data visually – see PennLibrary Facts http://metrics.library.upenn.edu/FACTS07.pdf
• Master Blog Communication System
• One blog per group (committee, task force, project)
• Put up agendas before meetings
• Minutes of meetings right after meetings
• Allows comments
• People can set up alerts
• What will the scholar in 2050 expect us to have saved? (Question from Betsy Wilson, UW)
• Need frameworks and models that reflect our values (Stephen Town, Univ. of York, UK
• Rick Luce had many good guidelines on being a successful organization
• Great if can work with consultants – one option is Jim Self (UofVA), Steve Hiller (UofWA), and Martha Kyrillidou (ARL) will come to libraries and help them set up better assessment programs (known as the Jim, Steve and Martha show)

Assessment of Place

• Mine LibQual comments to inform planning of space
• Space consultants can be useful
• Surveys – ask what they do in the library, how often they come, how long they stay, when
• Observation – visit different areas of the libraries and record how users use space – alone, in groups (2-3, 4+), what furniture they are using, if they are using personal or library materials, their laptop or library computers
• Facus groups to get at details of issues found by other methods
• Involve staff in findings and planning
• Identify affordable changes
• U of Chicago redid their wayfinding study – give novice library users 3 books to find and follow them around to see if they do find them – after the first study they redid signage and maps

Information Literacy / Instruction

• The Education Testing Service (ETS) test is now called iSkills
• Sounds like quite a few institutions are using this, but some of the preliminary results did not sound promising
• SAILS mentioned briefly
• Megan Oakleaf (Syracuse) and Lisa Hinchliffe from U of IL Champaign-Urbana did a study of 437 instruction librarians and asked if they assess their instruction sessions, if they have data, and if they have used the data. (228 respondents actually use the data) Some reasons for not doing this, besides lack of time and resources:
• Questions about whether the results actually measured IL
• Lack of knowledge and skills
• No centralized support or commitment to gather this data
• Lack of a conceptual framework
• One school developed a self-assessment tool
• Worked closely with teaching faculty
• Based on ACRL standards
• Students reflect on own research and learning process
• Some moving away from instruction on demand, as develop Info Lit program within the curriculum with faculty
• When Info Lit is imbedded in the general education program, e-portfolio systems (like our iWebfolio) can keep track of papers over the years

Reference

• READ (Reference Effort Assessment Data) Scale – with little effort gives insight into ref
• Six point scale given to each reference question answered
• 1: typical directional question that takes less than a minute
• 6: working with PhD student or faculty over hours or days
• 2-5 in between - have to train and calibrate across those who answer
  
• Keep track of questions on and off the desk
• Include in person (also WRAP), phone, e-mail, chat
• Good for scheduling staff
• Can be used in online system like Desk Tracker
• Cornell did systematic quantitative and qualitative analysis of it’s reference questions
• Transaction type, duration, mode, question content, date, time
• As a result have closed ref desk during summer hours in undergrad study library
• Thought of reference work as research assistant, information central & problem solver
• University of Pennsylvania reference consultation form
• Even more elaborate than at Cornell, but worth looking at for ideas
• Heard a few instances of reduced reference collections
• Idea – let’s keep track of what is used in our reference collection (even by us), probably by call number, so we can start weeding

Other assessment Tools and Methods

• Univ. of Rochester anthropological studies
• Learn from Rochester, but need to find how our students function
• Ask students to map out where they go during a typical day and when
• From their studies, as we well know, everyone has a cell phone, so make our phone number(s) more prominent – on home page, in stacks

Latvian Collection at University of Washington

I spent 15 years collecting Latvian books and other materials at the Latvian Studies Center (LSC) Library, but when Latvia became independent and the Latvian Program at Western Michigan University closed, the nationwide Latvian community could no longer maintain a library. I found a home for it in various institutions, but the primary one was at the University of Washington, which had just started a Baltic program. Michael Biggins, the head of the Slavic and East European division was willing to take it in. Since I was visiting Seattle and UW, I had to visit him and my “child”, as I used to call the LSC Library, before I had a real child. All 12,000 or so books I sent are cataloged, and Michael proudly showed how the Baltic history collection had grown from a couple of shelves to quite a few sections of books. I was most familiar with the literature section, which was shelved in about 16 sections, with about 4000 books. My original collection has been supplemented by regular additions from Latvia – about 250 new books a year. Having this solid Latvian collection has been leveraged to similarly grow the Lithuanian collection (9 sections in literature or about 2250 books), and Estonian is evolving too (4.5 sections or 1100 books). Of course, Latvian materials end up classified in various parts of the library, but the easiest sections to see were the DK’s for history and PG’s for literature, though I had also sent them microfilms and Latvian materials were in various sections of the N’s for art. Since there is a separate art library that focuses on North American and Western European art, this section is just for other art, often in the language of origin. Music too has Latvian materials and the choir director at UW is especially interested in Baltic choral music, so they have asked cataloging to provide additional tagging to be able to pull these out as a Baltic Choral Collection. The UW choir has visited the Baltic States with concerts, and has a Baltic repertoire. With the growth in interest in the study of cinema, they are also purchasing Baltic films.

I met the two catalogers that have worked on my collection – Jake and Nadia, and felt an instant affinity towards the people who had lovingly processed “my” books.

Besides getting books through an exchange program with the National Library of Latvia, they are also subscribing to 40-50 periodical titles. The newspapers in Latvia seem to be especially difficult to obtain, as the online access and archive are held by a private company – Lursoft, which does not allow for institutional subscriptions. The Estonians, on the other hand, have great free access to their news archives.

The Slavic & East European (including Baltic) collection does get used, circulating about 25,000 items (out of 400,000) per year.

I liked their pre-searched lists of new books. See their New Arrival in International Studies by country. You just choose your country of interest and see the books received in the last months, with a link to the catalog record.

One of the digital projects at UW is the William C. Brumfield Russian Architecture Collection. He is the eminent scholar on this in the states and has published numerous books, some in Russia. Brumfield teaches at Tulane in New Orleans, so Hurricane Katrina accelerated the motivation to get everything digitized quickly. There were challenges with metadata describing both the photograph and the building. They have gone with the METS system that lets them catalog the building as the core item, but also allows for full descriptions of the photo.

A woman at UW has photographed 12 members from each local Baltic community, included a short oral history, and an artifact from their own family albums. This exhibit will be traveling to the Baltic States next year and there is talk of expanding it.

Notes from Seattle

Another wonderful conference and visit to a great city. Barbara and I spent Sunday visiting the Olympic peninsula, beginning with a ferry ride across from Seattle. The day started overcast, but cleared up into a glorious sunny day. The environment is very different, dominated by evergreen trees, lushly green, and bodies of water all around. We stopped off at an authentic Native American store/gallery and a lavender farm, where we picked some lavender, before heading into the Olympic National Park. The center of the park is full of huge snow-capped mountains, so there is no road through the park. You can just drive around them and take various roads in towards the mountains. With our time limit, we did two of these drives.The first was to Hurricane Ridge, through an amazing old growth forest, up quite high with a magnificent vista of many of the mountains, including Mount Olympus itself. You could see that it and a few others are covered by glaciers. They have a lot of blizzards up there and the snow just gathers and never completely melts. As we took a hike around, we came to a couple of spots that still had snow. The other thing that was incredible up here was the variety of wildflowers – blue bells, blue lupines (one of my favorites of all time), Indian paintbrush, including a pink variety that is unique to this area, and many more.

The second drive was in along the Elwha River. We could drive in around 10 miles up the mountain and then we had a 2.5 mile gently sloping hike in to some natural hot springs. The drive was through a forest in areas thick with huge ferns and trees overgrown with moss. Once we started hiking we could really appreciate the immensity of the trees. We got to ford a few streams and finally got to the hot springs flowing out of the mountain side in a half a dozen places. At some, people had put up stone barriers to create a shallow basin to catch the hot water. The first and largest already had a few people in it, another had a single gentleman, but we found one we could have to ourselves. What a wonderful feeling after the long climb, even if we did smell a bit like sulfur afterward.

The only thing with ferries is that they have a limited capacity, and as many people were heading back to the city, there was a three hour wait at the ferry, so we ended driving around the southern tip of Puget Sound and getting back to the hotel fairly late.

The conference was held at the University of Washington student union, a 20 minute walk from our hotel. The campus is beautiful with interesting old (and new) buildings, huge trees and lush bushes and flowers everywhere. As one local said, things grow so well, you almost have to struggle to keep it trimmed back.

In the center is a huge fountain and over that on a clear day you can see Mount Ranier. It was really clear only on Monday, Tuesday morning I was still able to take a picture, after that it disappeared into the mists.

Barbara and I had different opportunities to see downtown Seattle. She attended the reception at the Oly mpic Sculpture Park, I walked around it later. We went into the Seattle Public Library (maybe a separate post), but only I got to see the guys throw fish at the Pike Place Fish Market.

The most surprising thing to both of us was the hilliness of Seattle, it reminded us of the steep streets of San Francisco – made for interesting driving. We left plenty of things to visit if we get back here for ACRL – the Space Needle, art museum, aquarium, Science Fiction museum, etc.

I do feel obligated to mention the weather. While we were there, we had beautiful, sunny, even hot weather. Since this was an assessment conference, they had “assessed” when to hold the conference and showed us charts on precipitation, where it is usually very high, except in July and August. The town isn’t all air conditioned, as I saw in a home I visited, or even the student union, where we had to stop using one of the ballrooms, as it was too hot. And it doesn’t get that cold. There aren’t heavy freezes, so people can grow palm trees, but on the other hand, it is hard to grow tomatoes, as there isn’t enough sun. I also understand that there is a lot of variation. The town of Sequim (pronounced Squim) is one of the sunniest places and a favorite retirement town. That’s where we visited the lavender farm.