From the 138th ARL Membership Meeting

ARL Committee on Statistics and Measurement
Wednesday, May 23, 2001
9:00 a.m. - 11:30 a.m.
Room: Windows West
Four Seasons Hotel
Toronto, Canada

Agenda

Note: The parenthetical times shown are estimates only to aid in moving the meeting along. If an issue warrants, we will take as much time as necessary.

1. Introductions and Overview

Welcome four new committee members: Don Bosseau, Jean Ehrhorn, Bob Hudson, Jennifer Younger (5 minutes)

Attachment 1: ARL Statistics and Measurement Program Plan 2001
Section 5.1 Statistics & Measurement Program [PDF]
Section 5.1.1 Annual Data Collection, Publications, and Workshop Activities [PDF]
Section 5.2 New Measures Initiative [PDF]
Section 5.2.1 LibQUAL+ Project [PDF]
Section 5.2.2 E-Metrics [PDF]

Outcome: Understanding and support of the Program Plan.

2. Approval of Minutes (5 minutes)

Attachment 2: Minutes of the Meeting of the ARL Statistics and Measurement Committee, October 2000 [PDF]

Outcome: Approval of minutes.

3. New Measures: E-Metrics

E-Metrics by Dr. Charles McClure (50 minutes)

The current ARL E-Metrics project is being conducted in three phases:

Phase I: Knowledge Inventory of ARL Libraries and organizing an ARL Working Group on Database Vendor Statistics (May 2000 - October 2000) * completed - see project web page http://www.arl.org/stats/initiatives/emetrics/index.shtml

Phase II: Development of statistics and performance measures (November 2000 - June 2001) in development and includes (a) vendor meeting (b) field-testing library data (c) field-testing vendor data (d) developing an institutional outcomes model.

Phase III: Institutionalizing Statistics and Performance Measures (July 2001-December 2001)

• Presentation at CNI Task Force Meeting, Washington DC, April 10, 2001
• Meeting in conjunction with ALA Annual, San Francisco, CA, June 15, 2001

Outcome: Understanding of project goals and update on progress

4. New Measures: LibQUAL+

LibQUAL+ by Fred Heath (15 minutes)

Funding was secured for a three-year period to continue this project with the ultimate goal of developing an ARL program of service quality assessment.

4a. LibQUAL+ - There are 43 libraries participating in spring 2001; currently another 40 libraries have expressed interest in participating in spring 2002; there is also interest from consortia like OhioLINK, associations such as AAHSL, and library groups like the University Library Group (ULG) for group participation.

4b. NSDL LibQUAL+ - ARL applied for funding to the National Science Foundation (NSF) for exploring the applicability of the LibQUAL+ protocol in the National Science Digital Library area.

4c. UK LibQUAL+ pilot - At the international level, there is also interest in exploring the LibQUAL+ process. Libraries in the UK would like to institute a LibQUAL+ pilot project; ARL and Texas A&M would like to explore the possibility of test sites in New Zealand or Australia to understand the international implications for user based assessment.

Outcome: Understanding current developments

5. New Measures: Higher Education Outcomes (HEO)

(15 minutes)

Interest is increasing for developing a project to support the investigation of measures for determining the library contribution to learning, research, and higher education outcomes. Ken Smith, Eller Distinguished Service Professor of Economics at the University of Arizona, and author of the ARL White Paper on Learning Outcomes http://www.arl.org/arldocs/stats/aboutstats/oct2001/HEOSmith.pdf was a featured speaker at the ACRL conference in Denver. (ARL is exploring avenues for working more closely with ACRL in promoting learning outcomes assessment. See agenda item 7.) Carla Stoffle will participate in a panel program on Academic Libraries and Assessment of Student Learning - Current and Future Needs for the American Association of Higher Education 2001 Assessment Conference to be held in Denver, Colorado, June 23-27. The convener is Amos Lakos (Waterloo). The other panelists include Hannelore Rader and Debra Gilchrist.

Some individual libraries already have engaged independently in the area of learning outcomes. Others may be interested in working together to follow a joint course of action as outlined in Ken Smith's white paper. Some of the ARL libraries identified as working in the area of learning outcomes on include:

• Arizona: University of Arizona, The Information Literacy Project
• California: University of California -- Berkeley
• Washington: University of Washington, UWired Program
• Wisconsin: University of Wisconsin -- Parkside

Attachment 3: Letter of acceptance to Amos Lakos [PDF]

Attachment 4: Next Steps on Learning Outcomes [PDF]

Outcome: Increase awareness of related developments and recommend ARL course of action

6. New Measures: Allocation of Staff Resources

Allocation of Staff Resources by Eileen Hitchingham (15 minutes)

The need to explore the applicability of a comprehensive cost accounting framework is being explored with a model originally developed and currently refined at Virginia Tech.

Outcome: Update on refining the methodology for cost allocation framework

7. ARL and ACRL Collaboration

(15 minutes)

There is an increasing overlap of interests between ARL and ACRL and these shared interests create an opportunity to explore the organizations could work more closely together. Some of the ACRL projects include:

• ACRL Library Statistics. Data collection has been completed for 1999-2000. ARL is already working closely with ACRL to eliminate unnecessary duplication of effort by exchanging data with institutional permission. ARL is also working on formalizing our relations with ACRL so that any revisions to the ARL Statistics survey instrument are incorporated into the ACRL version of the instrument. Last, there are many revisions in the interpretation of the definitions in the ARL survey that need to be communicated in a uniform way to the larger post-secondary education community. A formal mechanism for this communication is needed.

• ACRL has received an IMLS grant - Education & Training for $149,924.00 for a two year project will train academic librarians to work with faculty to design, implement, and evaluate tools for assessing student learning outcomes resulting from information literacy courses taught by librarians and faculty. See also ACRL's Institute for Information Literacy (ILL) web site for additional resources.

• ACRL has developed a one-page Student's Guide to Evaluating Libraries in Colleges and Universities that addresses Facilities, Resources and Services.

• One possible activity would be to create a joint collaborative forum like Create Change as an avenue to discuss outcomes assessment investigations in library settings and to promote projects and programs to the larger academic and research library communities.

Outcome: Define next steps for ARL and ACRL to work collaboratively on common issues of statistics and performance measurement

8. NISO/ISO Developments

(15 minutes)

8a. NISO held a successful invitational workshop regarding the revision of Z39.7 Library Statistics 1995 standard. The forum website is http://www.niso.org/news/events_workshops/stats.html. Eileen Hitchingham and Sherrie Schmidt attended the forum.

Attachment 5: NISO meeting report from April 2001 D-Lib Magazine In Brief

8b. ISO has a draft standard out for ballot and seeks feedback from interested parties. Draft standard is sent as a separate email attachment.

Attachment 6: Draft International Standard ISO/DIS 2789, Information and documentation -- International Library Statistics [PDF]

Outcome: Understanding of national and international standards activities

INFORMATION ITEMS:

9. Related Workshops/Presentations/Event Calendar

(a) ARL survey coordinators meeting (June 2001, San Francisco)

(b) ARL Statistics and Measurement Program (IFLA Presentation, August 2001, Boston) Julia Blixrud

(c) Building a Culture of Assessment in Libraries (Northumbria Pre-conference, August 2001, Pittsburgh) Shelley Phipps and Amos Lakos

(d) Conducting Focus Groups in Libraries workshop/online lyceum (not scheduled) Seeking VPO

(e) E-metrics presentation (CNI Spring Task Force meeting, April 2001, Washington, DC) Charles McClure, Bruce Fraser, and Sherrie Schmidt

(f) E-metrics Workshop for Project Participants (ALA Annual meeting, June 2001, San Francisco) Charles McClure and colleagues

(g) The 4th Northumbria International Conference on Performance Measures in Libraries (IFLA preconference, August 12-16, 2001, Pittsburgh, PA)

(h) Library Cost Allocation Model workshop (not yet scheduled) Robert Hayes

(i) Measuring Service Quality Online Lyceum workshop (June 25- August 3, 2001) oped by Danuta Nitecki and Toni Olshen

(j) "New Ways for Listening to Library Users" by Parasuraman, Colleen Cook and Bruce Thompson (May and September, 2001)

(k) Research Methods and Statistics workshop/online lyceum seeking VPO

(l) Role of Assessment in Advancing Diversity for Libraries led by DeEtta Jones, Julia Blixrud, and Martha Kyrillidou to be offered in the Fall, 2001 as an Online Lyceum course.

(m) User Survey for Academic Libraries workshop/online lyceum (not yet scheduled) seeking VPO

(n) XML workshop (not yet scheduled) Patrick Yott and colleagues at the University of Virginia

10. News from External Constituencies

(a) Academic Libraries Advisory Committee/NCES. Academic Libraries 1998 publication is almost finished; the datafile is available on the web, the 2000 data collection is being prepared. NCES plans to offer interactive data analysis capabilities of the Academic Libraries data. The Academic Libraries survey is not part of the IPEDS set of surveys but is been successfully managed under the Library Statistics Unit of NCES in 2000.

(b) CLIR/DLF Draft White Paper by Denise Troll on "How and Why are Libraries Changing" is available at http://www.diglib.org/use/whitepaper.htm. Denise Troll is seeking feedback on her white paper and will also be available to meet with ARL Statistics and Measurement Committee members in October.

(c) Outsell Inc. Briefing on "Today's Students, Tomorrow FGUs." Please provide feedback on the briefing to Martha Kyrillidou.

(d) Resource to develop a standard for Learning and Access for Museums, Archives and Libraries in the UK. See

(e) CENDI Report on Web Metrics and Evaluation: Current State of Implementation Among the CENDI Agencies. Attachment 7: CENDI report

(f) New Measures: Research Outcomes. Doug Jones is pursuing an investigation of research outcomes and working on a white paper that will be available later in the summer. He attended the NISO forum on Feb 15-16; and visited the ARL offices in DC on March 7.

(o) New Measures: Technical Services Cost Study by Dilys Morris and five test sites. Dilys as well as two representatives from the participating test sites (Flo Wilson and Christian Boissanas) will be available to make a presentation of the pilot projects and the software to the ARL Statistics and Measurement Committee Members and the ARL Survey Coordinators meeting at ALA in San Francisco.

(p) New Measures? In October 2000 one of the discussion sessions was on "Defining, Describing, Measuring and Evaluating Collections in the Electronic and Consortial Environment" Three documents related to that section are of interest to the ARL Statistics and Measurement Committee agenda and are attached:

Attachment 7: "Defining, Describing, Measuring and Evaluating Collections in the Electronic and Consortial Environment" Framing Questions [PDF]

Attachment 8: "Issues in Evaluating Research Library Collections in a Digital and Cooperative Age" by Merrily Taylor [PDF]

Attachment 9: Summary of the discussion session by Deborah Jakubs [PDF]