Association of Research Libraries (ARL®)

Carla Stoffle (Arizona)
Eileen Hitchingham (Virginia Tech)

Thirty-four people attended the discussion on outcomes and accreditation. Carla Stoffle, chair of the ARL Statistics and Measurement Committee, opened the discussion by reporting on some of the activities of the New Measures Initiative. The student learning outcomes project, which is an outgrowth of the document Ken Smith prepared for ARL, is an effort to identify measures that describe the library's contribution to those outcomes. It is broader than information literacy, but does incorporate some elements of it. A working group met earlier in the week and determined three areas in which action could be taken: 1) designing questions that could be added to national tests that libraries might share results to benchmark or identify best practice, 2) identifying common learning outcomes, particularly in general education curriculum, and preparing assessment tools that could be used by a variety of libraries to measure those outcomes, and 3) suggesting and designing training activities for library staff to work with faculty as well as learn how to become engaged with campus assessment efforts. Also, the ARL E-metrics Project team will continue in a next phase of their work to look at how to assess the impact of electronic resources on their user communities now that the project has defined and tested measures.

Discussion Summary

The Role of the Accreditation Agency

Accreditation agencies are moving towards broad guidelines instead of specific indicators and the presentation of the report is less of an inventory to one of a portfolio. Institutions set their own mission statements and describe how they will be achieving their objectives and how their success will be measured. If you aren't doing assessment, must at least have an assessment plan. Since this is a new way of doing accreditation, it is difficult for institutions to know how to prepare their accreditation response for this new environment and they are all looking for guidance. Some accreditation agencies are doing workshops to help institutions make this transformation. For libraries, it may be a good idea to get a hold of a self-study from a library that was part of a recent visit. Often, the accrediting agency will supply them. Some accreditation visits may include only a sample/selective visit. If the library is not to be included in the selective visit, it may be worth investigating why. It may only be that the selective visit is a result of a previous visit recommendation and there was no need to see the library again.

Several directors at the discussion reported their experience being on accrediting teams and Merrily Taylor serves as a commissioner for the Commission on Institutions of Higher Education (CIHE) of the New England Association of Schools and Colleges. She noted there continues to be some controversy around general education outcomes; i.e., those that cut across the curriculum. Agencies are also struggling with questions on the relationship between student outcomes and institutional effectiveness.

It was noted that the professional schools have defined specific outcomes (e.g., Music, Business) and they might be a fruitful area for investigation.

State Mandates

Some states have been moving to state mandated testing and state Boards, Trustees, or Regents are asking now for quality indicators rather than input measures. Specific states mentioned include Florida, Massachusetts, Missouri, Nebraska, and Virginia.

Campus Assessment Activities

Campus officials are looking for ways to ease measurement and often want only one quality indicator from the library (or any campus unit). LibQUAL+ could be one tool providing measurement information on a library's service.

Other Assessment Projects

Joan Lippincott reported that when the CNI assessment project was in place, they emphasized the importance of team-based collaboration. Libraries should continue to look for partners as they begin their assessment projects. The Transformative Assessment Project (TAP) is a joint project of the National Learning Infrastructure Initiative (NLII), the Flashlight Program of the TLT Group, and the Coalition for Networked Information (CNI). The project is designed to elicit new ideas about assessment practices and systems that will transform teaching and learning, and to help institutions of higher education put these ideas into action. A focus session has been rescheduled from September 2001 to March 2001 for interested institutions.

Obtaining Data

A short discussion surrounded how to gather data for external information. Professional schools do alumni surveys and perhaps libraries could add questions to those surveys since they would address graduates views of their campus experience. Other common surveys were mentioned and these are the ones the ARL Working Group will be investigating.

Other Observations

Those attending this discussion agreed on the importance of getting assessment into the local institutional culture. Although it is helpful to read other self-studies and assessment reports, libraries should not just take an instrument from someone else and copy it. Libraries should also become involved early in discussions regarding the campus accreditation activity. It is important that the library be seen as a campus unit integral to the success of the institution's mission.

Individuals wishing to keep up with this area are encouraged to consult the ARL New Measures website for outcomes. New materials and links for information on this topic will be announced and appearing there.

Session Overview

For the past 20 years, an assessment movement in higher education has been focusing on the importance of answering the question, are we making a difference? More recently, educational reform efforts supported by state and federal governments have also begun stressing the need to address the results of the educational process rather than the level of effort or inputs into that process. In light of the increased attention being placed on higher education institutions to illustrate their effectiveness, the six regional accrediting agencies in the United States are moving away from the use of quantitative criteria to one of qualitative assessment and outcomes based performance. In particular, the agencies and governments are focusing on the issue of how academic institutions demonstrate student learning. Some questions that arise for research libraries are:

What is the effect of a movement to outcomes based assessment on research libraries generally?

What changes can be seen on campuses that move to an outcomes based assessment environment?

What changes are occurring on individual ARL member institution campuses that illustrate the effect outcomes based performance assessment is having on campus culture?

What role are libraries playing in assisting universities to set institutional outcomes?

What assessment efforts are being carried out on campuses to determine if institutional outcomes are being met?

How can libraries provide evidence of their contribution to student learning?

How can libraries provide evidence of their contribution to the research enterprise?

For libraries beginning or in the midst of an accreditation process, what new or different requirements are being placed on libraries for reporting their contribution to the institution's reported outcomes?

What role could ARL play in helping libraries demonstrate their contributions to institutional outcomes?

Resources:

"Accreditors Revamp Policies to Stress Student Learning." Chronicle of Higher Education (July 7, 2000): A29. http://www.chronicle.com/weekly/v46/i44/44a02901.htm (registration required to access url)

Boyer Commission on Educating Undergraduates in the Research University. Reinventing Undergraduate Education: A Blueprint for America's Research Universities. Stony Brook: State University of New York at Stony Brook for the Carnegie Foundation for the Advancement of Teaching, 1998. http://notes.cc.sunysb.edu/Pres/boyer.nsf/

Gatten, Jeffrey. Academic Quality Improvement Project (AQIP) and Accreditation [powerpoint presentation given at the 4th Northumbria International Conference on Performance Measurement in Libraries & Information Services] (August 2001) http://www.arl.org/stats/north/program.html#tue

The Lombardi Program on Measuring University Performance http://thecenter.ufl.edu/mission.html

Wellman, Jane V. "Accreditors Have to See Past 'Learning Outcomes'" Chronicle of Higher Education (Sept. 22, 2000): B20. http://chronicle.com/weekly/v47/i04/04b02001.htm