One of the longest-running and most recognizable activities of the Association of Research Libraries (ARL)2 is the statistics program. Quantitative and descriptive statistics have been collected and published annually by ARL since 1961-62. The publication ARL Statistics describes the collections, expenditures, staffing, and service activities of the member libraries of ARL. Before 1962, annual statistics for university libraries were collected by James Gerould, first at Minnesota and later at Princeton. These data cover the years 1907-08 through 1961-62 and are now called the Gerould Statistics. The whole data series from 1908 to the present represents the oldest and most comprehensive continuing library statistical series in North America.
The ARL Annual Salary Survey, first published for 1972-73, currently compiles data for over 12,000 professional positions. Published annually since 1992-93, ARL Academic Law and Medical Library Statistics reports data on collections, expenditures, staffing, and user services in the law and medical libraries of ARL university members. Preservation Statistics, published annually since 1988-89, includes data tables on personnel, expenditures, conservation treatment, preservation treatment, and preservation microfilming, as well as an in-depth analysis of data by size of library.
In 2000 ARL Supplementary Statistics, with data on the size and kind of members' electronic resources, was first made available to the public, after being used since 1983 as a test for collecting information on new measures in libraries. Developing Indicators for Academic Library Performance: Ratios from the ARL Statistics, which presented 30 selected ratios that describe changes in internal library operations as well as resources per faculty and per student for the ARL university libraries over a two-year period, was published for the years 1992 - 99. Those ratios can now be generated from the interactive website <http://www.arl.org/stats>.
For many years the traditional statistics projects, ARL Statistics and the ARL Annual Salary Survey, were supported through volunteer efforts from member institutions. Kendon Stubbs, of the University of Virginia, served as the consultant for the main statistics and Gordon Fretwell, University of Massachusetts at Amherst, was the salary survey consultant. They were instrumental in establishing the data-collection activities and ensuring the consistent high quality of the data. In 1994, the program was expanded to include a full-time program officer and the data-gathering activities were transferred in-house. Activities increased substantially: additional surveys were undertaken; the statistics were made available over the web; more extensive custom reports were made available to members; and the Association became ever more active in other national library and higher education data gathering efforts.
A timeline of program activities <http://www.arl.org/stats/statsresources/timeline.shtml> highlights the Association's interest in statistics and descriptive data about research libraries. These data have been used both for comparative purposes and also to track the trends of investment in research libraries for the best part of the 20th century. The timeline also demonstrates the attempt by ARL over the years to respond to a need to look at more than descriptive data. In the early 1980s, the Statistics Committee began to look for ways to measure organizational performance objectively and began to collect supplementary statistical data to provide information on a variety of measures including those addressing access to information resources.
In 1999 the ARL Statistics and Leadership Committees and other interested members began what has become the ARL New Measures Initiative. This aims to develop methods to assess how well libraries meet institutional and user needs, and how well libraries use their resources and services. One area of interest focuses on how to measure user expectations and perceptions of library services. The history of the 1999-2000 LibQUAL+ pilot project and subsequent work is described by Cook and Heath in the paper that follows this one.
Another area of interest is how to measure the collection and use of electronic resources. The goals of the ARL E-Metrics Project are to develop, test, and refine selected statistics and performance measures to describe electronic services and resources in ARL libraries; to engage in a collaborative effort with selected database vendors to establish an ongoing means to produce selected descriptive statistics on database use, users, and services; and to develop a proposal for external funding to maintain the development and refinement of networked statistics and performance measures.
These and other new projects focus on higher education outcomes assessment, including the role that the library plays in support of learning, teaching, and research, identification of cost drivers, and applying the results of the ILL/DD Performance Measures cost study. Their progress can be followed at <http://www.arl.org/stats/initiatives/>.
In the 1990s, many of the annual surveys evolved into analytical products and services in unforeseen ways - primarily due to the use of new technologies ranging from innovations in data collection to electronic publishing of datasets, as well as derivative print publications. Probably the biggest challenge for any organization, including ARL, is the attempt to bring disparate web products and projects together in a way that complement and build on each other so that users maximize the benefits they receive.
An example of analytical work based on the ARL Annual Salary Survey includes a study of the age demographics of librarians published in 1996. Part of the analysis was revisited in 1999 and, with new data collection in 2000, work in this area continues with the purpose of developing retirement projections later in 2001. 3 The longitudinal data on salaries from ARL is also being further analyzed to give insights into the rate of salary increases in relation to various demographic characteristics of the population of librarians. 4 Since 1980 the ARL Annual Salary Survey has been a scalable survey as it is managed in a rather centralized fashion collecting a few basic variables for professionals and coordinated through the library personnel office that return to the ARL offices the data in a standardized Excel spreadsheet.
Figure 1 (Screen shot of the ARL Annual Salary Survey home page)
Additional analytical work has grown out of the ARL Statistics, a historical series of data on North American libraries describing major trends. 5 The ARL Statistics products are available to readers in three different basic forms:
(a) the printed publication, which includes an analytical introduction with a series of important and popular graphics describing major trends in libraries;
(b) the machine-readable version of the data, which comes in the form of two Excel files that users can download for their own use; 6 and
(c) the interactive WWW edition of the ARL Statistics, a set of highly functional interfaces for manipulating interactively on the Web the rich database that underlies the ARL Statistics.
The ARL Statistics Web Interactive edition 7 was one of the first electronic interactive products ever to appear on the Web. It is supported by one of the well-known electronic centers of the University of Virginia Library: the Geostat Center. (The former Social Science Data Center has now been folded into the new Geostat Center.) The first interactive edition of the ARL Statistics was created in 1994 by Kendon Stubbs and Paul Begen, and became publicly available in the early part of 1995. As one of the first interactive data analysis products it was presented at the National Center for Educational Statistics Summer Data Conference in Washington D.C. in June 1995 where Kendon Stubbs and Martha Kyrillidou talked about "Interactive Statistical Analysis on the WWW". A number of annual editions were supported by Patrick Yott as Director of the Social Science Data Center at the University of Virginia.
In 1994 with the advent of web browsers, the thinking was that books were being re-invented by transforming them from static objects into "books as performance" with the integration of existing media and the creation of dynamic publications. Every reader was viewed as an actor participating in creating information and unique experiences. Since then the interactive edition of the ARL Statistics has gone through a variety of enhancements; every year the availability of new data presented new opportunities to improve the interface by providing additional features. The 1999 edition is the brainchild of Spencer R. Graf, who worked on the interactive ARL Statistics while completing his Ph.D. in Economics at the University of Virginia. In the meantime, the evolution of the web since 1994 has transformed not only electronic publishing but many aspects of the daily work environment which are gradually being translated into web performances.
The ARL Statistics Interactive edition is an interface that builds upon the client/server architecture of the WWW and the strengths of various software applications. It currently runs on a UNIX server, utilizing protocols such as HTML and SGML, PERL and shell-scripting languages for submitting queries and building statistical programming code that can run a variety of statistical applications such as SAS, SPSS, and STATA in the background. The major challenge for the forthcoming web interactive editions of the ARL Statistics is to transfer this environment into an XML protocol and create a seamless process from data entry into data publishing.
The current interface has five top-level menu options for interactive data analysis:
(a) Institutional data that allows you to select up to seven institutions for peer group comparisons and download the data in a comma-delimited format for every year separately since 1963. Figure 2 shows the Institutional and variable data selection interface and Figure 3 is an example of the output derived from that interface.
(b) Descriptive statistics which allow to retrieve summary statistics on any one of the 48 variables or a ratio formed by a combination of any two of these variables for every year separately since 1963. Descriptive statistics are available for either all libraries or for a user-defined peer group, the latter selected through the "Advanced Descriptives Form." Figure 4 shows the descriptive statistics interface and Figure 5 a sample output from the interface on Figure 4.
(c) Ranked lists of institutions based on any one of the 48 variables or any ratio formed by any two of these variables for every year separately since 1963 with the option of placing a specific library in bold. Ranked lists are available for either all libraries or for a user-defined peer group selected through the "Advanced Listings Form." Figure 6 shows the interface that is used to create customized rank order tables and Figure 7 the output from the interface in Figure 6.
(d) Membership criteria, a set of options for displaying individual institutional data and graphs on the five variables comprising the ARL membership criteria dating back to 1986 when the index became available, as well as ranked lists of the variables comprising the ARL membership criteria index. Figure 8 shows an interactive map of all ARL libraries throughout North America. Figure 9 lists the output produced by clicking on one of the dots that represent libraries in Figure 8. And Figure 10 lists the data produced for the library selected in Figure 9.
(e) Interactive graphics for individual libraries or groups of libraries plotting data on a time series from 1963 to 1999, or other user-defined timespan, for up to six variables or ratios formed by two of these variables are also available. Figure 11 presents the output from a graph presenting summary data for many variables for one library and Figure 12 presents a graph of serial unit costs (one ratio variable) for three ARL libraries.
Many library directors and survey coordinators use the ARL Statistics web interactive edition to analyze the data for peer group comparisons and special data extractions for local budget justification purposes. In addition to this primary use, faculty and researchers have used the interactive edition for teaching purposes. ARL itself has built workshops and training opportunities related to Electronic Publishing of Datasets, a workshop offered once a year on how to publish numeric datasets on the WWW. The workshops are expanding with an offering on an introduction to XML as gradually options for integrating the variety of applications through XML are explored.
Major challenges that have arisen since the first electronic interactive edition of the ARL Statistics and are not yet resolved include:
In 1997 the annual ARL institutional data collection was done via the Web for the first time. Although electronic means of data collection were utilized before, they were diskette-based. The annual surveys were marked up in HTML and mounted on the web through an interface that was built on behalf of ARL by the Library Research Center at the University of Illinois at Urbana-Champaign. The web data entry interface has a PERL-scripted administrative interface that allows the survey administrator to mount new surveys, set up a password-protected environment, and edit the data using web forms. It has been used successfully to collect data and various enhancements have been added but still it is lacking live data editing capabilities.
As described in the next paper (by Cook and Heath) LibQUAL+ is an R&D project that attempts to assess the performance of research libraries as it is perceived by the user and is attempting to built a scalable web interface for user-based assessment. 8 The technology infrastructure used in LibQUAL+ is a database-driven design that relies on Cold Fusion and an SQL database that in turn interacts with analytical software such as SPSS. The technical process is described in a paper by Russell Thompson, one of the developers of the infrastructure currently located at Texas A&M. 9 The Spring 2000 pilot experience showed that technology is indeed maturing fast enough to allow libraries to build robust assessment mechanisms that move beyond the basic descriptive institutional assessment frameworks that have been so popular among libraries ever since 1908. The LibQUAL+ pilot proved that large-scale user-based assessment is a reality for libraries.
At an international symposium held in October 2000 experts discussed the various theoretical frameworks as well as the practical implications of user-based assessment in libraries. 10 Phillip Calvert said that, from a theoretical perspective, user based assessment has important and interesting international applications - as he had discovered from work done in comparing user perceptions between library users in New Zealand and China. Exploring the applicability of LibQUAL+ in the international library environment beyond North America is one of the next steps.
Looking backward through the 1990s, it seems clear that technology improvements in data collection, analysis and publishing have moved libraries much closer to the user. Yet the more knowledge expands, the more clear its limitations appear. Tying institutional assessment to user-defined outcomes is going to be a major challenge for most organizations, libraries and educational institutions in the digital, self-empowering environment created at the dawn of a new century. Issues related to the privacy of institutional and user records will become more prevalent as the development of pervasive infrastructures that may infringe upon user privacy appear. The real challenge for libraries may be indeed whether they can recognize their users and users their libraries, and, if so, in what ways. Will users at the end of the next century still recognize that it is @ the library where they find valuable resources?
Resources and services provided will be difficult to describe and assess as the information environment of users becomes increasingly complex. Both new and old assessment methods will have to be utilized in creating ways to describe multiple user and library realities. The more virtual and complex the environment becomes, the more pervasive and systematic assessment efforts need to be. The various avenues of development at ARL provide a promising springboard to prove how much such measures matter to the future of libraries.
Martha Kyrillidou is Senior Program Officer for Statistics and Measurement at the Association of Research Libraries, 21 Dupont Circle, Washington D.C. Martha@arl.org.
Martha is the editor of the major statistical publications ARL produces since 1994 and a research analyst on trends in research libraries with interdisciplinary training in evaluation and measurement and library and information science. She supports the work of the ARL Statistics and Measurement Committee by designing and implementing research, evaluation, and training activities and is one of the LibQUAL+ Project Managers. She has worked at the Library Research Center at the University of Illinois at Urbana-Champaign and at the Bureau of Research at Kent State University. Martha has also worked as a consultant with libraries in Greece, most closely with libraries at the Aristotle University of Thessaloniki.
The introduction traces the history of the Association of Research Libraries (ARL) statistics. The current range of published statistics is described. In recent years their presentation on the WWW has developed so that many are now available to the user interactively and for downloading. Further development is anticipated. Increased emphasis on quality and on the user is illustrated by the large scale LibQUAL+ Project.
1 Thanks to Julia Blixrud, ARL Director of Information Services, and Lee Anne George, ARL Program Planning Officer, for their work on the Historical introduction.
2The Association of Research Libraries (ARL) is a not-for-profit membership organization comprising 123 the libraries of North American (Canadian and U.S.) research institutions as of May 2001. ARL operates as a forum for the exchange of ideas and as an agent for collective action. It serves eight strategic objectives ranging from scholarly communication and information policies, access to research resources, technology, collection development, preservation, staffing, management, and performance measures.
3 Stanley Wilder, The Age Demographics of Academic Librarians: A Professional Apart (Association of Research Libraries, Washington D.C: 1995) and "The Changing Profile of Research Library Professional Staff" ARL 208-209 <http://www.arl.org/newsltr/208_209/chgprofile.shtml>
4 Martha Kyrillidou, "Salary Trends Highlight Inequities-Old and New" ARL 208-209 <http://www.arl.org/newsltr/208_209/saltrend.shtml> and "Educational Credentials, Professionalism, and Librarians" ARL 208-209 <http://www.arl.org/newsltr/208_209/edcred.shtml> and Julia Blixrud, "Back-Room and Front-Line Changes" ARL 208-209 <http://www.arl.org/newsltr/208_209/catref.shtml>.
5 Martha Kyrillidou, "Research Library Trends: ARL Statistics" Journal of Academic Librarianship (November 2000): 427-436.
6 Machine-Readable Files of ARL Statistics <http://www.arl.org/stats/annualsurveys/arlstats/mrstat.shtml>.
7 Kendon Stubbs, "Interactive Peer Group Comparisons Through the Web" ARL (April 1998): 11. <http://www.arl.org/newsltr/197/peergroup.shtml>
7 ARL LibQUAL+ Reader (forthcoming). Also, see the LibQUAL+ homepage <http://www.libqual.org/>
8 Russell L. Thompson, "Scalable Web-based User Survey Research Across Institutions: the LibQUAL+ Experience". See: <http://www.arl.org/stats/statsevents/msq2000thompson.shtml> (unpublished paper)
10 ARL Symposium on Measuring Library Service Quality. <; The majority of the paper presented at the symposium will be printed as the Spring 2001 Library Trends issue.
