Friday, June 22, 2007
Marriott at Metro Center
Montreal Room
Washington, DC
3:30 to 5:00 p.m.
At the ARL Survey Coordinators and SPEC Liaisons meeting held during the ALA convention in Washington DC, we covered important issues related to changes to the annual ARL Statistics.
Colleen Cook (Texas A&M), Chair of the Statistics and Assessment Committee walked survey coordinators and other interested ARL staff members though the recently adopted action agenda for new ways for ARL to measure collections.
These actions were:
+ the launch of a new Expenditures Index and its use for any public reports (reserving the Membership Criteria Index for non-public uses);
+ the development of a services-based index that combines three factors (collections, services, and collaborative relationships);
+ revision of definitions for collection-related data categories and experiment with a variety of new measures, including usage data, strength of collections, and service quality measures; and
+ collection of qualitative data to develop a profile of ARL member libraries.
The action agenda implements significant changes in ARL’s approach to measuring collections. The full details of the implementation plan as distributed at the ARL Business meeting and related documents are available on the ARL Web site at:
At the ARL Statistics and Survey Coordinators meeting held on January 19 in Seattle, Nancy Turner (Syracuse) made a presentation on “E-Metrics Services and ARL Statistics.” Her PowerPoint presentation is available at http://www.libqual.org/documents/admin/NancyTurner.ppt. The survey coordinators discussed the need for more systematic review of the evaluation methods available for electronic resources (e-metrics) and how some of the proposed revisions will be affecting the annual data collection activities. For more information, see Martha Kyrillidou, “The Impact of Electronic Publishing on Tracking Research Library Investments in Serials,” ARL: A Bimonthly Report, no. 249 (Dec. 2006): 6–7, http://www.arl.org/bm~doc/arlbr249serials.pdf.
Friday, June 23, 2006
Royal Sonesta Hotel
Evangeline Suite
3:30 to 5:00 p.m.
New Orleans, LA
At the survey coordinator meeting in January, there were round table discussions of the ARL Supplementary Statistics definitions. I would like to thank the following people for providing notes from the table discussions:
These notes (which can be found as a PDF file here) will be informing revisions of the ARL Supplementary Statistics definitions for next year.
We plan to continue our discussion at the upcoming ARL Survey Coordinators and SPEC Liaisons Meeting. In addition to discussing these issues, Damon Jaggars from the University of Texas will make a brief presentation on the experience of the Research Libraries Leadership Fellows and their learning coming out of a group project for assessing the 21st Century Research Library.
Minutes from the June 2006 Meeting
The Round Table discussion on the proposals forwarded from January are included here. These notes provide additional feedback on desired revisions to the ARL Supplementary Statistics (E-Metrics) survey. A request was made that any revisions be incorporated at least one year in advance of the mailing of the survey, so that libraries can adjust their practice before they are asked to report the data. This request will be honored and we will not be making changes to the 2005-06 Supplementary Survey though the proposals discussed to date are being reviewed by ARL staff and a revised survey will be available for comment by January 2007.
From: “Martha Kyrillidou” <martha@arl.org>
Sender: <statslmgr@arl.org>
Subject: notes from survey coordinators round tables by Jim Self
Date: Fri, 30 Jun 2006 15:27:55 -0400The following notes on the discussion held at the survey coordinators meeting are provided by Jim Self (University of Virginia):
- Questions 1-3, 7-8 should use terminology consistent with the regular statistics, i.e., they should say electronic ‘serials’ instead of electronic journals.
- The number of electronic serials should be de-duped, so the same journal is not counted repeatedly if it is available on multiple platforms.
- We are dubious as to the utility of questions #4 and #9—number and cost of electronic reference resources. We think the questions should be dropped.
From: dlowewin@mailer.fsu.edu
Subject: notes from survey coordinators meeting on 6/23/06
Date: Mon, 26 Jun 2006 23:18:57 -0400for number 16c. “Amen” get rid of it
question 1, are agreggators really on a 1:1 ratio to full text journals and to paper journals? When the formats create so many differences in the same title, why are we still counting all things as if they were print? Why do we concentrate on print vs. electronic? isn’t a journal still a journal no mater what its format?
The statistics we gather should be important to our users. Measure access available and dollars spent
On question 3. What about overlap titles? Should we only count unique titles? Is that even possible? There was also a thought at the table that free journals are becoming more and more scarce and will soon become extict or close to it. We did agree that the language should be the same language throught the entire question/definition.
Question 4 - We agree with Catherine Davidson’s last suggestion, have a list of what is an aggregator, and what is not so that all answers may be standardized, and therefore comparable to one another.
Question 5 - The point is counting the books, not how they are aquired, so yes, count e-books
7-10 - agreement that they coorespond to questions 1-5, but think that it should still be broken out by resource type.
11. to the note about separating staff questions out we say, it is still a question no matter who asks it.
12. a clearer definition here, possibly a list of what is what as in question 4 so that we are all measuring the same thing.
13-15. It is hard to compare to others if we are not all counting the same resources. Possibly also provide a list here so w can follow it, and all be on the same page.
The Point: We need a model, a definition or lists of what is what so that we are consistent across all libraries in what we are measuring, otherwise comparison is impossible.
-Dawn Lowe-Wincentsen
Florida State University
Thank you for your contributions as we are trying to resolve the challenge of describing electronic resources in an ever changing environment. Please reserve time in your calendar for the survey coordinators meeting at ALA
Midwinter.
Friday, January 20, 2006
The St. Anthony Hotel
Travis Room
3:30 to 5:00 pm
San Antonio, TX
We look forward to seeing you at the upcoming survey coordinators and SPEC liaisons meeting. Apart from a regular update regarding the status of the annual surveys and the SPEC surveys, we will focus the discussion on developments in the area of standardizing usage statistics for electronic resources and the ARL Supplementary Survey.
In particular, we will have two presenters, Tim Jewell (U. of Washington) and Oliver Pesch (EBSCO), who will share with us the latest developments in the “Standardized Usage Statistics Harvesting Initiative”(SUSHI)
http://www.library.cornell.edu/cts/elicensestudy/ermi2/sushi/
We will also engage in round table discussion regarding various definitional issues we are facing in the ARL Supplementary Statistics with
Friday, June 24, 2005
LeMeridien Hotel
Salon Rooms I and II
3:30 to 5:00 pm
Chicago, IL
We look forward to seeing you at the upcoming survey coordinators and SPEC liaisons meeting. Apart from a regular update regarding the status of the annual surveys and the spec surveys, we will continue to focus the discussion on the new ARL Supplementary Survey that attempts to collect data related to electronic resources (E-Metrics); the Visiting Program Officer activities of Steve Hiller (U. of Washington) and Jim Self (U. of Virginia); and the latest developments of the work of the Task Force on New Ways of Measuring Collections.
Minutes from the June 2005 Meeting
The ARL Survey Coordinators and SPEC Liaisons meeting took place at ALA in Chicago. A preliminary draft report on the new ARL Supplementary Survey – E-Metrics data collection was distributed to the participants. The meeting also included:
Friday, January 14, 2005
Omni Parker House
Kennedy Room
3:30 to 5:00 pm
Boston, MA
We look forward to seeing you at the upcoming survey coordinators and SPEC liaisons meeting. Apart from a regular update regarding the status of the annual surveys and the spec surveys, we will focus the discussion on the new ARL Supplementary Survey that attempts to collect data related to electronic resources (E-Metrics).
If any of you would like to share your stories ‘this is the way we are doing’ it or ‘that’s how we plan to do this next year’, we would like to hear from you. Please let us know if you would like to share your experiences with this new survey.
Minutes from the January 2005 Meeting
We discussed the process of collecting e-metrics supplementary statistics and enjoyed a presentation by Kris Brandriff and Kate Hayes, National Agricultural Library, Beltsville, MD, U.S. Department of Agriculture where they introduced a system they developed for doing the E-Metrics data colection. The powerpoint from their presentation demonstrating the system they have developed is available at:
Introducing EMET: NAL’s E-Metrics Database
http://www.libqual.org/documents/admin/ARL_Presentation2.ppt
and a sample report of what their system can produce is also available at: http://www.arl.org/stats/ARL_Report.pdf
A number of issues emerged from the discussion. Many libraries are finding that they need to establish a task force to coordinate this kind of data collection. Issues related to whether certain data elements need to be reported in a cumulative fashion or on an annual basis need to be clarified in the instructions. A closer link between the ARL Supplementary Statistics and the COUNTER Code of Practice needs to be developed. The need for software development and standards development that would facilitate the data collection across products, vendors and publishers needs to be explored. Issues related to the standardization of serial subscriptions and what we mean by serial subscription are going through an evolution – the notion of a ‘continuing resource’ is emerging in our bibliographic standards.
The meeting concluded with the annoucement that on Friday, June 24, 2005, in conjunction with ALA in Chicago we will be offering a workshop on Webmetrics from 9:30am to 3:30pm. More details about this event will be available soon. We will hold our regular survey coordinators and spec liaisons meeting from 3:30 to 5:00 as usual.
Friday, June 25, 2004
Embassy Suites International Drive
Citrus/Lake Room
3:30 to 5:00 pm
Orlando, FL
We will focus the discussion on the new ARL E-Metrics Supplementary Survey.
Jim Self (U. of Virginia) and Fern Brody (U. of Pittsburgh) will share with us how they have organized their units to collect the needed data elements and potential uses they foresee.
There are clearly a lot of definitional issues related to the new supplementary data collection as we are uncovering from the phone interviews, as well as, some proposed solutions. The ARL Statistics and Measurement committee discussed some of these issues during the May ARL Membership Meeting and provided advice on future steps. A more systematic conceptual understanding of these issues will be documented in a white paper during the coming months.
In the meantime, our goal is to continue to build consensus on the best way of defining and measuring the new electronic environment through the upcoming ARL Supplementary E-Metrics Survey that will be mailed to all members libraries later in the summer.
Resources from the June 2004 Meeting
Improving Statistics Collection at Pitt, by Fern Brody
http://www.libqual.org/documents/admin/Fern_Brody_ImprovingStats.ppt
The New Metrics at UVa, by Jim Self
http://www.libqual.org/documents/admin/ARLPresJune04x.ppt
Friday, January 9, 2004
San Diego Marriott
New York/Orlando Room
3:30 to 5:00 p.m.
San Diego, CA
This is a meeting for all survey coordinators but would like to focus our discussion and attention on E-Metrics related issues. We would like to ask you to bring with you examples of statistics that you use for describing electronic services and resources for a sharefair experience. Please send me an email if you plan to do so.
We would like to have a discussion on how you determine the cost-effectiveness of electronic resources at your library. Brinley Franklin, Director at the University of Connecticut and Chair of the ARL Statistics and Measurement Committee, has been invited to present at the summer 2004 IFLA conference on the topic of cost-effectiveness of electronic resources. We would like to have a sense of all of your perspectives on this topic as during difficult financial times budget cuts often affect the untouchable (?) electronic resources.
Friday, June 20, 2003
Toronto Convention Center
Room 715B
3:30-5:00 p.m.
Toronto, ON
Our focus will be a brief update on the developments of the SPEC and Statistics and Measurement Programs. The bulk of our discussion will focus on the recent decisions of the ARL Statistics and Measurement Committee that will impact the annual statistical surveys in 2004 (a year from now). For your information, a copy of the agenda of the ARL Statistics and Measurement Committee meeting held recently in Lexington, Kentucky, is located here.
January 24, 2003
Hawthorne Suites Hotel
Franklin B Room
3:30 pm to 5:00 pm
Philadelphia, PA
A brief update regarding ARL Annual, as well as, SPEC surveys will be followed by presentations regarding two ‘new’ New Measures Initiatives:
Lisa O’Connor and Julia Blixrud will describe SAILS, the most recently developed ARL New Measures Project. For additional information on SAILS, please see the following article in College and Research Libraries by Lisa G. O’Connor, Carolyn J. Radcliff, and Julie A. Gedeon “Applying Systems Design and Item Response Theory to the Problem of Measuring Information Literacy Skills” College & Research Libraries, Vol. 63, No. 6, November 2002.
Brinley Franklin will describe the work done at four academic health sciences libraries in relation to evaluating networked electronic services usage patterns and how this work will be modified for a pilot across a group of ARL libraries. See: “Networked Electronic Services Usage Patterns at Four Academic Health Sciences Libraries,” by Brinley Franklin and Terry Plum, delivered at the Northumbria Lite conference, August 22, 2002.
Friday, June 14, 2002
Sheraton Altanta Hotel
Atlanta 4/5 Room
4:30-6:00pm
Atlanta, GA
The largest part of the meeting will focus on presentations of the ARL
Emetrics project led by Sherrie Schmidt and Rush Miller and completed
under contract by the Information Management Use and Policy Institute of
the School of Information Studies at Florida State University.
Background information, project findings and detailed reports from the
pilot activities covering May 2000 to December 2001 are available here.
During the June 14 meeting various libraries that have been engaged in
the collection and testing of the proposed measures will present the
processes and mechanisms they have established to collect and utilize
data on electronic resources and services.
In particular, Irma Dillon (University of Maryland), Kurt Murphy
(Arizona State), and Gordon Fretwell (University of Massachusetts) will
present their institutional experiences.
Sue Phillips will also be available to discuss the latest developments
with the revision of the NISO Library Statistics standard.
Friday, January 18, 2002
Marriott Hotel
La Galerie V Room
4:30 - 5:30 p.m.
New Orleans, LA
We look forward to seeing you in conjunction with ALA Mindwinter for a
brief update and a presentation by Doug Jones on his ‘new measures’
investigation in relation to the library’s impact on research.
Friday, June 15, 2001
Hyatt Regency
Seacliff A Room
3:30 pm - 5:30 pm
San Francisco, CA
The meeting will feature a presentation from the technical services pilot cost
study currently under way at five institutions led by Dilys Morris;
updates on New and Old Measures and Spec Kit surveys.
January 12, 2001
Loews L’Enfant Plaza Hotel,
Monet III Room
4:00 - 6:00 pm
Washington, DC
AGENDA
Discussion on Higher Education Outcomes introduced by Carla Stoffle:
Karen Williams and Julia Blixrud will facilitate a discussion on learning
outcomes and Doug Jones will facilitate a discussion on research outcomes.
Update on Other New Measures and Related Projects and Annual Statistical
Surveys by Martha Kyrillidou
Update on Spec Kit Surveys by Lee Anne George
Friday, July 7, 2000
Chicago City Centre Holiday Inn
Ohio Room
4:30 to 6:30 p.m.
Chicago, IL
AGENDA
1. Welcome, Overview, Introductions
2. SPEC Survey update (Lee Anne George)
3. The Perennial Question: How to count electronic serials? (Julia Blixrud)
Please come prepared to share with us:
# what electronic products your library subscribes/buys;
# where in the ARL Statistics survey do you report them;
# (both in terms of expenditures and physicalities);
# what are some of the problems you encounter in ‘counting’ electronic serials.
NOTE: this was originally planned in February 2000 but due to time constraints
we were not able to discuss it at that time.
4. Update on New Measures Initiatives: The SERVQUAL pilot project
(Colleen Cook and Bruce Thompson)
Friday, January 14, 2000
Menger Hotel
Ballroom B
5:00 pm - 6:30 pm
San Antonio, TX
AGENDA
1. Welcome, Overview, Introductions
2. SPEC Surveys update (Lee Anne George)
3. ARL annual surveys update (Martha Kyrillidou)
4. The Perennial Question: How to count electronic serials? (Julia Blixrud)
Please come prepared to share with us:
what electronic products your library subscribes/buys
where in the ARL Statistics survey do you report them (both in terms of expenditures and physicalities)
what are some of the problems you encounter in ‘counting’ electronic serials
January 29, 1999
Crown Plaza Hotel
Constitution Room
4:30 pm - 6:00 pm
Philadelphia, PA
Discussion Topics:
1. Counting Electronic Resources: What’s Possible, What’s Not
2. Developing New Measures: Report from a Tucson Retreat
Updates from ARL:
1. Statistical Surveys
2. SPEC Surveys
3. General news
Friday, June 26, 1998
Marriott Metro Center
Salon A
4:30 p.m.
Washington, DC
AGENDA
I. Welcome and Introductions
Sharing of information on use of statistics on campus
II. Status of Annual Surveys
1. Annual Salary Survey
Additional questions on demographics
2. ARL Statistics
Eliminate question on reserves, clarify serials count
3. Law and Medical
Changes similar to main
4. Supplementary
Revisions to remove external hardware questions
5. Preservation
No changes, but optional questions remain
6. E&G Report
will not follow time series
III. Schedule for 1998-99
IV. Research Library Investments in Electronic Resources
Project Update - Tim Jewell
V. Counting Serials
Discussion paper from Julia Blixrud
VI. ARL Statistics and Measurement Program Plans for 1997
1. Workshops
2. Survey Coordinator Web Page
VII. Performance Measures
VIII. ARL activities update
Friday, January 9, 1998
New Orleans Convention Center
Room 40
4:30 - 6:00 p.m.
New Orleans, LA
AGENDA
Update on statistical publications Changes to the interactive ARL statistics web site Data gathering for electronic resources Group discussion of ideas for other measures
If you have any internal data gathering forms that you are willing to share, let me know.
** Deadlines are imminent for both the salary survey and for the main, law, and medical statistics.
To close the books on the datafile for salary, we need all corrections ASAP.
In order to develop the index calculation, we’d like to have all the main statistics data in before Christmas.
Thanks for your cooperation.
** The revised questionnaire for the supplementary statistics is in final production and will be distributed
and on the web site before the end of the month. It will have a deadline of February 28.