The Online Books Evaluation Project at Columbia University explores the potential for online books to become significant resources in academic libraries by analyzing (1) the Columbia community's adoption of and reaction to various online books and delivery system features provided by the Libraries over the period of the Project; (2) the relative life cycle costs of producing, owning and using online books and their print counterparts; and (3) the implications of intellectual property regulations and traditions of scholarly communications and publishing for the online format.

Online books might enhance the scholarly processes of research, dissemination of findings, teaching, and learning. Alternatively, or in addition, they might enable publishers, libraries and scholars to reduce the costs of disseminating and using scholarship. For example:

• If the scholarly community were prepared to use some or all categories of books for some or all purposes in an online format instead of a print format, publishers, libraries and bookstores might be able to trim costs as well as enhance access to these books.

• If online books made scholars significantly more efficient or effective in their work of research, teaching and learning so as to enhance revenues or reduce operating costs for institutions of scholarship, these books might be worth adopting even if their costs were no lower than those for their print counterparts.

• If an online format became standard, publishers might be able to offer affordable online access to books to institutions which would not normally have purchased print copies, thus both expanding convenient access to scholarship to members of those institutions and expanding publishers' revenues from these books.

The Columbia Online Books Evaluation Project is designed to learn about the scholarly community's enthusiasm for the online format in the near term and about features that users will demand, to project likely adoption patterns, and to estimate gains in operating effectiveness, revenue, and cost, if any, to be realized by publishers, funders of scholarship, libraries, and scholars. The Project confronts and explores a set of feasibility issues, including publishers' ability to provide books of various types and vintages in forms conducive to conversion to online formats and our ability to convert them to online books that will serve users' needs and preferences.

This paper focuses on the first of the Project's elements, user response, and reports on:

1. the conceptual framework for the Project; (Section 2)

2. background information on the status of the collection and other relevant Project elements; particularly design considerations; (Section 3)

3. our methodology for measuring adoption of online books by the Columbia community; (Section 4.1)

4. our current findings on relevant environmental factors, including access to online resources; (Section 4.2)

5. our current findings on use of online books and other online resources; (Section 4.3, 5.2)

6. our current findings on attitudes toward online books. (Sections 6 and 7)

The paper also reflects on our experience as a case study, specifically (1) problems encountered evaluating online resources (Section 6), and (2) problems encountered in producing online books (Section 3.2). The Project began in early 1995 with three reference works online; in autumn 1996 the first modern monographs became available to the Columbia University community which is the focus of the study.

Our current findings may be summarized as follows:

• Most if not all reference books are used more heavily online than in print. (Section 4.3.1)

• Early online reference books have experienced falling usage over time, substitution of use of a new delivery system for an old one, or a smaller rate of growth of use than might be expected given the explosion in access to and use of online resources in general. (Section 4.3.1)

In the early to mid-1990s, the novelty of online books may have brought users to the format somewhat without concern for their design, the utility of the delivery system, or the qualities of the books. With enhancement in delivery systems and expansion in the number of online books, being online is no longer a guarantee that a book will be used. New graphical delivery systems offer superior performance that is likely to draw scholars away from these early online resources provided via text-based systems increasingly, as access to those new delivery systems spreads. In addition, as more competing resources come online and either provide information that serves the immediate needs of a user better or offers a more attractive, user friendly format, scholars are less likely to find or to choose to use any single resource.

• Online scholarly monographs are available to and used by more people than their print counterparts in the library collection. (Section 4.3.2) Once a print book is in circulation, it is effectively unavailable to others for hours in the Reserve collection and weeks or months in the regular collection. An online book is always available to any potential user who has access to a computer with a Web browser.

• Being online may bring to a book scholars who would not have seen it otherwise. (Section 4.3.2) However, it is not yet clear whether their productivity or work quality will be significantly enhanced by such serendipity. The important concept of collation is transformed, in the networked environment, to a diversity of finding and navigational systems. As the online collection grows, browsing will require the focused use of online search tools rather than use of project-oriented Web pages.

• Data from the most recent 11 weeks, including the last half of the spring 1997 semester, suggest that when a social work book available in both print and online formats was used in a course, the share of students using the online version was at most one-quarter. (Section 4.3.4) We will track this rate of penetration for social work and other disciplines over the next semesters to see if students increase their rate of adoption.

• Some scholars, especially students in a course assigned a reading that is in the online collection, are looking at the online books in some depth, suggesting that they find value in this means of access. (Section 4.3.4) For example, in the most recent 11 weeks analyzed, the two most frequently used monographs averaged 9.6 and 7.7 hits (chapters) per unique user.

• Scholars residing off-campus are not using the online books from their homes to a significant degree. (Section 4.3.2.2) For the ten months from May 1996, only 11 percent of the hits on Columbia University Press monographs were dial-up connections. Scholars report (in our interviews) that the expense of dialing-in to campus or maintaining an Internet account, the lack of sufficiently powerful home computers and Web software, and the slowness of delivery of the Web over standard modems are key constraining factors.

• Students residing on campus may have Ethernet connections to the campus network - providing both speedy and virtually free access to the online collection. They are using online books, especially reference works, from their dorms during hours when the libraries are not open. Forty-two percent of the hits on The Oxford English Dictionary in the ten months from May 1996 were from computers with such residence hall connections. (Section 4.3.1.2)

• Some scholars perceive gains in the productivity and quality of their work in using online books, particularly reference books. Over half the respondents to our online survey (a small number) see the productivity and quality of their work using online resources to be as good or better than that achieved using paper resources. (Section 6.2.11)

• In surveys and interviews, students report that they particularly value easy access to the texts that are assigned for class and an ability to underline and annotate those texts. Students seek the ability to print out all or parts of the online texts that they are using for their courses, again indicating their desire to have the paper copy to use in their studying. Computer access to a needed text is not equivalent to having a paper copy (whole book or assigned portion) in one's backpack, available at any time and at any place. (Section 5.2.4)

If the effective choice is between borrowing a book from the library, probably on a very short term basis from Reserves, and accessing the book online, the student is facing a parallel situation of needing to photocopy or print out to obtain portable, annotatable media. However, the online book has the advantages of never being checked out when one wants to use it and of being accessible from a computer anywhere in the world at any time (as long as that computer has an Internet connection and a browser).

• In surveys and interviews we find that scholars value the ability to do searches, to browse, and to look up information in an online book quickly. They also like the ability to clip bits of the text and put them in an electronic research notes file. Willingness to browse and to read online for extended periods varies from person to person, but it does not seem to be widespread at this time.

• Scholars with easy access to a networked computer spend more time online and are more likely to prefer to use one of the forms of the online book. (Section 7) This suggests that, over time as such access achieves greater penetration in the scholarly community, online books will be achieve greater acceptance.

• In the most recent 11 weeks studied, 52 percent of the online book users who viewed our online survey responded to it, but only 15 percent of these users chose to click on the button taking them to the survey. (Section 6.2.14) Designing an online survey that is available to the reader without his taking action might enhance the response rate significantly.

2. CONCEPTUAL FRAMEWORK

The variables representing usage of a system of scholarly communication and research are at the same time effects and causes. Since scholars, the users of the system, are highly intelligent and adaptive, the effect of the system will influence their behavior, establishing a kind of feedback loop. As the diagram in Figure 1 shows, there are two key loops. The upper one, shown by the dark arrows, reflects an idealized picture of university administration. In this picture, the features of any system are adjusted so that, when used by faculty and students, they improve institutional effectiveness. This occurs in the context of continual adaptation on the part of the users of the system, as shown by the lighter colored arrows in the lower feedback loop.

All of this is constrained by the continual change of the environment, which affects the expectations and activities of the users, affects the kind of features that can be built into the system, and affects the very management that is bringing the system into existence. This interaction is shown by the dotted arrows in Figure 1.

Figure 1. Interrelation of Factors Involved in the Use and Impact of Online Books.

Our primary research goal, in relation to users, uses, and impacts, is to understand these relationships, using data gathered by library circulation systems, Internet servers, and surveys and interviews of users themselves.

[Table of Contents] [Next Page]