I was asked to talk today about the notion of “diffuse libraries,” a concept I advanced in a paper last summer for the Council on Library and Information Resources. The notion is at heart an attempt to make sense of the forces that have shaped and are shaping academic research libraries—and to detect, appreciate, and perhaps predict the ways in which library roles are changing (or should change) in a digital age.
In that “diffuse libraries” paper, I suggested there were fundamental, recent historical forces that are critical to understanding our current and emerging environment. The impact of these forces was brought home to me recently as I worked on a talk for another conference. The conference theme —“visions and nightmares” – was drawn from a Michael Gorman paper with the same title from the early 1990’s, and was of the so-called “crystal ball” genre. In researching a bit about prediction literature I revisited the often-cited paper by Wilf Lancaster, “Wither Libraries?” You may recall that Lancaster predicts (back in 1978) a paperless society and raises the question whether libraries will be needed in a future of largely electronic content.
Some of Lancaster’s predictions included:
In this world-view, Lancaster sincerely questioned whether libraries would be necessary. The relationship between user and content is simple and direct. What else is needed?
We can look at those predictions and be amused, or perhaps lament, what has and has not transpired in the intervening 25 years. Lancaster’s predictions represented some classic linear thinking, focusing on what would happen if the publication environment was merely extrapolated from print to electronic. The 1970’s were, after all, a time of vended index files, with structured command language for bibliographic queries -- a time when SDI’s and profiles were a very reasonable way to conceive of “searching.”
Where did Lancaster go wrong? With the benefit of hindsight, we can see what transpired in the 1980’s and 1990’s. There are, I think, two key forces in particular that changed everything. These are the birth of distributed computing and the emergence of open/collaborative models. I believe we can see phases of library evolution mirroring these developments as well. I plan to say a few words about these factors and their significance before exploring the notion of “diffuse libraries.”
We all know that in the 1990’s, distributed computing and the Web democratized technology by bringing it to the desktop. As a result, individuals and institutions now had the basic capabilities for publishing and creating “libraries.” This trend prompted an explosion of information goods and services. For libraries, it created both potential competitors (ask-Jeeves, Google, and Amazon are good examples) and potential partners
As distributed computing and applications developed, we also saw the emergence of standards and the maturation of more intelligent systems. The evolution of standards for creating, structuring, and disseminating digital content allowed libraries (and other content-rich organizations) to move away from proprietary methods of information access and management that characterized the early days of electronic information. It also set the stage for sharing and integrating distributed content.
The maturation of intelligent tools and systems that developed allowed invisible mediation to occur between content and user. So, for example, we began to see programs that transparently take a user of Amazon to a list of related sources or services based on user preferences. Or within libraries, we might see a user move seamlessly from a journal index to the full text articles online.
One could argue that these intelligent developments could lead to the perception that libraries have become irrelevant, since system capabilities can assume mediation functions previously provided within libraries and often involving human intervention and interaction. However, libraries could also harness these capabilities to build far more robust and useful information environments and to conceive of the library system as a part of a larger network environment. The challenge these capabilities pose is to create the desired seamlessness of resources without making the library’s role invisible.
I noted that there were two major forces or evolutionary phases absent in Lancaster’s crystal ball. The second major trend that Lancaster failed to predict is evidenced in the development of open paradigms and models in the late 1990's. For example the Open Source movement – the concept of software development wherein the source code is shared and development is collaborative – reflects a fundamental shift away from proprietary software and systems. These open models also began appearing in an interesting array of new applications and venues, such as the Open Knowledge Initiative to share learning technologies and the Open Law program as a collaborative approach to crafting legal arguments.
Themes of openness and collaborative exchange also emerged in the context of scholarly communication. Notably the Open Archives Initiative began with the intent to address concerns within the scholarly community about limitations in commercial journal publishing. This, in turn, gave birth to new conventions of disseminating information, such as e-print archives.
As information becomes more distributed and open models of exchange become more common, processes of scholarly communication have begun to show subtle and not-too-subtle changes that could dramatically alter the library’s relationship with content creators, and publishers. Significantly within these trends there is also evidence of a critical shift underway in some areas from publication as product to "publication" as process.
I’m sure you’ve seen examples where commentary on a peer reviewed article might be incorporated in a journal service. We’ve also begun to see online discussions and conferences that are in themselves a real-time record of scholarly activity. These new dynamic or cumulating activities alter the classic characteristics of publication.
In another contribution to the “predictive genre,” Computer Scientist Hal Berghel has forecast that this shift will become increasingly prominent. He notes:
By 2100, our current view of electronic publications as copyright-able artifacts will be viewed primarily as a historical allegiance to a pre-participatory, non-interactive, essentially dull and lifeless era of publishing – an era in which one thought of digital libraries…as a collection of linked “things” rather than articulated processes and procedures. The current digital publication will be a relic, an obscure by-product from the horse and buggy age of digital networks.
Publication as “articulated process and procedures.” This is a frightening statement for libraries that are currently in the business of managing fixed objects, archives of copyright-able works. As this trend toward open models unfolds (possibly giving way to very dynamic, iterative processes), there are likely to be further changes in the library’s functions and its relative position in the continuum from content creation to distribution to management. The library’s engagement further upstream in the communication process is a new role, a shift from traditional focus on managing the end-products of these systems.
In this second phase in the evolution of library roles, the library starts to engage in collaboration as a strategy to address its core mission of building collections, maintaining access, and providing service. Building on distributed structures, the library begins to involve other stakeholders in fulfilling its functions, and in fact sustaining relationships among stakeholders becomes an essential activity. One definition of collaboration offered by IMLS’s Robert Martin notes that collaboration is the process of moving cooperation to a point of synergy. In other words, we’re not just talking about joint projects between various stakeholders when we use the “c” word – rather we’re talking about synergy, a mutually advantageous conjunction of participants or elements (such as resources or expertise).
With these distributed tools and open and collaborative methods, the library has the potential to become more involved at all stages and in all contexts of knowledge creation, dissemination, and use. Rather than being defined by its collections or the services that support them, the library can become a diffuse agent within the broader community.
What do I mean by “diffuse” roles or “diffuse libraries”? In physics, “diffuse” refers to the spreading out of elements, an intermingling (though not a combining) of molecules. Applying this analogy to libraries, we see the library becoming more deeply engaged in the fundamental mission or goals of the institution or community in ways in which library contributions are more broadly represented and more closely intertwined with the other stakeholders in these activities. The library becomes a collaborator within the broader environment, yet retains its distinct identity.
Back in 1999 at ACRL, I was a panelist commenting on a talk by Cliff Lynch. Cliff’s talk highlighted the changes underway in the so-called “scholarly canon” – the introduction of new genres, real time media, online conferences and the like that would challenge our prevailing practices for managing the record of scholarly activity. In my remarks, I suggested we needed a new metaphor for describing libraries in light of these changes – the movement from product to process that was evidenced in his examples.
University presidents often refer to the library as the “heart of the university” (the vital organ metaphor)– or perhaps as the “jewel of the crown” (too precious). I suggested to the audience that we needed to conceive of ourselves differently and suggested that a far more domestic metaphor of library as fabric would capture the changing nature of the library.
The fabric metaphor suggests that the library has the potential and the mission to provide functions that are diffuse and inextricably interwoven with the community of stakeholders. That is, at heart, what a diffuse library is all about – becoming an integral part of the community, of weaving library expertise throughout the learning, teaching, research, and service functions of the institution. Four years later, that metaphor still feels right to me.
Overall, we can characterize the trends to diffuse libraries as a series of shifts. Library roles are shifting from:
In the longer CLIR paper, I explored classic roles of libraries and highlighted areas where we are beginning to see evidence of diffuse characteristics. Let me provide a few examples here for those of you unfamiliar with the concept, and then I’d like to move on to talking about future stages of diffuse libraries—realizing the “library as fabric.”
Obviously a primary, classic role relates to collection building. Libraries have been in the collection business for centuries and are defined largely by the functions of collection development and management—i.e., the continuum of processes to select content appropriate for a particular community, make it accessible, manage it, and preserve it. These discrete functions have been viewed as necessary components of good collection stewardship, and they have obvious definition in a physical context. Libraries bought books (which they then owned), organized them, made them available through library facilities, and took steps to ensure the longevity of the volumes for future use.
While there are certainly some models in the electronic domain that preserve this continuum, there are others that unbundle the discrete component functions of traditional collection development and management responsibilities. One model (that has been realized in many digital library projects) relates to the notion of content federation.
Federated systems allow content that is physically distributed on the network (i.e., managed by different providers and delivered via different systems) to be brought together and used as an integrated collection. Individual content managers retain ownership and governance over each discrete collection, but the content is made accessible under the principles of the larger federating system.
Whereas traditional models bring content control to the library and create a central access strategy, the federating model balances distributed content and collection-specific functionality with cross-collection functionality and tools.
In the federated process, the library needs to work with content providers to negotiate content, often there is a mapping of metadata or use of other tools to bring metadata together. A key feature, however (that is characteristic of diffuse models) is that the library is engaged in building, sustaining, and developing relationships with distributed partners, in this case the distributed content providers.
In the federated model, the library controls neither the content nor the permanence of these resources. To the extent that component collection databases are dynamic and subject to the decisions of the distributed collection managers, the library must forgo its traditional archiving role. In fact, it loses a good deal of control, which introduces risk into the information environment. However, this also prompts a new responsibility for the library in influencing and educating individual content providers, the institution, and the community about the requirements for preservation and archiving.
We see in this example, the emergence of expertise as the “coin of the realm.” The library holds fundamental expertise with respect to understanding content, its structure, and the characteristics of an access system necessary for effective use.
Other examples of libraries sharing collection and content expertise involve libraries serving as publishers or at least playing a much more active role in disseminating content. We have HighWire Press at Stanford, Columbia University's work with EPIC, Johns Hopkins and MUSE. I think we could also categorize work with institutional repositories in this context. California Digital Library’s work with e-prints, MIT’s D-Space, and Ohio State’s Knowledge Bank all fall in this arena of creating the infrastructure to manage and services to exploit the intellectual goods of an institution. Along the way, too, we see these repositories evolving around specific disciplines and creating an environment for sharing within a subject domain.
Let’s move on to issues surrounding Information Access. Organizing and providing access to information is another classic role. The twin functions of cataloging and classification have allowed published works to be fixed in a framework of knowledge and to be given multiple access points for retrieval, a combination that has supported inquiry well over time. These descriptive functions have brought predictability and a cumulative order to vast amounts of material.
Traditional access techniques have been largely undifferentiating and unintrusive; that is, all materials added to libraries have generally had the same descriptive treatment and the functionality or structure of the works themselves have not been materially altered by these processes. In the continued evolution of description for resource discovery, we’ve seen richer descriptive metadata apparatus developed that takes advantage of the distinct characteristics of digital resources. We’ve also seen library communities engaged with discipline communities to deepen the metadata structures for particular domains of interest.
These practices sustain the library’s focus on description as a primary strategy for access. As the distributed environment and tools evolve, however, we've begun to see a shift in the role of the library in providing access. The past focus on uni-dimensional descriptive techniques to bring order and control to local resources has given way to multi-dimensional techniques that exploit distributed resources and systems through techniques that actually penetrate and operate on the content. Resource discovery is only one aspect of what access is all about in this broader context.
As a prime example, the work within the Open Archives Initiative to develop interoperability standards for content dissemination provides the impetus to rethink the library's role. The functions of description and access have been de-coupled and libraries can both provide metadata to be harvested by others and/or create services on top of harvest-able metadata from others.
We are beginning to see models of a layered architecture for digital libraries wherein both generic and specialized access services are designed on top of (or to act upon) distributed digital repositories. These services might include subject specific access services or format-based services (e.g., GIS or EAD), or function-based services (e.g., course content services). In this model, content can be re-purposed for multiple audiences, uses, tools, a model that Lorcan Dempsey references in his notion of “recombinant libraries.” As we unbundle metadata, services, and tools, these resources become building blocks for constructing applications. This broader framework for access obviously requires the library to have a far more intimate understanding of the needs for each of these purposes and to understand the relationships between and among resources. Access, like content, is unbundled into component processes. Access becomes an array of functions the library develops to act upon distributed content with its associated descriptive apparatus.
Many of you sent me wonderful examples of diffuse library service roles, for example, instances where the library partners to provide information literacy curricula (e.g., Washington University’s Wanda program, or University of Washington’s UWired efforts). Many libraries offer online or interactive reference services or participate in emerging programs for cooperative reference services. The latter is interesting for the efforts to profile individual and institutional specialization in systematic and structured ways to achieve coordination among a worldwide network of participants.
A key feature within coordinated, distributed reference services is the potential to make access to expertise ubiquitous, no longer available only within libraries. That sets the stage for re-purposing expertise as well –i.e., determining how best and when to offer it as part of broader library systems of access. Several institutions (e.g., Virginia Tech, Michigan, and Minnesota) have created models whereby librarians with specialized expertise serve within the physical context of academic departments to expose the community to that expertise and to incorporate it into teaching and research functions of academic departments.
While the traditional notion of library services focuses on user-initiated requests within a library facility, the more diffuse constructs bring reference and specialized expertise to a wide range of contexts, within libraries and within virtual contexts. Ultimately, the library's presence becomes more pervasive and its services more fully integrated into the processes of learning and research.
When I originally drafted the report a year ago, I had trouble categorizing library functions. Clearly, our traditional constructs of collection, access, service are changing all the time and increasingly it is hard to characterize library activity as purely within one of these domains. This difficulty undoubtedly says something about the maturation of library models.
Let me try to offer two examples wherein libraries have launched major initiatives that defy neat categorization and are transformative – that is, they build on unique library expertise, and seek to play a role that catalyzes as well as supports the interests and assets of the scholarly community. They offer models that typically incorporate the building blocks we’ve talked about: distributed resources, federating and harvesting techniques, service layer architectures, and relationship management.
Cornell University has recently created a Digital Consulting and Production Service (DCAPS) agency, bringing expertise in creation and conversion of content, access and metadata applications, copyright permission support, information management and retrieval infrastructure services, and potentially archiving of content. Cornell’s DCAPS is part of a larger planning effort to potentially leverage Library assets toward assisting smaller libraries, university presses, publishers, and others. This new service is conceived as a strategy to transform information services within higher education. For example, the DCAPS facility might at some point be an incubator for editorial offices or could create the venue for ensuring sustainable institutional repositories over time. Cornell’s Ross Atkinson has suggested that the library’s purpose is not to provide access to information, but rather to “facilitate its production.” He notes that “provision of access is just a step along the production path.” By creating services in support of the production chain, the library is ensuring that distributed content can be recombinant and can be sustained.
The University of Virginia's concept of information communities in its Library of Tomorrow planning process offers another interesting new diffuse model. Each information community project draws on harvesting protocols to bring together distributed content that represents disparate types of information (text, data, media, images). The target audience, or community, is defined as a group of scholars, students, researchers, librarians, information specialists, and citizens with a common interest in a particular thematic area. Each project brings together participants (as providers or users) from a particular subject domain. For example, the American Studies Information Community is being undertaken collaboratively with other institutions and content providers (e.g., Thomas Jefferson Foundation, Virginia Tech University, and the Smithsonian Museum of American Art).
The University of Virginia describes these information communities as “learning and teaching environments in which subject-driven websites are developed around print and digital versions of our collections and the teaching interests of our faculty members...Information communities will foster interdisciplinary and collaborative research and publication amongst scholars with common interests.”
This model is interesting because it takes advantage of a distributed collection model and a range of partners. The descriptive techniques will reflect enhanced attributes appropriate to the subject area and the diverse formats in the distributed collections; harvesting tools will be used to federate content, and analytic tools will be incorporated to add value to the content and to stimulate collaboration. Perhaps most significant, the system and services are explicitly designed to serve a social role as a catalyst for an interdisciplinary community. This is a far more intrusive role than is provision of access and content alone.
One could imagine variations that bring in institutional content or integrate online services. This integration of content, services, data begins to mirror the construct of a collaboratory, for focused research communities. Collaboratories have been defined as "tool-oriented computing and communication systems to support scientific collaboration" (National Research Council 1993). For example, the Space Physics and Aeronomy Research Collaboratory (SPARC), provides an online knowledge environment for atmospheric scientists worldwide. SPARC incorporates the ability to control remote telescopes and instrumentation, to review and collaboratively analyze observational data of atmospheric events, to create and archive vast amounts of research data, and to use tools to manipulate the data. These types of robust information environments are also envisioned in the recent Cyberinfrastructure report.
To the extent that libraries begin to develop access techniques in response to a community and to support the potential development of collaboratories for these communities, we see them assuming a far more integral role within the scholarly arena. The role of the library moves from manager of scholarly products to that of participant in the scholarly communication process.
I hope these examples have helped make real the notion of diffuse libraries. Enabled by distributed resources and open methodologies, we see that the functions and roles of libraries have been transformed. While there may be areas where we can see legacy roles, in many cases, the library realizes new roles that are essentially expertise-based.
Diffuse libraries are increasingly social agencies – that is, they are called upon to comprehend and engage the needs of a community. As our longstanding notions of publication move to “articulated processes and procedures” the library is challenged to be a player in these processes--whether it involves knowledge creation, management, dissemination, or use. The library is not only a player, but possibly an agent of change.
Diffuse libraries. Recombinant libraries. Libraries as social agents. These are powerful concepts.
Let me return to my domestic metaphor: libraries as fabric. If the library is truly integrated with activities of the community, the library’s reach must be sufficiently interwoven such that the warp and weft of the community environment are weakened when tension is created by constraints on the library, or embellished when the library’s strengths are realized more broadly. Weaving the fabric ensures that the answer to Lancaster’s question -- “Whither libraries”? -- is one in which the library has tremendous potential for the future. The library adds durability, it adds shape, it adds seamlessness to an increasingly distributed and open world.
This talk draws on a report prepared for the Council on Library and Information Resources: Diffuse Libraries: Emergent Roles for the Research Library in the Digital Age (2002), http://www.clir.org/pubs/abstract/pub108abst.html.