The Digital Object Identifier (DOI) is an important emerging international standard for identification of published material online. It forms the foundation layer of a set of technologies that will enable commerce in published material on the Internet so that copyright is protected, content creators can be compensated for their work, and consumers can benefit from technology that is sophisticated, yet seamless and easy to use.

This is the story of the genesis of the DOI, told by a participant in the project. I will explain, from my perspective, what the book and journal publishers who developed the DOI were trying to do, what decisions they made, and why. They started out wanting to overcome all of the myriad obstacles to successful online commerce in content, but they quickly decided that that was inadvisable. Instead, they decided to start with a single core issue — that of content identification — and set up a framework for building upon it. The increasing momentum in the industry surrounding the DOI shows that those decisions were the right ones.

The Dilemma of Publishing for Profit on the Internet

Our story begins in autumn 1994, when the Association of American Publishers (AAP), the trade association for U.S.-based book and journal publishers, established its Enabling Technologies Committee. They wanted to solve a range of problems that, taken together, posed a set of tradeoffs and conundrums about protecting intellectual property and sustaining commerce on the Internet.

The Internet is well known as a threat to copyrighted material because it enables global ubiquity of information through technologies that allow people to make as many perfect copies of content as desired. On the other hand, the Internet enables many new types of "infomediary" second-tier content businesses that can create new revenue streams for publishers. Those businesses are analogous to bookstores, book clubs, clipping services, or libraries in the real world: they can retail, aggregate, repackage, or redistribute content online.

Such businesses provide value by offering information in forms that consumers find convenient. Therefore, publishers need to encourage those new types of businesses. To foster their growth, publishers can make it easy to do business electronically with all publishers in the same way — in other words, to offer some degree of interoperability among publishers in the online environment.

But how much interoperability is feasible without compromising intellectual property? The more interoperability that exists among electronic publishing formats and systems, the easier it becomes to make unauthorized perfect copies of copyrighted information, because information can flow more smoothly and automatically across systems. Each publisher who thus far has tried to sell information online has had to adopt proprietary technology that is unlikely to be compatible with that of any other publisher. Those proprietary systems have the "advantage" — as far as copyright protection is concerned — of being more difficult to access.

Interoperability among publishers presents a classic threat-versus-opportunity tradeoff: protection through restriction versus enabling commerce through access. In meetings of the Enabling Technology Committee, that tradeoff was represented as the "lawyers vs. marketers" dilemma. The lawyers wanted to protect copyright at all costs, and did not worry about alienating customers by restricting information, whereas the marketers wanted to put material out on the Internet at all costs, and did not worry about losing revenue through "bleed" (unauthorized copying).

This protect-vs.-enable tradeoff is surely one of the most fascinating conundrums of the digital age. At one extreme of the continuum is the "information wants to be free" dogma espoused by the Electronic Frontier Foundation, which helps neither publishers nor second-tier content businesses.

At the other extreme is proprietary, end-to-end content protection technologies that store content in encrypted state and enforce a "pay-per-view" business model. Those technologies, which have their origins in the CD-ROM market, have embraced the Internet as a distribution medium. Companies like Portland Software, InterTrust, TragoeS, and IBM (through its InfoMarket group) have been developing systems that enclose content in a secure "container" and allow access only in particular conditions — though not necessarily limited to pay-per-view.

Those secure container technologies can force publishers into situations that inconvenience and ultimately alienate customers. A user who wants to get at a piece of content has to figure out who the publisher is, whose client application or "plug-in" they have to use, what forms of payment are acceptable, and so on. That is analogous to — but even worse than — the current fragmentation in the Internet video market, where you need to figure out whether the video clip you want to watch is in VXtreme WebTheatre, RealNetworks RealVideo, VDOnet, MPEG-1, or some other format.

Such a situation has other negative implications for publishers. In many market segments it would favor those who use unrestrictive technology to distribute content — including those who garner revenue solely through advertising. In other words, publishers who cannot or choose not to solicit advertising may have to compete with those who publish information free — further supporting the "bad money drives out good" economic theory. A well-known example of this in the non-digital world is school books that consumer-product companies sponsor and give away to schools.

Open Standards Provide a Solution

The intent of the AAP's Enabling Technologies Committee was to support copyright protection, while ameliorating inconvenience to users, by supporting technology that promotes interoperability. One way to accomplish that would have been for the AAP to work with one vendor to develop a solution that satisfied everyone and somehow adopt it industry-wide. But that's not a good idea, because it would create a monopoly situation that not only would offer no incentives for the chosen vendor to improve its solution over time, but would also leave the AAP wide open to being sued for anticompetitive practices.

We decided it would be far better to develop a set of interoperable standards on which technology vendors could build components, such as client-side player/viewers, content servers, encryption schemes, and transaction-processing tools. Such standards should be open, meaning that they are published, comprehensive, and in sufficient detail to implement. That would engender a competitive market, for both client applications and server software, that favors users and publishers alike by offering them choice, quality, and value — analogous to the market for "IBM compatible" PCs or Web authoring tools. It would also give publishers maximum flexibility to take a position between the two extremes of free information and proprietary solutions, one that minimizes obstacles between customers and content while maximizing intellectual-property protection.

The AAP's true goal was to standardize on some technology that would promote copyright protection. An online-content transaction that supports copyright protection would have to include various pieces of information in the communication between publisher/server and customer/client, such as:

1. Content identification, a "tag" on each item that identifies it uniquely and over time.

2. Content description, such as bibliographic data and key words, that enables searching and browsing.

3. Access rights that state, for example, whether a user may view, print, or copy an item, whether he or she is buying, borrowing, or accepting transference of rights, and under what conditions those rights are conveyed.

4. Display formats, such as HTML, PDF, or XML for page-like content, or other formats for sound and video.

5. Content protection schemes, such as encryption and watermarking.

6. Financial-transaction information, such as account numbers, prices, etc.

But then the question was, What should we standardize on? The Committee decided to build a foundation and work up. It became clear fairly quickly that the most basic part of that foundation was content identification, just as ID numbers are the most basic components of inventory or personnel tracking systems. The other components could come later.

We decided that financial-transaction standards (like Secure Electronic Transaction, or SET, developed by MasterCard and Visa) would be out of scope: Such standards would clearly be developed for a range of electronic-commerce applications much broader than publishing. Content protection schemes were problematic for a number of reasons. Most importantly, all such schemes are proprietary from end to end, meaning that the AAP could not adopt them as "standards" because of the monopolistic considerations. Also, it was (and still is) unclear to publishers which of the technologies, if any, had the best combination of user convenience, piracy deterrence, applicability to publishers' business models, and legal support.

We didn't address display formats standards, either. The issues there are relatively straightforward: HTML and PDF are mature and supported directly in many standard Web browsers; XML (Extensible Markup Language) is an interesting emerging document-markup standard, and support for it is built into the latest version of Microsoft's Internet Explorer. Furthermore, the committee did not want to limit itself to the kinds of information that could be displayed in page-like formats.

Even though the committee chose not to consider access rights and content description immediately, we agreed that they are necessary requirements of sophisticated electronic-publishing schemes and therefore that we would revisit them.

Access Rights

Access rights standards are the beasts in the closets of many publishing companies. The committee's consultant, Christopher Burns, surveyed a variety of publishing companies about their perceived needs for rights and permissions management. The answers were surprising and dismaying: Most publishing CEOs felt that they had no problem in that area, while a few others claimed that they understood the problem but found it too difficult and expensive to solve. None admitted that they needed to get their rights-management houses in order or claimed to be working on a solution.

My personal experience suggests that these publishers are acting short-sightedly. What is the problem exactly? Many publishers negotiate for rights to use other publishers' content in various ways, such as excerpting and electronic distribution. Performing those transactions usually requires checking a contract to see if the use is allowed, and if so under what conditions, then generating and signing a letter, and perhaps issuing payment. That is a paperwork-intensive, manual process. At the same time, end-users also engage in rights transactions with publishers, such as buying content, borrowing it (buying with a time limit), getting it for free, or some complex combination (e.g., first 10 views free, then it costs a nickel; or $10 for a month, $5 per month thereafter).

It's possible to treat both publisher-to-publisher and publisher-to-customer transactions as special cases of the more general concept of content access-rights transactions. That is a good idea, because the potential for flexible business models in the online world requires that such transactions be fully automated so that they can be performed on the fly. To do that, publishers would need to make rights conform to a template or rights model from which all possible types of rights transactions can be derived; then, all rights transactions could be automated in a uniform way.

To truly enable flexible, standards-based online publishing, all publishers should ideally adopt the same rights model — a "universal" rights model that can adapt to any conceivable type of transaction. Mark Stefik, a scientist at Xerox PARC research labs, has done some interesting work on a universal rights model[1] Some committee members decided to get together on their own, outside the auspices of the AAP, to attempt to solve the problem[2]; those efforts did not bear fruit because we could not agree on a common data model — a way of describing how rights are cleared at each publishing company.

Content Description

The area of content description is also, to some degree, a problem that publishers must solve internally. Solving it requires adopting a metadata model, a collection of attributes with which a publisher can describe all of its content in ways that guarantee its ease of accessibility through browsers and automated methods (such as interfaces to Web services). This article, for example, might have these metadata attributes, among others:

After a publisher decides on a metadata model, it can — analogously to the rights management situation — build a database of metadata for every content item it has. But unlike publishers' reaction to the rights-management problem, publishers are interested in building such metadata databases. They are precursors to digital content-management systems, which are of great interest to publishers today because of their potential in reducing inventory and cycle time, and in facilitating revenue opportunities through content repurposing.

Standards for bibliographic metadata exist, the best-known being the Dublin Core. But in general, metadata models that are truly useful to publishers need to be far more detailed than bibliographic models are. The more detail there is (i.e., the more metadata attributes there are), the more difficult it to standardize, because necessary attributes vary widely from one type of publishing (e.g., medical journal articles) to another (e.g., mystery fiction) — and even more widely among content business outside of book and journal publishing, such as motion pictures, sports videos, and music. There are several efforts in various media businesses to address standardization of metadata[3], but the AAP Committee has not, thus far, decided to adopt any one of them.

Content Identification

The AAP Enabling Technologies Committee decided that standardized content identification was the most basic requirement. Given the committee's decision to back open standards, the dilemma was how much to standardize over the long term. There would be a tradeoff between the strength of the standard and the breadth of its acceptance among publishers. If the standard were too weak, then everyone might accept it in the short term, but its lack of perceived value would kill its momentum over the longer term. On the other hand, if the standard were too comprehensive, then it might have plenty of value, but publishers would be reluctant to accept it: details would be more likely to be controversial, resulting in arguments that drag on forever, and adherence would be too costly and disruptive.

So the AAP adopted its strategy of standardizing content identification as a kind of "hook" to get publishers to cooperate, then waiting until after that standard achieved momentum to move on to other issues, such as content description. Yet even now, content-description issues are being recognized as a necessary part of the overall electronic publishing architecture. The current DOI project committee is investigating the use of various metadata element sets (including Dublin Core) as a means of building a bibliographic database as an optional adjunct to the DOI directory itself.

(Rights and permissions issues may become important for the DOI later, when more electronic commerce infrastructure gets put in place that can support the more complex, transaction-oriented online business models that demand rights and permissions management.)

The Digital Object Identifier

Once the Enabling Technologies Committee established its strategy, it spawned a subcommittee that designed the identification scheme now known as the Digital Object Identifier. We decided to create an identification scheme that was maximally flexible and could coexist with all of the other systems that we envisioned would be involved in electronic publishing, as well as with any future standards we might define for areas such as content description and access rights. It also had to recognize the unique nature of publishing rights: Ownership may change over the life of a copyright as intellectual properties are transferred from author to publisher and from publisher to publisher. In all, we came up with these requirements:

• The identifiers would be as "dumb" as possible, i.e., they would have no intrinsic meaning.

• Identifiers must uniquely identify content items; there can be no duplication.

• There could be infinitely many identifiers, with no limitation. Publishers will want to assign identifiers to individualized, potentially ephemeral "products" created on the fly.

• It would be a "meta-scheme" that would subsume any existing scheme that publishers might be using already, such as ISBN, ISSN, BICI, SICI, etc., as well as Web URLs.[4]

• Unlike the schemes mentioned above, it could be used for content of any type, including static print-like material, services (e.g., subscriptions, time-based access to content), software, audio, video, etc.

• It could also apply to objects of any granularity, ranging from the online equivalents of multi-volume sets down to individual paragraphs or illustrations. Granularity decisions would be left entirely up to the publisher. Furthermore, an identifier c ould apply to a large object composed of smaller objects, each of which has its own identifier.

• An identifier would persistently refer to a content item regardless of its physical location or ownership, even if either are transferred to another publisher.

DOI Syntax

The resulting scheme was called the Digital Object Ide ntifier. DOIs actually are in line with the Internet Engineering Task Force's preliminary specifications for Uniform Resource Names (URNs). URNs are more general than standard Web URLs; they also have the properties of persistence over time (i.e., they do not become invalid) and high availability (e.g., they do not depend on a single Web server being up and running).

DOIs have a simple syntax, which is shown in Figure 1.

Figure 1: Syntax of the Digital Object Identifier.

The DOI has two parts: the prefix and the suffix.

The registrant prefix (1016 in this example, corresponding to Elsevier Science) is a number that corresponds to the publisher (or other intellectual property owner) that assigns the DOI. The prefix is prepended with a number (10 in this example) that can denote the directory that assigned the prefix to that publisher, followed by a period (.).

A slash (/) separates the prefix and the suffix.

The suffix begins with an optional code designator in square brackets, which can be helpful as a signal to software (or human users) that the following item ID uses a known identification scheme. Item IDs can be any character string, although the current implementation has a limitation of 128 characters. Almost any printable character is valid in item IDs. (There are a few minor limitations when DOIs are embedded within URLs.) URLs can even be embedded within item IDs. In the example in Figure 1, the suffix is [SICI]S1384107697000225. The [SICI] signals that the following item ID is a Serial Item and Contribution Identifier, while the item ID itself is S1384107697000225.

The Directory: Making DOIs Work

DOIs work by means of a directory that resolves them to Internet addresses, which are Internet URLs. URLs are typically Web addresses (HTTP), but they can also be addresses for such things as FTP and gopher servers. The DOI System will ultimately have a global, distributed, highly available network of directories, but currently, there are three directory servers: one in Reston, Virginia, the others in California, all run by the Corporation for National Research Initiatives, with directory ID 10. In the future, third parties — such as publishers, "infomediary" Web sites, and other value-added service providers — may be able to run their own directories in the network. If that capability is built into the underlying technology, then each will get its own directory number.

Each directory has a manager who assigns DOI registrant prefixes and establishes password-protected interfaces to publishers and other intellectual-property owners. A publisher can get a single prefix or many, e.g., one for each division or imprint. After a publisher receives its prefix, it can deposit full DOIs with the directory. To deposit a DOI, the publisher must supply both the DOI and a URL. Publishers can register one DOI at a time or, more commonly, in batches. (Some publishers have already registered tens of thousands of DOIs in one step.)

The URL does not necessarily point to the content item itself. At a minimum, it should point to a response page that tells the user — or a piece of software — what to do next. Here are some examples of what a response page can do:

• Show some information about the content item, such as author, title, publication date, an abstract, perhaps a GIF image of the cover, and an offer to sell the item to the user — like book description pages on Amazon.com[5].

• Provide an order form for subscribing to a journal.

• Offer to transmit the content itself in a secure container.

• Download a Java applet that runs on the user's machine.

• Give information that the DOI refers to an out-of-date publication, perhaps with an offer to link to the most recent version.

A DOI never changes once it is registered. However, a publisher can change the URL corresponding to a DOI in the directory. That is one of the most important advantages of DOIs — they are persistent identifiers that provide a buffer between a reference to an object and its physical location. If a publisher restructures its Web server, installs a new content-management system, or changes its internal organization of content in some other way, it need simply change the URLs corresponding to DOIs it has registered.

DOIs don't change even if a publisher sells or otherwise transfers ownership of content items to another organization. Even the prefixes stay the same. The prefix of a DOI is primarily meant to ensure that the directory issues no duplicate DOIs, and that publishers have maximum freedom to assign whatever item IDs they wish. Therefore, DOIs can find content even if — as happens all too frequently — product lines are bought and sold among publishers.

Using DOIs

DOIs can be used in various ways. At the most minimal level, users can enter DOIs by hand (or cut and paste them) into a lookup screen [formerly http://www.doi.org/lookup.html] at the DOI directory's Web site and get them resolved to URLs. But a more typical way of using DOIs is to embed them in URLs for simple, click-to-resolve functionality. Currently, a URL of the form http://dx.doi.org/DOI initiates the resolution process when invoked.

As a very simple example, http://dx.doi.org/10.1000/9 contains the DOI 10.1000/9, which belongs to the DOI Foundation (prefix 1000) and refers to the DOI Frequently Asked Questions document (item ID 9). In this case, the DOI resolves directly to the document itself at URL http://www.doi.org/faq.html. But if The International DOI Foundation wanted to charge for access to that document, offer to put you on its mailing list, inform you that the document is obsolete, or even just ensure the longevity of the reference, it could use a DOI and embed it in a dx.doi.org URL.

If the DOI Foundation were to transfer ownership of that document to another organization, the DOI for the document would stay the same, even though the URL to which it resolves might change. For example, let's say that the International Federation of Reproduction Rights Organisations (IFRRO) took over responsibility for the DOI. Then the document might move to IFRRO's Web site, and the URL in the DOI directory would need to be changed to something like http://www.copyright.com/ifrro/doi/faq.html. But the DOI would still be 10.1000/9.

Figure 2 shows how the DOI directory resolves URLs.

Figure 2: Resolving a DOI through the directory.

In step 1, a user is looking at an article on her Web browser that refers to the September 1997 issue of EJournal, published by GiantSteps Publishing Co. The article has a URL containing a DOI, http://dx.doi.org/10.1346/ejournal-09-97. The DOI prefix 1346 is owned by GiantSteps. When the user clicks on that URL, the Web browser goes to dx.doi.org to access the DOI directory.

In step 2, DOI Directory 10 looks up DOI 10.1346/ejournal-09-97 and finds the URL http://www.giantsteps.com/ejournal/purchreq?09-97. That happens to be a CGI script on GiantSteps' Web site that constructs a response page asking the user if he or she would like to purchase the journal issue (supplied as argument after the ? in the URL).

In step 3, The DOI Directory fires off that URL, which reaches GiantSteps' Web server.

Finally, in step 4, GiantSteps' Web server sends a response page asking the user if she would like to purchase the September 1997 issue of EJournal.

DOIs can be used in ways other than embedding in URLs. For example, a "secure container" may contain a DOI, which it can use to access content once the user has satisfied the security or payment requirements. We have already been discussing with some of the secure container software vendors the possibility of incorporating the DOI into their technologies. As another example, a library lookup service that uses Dublin Core bibliographic data could populate the Resource Identifier elements of its database with DOIs and use them to retrieve items found in bibliographic searches.

DOI Status and Outlook

The DOI model happened to be a perfect fit to technology developed by the Corporation for National Research Initiatives (CNRI) called the Handle System. CNRI is a not-for-profit organization in Reston, Virginia, that was founded by Robert Kahn and Vinton Cerf, two of the fathers of the Internet. It engages in research and development intended to advance the technology of the National Information Infrastructure.

CNRI adapted the Handle System for use with DOIs, and the result was the current DOI Directory. Nine publishers voluntarily participated in a prototype phase from July through mid-October 1997, committing their own resources to create DOIs for their material. At the end of that phase, those publishers had deposited over a quarter million DOIs. Then, three years after the AAP Enabling Technologies Committee first convened, the system went "live" to the world and was presented at the Frankfurt Book Fair in October 1997, to overwhelmingly positive reaction. At this point, the directory is ready to accept online applications for DOI prefixes and to register DOIs; the DOI Web site has a "Getting Started" page for those who are interested. CNRI is continuing development on the DOI System.

The DOI initiative has expanded beyond the purview of the AAP. The International DOI Foundation (IDF), a not-for-profit organization, was incorporated on October 10th, 1997, with offices in the United States and Switzerland. An international Board of Directors was appointed to oversee the Foundation. Under the chairmanship of U.S. Representative Charles Ellis (John Wiley & Sons), the Board currently comprises Jean-Manuel Bourgois (Editions Vuibert, France), Dietrich Goetze (Springer-Verlag, Germany), Robert Kiernan (Routledge, U.K.), and Herman Spruijt (Elsevier Science BV, Netherlands). Carol Risher of AAP is the IDF's acting General Manager. The DOI has been endorsed by such international organizations as the International Publishers Association (IPA) and the International Association of Scientific, Technical & Medical Publishers (STM).

The cat is out of the proverbial bag now. Although the members of the various committees and subcommittees worked assiduously to ensure publishers' cooperation, the coming months will be crucial in determining its success. Will its relatively modest level of standardization prove valuable for publishers? Will other media industries want to embrace the technology? (Already some, like the music and motion picture industries, have begun to look at it.) Will it attract a critical mass of commercial technology vendors? Those are among the yardsticks by which the publishing community will measure the DOI's success.

Perhaps the real value of the DOI is that it has brought publishers and technologists to the same table. Mutually beneficial discussions have already taken place about the nature of the problem, the business risks and opportunities, the magnitude of work publishers must do to be successful online, and the opportunity for technology vendors. The DOI is not a silver bullet that will eliminate the urgent need for publishers to get their houses in order with respect to intellectual-property management — although, with the establishment of a mini-industry of technology vendors building DOI-compatible electronic publishing solutions, they should find their burdens eased and their time to market shortened.

At a minimum, the DOI helps ensure that publishers will dance to the same rhythm. By adopting the DOI, publishers are making a statement akin to the one the great publisher Benjamin Franklin made at the signing of the Declaration of Independence, "We must all hang together, or assuredly we shall all hang separately." Adopting electronic-publishing solutions based on common open standards increases the likelihood that publishers' capital investments will pay off. It helps ensure that publishers will be able to extend their franchises into cyberspace, where much of the world of information is surely going.

Footnotes

1. Mark Stefik, "Letting Loose the Light: Igniting Commerce in Electronic Publication," in Internet Dreams: Archetypes, Myths, and Metaphors (Cambridge, Mass.: MIT Press, 1996), 219-54.

2. That group, of which the author was a member, called itself the Rights And Permissions Group, or RAP Group. They gave a presentation, "Two Sides of the Coin: Publishers' Requirements for Digital Intellectual Property Management," at the Inter-Industry Forum on Technology-Based Intellectual Property Management, Washington, D.C., March 1996. Copies of slides are available from the author.

3. The most ambitious of those is the Media Asset Management Group, which was announced at the Seybold San Francisco conference in October 1997. MAMG intends to design standard metadata models for various media industries.

4. A good summary of the bewildering array of existing and proposed identification schemes is Brian Green and Mark Bide's Unique Identifiers: a brief introduction [formerly http://www.bic.org.uk/bic/uniquid.html], written for Book Industry Communication and EDItEUR.

5. Currently, Amazon.com does not use DOIs, but they do use ISBNs to accomplish a similar purpose.

Bill Rosenblatt is Market Development Manager for Media and Publishing at Sun Microsystems. His responsibilities are to increase Sun's presence in the media and publishing markets through partnerships with software vendors and industry evangelism. He joined Sun in 1996 as an Enterprise IT Architect, providing technology strategy consulting to Sun's major media industry customers. He is a founder of Sun's Digital Media Solutions Center in New York City.

Before joining Sun, Bill was Director of Publishing Systems at the Times Mirror Company, the Los Angeles-based publishing concern. His experience in digital media began at Moody's Investors Service, where he developed an electronic-publishing architecture based on Lotus Notes, Adobe Acrobat, and relational database technologies.

He is an author and editor for the book publisher O'Reilly Associates: his titles including Learning the Korn Shell and Learning GNU Emacs; he is a contributor to the book Electronic Publishing Strategies, published by Pira International Ltd., and he serves on the Editorial Board of Vista Computer Services' Publishing in the 21st Century series. He has been a columnist for the magazine SunWorld for over four years.

Bill holds a B.S.E. in Electrical Engineering and Computer Science from Princeton University and an M.S. in Computer Science from the University of Massachusetts.