RFC 1729

Using the Z39.50 Information Retrieval Protocol in the Internet Environment 💖

Okay bestie, here's the header rundown 🎀 This is RFC 1729, from the Network Working Group, written by C. Lynch of the University of California Office of the President. It's Category: Informational, so it's a guide and not a standard, and it's dated December 1994. The header doesn't list anything it obsoletes or updates ✨

Status of this Memo 🌸

This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited.

Summary 🦋

This memo explains one way to implement the ANSI/NISO Z39.50-1992 Standard for Information Retrieval over TCP/IP, which is already in wide use by the Z39.50 implementor community. It's giving practical, real-world energy 💅

Introduction 🧚

Okay so, Z39.50 is a US national standard for computer-to-computer information retrieval 🌈 It was first adopted in 1988 [1] and then extensively revised in 1992 [2]. The National Information Standards Organization (NISO) developed it. NISO is an ANSI-accredited standards body that serves the publishing, library, and information services communities.

There's also a closely related international standard, ISO 10162 (service definition) [3] and 10163 (protocol) [4], which people casually call Search and Retrieve or SR. SR reached full International Standard (IS) status in 1991. Work is still going on within ISO Technical Committee 46 Working Group 4 Subgroup 4 to move various extensions to SR through the international standards process 🔥 The international standard is basically a compatible subset of the current US Z39.50-1992 standard.

Z39.50 is an applications layer protocol in the OSI reference model 🫶 It assumes lower-level OSI services are there, especially the presentation layer [5], and also the OSI Association Control Service Element (ACSE) [6], which lives inside the application layer.

Now here's the tea ☕ Lots of institutions decided to run this protocol directly over TCP/IP instead of building it in an OSI environment. They also skipped the existing techniques that give full OSI services at and above the OSI Transport layer on top of TCP connections, like RFC 1006 [7] and the ISO Development Environment software. Their reasons were the size and complexity of OSI implementations, the lack of mature OSI software for the full range of computing environments they used, and a feeling that the OSI application layer's architecture was relatively unstable (as opposed to specific application protocols like Z39.50 itself). The biggest worry was that the complexity of the OSI upper layers would outweigh the pretty meager extra functionality they'd get 🙈 So, for better or worse, they decided to put Z39.50 directly on top of TCP. They understood this choice might get revisited someday 💜

From 1991 to 1993, a group of implementing institutions joined the Z39.50 Interoperability Testbed project (often called "ZIT") under the Coalition for Networked Information (CNI) ✨ Their main goal was to encourage lots of interoperable Z39.50 implementations running over TCP/IP on the Internet. By mid-1993, several independent Z39.50 implementations were up and running and could talk to each other across the Internet 🎀

The Library of Congress is the Z39.50 Maintenance Agency for NISO, and it keeps a registry of implementors [8]. That registry includes members of the Z39.50 interoperability testbed 📚

This document describes implementation decisions made by current Z39.50 implementors on the Internet. These decisions have been proven within the ZIT project and are used by most members of the Z39.50 Implementors' Group (ZIG) 🌸 ZIG is an informal group that meets quarterly to talk about implementation and interoperability issues and to develop extensions for future versions of the standard. This document is meant as a guide for other implementors who want interoperable Z39.50 implementations over TCP/IP. It focuses on TCP/IP issues. It doesn't cover other potential interoperability problems, or agreements the implementors have reached to solve them. It does include a few notes about extensions to the existing Version 2 protocol that the implementor community uses and that have interoperability implications 💅

If you're a potential implementor, subscribe to the Z3950IW LISTSERV [9]. That's where you'll find info on the protocol, the extensions in development, and details about current implementations 🦋

Unless noted otherwise, the version discussed here is ANSI/NISO Z39.50-1992, sometimes called Z39.50 Version 2. The obsolete original version is called Z39.50-1988 or Z39.50 Version 1 🥀 The approach defined here should also work for future versions of the protocol, perhaps with some minor changes, and especially for Version 3, which is still under development. This document will probably be updated to cover new versions of the base Z39.50 protocol once they become stable ✨

Encoding 🌈

Z39.50 specifies its application protocol data units (APDUs) in Abstract Syntax Notation One (ASN.1) [10]. These APDUs include EXTERNAL references to other ASN.1 and non-ASN.1 objects, like the ones that define the record transfer syntaxes used in a given application association 🧚

The standard Basic Encoding Rules (BER) [11] get applied to the ASN.1 structures defined by Z39.50, which turns them into a byte stream you can send across a TCP/IP connection. The only restriction on using BER to make this byte stream is that direct, rather than indirect, references must be used for EXTERNAL objects. This is needed because TCP/IP has no presentation context to support indirect reference 😭 An implementation that follows this spec and runs over TCP/IP should produce a valid byte stream according to the Z39.50 standard, meaning the same byte stream could be handed to an OSI implementation. But not every byte stream BER can make from the Z39.50 APDUs in an OSI environment will be legit under this spec. This spec defines a subset of the byte streams that are valid in a pure OSI environment, and it leaves out the ones that use indirect reference for EXTERNAL objects 👑

All other BER features should be tolerated by Z39.50 implementations running over TCP/IP, including the ability to accept indefinite length encodings. Still, it's preferable that implementations don't generate those encodings, since they've caused problems for some ASN.1/BER parsers 🙈 Also, at least to the best of the author's knowledge, no implementations currently use ASN.1/BER representations of floating point numbers. Integers with scaling factors have been used for those purposes instead ✨ One more thing: Z39.50 version 2 doesn't really address character set encoding issues. Those questions, and how they interact with ASN.1/BER support for multiple character sets, are under active discussion as part of the effort to develop Z39.50 version 3 💖

Connection 🎀

In the Internet environment, the Internet Assigned Numbers Authority [12] has assigned TCP Port 210 to Z39.50. To start a Z39.50 connection to a server in the TCP/IP environment, a client simply opens a TCP connection to port 210 on the server. As soon as the TCP connection is established, the client sends a Z39.50 INIT APDU, encoded with BER as described above 🦋

Heads up, bestie 🚨 There's a huge installed base of Wide Area Information Server (WAIS) systems out there. The original WAIS software used Z39.50 Version 1 with some extensions. Version 1 didn't use BER encoding, and its INIT APDUs look totally different from Version 2 INIT APDUs. Implementations of Z39.50 should at least be ready to reject WAIS-type INIT APDUs gracefully 💀

Some implementations spot WAIS-type INIT APDUs and switch to the Version 1 variant that WAIS uses. That gives them backwards compatibility with the existing base of WAIS clients 🫶 The usual way to check whether a client is WAIS, instead of Z39.50 Version 2, is to see if the first byte sent on the connection is an ASCII zero, since that means it's a WAIS client. (In version 1 of WAIS, bytes 0-9 of the first PDU hold an ASCII packet length, and the lowercase ASCII string "wais" shows up starting at byte 12 🧚)

Work is underway to specify a WAIS profile for use with Z39.50 version 2 [13]. It's expected to be issued as a Z39.50 Applications Profile through the NIST OIW Library Automation Special Interest Group. That profile is expected to be compatible with the layering defined in this RFC 🌈

---

Part 1 of 2 is a wrap, bestie 🦋 Next up is Service Mappings, where the TCP CLOSE and ABORT stuff gets fun 🔥

Contexts ✨

okay bestie, the second the TCP connection is up on TCP port 210, the client and the server should both assume that the application context from Appendices A and B of the Z39.50-1992 standard is already in place 💅. Those are the ASN.1 definitions of the Z39.50 APDUs, plus the transfer syntax you get by applying the BER to those APDUs 🎀.

Implementations can reasonably expect the diagnostic set BIB-1 to be supported. If resource control is being used, the resource report format BIB-1 is supported too 🦋. That's the baseline everyone can count on, ngl.

Now, if there's no presentation negotiation mechanism, clients and servers should be cautious 🙈 about using alternative attribute sets, diagnostic record formats, resource report formats, or other objects defined by optional EXTERNALs in the Z39.50 ASN.1, such as authentication parameters. Only do it if there's known prior agreement to support them. Of course, either participant in an association can reference one of these objects by object ID in an APDU, but there's no guarantee the other partner will understand it. Robust implementations should be prepared to run into unknown or unsupported object IDs and generate appropriate diagnostics 🔥. Over time, the default, commonly known pool of object IDs may be expanded (for example, to support authentication parameters) 🌈.

Implementors should refer to the document [14] issued by the Z39.50 maintenance agency in June 1992 for more details on the assumed contexts and object identifiers 📚.

Record syntaxes are a seriously big practical problem 💀. In the OSI environment, the partners in a Z39.50 association are assumed to agree on which record syntaxes they both commonly know. They can do this through presentation negotiation as part of association establishment, or later and dynamically as part of the PRESENT process, through the use of the alter presentation context function at the presentation layer. There's a preferred record syntax parameter the client can supply to guide this negotiation. A bunch of registered record syntaxes exist. Some are based on ASN.1, and others use formats like the MARC standard for the interchange of machine readable cataloging records. MARC predates ASN.1 but is still widely implemented 🍓. In the TCP/IP environment, if the server can't supply the record in the preferred syntax, it has no guarantee the client will understand any other syntax the record might be sent back in. It also has no means of negotiating those syntaxes 😭.

Several proposals have been floated to fix this 💖. The one that will likely be part of Z39.50 Version 3 is to replace the preferred record syntax parameter with a list of prioritized preferred syntaxes supplied by the client. It would also add a flag saying whether the server is allowed to substitute a record syntax that isn't on the client's list ✨. The currently proposed ASN.1 for this extension is upwards compatible with Z39.50 Version 2, though the details are still being discussed within the Z39.50 Implementor's Group. As the Version 3 ASN.1 becomes stable in this area, Z39.50 servers are encouraged to accept the extended ASN.1 for generalized preferred record syntax 🫶.

The extensibility rules for Z39.50 negotiation let clients and servers negotiate the use of Z39.50 Version 2 plus the generalized preferred syntax feature from Version 3. So a client could support the generalized preferred record syntax and propose it to any server. If the server rejects the proposal, the client can revert to the Version 2 preferred syntax feature 🔄.

A second option, which isn't incompatible with the Version 3 extension, would be to adopt a convention for TCP/IP implementations. Under this convention, the server doesn't return a record in a syntax that isn't on the preferred record syntax list the client provided. Instead, it returns a diagnostic record saying a suitable record transfer syntax wasn't available 🧚. This strategy could be viewed as simply implementing a subset of the Version 3 solution, and implementors of servers should consider it as a possible interim measure 🌸.

Other Interoperability Issues ✨

Version 3 will include an "other" data field in each APDU, which can be used to carry implementation-specific extensions to the protocol 💜. A number of implementations are already using this field. Interoperable implementations might be wise to include code that at least ignores the presence of such fields, rather than treating their presence as an error, since that's in contravention of the standard 🙃.

References 📚

[1] National Information Standards Organization (NISO). American National Standard Z39.50, Information Retrieval Service Definition and Protocol Specifications for Library Applications (New Brunswick, NJ: Transaction Publishers; 1988).

[2] ANSI/NISO Z39.50-1992 (version 2) Information Retrieval Service and Protocol: American National Standard, Information Retrieval Application Service Definition and Protocol Specification for Open Systems Interconnection, 1992.

[3] ISO 10162 International Organization for Standardization (ISO). Documentation -- Search and Retrieve Service Definition, 1992.

[4] ISO 10163 International Organization for Standardization (ISO). Documentation -- Search and Retrieve Protocol Definition. 1992.

[5] ISO 8822 - Information Processing Systems - Open Systems Interconnection - Connection Oriented Presentation Service Definition, 1988.

[6] ISO 8649 - Information Processing Systems - Open Systems Interconnection - Service Definition for the Association Control Service Element, 1987. See also ISO 8650 - Information Processing Systems - Open Systems Interconnection - Protocol Specification for the Association Control Service Element, 1987.

[7] Rose, M., and D. Cass, "ISO Transport Layer Services on Top of the TCP, Version 3", STD 35, RFC 1006, Northrop Research and Technology Center, May 1987.

[8] Registry of Z39.50 Implementors, available from the Z39.50 Maintenance Agency (Ray Denenberg, ray@rden.loc.gov)

[9] To subscribe to the Z39.50 Implementor's Workshop list send the message: SUB Z3950IW yourname to: LISTSERV@NERVM.NERDC.UFL.EDU (or NERVM.BITNET). Current drafts of the Version 3 Protocol document are available through the Library of Congress GOPHER server, MARVEL.LOC.GOV.

[10] ISO 8824 - Information Processing Systems - Open Systems Interconnection - Specifications for Abstract Syntax Notation One (ASN.1), 1987

[11] ISO 8825 Information Processing Systems - Open Systems Interconnection - Specification of Basic Encoding Rules for Abstract Syntax Notation One (ASN.1) 1987

[12] Reynolds, J., and J. Postel, "Assigned Numbers", STD 2, RFC 1700, USC/Information Sciences Institute, October 1994.

[13] WAIS Profile of Z39.50 Version 2, Revision 1.4, April 26, 1994, available from WAIS Inc.

[14] Registration of Z39.50 OSI Object Identifiers (Z39.50-MA-024), available from the Z39.50 Maintenance Agency (Ray Denenberg, ray@rden.loc.gov).

---

Security Considerations 🔐

Okay, so this document doesn't discuss security considerations 😬. It should be noted, though, that the Z39.50 protocol includes mechanisms for authentication and security, and implementors should review them.

🚨 Implementors should review the Z39.50 authentication and security mechanisms. Don't skip this one, bestie.

Author's Address 🎀

Clifford A. Lynch
University of California, Office of the President
300 Lakeside Drive, 8th Floor
Oakland, CA 94612-3550

Phone: (510) 987-0522
EMail: clifford.lynch@ucop.edu

That's the whole RFC, bestie! 🌈🦋💖