RFC 971
Survey of data representation standards
RFC 971: A SURVEY OF DATA REPRESENTATION STANDARDS 🎀
Okay bestie, this one is from January 1986 and it's written by Annette L. DeSchon from ISI 💅. It's RFC number 971 in the Network Working Group series. The Status section makes it clear this is a discussion doc, not a standard, and the header doesn't have any Obsoletes or Updates line, so it doesn't replace or update anything 🌸.
Status of This Memo
Quick disclaimer 🙈: this RFC talks about data representation conventions in the ARPA-Internet and suggests some possible fixes. None of the proposals in it are standards for the ARPA-Internet yet. The hope is that the community reaches a general consensus on the right approach, and that eventually leads to actual ARPA-Internet standards. Distribution of this memo is unlimited ✨.
1. Introduction ✨
This report compares several data representation standards that people are already using 🦋. The ones it covers are the CCITT X.409 recommendation, the NBS Computer Based Message System (CBMS) standard, the DARPA Multimedia Mail system, the Courier remote procedure call protocol, and the SUN Remote Procedure Call package. Together these are the "system type definitions" being discussed.
So the main goal is to see how the CCITT standard compares to the others. CCITT is gaining wide acceptance internationally, and the others came out of email, distributed interprocess communication, and remote procedure call work 💖. The CCITT X.409 recommendation is called "Presentation Transfer Syntax and Notation." It's an international standard and part of the X.400 series Message Handling Systems (MHS) specifications [1]. Both NBS and ISO have adopted it, and some commercial orgs have announced plans to support a CCITT interface for email 🍓.
The NBS CBMS standard came first. It was published as a Federal Information Processing Standard (FIPS Publication 98) in 1983 [3]. The DARPA Multimedia Mail system is an experimental email system used in the DARPA Internet [2,4,5]. It creates and sends messages that include text, graphics, stored speech, and images, and it has been implemented on several very different machines 🌈. Courier is the XEROX network systems remote procedure call protocol [7]. And the SUN Remote Procedure Call package builds "network pipes" between UNIX machines [6] 🧚.
2. Background
This section gives a quick overview of the basic terminology and approach of each data representation standard. Think of it as the tea before the drama ☕.
2.1. Interprocess Communication Standards
Standards built for distributed interprocess communication or remote procedure call between like machines usually like types that map easily onto the types of the programming language running on the system 👑. For example, the XEROX Courier types look a lot like the types in the Mesa programming language, and the SUN Remote Procedure Call types look like C types 💜. The upside of using like machines is that the external data representation can be defined so that converting to and from the local format takes very little work ✨.
2.1.1. Courier
Courier's standard data types define the data objects that get sent back and forth between system elements running the Courier remote procedure call protocol 💅. The "standard representation" is the encoding of the data that actually gets transmitted. The "standard notation" covers how higher-level applications should interpret that data 🎀. The standard representation encodes the value of an object, but the type of the object is figured out by whatever software generates or interprets the representation. So the value is in the bits, and the type lives in the software 🫶.
2.1.2. SUN Remote Procedure Call Package
The SUN Remote Procedure Call package has routines that let a process on one UNIX machine consume data produced by a process on another UNIX machine 🦋. This is called a "network pipe," and it's basically an extension of the standard UNIX pipe 🌸. The "eXternal Data Representation (XDR)" standard defines the routines used to encode, or "serialize," data for transmission, and to decode, or "deserialize," it for local interpretation 🔥. Honestly the syntax makes it sound more like "remote interprocess communication" than "remote procedure call," so the name is kind of iconic in its own confusing way 💀.
2.2. Message Standards
The message-oriented standards, including DARPA Multimedia Mail, NBS CBMS, and CCITT X.409, tend to prefer more general, highly extensible type definitions 🌈. This probably has to do with the expectation that a system will include lots of different machines programmed in lots of different languages 🧚.
2.2.1. DARPA Multimedia Mail
The DARPA Multimedia Mail system was developed for the DoD Internet community 💖. The set of data elements used in the Multimedia Message Handling Facility (MMHF) is called its "presentation transfer syntax" ✨. How each element is encoded depends on the data type it holds. Every element starts with a one-octet element-code 🎀.
Some elements have a fixed length. For example, an INTEGER element is five octets total: one for the element-code and four for the "value component" 🍓. Other elements can vary in length. The TEXT element is made of a one-octet element-code, a three-octet count of the characters that follow, and then a variable number of octets, each holding one right-justified seven bit ASCII character 💅. The element-code and the length together make up the "tag component" 🦋.
A "base data element" is self-contained, while a "structured data element" is built out of other data elements 👑. The LIST element builds structures out of other elements. Its tag component is a one-octet element-code, a three-octet count of the octets that follow, and a two-octet count of the elements that follow 🌸. The PROPLIST element builds a structure made of an unordered set of name-value pairs 🔥. Its tag component is a one-octet element-code, a three-octet count of the octets that follow, and a one-octet count of the name-value pairs in the PROPLIST 💜. Both LIST and PROPLIST are followed by an ENDLIST element 🫶.
2.2.2. NBS Computer Based Message System
The NBS Computer Based Message System (CBMS) standard was built to specify the format of a message at the interface between different computer-based message systems 🎀. Each data element is made of "components." There are five possible types: the "identifier octet," the "length code," the "qualifier," the "property-list" component, and the "data element contents" 🌈.
Every data element has an identifier octet and a length code 🧚. The identifier octet holds a one-bit flag that says whether the data element has a property-list, plus a code that identifies the data element and says whether it has a qualifier 💖. In the NBS standard, the property-list is attached to a data element and holds properties like a "printing-name" or a "comment" ✨. What the qualifier means depends on the data element code 🍓. The length code tells you how many octets follow, and it's between one and three octets long 🦋.
Each data element is either a "primitive data element," which holds one basic piece of information, or a "constructor data element," which holds one or more data elements 👑. The "field" data element is itself a constructor, and it uses a qualifier component that holds a "field identifier" to say which specific field is being represented in a message 💅.
2.2.3. CCITT Recommendation X.409
The CCITT recommendation X.409 defines the notation and the representation technique used to specify and encode the Message Handling System (MHS) protocols 💜. Here's how it describes encoding type definitions 🌸. A data element has three components: the "identifier" (which is the type), the "length," and the "contents" 🔥. An element and its components are made of a whole number of octets 🫶.
An identifier has a "class" ("universal," "application-wide," "context-specific," or "private-use"), a "form" ("primitive" or "constructor"), and the "id code" 🎀. There's a convention for both single-octet and multi-octet identifiers 🌈. The length says how long the contents are, in octets, and the length field can itself vary in size 🧚. There's also an "indefinite" length value. That means no length is given for the contents, and the contents end with an "end-of-contents" (EOC) element instead 💖. In X.409 you can tell whether a data element is a primitive or a constructor from the form part of its identifier ✨. And you can "tag" the data by attaching meaning to an id code inside a specific application 🍓.
Part 2 is coming, so stay tuned bestie 🫶🦋✨
3. Implicit Versus Explicit Representation ✨
okay so this section is about two ways to pack up data for the wire, and it's giving packing strategy 🧳💅
SUN Remote Procedure Call and XEROX Courier both go implicit 🤫. That means the sender and receiver agree in advance on what the type definitions are for a given kind of call or message. The parameters (or message fields) just show up in a predefined order, and each one already has a predefined type. So the data can't really get reformatted into the local form until it reaches a process that knows the types of the specific parameters. At that point, system routines convert from the external format to the local format. If your system is homogeneous, there might be barely any conversion to do 🙌. Bonus: you don't pay the overhead of sending the type definitions along with the data.
Then there's explicit 📦. The DARPA Multimedia Mail system, the NBS CBMS standard, and the CCITT X.409 recommendation all encode the type definitions right into the message. This has some real perks 💖:
- A low-level receiver process in the destination host can convert the data from the standard form to the local form as it's received. That can be more efficient, because the destination host can skip passing around data that doesn't line up with its local word boundaries.
- It's flexible for the future 🦋. The overall length is part of the type definition, so a host can deal with data of types it doesn't really understand, or just ignore it.
- Since interpretation doesn't depend on position, message fields (or parameters) can be reordered or optionally left out.
The downsides are also very real 💀. Assuming no field could ever be omitted, the external representation of the message may end up longer than it would have been with an implicit representation. And extra time can get eaten up by converting between the external and local formats, since the external format almost certainly won't match the local format for any of the participants.
4. Data Representation Standards Scorecard 📊
this is the comparison table 🍓 It lists the data elements defined for each of the standards we've been talking about, so you can get a general feel for which types each standard has. Heads up though: putting types in the same group doesn't mean one type matches up exactly with another one 🙈. Where it applies, the identifier code is in parentheses after the name of the data element. In the NUMBER column, "S" means signed, "U" means unsigned, "V" means variable, and the number is how many bits there are. So "Integer S16" means a "signed 16-bit integer" ✨.
Type CCITT MMM NBS XEROX Sun
-----------------------------------------------------------------------
END | End-of- | ENDLIST | End-of- | -- | --
| Contents | (11) | Constructor| |
| (0) | | (1) | |
| | | | |
PAD | Null (5) | NOP (0) | No-Op (0) | -- | --
| | PAD (1) | Padding | |
| | | (33) | |
| | | | |
RECORD | Set (17) | PROPLIST | Set (11) | -- | --
| | (14) | | |
| Sequence | LIST (9) | Sequence | Sequence | Structure
| (16) | | (10) | |
| | | | Record |
| | | Message | |
| | | (77) | |
| -- | -- | -- | Array | Fixed Array
| | | | | Counted Array
| "Choice" | -- | -- | Choice |Discriminated-
| "Any" | | | | Union
| | | | |
| "Tagged" | "name" | Field (76) | -- | --
| | |Unique-ID(9)| |
| -- | SHARE-TAG | -- | -- | --
| | (12) | | |
| | SHARE-REF | | |
| | (13) | | |
| | | | |
| -- | -- | Compressed | -- | --
| | | (70) | |
| -- | ENCRYPT | Encrypted | -- | --
| | (14) | (71) | |
Type CCITT MMM NBS XEROX Sun
-----------------------------------------------------------------------
BOOLEAN| Boolean(1)| BOOLEAN(2)| Boolean(8) | Boolean | Boolean
| | | | |
NUMBER | Integer(2)| EPI (5) | Integer(32)| Integer | Integer
| SV | SV | SV | S16 | S32
| | INDEX (3) | | Cardinal | Unsigned Int
| | U16 | | U16 | U32
| | INTEGER(4)| |Unspecified|Enumeration
| | S32 | | 16 | 32
| | | | Long Int |Hyper Integer
| | | | S32 | S64
| | | | Long Card |Uns Hyper Int
| | | | U32 | U64
| | | | | Double Prec
| | | | | 64
| -- | FLOAT (15)| -- | -- | Float Pt
| | 64 | | | 32
| | | | |
BIT- | Bit String| BITSTR(6) | Bit-String | -- | --
STRING| (3) | | (67) | |
| Octet- | -- | -- | -- | Opaque
| String(4)| | | |
| | | | |
STRING | IA5 (22) | TEXT (8) | ASCII- | String | Counted-
| | | String (2)| | Byte String
| | NAME (7) | | |
| Numeric | | | |
| (18) | | | |
| Printable | | | |
| (19) | | | |
| T.61 (20) | | | |
| Videotex | | | |
| (21) | | | |
Type CCITT MMM NBS XEROX Sun
-----------------------------------------------------------------------
OTHER | UTC Time | -- | Date (40) | -- | --
| (23) | | | |
| Gen Time | | | |
| (24) | | | |
| -- | -- | Property- | -- | --
| | | List (36)| |
| -- | -- |Property(69)| -- | --
| | | | |
| -- | -- | -- | Procedure | --
| | | | |
| -- | -- | Vendor- | -- | --
| | | Defined | |
| | | (127) | |
| | | Extension | |
| | | (126) | |5. Conclusions 🎀
okay the big takeaway 👑 Of the standards we surveyed, the CCITT approach (X.409) has already gained wide acceptance. For a system that's going to include a bunch of dissimilar hosts, which is totally the case for an Internet application, a standard with explicit representation like CCITT X.409 would probably work really well 🫶. Using CCITT X.409, you can construct most of the data elements specified for the other standards, with the possible exception of the "floating point" type.
BUT 😭 some of the flexibility built into this standard, like the "private-use class," might lead to ambiguity and a lack of coordination between implementors at different sites. So if something like CCITT were ever used in an Internet experiment, a fully defined (but large) subset would probably have to be picked. Not me being dramatic, it's just how it is 💅.
6. References 📚
[1] "Message Handling Systems: Presentation Transfer Syntax and Notation", Recommendation X.409, Document AP VIII-66-E, International Telegraph and Telephone Consultative Committee (CCITT), Malaga-Torremolinos, June, 1984.
[2] J. Garcia-Luna, A. Poggio, and D. Elliot, "Research into Multimedia Message System Architecture", SRI International, February, 1984.
[3] "Specification for Message Format for Computer Based Message Systems", FIPS Pub 98 (also published as RFC 841), National Bureau of Standards, January, 1983.
[4] J. Postel, "Internet Multimedia Mail Transfer Protocol", USC Information Sciences Institute, MMM-11 (RFC-759 revised), March, 1982.
[5] J. Postel, "Internet Multimedia Mail Document Format", USC Information Sciences Institute, MMM-12 (RFC-767 revised), March, 1982.
[6] "Extended Data Representation Reference Manual", SUN Microsystems, September, 1984.
[7] "Courier: The Remote Procedure Call Protocol", XSIS-038112, XEROX Corporation, December, 1981.
📦 where this page comes from
- ietf: rfc971 1986-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms