RFC 910

Multimedia Mail Meeting Notes 💌✨

Okay bestie, this is RFC 910, Multimedia Mail Meeting Notes 🎀 It's from the Network Working Group, written by Harry Forsdick at BBN Laboratories, and it's dated August 1984. The doc is basically a meeting report, which the Status section makes clear, and it doesn't say it obsoletes or updates anything 🦋

Status of this Memo 📝

This memo is a report on a meeting about the experimental multimedia mail system, and kind of a status report on that experiment too 🧚 Distribution is unlimited, so share it with everyone 💜

1. Introduction ✨

So a meeting happened at Bolt Beranek and Newman on 23-24 July 1984 🌸 The point was to check in on groups building multimedia mail systems and talk through a bunch of stuff about where multimedia systems should go next. Representatives came from BBN, ISI, SRI and Linkabit 👑 The attendee list is at the end of the note, supposedly 🙈

The outcome is a series of agreements that will get folded into the next round of multimedia mail experiments, plus a list of things to follow up on later 🔥

Note: this doc references notes in a series about multimedia mail. They're online in the directory [USC-ISIF]<MMM>, and the files are named MMM-N.TXT, where N is the note number. The file MMM-INDEX.TXT lists every note in the series. You can grab these public files over FTP with the username ANONYMOUS and the password GUEST 💅

2. Review of Status 🌈

Each organization gave a status report on the work they'd done over the last year 🫶

2.1. BBN 🍓

The first version of Diamond is done and runs on the Jericho workstation, which is iconic 👑 Diamond can currently swap compound documents that contain text, graphics, images, voice and spreadsheet/charts 😭 They showed a demo of the user's view of Diamond messages and message management, plus how the components of this distributed system talk to each other 🦋

Right now Diamond uses the TOPS-20 implementation of MPM for inter-cluster message transport. The plan is to bring an MPM implementation for the Sun Workstation into Diamond too 💖 The current focus is porting Diamond to the Sun Workstation. A first Sun version is nearly finished, and the document editor was demoed running on the Sun 🔥 Diamond will be used in the ADDCOMPE testbed, with around 100-200 users expected in the next year or so 🎀 Future plans include building on what they've learned in multimedia conferencing, pushing multimedia into other application areas, and applying Diamond's distributed architecture to other application areas as well ✨

2.2. ISI 💜

A brand new effort just kicked off to build a user interface on a Xerox 1108 (Dandelion) workstation 🌸 All of the implementation is in Interlisp, which is giving retro and I'm here for it 🧚 Initial work is done on IP and TFTP for the 1108, plus a document editor that uses the Interlisp-D window system 💅 The user interface work they did on the Perq is winding down. The MPM on TOPS-20 is basically complete, with MPM to SMTP mail conversion added, and no major changes are expected 🫶 The TOPS-20 MPM will serve as the message transport for the 1108 user interface, and TFTP will be used to move messages from the 1108 to the TOPS-20 🦋

2.3. SRI 🌈

The SRI multimedia mail system has three parts: the Multimedia Mail Handler (MMH), which is the user's interface for managing mail; the Structure Editor (SE), which is used to view and compose multimedia messages; and the MPM for mail transport 👑 It's implemented on the Sun Workstation. The first release of MPM on the Sun will be ready for distribution at the end of this summer 💖 The SE views and composes structures of multimedia objects, and right now it supports text, voice and graphics ✨

SRI also has another effort, which is about integrating applications to run in the ADDCOMPE testbed 🔥 Diamond will be the first application to use multimedia data in the testbed 🎀 SRI wants to look into the issues around multimedia systems, so they can figure out how multimedia data could be used in other applications that might end up in the testbed 🧚

2.4. Linkabit 🍓

Linkabit just got a contract to work on protocol evaluation, problems that come up in a large internet environment, and new real-time end-to-end services 💅 They'll be working with Sun workstations 🫶 Their areas of interest are protocols, multimedia conferencing and domains 💜

3. Discussions and Agreements 🎀

3.1. Conversion to the New Multimedia Syntax 🌸

Everyone generally agreed that future experiments would all adopt the revised syntax for multimedia mail, as described in the Final Report to SRI Project 5363 🧚 It was also agreed that RFC767 should be revised to reflect these changes 👑 The switchover should happen as soon as possible, and it should be completed by October 1, 1984 💖

3.2. Graphics Representation 🦋

There was a wide-ranging discussion about how graphics should be represented in multimedia documents 🌈 The group generally agreed that the first kind of graphical object to go into multimedia messages would be a simple line drawing, like the ones made by the many "draw" programs (e.g. LisaDraw) out there right now ✨ They looked at the two existing standards (ACM-CORE and GKS) and the interim protocol used in the Diamond system 🔥 Two major problems with the existing standards came up:

• In both ACM-CORE and GKS, grouping is inadequate or non-existent 😭 That makes it hard or impossible to build a composition out of several graphical objects and then manipulate that composite as a single graphical object 💀
• Neither ACM-CORE nor GKS has specified a standard method for representing graphical drawings in memory (e.g. long-term file storage) 🙈 That's one of the most important parts of a graphical standard for multimedia mail, and it's a huge gap 😤 So far, graphical standards have focused on driving devices in an independent manner, not on storing graphics in a standard representation 💅

Diamond's representation for graphical objects got presented next. The protocol is documented in MMM-18 and MMM-23 📚 If you want hardcopies of the diagrams in those documents (sigh), you can send a request to Travers@BBN 🥲

The discussion then moved to how much effort it would take to switch from one representation to another 🌸 The group generally agreed that, compared to the whole editor used to manipulate graphical objects (e.g., the "draw" program), the part that reads or writes objects to and from files is relatively simple 🧚 Most draw programs have a unique internal representation that gets built when reading the file representation and is used as the source when writing it back out 👑 So it's just this relatively small piece of a graphics editor that gets affected by switching from one file representation to another 💖 That made adopting one interim representation now, and planning to move to a standard representation once the standards work solves the problems above, feel reasonable ✨

After a lot of examination of the issues, the group reached these decisions 🔥

1. The representation for graphics used in Diamond and documented in MMM-18 and MMM-23 will be adopted as an interim representation for graphics in multimedia mail 🎀 It'll be known as the MMGraphics1 protocol 💜
2. They'll actively track development of the GKS standard, and also look at a GKS-subset that was developed by Sandia Labs 🦋
3. They agreed to eventually settle on an adopted international standard 🌈

3.3. Document Presentation Semantics ✨

okay so there was this presentation about the ideas in MMM-22: "A Format for the Construction of Multimedia Messages" 📄💖 and the discussion hit two big questions:

• Presenting documents on different display devices 🖥️ (different dimensions, resolutions, available fonts, etc.)

The vibe was that there's no single set of fonts or page sizes that works for everything 😭. The strong feeling was that as long as the display surface is a reasonable size, a document should look "correctly" formatted. So instead of the originator setting hard page boundaries, the originator's intent about formatting and grouping of objects should be kept and used when the document is actually shown. A window on a bitmap display and a hardcopy page printer are both examples of display devices 🦋.

• Representing the documents that already exist on paper 📚 as well as documents that can use the new powers of electronic creation, storage and presentation ⚡.

Several points were made 💅:

1. One of the first goals for multimedia systems should be representing the kinds of documents that are everywhere in the hardcopy world 🌸. People already know these documents and will expect multimedia systems to at least handle them.
2. While we borrow the hardcopy model for electronic documents, we should NOT throw away the huge potential of electronic documents that are way more reactive 🔥. For example, a document where a graphical figure and a textual explanation of that figure are linked, so the figure stays visible as long as you're reading the explanation. Iconic 🤩.
3. In lots of situations, being able to carry away a paper copy of a document is a requirement, even if the document wasn't mainly meant to be presented in hardcopy 🖨️.

The following agreements were made 🫶:

1. A method for recording the author's intent about presenting a document should be developed 💜. This representation would defer final presentation decisions about size, resolution and fonts to the reader's document presenter.

Topics this representation will cover include:

• Grouping of objects
• Limited Font attributes (e.g., normal, bold, italic)
• Headings, Footings
• Sectioning

Of course, the reader's document presenter is free to ignore any of the author's intentions it can't deal with 🙈. The whole point is to record what the author wants, but hold off on the final decisions about how to use those intentions until the reader's capabilities are known 🧚.

This representation will sit somewhere between the rather loose spatial and temporal positioning commands already in the protocol (Sequential, Simultaneous and Independent) and the absolute positioning of a system that defines rigid page boundaries and absolute positions for object placement on a page 🎀.

2. We will NOT try to make this representation handle all of the issues that modern document formatting systems handle 🙅‍♀️. Instead, we'll skim off some of the most useful ideas and maybe limit the flexibility that's present in those complex formatting systems ✨.
3. The document representation will be able to describe relationships between objects that use the capabilities of electronic document presentation 🦋. That includes simultaneous presentation (two objects that are visible at the same time) and overlay presentation (two objects, possibly transparent, that occupy the same area in a document, and which may also be separated under viewer control) 👑.
4. Methods should be developed for all aspects of the document representation so it can be presented in hardcopy form 🖨️. This applies both to electronic documents that fit the traditional hardcopy model and to those that use the more reactive features of the representation 💕.

---

3.4. Directory Service 🗂️

There is an increasing need to find out attributes of users, hosts and domains across the DARPA Internet 🌐. For example, when composing the header fields of a message, it's useful to be able to ask about the mail box location of the person the message is addressed to 📬. It's also needed to figure out what services a host provides, so that requests that will never be satisfied can be avoided 🙅‍♀️.

The group felt that the work on the Internet Domain system (being done at ISI and Berkeley) would answer some of these problems 💡. We should look at its design documents to see how that system might help us (see RFCs 882 and 883) 📖. The WhoIs server is useful, but only for information about the text mail box of a person (see RFC812) 🙈.

---

3.5. New Media Types 🎨

The discussion covered three topics: a proposal for a new media type, ideas for other new media types, and provisions for dealing with unknown media types 🧩.

A description of the Diamond SpreadSheet/Chart media type was presented 📊. It's documented in MMM-24. In this media, you can represent a table containing numbers, labels, dates and formulas 🔢. The unique part is that the spreadsheet model AND the data are transmitted 🤯. The reader of a document with a spreadsheet object can test what effect different data would have on the conclusions the spreadsheet suggests. A spreadsheet can show up as a table and/or as one of several business charts (line graph, scatter graph, bar chart or pie chart) 📈. Rulings can be added to the tabular representation so it can look super sophisticated 💅. During the discussion, someone pointed out that a minimal implementation could just ignore the formulas and present only the values of the cells, which still allows a minimal presentation of the tabular and chart info 🌸.

Ideas for new media types included:

Form 📝

A set of fields that are Name-Value pairs 💜. Forms can be used for presentation and/or for collecting information. Filling out a form might (with user approval) trigger sending the completed form to the appropriate person who handles such forms 💌.

Animated Graphics 🦋

A line drawing that has temporal information encoded in how its components are presented ✨. Parts of a graphics object could move around the object during its presentation. For example, an arrow could move about a map showing a route to follow 🗺️. There was some discussion about how this would interact with other media. For example, how could an arrow moving around a map be coordinated with voice instructions on how to get from one place to another 🎧? No decisions were made about how best to do this 🤷‍♀️.

Finally, we agreed that all of our systems should be ready to accept (and possibly ignore) media types that aren't implemented yet 🫶. The common way to handle this is to include a statement of the form "An object of type <Type> appears here" 🪧. With the regularized syntax that's been adopted, many of the common attributes of all object types will be understandable, even if the actual type isn't implemented 🧠. In Diamond, we'd like to use the MPM to transfer Diamond messages between Diamond and non-Diamond clusters 🚀. Right now, if we included a spreadsheet in one of these messages, all the other multimedia mail implementations would probably end up in the debugger 💀 when they tried to process our messages, instead of just indicating that there was something they didn't quite understand 😭.

---

3.6. MPM Support 🛰️

By the end of the summer there will be two implementations of the MPM: one on TOPS-20 and one on the Sun Workstation 🖥️. We agreed to try to set up the following operational MPMs 💖:

Organization  Host          MPM Implementation

ISI           ISIF          TOPS-20
ISI           ISIB          TOPS-20
SRI           ?             Sun Workstation
BBN           ?             Sun Workstation
DARPA         ?             Sun Workstation
Linkabit      DCN6          Sun Workstation

The idea behind this agreement is to get wide geographic coverage 🌍. That lets us use multimedia mail on a regular basis and test the impact of realistic use of multiple communicating MPMs over the Internet 🦋.

---

3.7. Floating Point Data Type 🔢

In the representation for data defined in RFC759, there's no way to represent floating point numbers 😭. We agreed that a new data type should be added, called Float64, which is the 64-bit IEEE standard floating point number representation ✨.

---

3.8. Captions 🏷️

The idea of including a text caption as an optional property of every object was discussed 💅. A caption would be useful for a few reasons 🎀:

• For media like voice that don't have an implicit visual representation, it's useful to include a caption saying something about the object 🎧. The caption can serve as a visual indication that the non-visual object is there 👀.
• When an implementation of a multimedia message system doesn't support a given media type, it can be useful to give some info about the object in the form of a text passage 📝.
• In some situations, it's important to present an outline of a document 🗒️. Captions attached to each object could be used to generate a shortened abstract of the document 🧁.

We agreed to add to all object types an optional property whose name is Caption and whose value is of type Text String 💜.

---

3.9. More Users of Multimedia Mail 📣

We need to use multimedia mail more so we can gain experience with the issues that need attention 💪. This can be done by:

• Encouraging more sites to join the experiments 🙌. There are several possible sites with Sun workstations that could be set up to run an MPM and one of the multimedia message systems 🖥️.
• Making the MPMs perform translations to and from SMTP text-only mail 📨. At BBN, the Diamond Import/Export component does translations in both directions, and this has proved super useful for testing our system 🔁. Also, including statements like <Graphics appears here> might spark interest from text-only mail recipients 👀, although care should be taken not to offend anybody with this kind of "class differentiation" 🙈.

To the extent possible, the Sun Workstation MPM will be modified to perform translations to and from SMTP mail 🛠️. The TOPS-20 MPM already translates from multimedia mail to text-only mail 💻. It may be possible to add translation in the other direction too 🌈.

3.10. Multimedia Exploder Mailing List 📬💖

okay so a mailing list for Multimedia Mail is getting set up at ISI! 🌸 It's going to be the "exploding" kind, which means when you send one message to the list, literally everybody on it gets their own copy 💅 main character energy for your inbox. 🦋 To join or leave, email MMM-People-Request@USC-ISIF.ARPA 🫶 and to actually send to everyone, the exploder mailbox is MMM-People@USC-ISIF.ARPA ✨

3.11. Next Experiment 🧚

the next experiment is happening in January 1985 🎀 and we're going to try to show off these new features:

• Use of the revised multimedia syntax described in section 3.1. 💜
• Inclusion of Graphics objects, in addition to Text, Images and Voice. 🌈
• Use of the, as yet unspecified, document presentation semantics described in section 3.3. 🔥 (still not specified yet, which is giving suspense 😭)
• Use of the Sun Workstation MPMs. 💻✨

4. Further Actions 📋👑

ngl there's a whole to-do list coming out of all these agreements 😭 the dates are ones that seemed reasonable, so here's the lineup:

Revision of RFC759 to include Float64 data type.
Person:  Greg Finn and Jon Postel.
Due Date: 1 September 84.

Conversion to the new Multimedia Syntax
Person:  All groups.
Due Date: 1 September 84.

Revision of RFC767 to reflect revised Multimedia Syntax and
optional Caption property
Person:  Jose Garcia-Luna and Jon Postel
Due Date: 1 October 84.

Specification of Document Presentation Semantics (Section 3.3)
Person:  Harry Forsdick
Due Date: 1 October 84.

Acquisition of GKS and GKS-subset documentation
Person:  Lou Schreier
Due Date: 1 September 84

Completion of initial implementation of Sun Workstation MPM
Person:  Andy Poggio
Due Date: 15 September 84

Multimedia Exploder Mailing List
Person:  Greg Finn
Due Date: 15 August 84       < COMPLETED >
Addition of MPM<==>SMTP translation logic to Sun Workstation MPM
Person:  Mike O'Connor
Due Date: 1 November 84

Demonstrate Text-Graphics-Image-Voice Document Exchange
Person:  All
Due Date: January 85

🫶 The Mailing List Glow-Up

the exploder mailing list one is already marked as COMPLETED 🎉 Greg Finn handled it on 15 August 84, so we love to see it ✨

5. Attendees 🌷💅

the whole squad who showed up, with their networks, emails, and phone numbers, all listed out so everyone can reach their bestie 📞💖

Harry Forsdick     BBN       Forsdick@BBN       (617) 497-3638
David L. Mills     Linkabit  Mills@ISID         (703) 734-9000
Louis Schreier     SRI       Schreier@SRI-SPAM  (415) 326-6200
Philip Au          SRI       Psa@SRI-SPAM       (415) 326-6200
Greg Finn          ISI       Finn@ISIF          (213) 822-1511
Mike O'Connor      Linkabit  OConnor@DCN9       (703) 734-9000
Ray Tomlinson      BBN       Tomlinson@BBN      (617) 497-3363
Ginny Travers      BBN       Travers@BBN        (617) 497-2647
Terry Crowley      BBN       TCrowley@BBN       (617) 497-2677
Andy Poggio        SRI       Poggio@SRI-TSC     (415) 859-5094
Jose Garcia-Luna   SRI       Garcia@SRI-TSC     (415) 859-5647
George Robertson   BBN       GRobertson@BBN     (617) 497-3632

---