RFC 2502

Limitations of Internet Protocol Suite for Distributed Simulation the Large Multicast Environment

📅 published February 1999, from ietf

RFC 2502 🎀 Limitations of Internet Protocol Suite for Distributed Simulation in the Large Multicast Environment

Okay so this one is from February 1999 and it's filed under the Network Working Group 💖. The authors are M. Pullen (George Mason University), M. Myjak (The Virtual Workshop), and C. Bouwens (SAIC). It's Informational, so it's giving "here's some useful info" energy, not a standard 🫶. The header doesn't list anything it obsoletes or updates, so it's standing on its own ✨.

Status of this Memo ✨

This memo gives information to the Internet community 🌸. It is not an Internet standard of any kind, and it can be shared freely with no limits 🦋.

Copyright (C) The Internet Society (1999). All Rights Reserved. 💜

Abstract 🔥

Bestie, the Large-Scale Multicast Applications (LSMA) working group was chartered to help build Internet protocols for large multicast apps, including real-time distributed simulation, through consensus-based development 🧚. This memo spells out the services LSMA found to be required, plus the parts of the Internet protocols that LSMA found need more development to meet those requirements 💅.

1. The Large Multicast Environment ✨

The Large-Scale Multicast Applications working group (LSMA) was formed to build a consensus-based set of requirements for Internet Protocols so they can support Distributed Interactive Simulation (DIS) [DIS94], its successor the High Level Architecture for simulation (HLA) [DMSO96], and related apps 🎀. These apps need to distribute real-time data across a shared wide area network in a scalable way. We're talking a few hosts up to tens of thousands, all swapping state data fast enough and reliably enough to keep a three-dimensional virtual, visual environment with tons of moving objects feeling alive 🤯. The network running something like this has to be capable of multicast [IEEE95a,IEEE95b] 💜.

Distributed Interactive Simulation is a whole family of protocols used to trade information about a virtual environment between hosts in a distributed system, where each host is simulating how objects in that environment behave 🦋. The objects can physically interact and sense each other visually and other ways (infrared, etc.) 👀. DIS was built by the U.S. Department of Defense (DoD) for military training, rehearsal, and other purposes. You can find more on DIS in [SSM96] 🫶.

Here's what really drives the network requirements: distributed simulation is meant to work with humans, sending output to them and taking input from them across distributed simulators in real time 😭. That means latency between hosts has to be super tight 💅. Any practical network also has to do multicasting so all the data reaches all the participating simulators 🔥. Big distributed simulations are expected to group hosts into multicast groups based on who shares the same sensor inputs in the virtual environment ✨. That can mean needing thousands of multicast groups, with objects hopping between groups in big numbers at high rates 🌈. Since the number of simulators is known ahead of time, and their max output rate (in packets per second and bits per second) is specified, the overall total data rate (the sum of all multicast groups) is bounded 🍓. But the data rate in any one particular group can't be predicted, and it can change pretty quickly during the simulation 🙈.

DIS real-time flow is made of packets around 2000 bits long, sent at rates from .2 packets per second per simulator up to 15 packets per second per simulator 💖. This info is deliberately redundant and usually goes over a best-effort transport protocol (UDP) 🎀. Sometimes it's also compressed 🧚. The accuracy needed for latency and for physical simulation depends on what the simulation is for, but it generally has to be at least enough to satisfy human perception 👑. For example, in tightly coupled simulations like high-performance aircraft, the max acceptable latency is 100 milliseconds between any two hosts ⭐. Every so often, events (e.g. collisions) can happen that need some data sent reliably, on a unicast basis, to any other host in the system 💀.

The U.S. DoD has a goal to build distributed simulation systems with up to 100,000 simulated objects 🫶. Lots of those objects are computer generated forces that run with minimal human involvement, acting as an opposing force or simulating friendly forces that aren't available to participate 🔥. DoD would like to run these simulations over a shared WAN 💅. Beyond DoD, lots of people think distributed simulation could get commercialized as entertainment 🌸. How big an entertainment system could get is hard to predict, but it could conceivably go past the DoD goal of 100,000 😱.

The High Level Architecture (HLA) is a DoD development that goes beyond DIS, aiming to bring DIS and other kinds of distributed simulation into one unifying system paradigm 💜. From a distributed systems standpoint, HLA is way more sophisticated than DIS 👑. For instance, attributes of distributed objects may be controlled by different simulators ✨. From the supporting network's point of view, the main difference is that HLA doesn't call for redundant transmission of object attributes 🌈. Instead, it specifies a "Run Time Infrastructure" (RTI) that's responsible for transmitting data reliably, and the RTI may choose to do that in different ways, including redundant transmission over best-effort protocols 🦋. It's fair to say any network that can meet DIS's needs can support HLA using DIS-like redundant transmission. But that approach ignores the possibility that, under HLA, a mix of redundant and reliable transmission can make way better use of network resources than DIS can 🧚. Even though HLA, like DIS, doesn't specify a multicasting network, it has similar requirements for many-to-many transmission of object attributes at rates over one update per object per second, and those can't be met without multicasting 🔥. HLA also calls for transmitting semantically organized data (for example, groups of objects with similar capabilities, like tanks or aircraft) in this many-to-many context 💖.

One solution people have used for these challenges is to aggregate the contents of many multicast groups into a single multicast transmission [PuWh95, CSTH95] 🎀. Called "dual-mode" or "bi-level" multicast, this takes advantage of the fact that even though the traffic in any one multicast group can vary a lot, the aggregate of all transmissions is bounded 🍓. If all the traffic is aggregated into one large flow, an underlying ATM network can create multicast SVCs with acceptable QoS to support the requirement 💅. It also limits the network control problem of group joins, since the joins happen among dedicated collections of routers and across the dedicated SVCs, instead of competing with other LSMAs that might share the same network 🌸. But it does all this at the cost of adding a new, nonstandard aggregation element to the network that's a hybrid of the Internet and ATM protocols 🙈. Below, we address the requirement to get this same result using a purely IP network with aggregated reservation via RSVP ✨.

The defense distributed simulation community has built a bunch of multicast-capable networks for different simulated exercises 🫶. These ranged from tens to hundreds of simulated objects, spread across anywhere from two to twenty sites 🧚. As the number of objects grew, they found that building multicasting networks that could potentially support thousands of simultaneous multicast groups with large group change rates is a hard problem 😭. This defense problem comes before similar problems we can expect in commercial networks 👑. So the sections below describe the services required, and the shortcomings found when using today's Internet protocols to provide them. The goal is to inform the IETF so it can produce protocols that meet the needs in these areas 💜🦋.

2. Distributed Simulation (DIS and HLA) network service requirements. 🌈

  • a. real-time packet delivery, with low packet loss (less than 2%), predictable latency on the order of a few hundred milliseconds, after buffering to account for jitter (variation of latency) such that less than 2% of packets fail to arrive within the specified latency, in a shared network 🔥
  • b. multicasting with thousands of multicast groups that can support join latencies of less than one second, at rates of hundreds of joins per second 🍓
  • c. multicasting using a many-to-many paradigm in which 90% or more of the group members act as receivers and senders within any given multicast group 🙈
  • d. support for resource reservation; because of the impracticality of over-provisioning the WAN and the LAN for large distributed simulations, it is important to be able to reserve an overall capacity that can be dynamically allocated among the multicast groups 💅
  • e. support for a mixture of best-effort and reliable low-latency multicast transport, where best-effort predominates in the mixture, and the participants in the reliable multicast may be distributed across any portion of the network 🌸
  • f. support for secure networking, in the form of per-packet encryption and authentication needed for classified military simulations 🔐

3. Internet Protocol Suite facilities needed and not yet available for large-scale distributed simulation in shared networks 🧚

These derive from the need for real-time multicast with established quality of service in a shared network. (Implementation questions are not included in this discussion. For example, it is not clear that implementations of IP multicast exist that will support the required scale of multicast group changes for LSMA, but that appears to be a question of implementation, not a limitation of IP multicast.) ✨

3.1. Large-scale resource reservation in shared networks ✨

Okay bestie, so the Resource reSerVation Protocol (RSVP) is all about setting up flow-based info for managing information flows at pre-committed performance levels 💅. Everyone agrees real-time systems like the HLA RTI need this, but people have raised concerns about whether RSVP can scale 😬 and whether it can handle highly dynamic flow control changes.

Here's the tea 🫖: for existing RTI capabilities, LSMA needs rapid change of group membership, not rapid change of group reservations. Why? Because in existing RTIs, the aggregate requirement for all groups in a large scale distributed simulation is static 🧊. BUT the current RSVP draft standard for LSMA doesn't support aggregating reservation resources for groups of flows, so it doesn't meet what existing RTIs need. Plus, at least one RTI development is underway that wants individual, dynamic reservations for large numbers of groups 👀. That means we'll need a dynamic resource reservation capability that scales to thousands of multicast groups 🔥.

Also, RSVP only handles communicating the specs of the required information flows between simulators and the network, and within the network 🛜. Distributing routing information among routers is a totally different job, handled by routing protocols like Multicast Open Shortest Path First (MOSPF) 🗺️. So for effective resource reservation in a large shared network, we might need a routing protocol that picks paths within the context of a quality of service requirement ✨. One example is the proposed Quality Of Service Path First (QOSPF) routing protocol [ZSSC97]. The catch? Defining that resource-sensitive routing requirement will be tough before LSMA networks are actually deployed with RSVP 😭.

Multicast is so efficient because the network can recognize and copy information packets destined for a group of locations 🤯. The network takes on the job of making duplicate copies for all destinations, which greatly cuts down the amount of info flowing into and through the network. It's giving efficiency 💖.

When IP multicast runs over Ethernet, packets get transmitted once and every receiver picks them up using Ethernet-layer multicast addressing, so there's no packet replication needed 🙌. But with wide-area Asynchronous Transfer Mode (ATM), you can't yet take advantage of data link layer multicast beyond a single Logical IP Subnet (LIS) 💀. This seems to happen for two reasons: (1) the switching models of IP and ATM are different enough that fixing it would need a pretty complex solution, and (2) nobody has shown a clear application requirement for IP multicast over ATM multicast that provides packet replication across multiple LIS. Distributed simulation is literally an application with that requirement 🎀.

3.3. Hybrid transmission of best-effort and reliable multicast 🌈

In general, the Internet protocol suite uses the Transmission Control Protocol (TCP) for reliable end-to-end transport, and the User Datagram Protocol (UDP) for best-effort end-to-end transport, including all multicast transport services 🧚. TCP can only do unicast, which is a bummer for us 😤.

The IETF has seen proposals for several reliable multicast transport protocols lately (see [Mont97] for a summary) 📚. The big issue is the congestion problem tied to delivery acknowledgments, which has made real-time reliable multicast transport infeasible up to now 🙈. Out of roughly 15 attempts to build a reliable multicast transport, every single one had some problem with positive receipt acknowledgments (ACK) or negative acknowledgments (NAK) 💅. Honestly, it seems clear there won't be one single solution for reliable multicast, just a bunch of solutions tailored to different application domains 🍓. Approaches using distributed logging seem especially promising for distributed simulation 🔥.

In the DIS/HLA environment, five different transmission needs can be identified:

  1. Best-effort low-latency multicast of object attributes that often change continuously, for example position of mobile objects 🏃‍♀️
  2. Low-latency reliable multicast of object attributes that don't change continuously but may change at arbitrary times during the simulation, for example object appearance (The important thing about this category is that the receiver only needs the latest value of any attribute ✨.)
  3. Low-latency, reliable unicast of occasional data among arbitrary members of the multicast group (This form of transmission was specified for DIS "collisions" 💥. It's not in the current HLA specification, but it might be a good idea to include it there. The requirement is occasional transaction-like exchange of data between two arbitrary hosts in the multicast group, with low latency that makes TCP connections impractical 🙅‍♀️.)
  4. Reliable but not necessarily real-time multicast distribution of supporting bulk data, such as terrain databases and object enumerations 🗂️; and
  5. Reliable unicast of control information between individual RTI components (this one is met by TCP 💜).

All of these transmissions happen inside the same large-scale multicasting environment 🌸. The idea of integrating categories (1) and (2) into a single selectively reliable protocol was proposed by Cohen [Cohe94] 💡. Pullen and Laviano implemented that concept [PuLa95] and demonstrated it within the HLA framework [PLM97] as the Selectively Reliable Transmission Protocol (SRTP) for categories (1) through (3) 🦋. Category (4) could be handled by a reliable multicast protocol like the commercial multicast FTP offering from Starburst [MRTW97], but adequate congestion control hasn't been demonstrated in any such protocol yet 😬. People have also talked about using the Real-Time Streaming Protocol (RTSP) for this, but since the databases must be transmitted reliably and RTSP uses a best-effort model, it doesn't look applicable 🙃.

So in summary, it's clear that a hybrid of best-effort and reliable multicast is needed to support DIS and HLA (not necessarily all in the same protocol), and that the low-latency, reliable part of this hybrid is not available in the Internet protocol suite 💔.

3.4. Network management for distributed simulation systems 🛠️

Coordinated, integrated network management is one of the trickiest parts of a large distributed simulation exercise 😵‍💫. The network management techniques that have successfully supported the growth of the Internet for the past several years could be expanded to fill this need 🌐. The technique is based on a primitive called a Management Information Base (MIB) that gets polled periodically at very low data rates 📊. The receiver of the poll is called an Agent, and it's collocated with the remote process being monitored. The Agent is kept simple so it doesn't eat up lots of resources 🫶. The requesting process is called a Manager, and it's typically located elsewhere on a separate workstation 🖥️. The Manager talks to all of the Agents in a given domain using the Simple Network Management Protocol (SNMP) 📡.

It seems SNMP is really well suited for distributed simulation management, on top of managing the underlying simulation network resources 👑. Creating a standard distributed simulation MIB format would let the simulation community use the whole collection of powerful, off-the-shelf network management tools built around SNMP 🧰. Iconic, honestly 🤌.

3.5. A session protocol to start, pause, and stop a distributed simulation exercise ⏯️

Coordinating the start, stop, and pause of large distributed exercises is a complex and difficult task 😩. The Session Initiation Protocol (SIP), recently proposed by the Multiparty Multimedia Session Control (MMUSIC) working group, does something similar for managing large scale multimedia conferences 🎤. As proposed, SIP seems to offer enough extensibility to be used for exercise session control, if it gets standardized by the IETF 🥺.

3.6. An integrated security architecture 🔐

It looks like this requirement will be met by IPv6 deployment 🎉. One shortcoming of the current Internet Protocol (IPv4) implementation is the lack of integrated security 😤. The new IPv6 protocol requires implementers to follow an integrated security architecture that provides the required integrity, authenticity, and confidentiality for use of the Internet by communities with stringent security demands, such as the financial community 💰. The possibility that the IPv6 security architecture could meet military needs, when combined either with military cryptography or government-certified commercial cryptography, is worth further study 🧐.

3.7. Low-latency multicast naming service 🏷️

Name-to-address mapping on the Internet is done by the Domain Name Service (DNS) 📖. DNS has a distributed architecture tuned for the needs of unicast networking with reliable transmission (TCP), and that's not considered a problem as long as its latency is on the order of a second or more 👍. But distributed simulation needs agile movement among multicast groups, which implies a need for name-to-multicast-address mapping with latency under one second for name resolution and group join combined ⚡. This problem has been worked around in military simulations by using group IP addresses rather than names 🪖. Military simulations may be fine communicating with a known mapping from grid squares to multicast groups, but growth of distributed simulation into commercial entertainment can't be built on that simple capability 🎮. Players in distributed entertainment simulations will want to be organized symbolically by virtual world and role 💜. So a low-latency multicast naming service will be required 💅.

3.8. Inter-Domain Multicast Routing for LSMA 🌍

While military LSMAs typically take place within a single administrative domain, future entertainment LSMAs can be expected to involve heavy inter-domain multicast traffic, since players may be supported by multiple service providers 🌈. Standardized protocols that can support large numbers of multicast flows across domain boundaries will be needed for this 🫶. Current work to create a Border Gateway Multicast Protocol (BGMP) shows promise of meeting that need ✨.

4. References 📚

okay bestie, here's the whole reading list ✨ every source this RFC leans on, so you can go dig if you're curious 🫶

  • [CSTH95] Calvin, J., et. al., "STOW Realtime Information Transfer and Networking Architecture," 12th DIS Workshop on Standards for the Interoperability Distributed Simulations, March 1995.
  • [Cohe94] Cohen, D., "Back to Basics," Proceedings of the 11th Workshop on Standards for Distributed Interactive Simulation, Orlando, FL, September 1994.
  • [DIS94] DIS Steering Committee, "The DIS Vision," Institute for Simulation and Training, University of Central Florida, May 1994.
  • [DMSO96] Defense Modeling and Simulation Office, High Level Architecture Rules Version 1.0, U.S. Department of Defense, August 1996.
  • [IEEE95a] IEEE 1278.1-1995, Standard for Distributed Interactive Simulation - Application Protocols
  • [IEEE95b] IEEE 1278.2-1995, Standard for Distributed Interactive Simulation - Communication services and Profiles
  • [MRTW97] Miller, K., et. al. "StarBurst Multicast File Transfer Protocol (MFTP) Specification", Work in Progress.
  • [Mont97] Montgomery, T., Reliable Multicast Links webpage, http://research.ivv.nasa.gov/RMP/links.html
  • [PuLa95] Pullen, M. and V. Laviano, "A Selectively Reliable Transport Protocol for Distributed Interactive Simulation", Proceedings of the 13th Workshop on Standards for Distributed Interactive Simulation, Orlando, FL, September 1995.
  • [PuWh95] Pullen, M. and E. White, "Dual-Mode Multicast: A New Multicasting Architecture for Distributed Interactive Simulation," 12th DIS Workshop on Standards for the Interoperability of Distributed Simulations, Orlando, FL, March 1995.
  • [PLM97] Pullen, M., Laviano, V. and M. Moreau, "Creating A Light-Weight RTI As An Evolution Of Dual-Mode Multicast Using Selectively Reliable Transmission," Proceedings of the Second Simulation Interoperability Workshop, Orlando, FL, September 1997.
  • [SPW94] Symington, S., Pullen, M. and D. Wood, "Modeling and Simulation Requirements for IPng", RFC 1667, August 1994.
  • [SSM96] Seidensticker, S., Smith, W. and M. Myjak, "Scenarios and Appropriate Protocols for Distributed Interactive Simulation", Work in Progress.
  • [ZSSC97] Zhang, Z., et. al., "Quality of Service Path First Routing Protocol", Work in Progress.

4. Security Considerations 🔐

security is the big serious part, and it lives over in section 3.6 🛡️ the security architecture stuff is where all the drama is, so go read that one with your full attention 😭

5. Authors' Addresses 🏡

here's where to find the authors, if you want to say hi 💌

J. Mark Pullen
Computer Science/C3I Center
MS 4A5
George Mason University
Fairfax, VA 22032

EMail: mpullen@gmu.edu


Michael Myjak
The Virtual Workshop
P.O. Box 98
Titusville, FL 32781

EMail: mmyjak@virtualworkshop.com


Christina Bouwens
ASSET Group, SAIC Inc.
Orlando, FL

EMail: christina.bouwens@cpmx.mail.saic.com

Copyright (C) The Internet Society (1999). All Rights Reserved.

okay so the legal bit, but make it cute 🎀 anyone can copy and share this document, and can make derivative works that comment on it, explain it, or help people implement it, and they can publish and distribute those in whole or in part with no restriction, as long as the copyright notice and this paragraph stay on every copy and every derivative work. the document itself, though, can't be changed in any way, like taking out the copyright notice or the references to the Internet Society or other Internet organizations. the exceptions are developing Internet standards (in which case the copyright procedures in the Internet Standards process have to be followed) or translating it into languages other than English 🌍

the permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns 💜

this document and the information in it is provided on an "AS IS" basis, and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE 💅

📦 where this page comes from