RFC 1077

Critical issues in high bandwidth networking

๐Ÿ“… published November 1988, from ietf

RFC 1077: Critical Issues in High Bandwidth Networking ๐ŸŽ€

Okay bestie, this is Part 1 of 10 ๐ŸŒธ and the title block is giving main character energy โœจ. It's from the Network Working Group, specifically the Gigabit Working Group, and the editor is B. Leiner ๐Ÿ’…. It's dated November 1988 ๐Ÿ“…. The header doesn't list anything it obsoletes or updates ๐Ÿงš.


Status of This Memo โœจ

This memo shares the results of a working group on High Bandwidth Networking ๐ŸŒˆ. It's for your information, and you're encouraged to comment on the issues presented ๐Ÿซถ. Distribution of this memo is unlimited ๐Ÿฆ‹.

ABSTRACT ๐Ÿ’–

So Maj. Mark Pullen and Maj. Brian Boesch of DARPA asked for an ad-hoc working group ๐Ÿ‘‘ to put together recommendations on the research needed for a ubiquitous high-bandwidth network. This is the kind of network the FCCSET recommendations for Phase III talked about, and it's a big deal ngl ๐Ÿ”ฅ.

This report lays out a set of research topics meant to build the technology base for a bunch of interconnected networks that can deliver high-bandwidth capabilities ๐Ÿ“. The research focus builds on ongoing work and adds basic and applied pieces to it ๐Ÿ’œ. The major activities are:

  • building and demonstrating a gigabit backbone network ๐Ÿฆ‹
  • building and demonstrating a set of interconnected networks with gigabit throughput and solid management techniques ๐ŸŒธ
  • building and demonstrating the overall architecture that lets users actually access all that bandwidth ๐Ÿ‘‘

1. Introduction and Summary ๐Ÿฆ‹

1.1. Background ๐ŸŒˆ

The computer communications world is evolving toward high-bandwidth capability and high-bandwidth requirements, and honestly it's giving everything at once ๐Ÿ˜ญ. The recent FCCSET Committee on High Performance Computing workshop [1] found a bunch of areas that need extremely high-bandwidth networking to support the scientific research community ๐Ÿ”ฌ. These range from remote graphical visualization of supercomputer results all the way to moving high-rate sensor data from space down to ground-based scientists ๐Ÿ›ฐ๏ธ. Military command and control (C2) has the same needs, since it has to quickly access and act on data from real-time sensors ๐Ÿซก.

The workshop also identified a need for switched high-bandwidth service of more than 300 Mbit/s to a single user ๐Ÿ’ฅ. It also found a need for service in the range of a Mbit/s on a low-duty-cycle basis for millions of researchers ๐Ÿคฏ. Add in the needs of military and commercial users, and the aggregate requirement adds up to many billions of bits per second ๐Ÿ’€. The results of this workshop went into a report by the FCCSET [2] ๐Ÿ“‘.

Fortunately, technology is moving fast too โšก. Even today, the installed base of fiber optics lets us think about aggregate bandwidths in the range of Gbit/s and beyond, but only across limited geographical regions โœจ. People at the workshop estimated there will be raw bandwidth approaching terabits per second, which is absolutely unhinged in the best way ๐Ÿงš.

The big question is how to use all this raw bandwidth to meet the workshop's requirements. The first is bandwidth on the order of several Gbit/s for individual users ๐Ÿ’–. The second is modest bandwidth on the order of several Mbit/s for a large number of users, delivered cost-effectively by aggregating their traffic ๐Ÿซถ.

DARPA has been a central player in developing packet-oriented communications through its research funding ๐Ÿ‘‘. That's been hugely beneficial to the U.S. military in terms of survivability and interoperability ๐ŸŽ€. DARPA-funded work gave us a lot of iconic stuff: the ARPANET, the first packet-switched network ๐Ÿฆ‹; the SATNET, MATNET and Wideband Network, which showed efficient use of shared-access satellite channels between geographically diverse sites ๐Ÿ›ฐ๏ธ; packet radio networks for mobile tactical environments ๐Ÿ“ก; the Internet and TCP/IP protocols for interconnection and interoperability between heterogeneous networks and computer systems ๐ŸŒ; electronic mail ๐Ÿ’Œ; and a lot of advances in network security, privacy, authentication and access control for distributed computing environments ๐Ÿ”.

Recognizing DARPA's past accomplishments and its desire to keep leading on these issues, this document recommends research topics in gigabit networking ๐Ÿ’…. It's meant to be an organized compendium of the critical research issues that need to be solved to build the technology base for this kind of high-bandwidth ubiquitous network ๐ŸŒˆ.

1.2. Ongoing Activities ๐Ÿ“

The OSTP report mentioned above recommended a three-phase approach to getting the required high-bandwidth networking for the scientific and research community ๐Ÿ“‹. Some of this work is already well underway ๐Ÿƒโ€โ™€๏ธ. An ad-hoc committee, the Federal Research Internet Coordinating Committee (FRICC), is coordinating the interconnection of the current wide area networking systems in the government ๐Ÿ›๏ธ. The networks involved include those of DARPA, the Department of Energy (DoE), the National Science Foundation (NSF), the National Aeronautics and Space Administration (NASA), and the Department of Health and Human Services (HHS) ๐Ÿซถ.

Following Phases I and II of the OSTP report, this activity will provide an interconnected set of networks to support research and other scholarly pursuits, and it'll be a foundation for future networking for this community ๐ŸŒธ. The networking is getting upgraded through shared increased bandwidth, with current plans to share a 45 Mbit/s backbone ๐Ÿ’ซ. It's also getting coordinated interconnection with the rest of the world ๐ŸŒ. In particular, the FRICC is working with the European networking community through another ad-hoc group, the Coordinating Committee for Intercontinental Research Networks (CCIRN), to set up effective US-Europe networking ๐Ÿ‡ช๐Ÿ‡บโœจ.

But as the OSTP recommendations point out, the bandwidth needed in the future is way past what public, private, and government networks currently have planned ๐Ÿ˜ณ. Getting to gigabit networking will take a strong research effort ๐Ÿ”ฌ. There's a lot of ongoing research in relevant areas that we can draw from ๐Ÿ“š, especially in high-bandwidth communication links, high-speed computer switching, and high-bandwidth local area networks ๐Ÿ’œ. Appendix A has some pointers to current research efforts ๐Ÿงš.

1.3. Document Overview ๐Ÿงš

This report lays out research topics for building the technology base for an interconnected set of networks that can deliver the high-bandwidth capabilities we just talked about ๐Ÿ’–. The research focus builds on ongoing work and adds basic and applied components ๐ŸŒˆ. The major activities are developing and demonstrating a Gigabit Backbone network (GB) [3] ๐Ÿฆ‹, developing and demonstrating an interconnected set of networks with gigabit throughput and proper management techniques ๐Ÿซถ, and developing and demonstrating the overall architecture that lets users access this high bandwidth ๐Ÿ‘‘.

Section 2 covers the functional and performance goals, plus the benefits these would bring to the ultimate users of such a system ๐ŸŽฏ. Section 3 is where the critical research issues needed to hit those goals get discussed ๐Ÿ”. It's organized into the major technology areas that need work: general architectural issues, high-bandwidth switching, high-bandwidth host interfaces, network management algorithms, and network services ๐Ÿ—‚๏ธ. Some of the discussion includes examples of ongoing relevant research or possible approaches ๐Ÿ’ก. These examples are there to clarify the issues, not to push any particular approach ๐Ÿ™…โ€โ™€๏ธ. Section 4 discusses how this research relates to other ongoing activities, plus the best methods for pursuing it ๐ŸŒธ.

2. Functional and Performance Goals ๐Ÿ‘‘

Here we assess the types of services a GN should provide to its users โœจ. The GN is four or five orders of magnitude faster than current networks, so this is a big glow-up ๐Ÿ˜ค. Where we felt there'd be a significant performance impact, we estimated how much bandwidth is needed and how much delay is allowable to provide these services ๐Ÿ“Š.

2.1. Networking Application Support ๐Ÿซถ

The GN is envisioned to support all of the following types of networking applications ๐ŸŒˆ.

Currently Provided Packet Services

It's important that the network give users the equivalent of services that already exist in packet-switched networks ๐Ÿ’Œ. Those include interactive data exchange, mail service, file transfer, and online access to remote computing resources, among others ๐Ÿ’ป. The network should also let users expand into more advanced services as they become available ๐Ÿš€.

Multi-Media Mail

This capability lets users take advantage of different media types, like graphics, images, voice, and video as well as text and computer data, when they send messages ๐ŸŽจ. That makes message exchange way more effective ๐Ÿ’….

Multi-Media Conferencing

Conferencing means exchanging large amounts of information in short periods of time ๐ŸŽฅ, so it needs high bandwidth with low delay ๐ŸŒŠ. We estimate bandwidth would range from 1.5 to 100 Mbit/s, with end-to-end delay of no more than a few hundred msec. โฑ๏ธ.

Computer-Generated Real-time Graphics

Visualizing computer results in the modern supercomputer world needs tons of real-time graphics ๐Ÿ“ˆ. That takes about 1.5 Mbit/s of bandwidth and no more than several hundred msec. of delay ๐Ÿฆ‹.

High-Speed Transaction Processing

One of the most important reasons to have an ultra-high-speed network is to take advantage of supercomputing capability ๐Ÿ’ฅ. There are several scenarios where that could be used ๐ŸŽญ. For example, a non-supercomputer might need a supercomputer to do some processing and return intermediate results that get used for further processing ๐Ÿง . Or several supercomputers operating in tandem might periodically exchange results, like in battle management, war gaming, or process control applications โš”๏ธ. In those cases, super short response times are necessary to get through as many as hundreds of interactions in real time ๐Ÿ˜ญ. That needs very high bandwidth, on the order of 100 Mbit/s, and minimum delay, on the order of hundreds of msec. ๐Ÿ’œ.

Wide-Area Distributed Data/Knowledge Base Management Systems ๐ŸŒ

Computer-stored data, information, and knowledge is literally spread all over the country for a bunch of different reasons ๐Ÿ—บ๏ธโœจ Being able to run complex queries, updates, and report generation as though lots of large databases are ONE system would be extremely powerful ๐Ÿ’… but it needs low-delay, high-bandwidth communication for interactive use ๐Ÿซถ The Corporation for National Research Initiatives (NRI) has been hyping up the idea of a National Knowledge base with these characteristics ๐ŸŒˆ The cute approach here is to cache views at the user sites, or close by, so repeated queries and multi-relation processing for relations on different nodes run efficiently ๐Ÿงš

But here's the catch ๐Ÿ˜ญ With caching, a processing activity may hit a miss in the middle of a query or update, so it gets delayed by however long it takes to retrieve the missing relation or portion of relation ๐Ÿ’€ To keep the overhead for cache directories low (at both the server and client sites), the unit of caching should be large, like a megabyte or more ๐Ÿ“ Also, to keep things consistent at the caching client sites, server sites need to multicast invalidations and/or updates ๐Ÿ”ฅ Replicating the data makes communication requirements even higher ๐Ÿ“ˆ

The critical parameter here is latency for cache misses and consistency operations โญ Say the distance between sites is on average 1/4 the diameter of the country ๐Ÿ—บ๏ธ then a one Gbit/s data rate is required to reduce the transmission time so it's roughly the same as the propagation delay, which is around 8 milliseconds for this size of unit ๐ŸŽ€ Also, note this app is supporting way more sophisticated queries and updates than you'd normally see in transaction processing, so it needs a larger amount of data to be transferred ๐Ÿ’–


2.2. Types of Traffic and Communications Modes โœจ

Okay, different types of traffic can come with different constraints when it comes to throughput, delay, delay dispersion, reliability, and sequenced delivery ๐Ÿฆ‹ Table 1 sums up some of the main characteristics of several different types of traffic ๐Ÿ’œ

   Table 1: Communication Traffic Requirements

   +------------------------+-------------+-------------+-------------+
   |                        |             |             | Error-free  |
   | Traffic                | Delay       | Throughput  | Sequenced   |
   | Type                   | Requirement | Requirement | Delivery    |
   +------------------------+-------------+-------------+-------------+
   | Interactive Simulation | Low         |Moderate-High| No          |
   +------------------------+-------------+-------------+-------------+
   | Network Monitoring     | Moderate    | Low         | No          |
   +------------------------+-------------+-------------+-------------+
   | Virtual Terminal       | Low         | Low         | Yes         |
   +------------------------+-------------+-------------+-------------+
   | Bulk Transfer          | High        | High        | Yes         |
   +------------------------+-------------+-------------+-------------+
   | Message                | Moderate    | Moderate    | Yes         |
   +------------------------+-------------+-------------+-------------+
   | Voice                  |Low, constant| Moderate    | No          |
   +------------------------+-------------+-------------+-------------+
   | Video                  |Low, constant| High        | No          |
   +------------------------+-------------+-------------+-------------+
   | Facsimile              | Moderate    | High        | No          |
   +------------------------+-------------+-------------+-------------+
   | Image Transfer         | Variable    | High        | No          |
   +------------------------+-------------+-------------+-------------+
   | Distributed Computing  | Low         | Variable    | Yes         |
   +------------------------+-------------+-------------+-------------+
   | Network Control        | Moderate    | Low         | Yes         |
   +------------------------+-------------+-------------+-------------+

The topology among users can be one of three types: point-to-point (one-to-one connectivity), multicast (one sender and multiple receivers), and conferencing (multiple senders and multiple receivers) ๐Ÿ‘ฏ There are three types of transfers that can take place among users: connection-oriented network service, connectionless network service, and stream or synchronous traffic ๐ŸŽ€ Connection and connectionless services are asynchronous.

Here's the vibe: a connection-oriented service assumes and provides for relationships among the multiple packets sent over the connection (e.g., to a common destination), while connectionless service assumes each packet is a complete and separate entity unto itself ๐Ÿซง For stream or synchronous service, a reservation scheme is used to set up and guarantee a constant and steady amount of bandwidth between any two subscribers ๐ŸŒธ

2.3. Network Backbone โœจ

The GB needs to be of high bandwidth to support a large population of users, and additionally to provide high-speed connectivity among certain subscribers who may need that capability (e.g., between two supercomputers) ๐Ÿ–ฅ๏ธ These users may access the GN from local area networks (LANs) directly connected to the backbone, or via high-speed intermediate regional networks. The backbone must also minimize end-to-end delay to support highly interactive high-speed (supercomputer) activities ๐Ÿ”ฅ

It is important that the LANs that will be connected to the GN be permitted data rates independent of the data rates of the GB ๐Ÿ™Œ LAN speeds should be allowed to change without affecting the GB, and the GB speeds should be allowed to change without affecting the LANs. That way, development of the technology for LANs and the GB can proceed independently ๐Ÿงš

Access rate requirements to the GB and the GN will vary depending on user requirements and local environments ๐ŸŒˆ Users may need access rates ranging from multi-kbit/s for terminals or personal computers connected by modems, up to multi-Mbit/s and beyond for powerful workstations, and up to the Gbit/s range for high-speed computing and data resources ๐Ÿ’ซ

2.4. Directory Services โœจ

Directory services similar to those found in CCITT X.500/ISO DIS 9594 need to be provided ๐Ÿ“’ These include mapping user names to electronic mail addresses, distribution lists, support for authorization checking, access control, and public key encryption schemes, multimedia mail capabilities, and the ability to keep track of mobile users (those who move from place to place and host computer to host computer) ๐Ÿฆ‹ The directory services may also list facilities available to users via the network. Some examples are databases, supercomputing or other special-purpose applications, and on-line help or telephone hotlines โ˜Ž๏ธ

The services provided by X.500 may require some extension for GN. For example, there is no provision for multilevel security, and the approach taken to authentication must be studied to ensure that it meets the requirements of GN and its user community ๐Ÿ”

2.5. Network Management and Routing โœจ

The objective of network management is to ensure that the network functions smoothly and efficiently. It consists of the following: accounting, security, performance monitoring, fault isolation, and configuration control ๐Ÿ—‚๏ธ

Accounting ensures that users are properly billed for the services that the network provides ๐Ÿ’ธ Accounting enforces a tariff, and a tariff expresses a usage policy. The network need only keep track of those items addressed by the tariff, such as allocated bandwidth, number of packets sent, number of ports used, etc. Another type of accounting may need to be supported by the network to support resource sharing, namely accounting analogous to telephone "900" numbers ๐Ÿ“ž This accounting, performed by the network on behalf of resource providers and consumers, is a pragmatic solution to the problem of getting users and consumers into a financial relationship with each other, which has stymied previous attempts to achieve widespread use of specialized resources ๐Ÿ’…

Performance monitoring is needed so that the managers can tell how the network is performing and take the necessary actions to keep its performance at a level that will provide users with satisfactory service ๐Ÿ“Š Fault isolation using technical control mechanisms is needed for network maintenance ๐Ÿ› ๏ธ Configuration management allows the network to function efficiently โœจ

Several new types of routing will be required by GN ๐Ÿ›ฃ๏ธ In addition to true type-of-service, needed to support diverse distributed applications, real-time applications, interactive applications, and bulk data transfer, there will be a need for traffic controls to enforce various routing policies. For example, policy may dictate that traffic from certain users, applications, or hosts may not be permitted to traverse certain segments of the network ๐Ÿšง Alternatively, traffic controls may be used to promote fairness; that is, to make sure that a busy link or network segment isn't dominated by a particular source or destination โš–๏ธ The ability of applications to reserve network bandwidth in advance of its use, and the use of strategies such as soft connections, will also require development of new routing algorithms ๐Ÿง 

2.6. Network Security Requirements โœจ

Security is a critical factor within the GN, and it's one of those features that are difficult to provide ๐Ÿ”’ It is envisioned that both unclassified and classified traffic will utilize the GN, so protection mechanisms must be an integral part of the network access strategy. Features such as authentication, integrity, confidentiality, access control, and nonrepudiation are essential to provide trusted and secure communication services for network users ๐Ÿ›ก๏ธ

A subscriber must have assurance that the person or system he is exchanging information with is indeed who he says he is. ๐ŸŒˆ Authentication provides this assurance by verifying that the claimed source of a query request, control command, response, etc., is the actual source ๐Ÿ•ต๏ธ Integrity assures that the subscriber's information (such as requests, commands, data, responses, etc.) is not changed, intentionally or unintentionally, while in transit or by replays of earlier traffic ๐Ÿ” Unauthorized users (e.g., intruders or network viruses) would be denied use of GN assets through access control mechanisms, which verify that the authenticated source is authorized to receive the requested information or to initiate the specified command ๐Ÿšซ In addition, nonrepudiation services can be offered to assure a third party that the transmitted information has not been altered ๐Ÿ“ And finally, confidentiality will ensure that the contents of a message are not divulged to unauthorized individuals ๐Ÿคซ Subscribers can decide, based upon their own security needs and particular activities, which of these services are necessary at a given time ๐ŸŽ€


3. Critical Research Issues โœจ

In the section above, we discussed the goals of a research program in gigabit networking, namely to provide the technology base for a network that will allow gigabit service to be provided in an effective way ๐Ÿš€ In this section, we discuss those issues which we feel are critical to address in a research program to achieve such goals ๐Ÿ’œ

3.1. General Architectural Issues โœจ

okay so, in the last generation of networks, everyone assumed bandwidth was the scarce resource ๐Ÿฅบ, so the whole switch design was basically built around managing and allocating it super carefully. the most basic change in the next-gen network? the trunks are getting faster way faster than the switching elements are ๐Ÿš€. that's the big plot twist.

this shift in speeds is showing up in a bunch of ways ๐Ÿ’…. in most current LAN designs, where bandwidth isn't expensive, the move was to trade away effective bandwidth use for a simpler switching technique โœจ. networks like Ethernet use broadcast as the normal distribution method, which basically means you don't really need a switching element at all ๐Ÿ™ˆ.

as we look at even higher-speed networks, especially ones where bandwidth is still the expensive part ๐Ÿ’ธ, we need new switching options that let us use bandwidth effectively without the switch itself becoming the bottleneck ๐Ÿšง.

so the central thrust of new research is to explore new network architectures that actually fit these very different speed assumptions ๐Ÿง .

computer communications have been seriously warped by wide-area networking ๐Ÿ˜ญ. it's normally high cost, low speed, high error rate, and large delay. honestly, the time is ripe for a full revolution in thinking, technology, and approaches ๐ŸŽฌ, kind of like how VCR technology was a revolution compared to 8 and 16 mm. film technology ๐Ÿ“ผ.

fiber optics is clearly the enabling tech for high-speed transmission ๐ŸŒˆ, and it's so good that people now expect the switching elements to be what holds the data rates down ๐Ÿ˜ค. both conventional circuit switching and packet switching have real problems at higher data rates ๐Ÿ’€. for instance, circuit switching needs increasing delays for FTDM synchronization to handle skew ๐Ÿซ . in the case of packet switching, traditional approaches need way too much processing per packet to handle the tremendous data flow. the problem for both switching regimes is the "intelligence" in the switches, which in turn requires electronics technology ๐Ÿ”Œ.

besides intelligence, another problem for wide-area networks is storage ๐Ÿ“ฆ, both because it ties us to electronics (for the foreseeable future) and because it produces instabilities in a large-scale system. (See, for instance, the work by Van Jacobson on self-organizing phenomena for self-destruction in the Internet ๐Ÿ‘€.) techniques are required to eliminate dependence on storage, such as cut-through routing โœ‚๏ธ.

overall, high-speed WANs are the greatest agents of change ๐Ÿ”ฅ, the greatest catalyst both commercially and militarily, and the area ripe for revolution ๐Ÿ‘‘. judging by the attributes of current high-speed network research prototypes, WANs of the future will be photonic, multi-gigabit networks with enormous throughput, low delay, and low error rate โœจ๐ŸŒธ.

a zero-based budgeting approach is required to develop the new high-speed internetwork architecture ๐Ÿ“‹. that is, the time is ripe to significantly rethink the Internet, building on experience with this system ๐Ÿ’ญ. the issues of concern are manageability, understanding evolvability, and support for the new communication requirements, including remote procedure call, real-time, security, and fault-tolerance ๐Ÿ›ก๏ธ.

the GN must be able to deal with two sources of high-bandwidth requirements ๐ŸŽฏ. first, there will be some end devices (computers) connected more or less directly to the GN because of their individual requirements for high bandwidth (e.g., supercomputers needing to drive remote high-bandwidth graphics devices) ๐Ÿ–ฅ๏ธ. second, the aggregate traffic from large numbers of moderate-rate users (estimates are roughly up to a million potential users needing up to 1 Mbit/s at any given time) adds up to a high-bandwidth requirement in total on the GN ๐ŸŒŠ. the statistics of such traffic are different, and there are different possible technical approaches for dealing with them, so an architectural approach for handling both must be developed ๐Ÿงฉ.

overall, the next-generation architecture has to be, first and foremost, a management architecture ๐Ÿ’ผ. the directions in link speeds, processor speeds, and memory solve the performance problems for many communication situations so well that manageability becomes the predominant concern ๐Ÿ’…. (in fact, fast communication makes large systems more prone to performance, reliability, and security problems ๐Ÿ˜ฌ.) in many ways, the management system of the internetwork is the ultimate distributed system ๐ŸŒ. the solution to this tough problem may well require the best talents from the communications, operating systems, and distributed systems communities, perhaps even drawing on database and parallelism research ๐Ÿค“.


3.1.1. High-Speed Internet using High-Speed Networks โšก

the GN will need to take advantage of a multitude of different and heterogeneous networks, all of high speed ๐ŸŒŸ. in addition to networks based on the technology of the GB, there will be high-speed LANs. a key issue in the development of the GN will be coming up with a strategy for interconnecting such networks to provide gigabit service on an end-to-end basis ๐Ÿ”—. this will involve techniques for switching, interfacing, and management (as discussed in the sections below), coupled with an architecture that allows the GN to take full advantage of the performance of the various high-speed networks ๐Ÿš€.

3.1.2. Network Organization ๐ŸŒท

the GN will need an architecture that supports the need to manage the system as well as obtain high performance ๐Ÿ“. we note that almost all human-engineered systems are hierarchically structured from the standpoint of control, monitoring, and information flow ๐Ÿ—‚๏ธ. a hierarchical design may be the key to manageability in the next-generation architecture ๐Ÿ”‘.

one approach is to use a general three-level structure, corresponding to interadministrational, intraadministrational, and cluster networks ๐ŸŽ€. the first level interconnects communication facilities of truly separate administrations, where there is significant separation of security, accounting, and goals ๐Ÿฐ. the second level interconnects subadministrations, which exist for management convenience in large organizations. for example, a research group within a university may function as a subadministration ๐ŸŽ“. the cluster level consists of networks configured to provide maximal performance among hosts which are in frequent communication, such as a set of diskless workstations and their common file server ๐Ÿ–ฅ๏ธ. these hosts are typically, but not necessarily, geographically collocated ๐Ÿ“. for example, two remote networks may be tightly coupled by a fiber optic link that bridges between the two physical networks, making them function as one ๐Ÿชข.

research along these lines should study the interorganizational characteristics of communications, such as those being investigated by the IAB Task Force on Autonomous Networks ๐Ÿ”ฌ. based on current results, we expect that such work would clearly demonstrate that considerable communication takes place between particular subadministrations in different administrations, so communication patterns are not strictly hierarchical ๐ŸŒ€. for example, there might be intense direct communication between the experimental physics departments of two independent universities, or between the computer support group of one company and the operating system development group of another ๐Ÿงช. in addition, (sub)administrations may well also require divisions into public information and private information ๐Ÿ”.

3.1.3. Fault-Tolerant System ๐Ÿ›Ÿ

although the GN will be developed as part of an experimental research program, it will also serve as part of the infrastructure for researchers who are experimenting with applications which will use such a network ๐Ÿง‘โ€๐Ÿ”ฌ. the GN must have reasonably high availability to support these research activities ๐Ÿ“ถ. in addition, to facilitate the transfer of this technology to future operational military and commercial users, it will need to be designed to become highly reliable ๐Ÿ›ก๏ธ. this can be accomplished through diversity of transmission paths, the development of fault-tolerant switches, use of a distributed control structure with self-correcting algorithms, and the protection of network control traffic ๐Ÿงฏ. the architecture of a GN should support and allow for all of these things โœ….

3.1.4. Functional Division of Control Between Network Elements ๐Ÿง 

current protocol architectures use the layered model of functional decomposition, first developed in the early work on ARPANET protocols ๐Ÿฅฐ. the concept of layering has been a powerful one, allowing dramatic variation in network technologies without requiring the complete reimplementation of applications ๐Ÿฐ. the concept of layering has also had a first-order impact on the development of international standards for data communication, witness the ISO "Reference Model for Open Systems Interconnection." ๐ŸŒ

unfortunately, however, the powerful concept of layering has been paired, both in the DoD Internet work and the ISO work, with an extremely weak concept of the interface between layers ๐Ÿ˜ค. the interface designs are all organized around the idea of commands and responses plus an error indicator ๐Ÿ“ž. for example, the TCP service interface provides the user with commands to set up or close a TCP connection, and commands to send and receive datagrams ๐Ÿ“จ. the user may well "know" whether they are using a file transfer service or a character-at-a-time virtual terminal, but can't tell the TCP ๐Ÿ™ƒ. the underlying network may "know" that failures have reduced the path to the user's destination to a single 9.6 kbit/s link, but it also can't tell the TCP implementation ๐Ÿข.

all of the information that an analyst would consider crucial in diagnosing system performance is carefully hidden from adjacent layers ๐Ÿ•ต๏ธ. one "solution" often discussed (but rarely implemented) is to condense all of this information into a few bits of "Type of Service" or "Quality of Service" request flowing in one direction only, from application to network ๐Ÿ—œ๏ธ. it seems likely that this approach cannot succeed, both because it applies too much compression to the knowledge available and because it does not provide two-way flow ๐Ÿšซ.

we believe it to be likely that the next-generation network will require a much richer interface between every pair of adjacent layers if adequate performance is to be achieved ๐Ÿ’–. research is needed into the conceptual mechanisms, both indicators and controls, that can be implemented at these interfaces and that, when used, will result in better performance ๐Ÿ”ฌ. if real differences in performance can be observed, then the implementors of every layer will have a strong incentive to make use of the mechanisms ๐Ÿซถ.

We can observe the first glimmers of this kind of coordination between layers in current work ๐Ÿ‘€๐Ÿ’–. For example, in the ISO work there are 5 classes of transport protocol that are supposed to cover a range of matches between application needs and network capabilities ๐Ÿงฉ. Unfortunately, the class of transport protocol gets chosen statically, by the implementer, instead of dynamically ๐Ÿ˜ญ. The DARPA Wideband net offers a choice of stream or datagram service, but a given host typically uses just one or the other the whole time, which is static again ๐Ÿ’€. So the research we believe is needed isn't really about how to provide alternatives. It's about how to provide them and choose among them on a dynamic, real-time basis โœจ๐Ÿฆ‹.

3.1.5. Different Switch Technologies ๐ŸŒˆ

So one way to chase high-performance networking is to build a technology that's basically a standalone demo, without worrying about hooking up to other networks ๐Ÿ™ƒ. Honestly, those experiments can be super valuable for quickly exploring the design space ๐Ÿ”. Still, our experience with the Internet project kind of says that a primary research goal should be an architecture that lets a number of different switching technologies interconnect ๐Ÿ”ฅ.

The Internet project was a massive win mostly because it could scoop up a bunch of new and preexisting network tech ๐Ÿ“: various LANs, store and forward switching networks, broadcast satellite nets, packet radio networks, and more. That meant it decoupled the protocols from any particular technology base ๐Ÿงš. The tech base kept evolving like crazy, but the Internet protocols themselves gave everything a stability that led to their success ๐Ÿ‘‘.

The next-generation architecture must similarly deal with a diverse and evolving technology base. We see "fast-packet" switching being developed now (for example in B-ISDN), and we see photonic switching and wavelength division multiplexing as more advanced technologies ๐Ÿš€. We must divorce our architecture from dependence on any one of these ๐Ÿ’….

At the host interface, we must divorce the multiplexing of the medium from the form of data that the host sees. Today the packet is used both as a multiplexing and an interface element ๐Ÿ“ฆ. In the future, the host may see the network as a message-passing system, or as memory ๐Ÿง . At the same time, the network may use classic packets, wavelength division, or space division switching โœจ.

A number of basic functions must be rethought to provide an architecture that isn't dependent on the underlying switching model. For example, our transport protocols assume that data will be lost in units of a packet. If part of a packet is lost, we discard the whole thing ๐Ÿ—‘๏ธ. And if several packets are systematically lost in sequence, we may not recover effectively ๐Ÿ˜ฌ. There must be a host-level unit of error recovery that's independent of the network. This sort of abstraction must be applied to all the aspects of service specification: error recovery, flow control, addressing, and so on ๐Ÿซถ.


3.1.6. Network Operations, Monitoring, and Control ๐ŸŽ€

There's a hierarchy of progressively more effective and sophisticated techniques for network management that applies regardless of network bandwidth and application considerations ๐Ÿชœ:

  1. Reactive problem management
  2. Reactive resource management
  3. Proactive problem management
  4. Proactive resource management.

ngl, today's network management is mostly reactive, not proactive ๐Ÿ˜ฌ. Problem management kicks off only after users complain about service outages, and resource allocation decisions get made once people are already complaining about deteriorating quality of service. So today's network management systems are stuck at step 1 or maybe step 2 of the hierarchy ๐Ÿ’€.

Future network management systems, though, will provide proactive problem management ๐Ÿ”ฎ. That means diagnosing problems and restoring service before users even become aware something was wrong. They'll also provide proactive resource management, which is dynamic allocation of network bandwidth and switching resources so that an acceptable level of service is continuously maintained ๐Ÿ’œ.

The GN management system should be expected to provide proactive problem and resource management capabilities ๐Ÿ’…. It'll have to do that while contending with three important changes in the managed network environment:

  1. More complicated devices under management
  2. More diverse types of devices
  3. More variety of application protocols.

Performance under all of that is going to require us to seriously re-think how a network management system handles the expected high volumes of raw management-related data ๐Ÿ“Š. It'll become especially important for the system to provide thresholding, filtering, and alerting mechanisms that can save the human operator from drowning in data ๐ŸŒŠ, while still permitting access to details when diagnostic or fault isolation modes are invoked ๐Ÿ”.

The presence of expert assistant capabilities for early fault detection, diagnosis, and problem resolution will be mandatory ๐Ÿง . These capabilities are highly desirable today, but they'll be essential for handling the complexity and diversity of devices and applications in the Gigabit Network โœจ.

Besides dealing with complexity, automation is literally the only hope for controlling and reducing the high costs of daily management and operation of a GN ๐Ÿค–๐Ÿ’ธ.

Proactive resource management in GNs must be better understood and practiced ๐Ÿ“š, at first as an effort that requires human intervention and direction. Once that's achieved, it too must become automated to a high degree in the GN ๐Ÿซถ.


3.1.7. Naming and Addressing Strategies ๐Ÿท๏ธ

Current networks, both voice (telephone) and data, use addressing structures that tie the address super closely to the physical location on the network ๐Ÿ“. The address identifies a physical access point, rather than the higher-level entity (computer, process, human) attached to that access point ๐Ÿ™ƒ. In future networks, this physical aspect of addressing must be removed.

Think about finding the person you want on today's telephone network ๐Ÿ“ž. If they're not at their listed number, you might need a few preliminary calls where someone gives you advice about who to call next. That works fine when a human is placing the call, since humans can cope with arbitrary conversations ๐Ÿ™Œ. But if a computer is placing the call, getting the correct address has to be built into the architecture as a core network service ๐Ÿค–.

Since mobile hosts, hosts that are connected to multiple networks, and replicated hosts are totally reasonable to expect ๐Ÿงณ, the issue of mapping to the physical address must be properly resolved ๐Ÿ’ฏ.

So to let the network maintain the dynamic mapping to the current physical address ๐Ÿ”„, high-level entities need a name (or logical address) that identifies them independently of location. The network maintains the name and maps it to the current physical location as a core network service โœจ. For example, mobile hosts, multi-network hosts, and replicated hosts would have static names whose mapping to physical addresses (many-to-one, in some cases) changes with time ๐Ÿฆ‹.

Hosts aren't the only things whose physical location moves around ๐Ÿƒโ€โ™€๏ธ. Users' electronic mail addresses change. Within distributed systems, processes and files migrate from host to host. In a computing environment where robustness and survivability matter, entire applications may move about, or they may be redundant ๐Ÿงš.

This needed function has to be considered alongside the mobility and address resolution rates, especially if every address in a global data network worked this way ๐ŸŒ. The distributed network directory discussed elsewhere in this report should be designed to provide the necessary flexibility and responsiveness ๐Ÿ—‚๏ธ. The nature and administration of names must also be considered ๐Ÿ“.

Names that are arbitrary or unwieldy would be barely better than the addresses used now ๐Ÿ˜ฉ. The name space should be designed so that it can easily be partitioned among the agencies that will assign names. The structure of names should facilitate, rather than hinder, the mapping function ๐Ÿ™Œ. For example, it would be hard to optimize the mapping function if names were flat and unstructured ๐Ÿ’….


3.2. High-Speed Switching โšก

The term "high-speed switching" refers to changing the switching at a high rate ๐ŸŽ๏ธ, rather than switching high-speed links, because the latter isn't difficult at low speeds (consider, for example, manual switching of fiber connections, lol). The switching regime chosen for the network determines various aspects of its performance, its charging policies, and even its effective capabilities ๐ŸŽ€. As an example of the latter, it's difficult to expect a circuit-switched network to provide strong multicast support ๐Ÿ™ˆ.

A major debate is the choice between packet switching and circuit switching ๐ŸฅŠ. It's a key research issue for the GN, and that includes considering whether combinations of the two approaches are feasible ๐Ÿงฉ.

3.2.1. Unit of Management vs. Multiplexing ๐Ÿ“ฆ

With very high data rates, either the unit of management and switching must be larger or the speed of the processor elements for management and switching must be faster ๐Ÿƒโ€โ™€๏ธ. For example, at a gigabit, a 576 byte packet takes roughly 5 microseconds to be received ๐Ÿคฏ, so a packet switch must act extremely fast to avoid being the dominant delay in packet times. Moreover, storage time for the packet in a conventional store and forward implementation also becomes a significant component of the delay. So for packet switching to remain attractive in this environment, it appears necessary to increase the size of packets (or switch on packet groups), do so-called virtual cut-through, and use high-speed routing techniques such as high-speed route caches and source routing ๐Ÿ”ฅ.

Alternatively, for circuit switching to be attractive, it must provide very fast circuit setup and tear-down ๐ŸŽข to support the bursty nature of most computer communication. This problem is rendered difficult (and perhaps impossible for certain traffic loads) because the delay across the country is so large relative to the data rate ๐ŸŒŽ. That is, even with techniques like so-called fast select, bandwidth is reserved along the path by the circuit for almost twice the propagation time before being used โณ.

With gigabit circuit switching, since it isn't feasible to physically switch channels, the low-level switching is most likely doing FTDM on micro-packets, as is currently done in telephony ๐Ÿ“ž. Performing FTDM at gigabit data rates is a challenging research problem if the skew introduced by wide-area communication is to be handled with reasonable overhead for spacing these micro-packets ๐Ÿงฉ. Given the lead and resources of the telephone companies, this area of investigation should, if pursued, be pursued cooperatively ๐Ÿซถ.

3.2.2. Bandwidth Reservation Algorithms ๐ŸŽŸ๏ธ

Okay so some apps, like real-time video, need a sustained high data rate stream for a long time, like minutes if not hours ๐ŸŽฅ. Intuitively, it's giving "reserve it early." Pre-allocating the bandwidth they need would minimize the switching load on the network and guarantee the bandwidth is actually there ๐Ÿ’…. But more research is needed to figure out whether bandwidth reservation is worth it, especially alongside the different switching technologies ๐Ÿ”. There's some concern that reservation might need excessive intelligence in the network, which could reduce performance and reliability ๐Ÿ˜ฌ. Also, reservation opens a brand new denial of service option for an intruder or malicious user ๐Ÿ’€. So work here has to move together with switching technologies and capabilities, plus security and reliability requirements ๐Ÿ”.

3.2.3. Multicast Capabilities โœจ

It's pretty widely accepted that multicast should be offered as a user-level service, like RFC 1054 describes for IP ๐Ÿ“ก. Figuring out the best way to support it at the network layer and below still needs research ๐Ÿง. It's fairly clear the GN will be built from point-to-point fiber links that don't give you multicast/broadcast for free ๐Ÿงต. The most conservative option is no support at all, where each host or gateway fakes multicast by sending several individually addressed packets ๐Ÿ“ฆ๐Ÿ“ฆ๐Ÿ“ฆ. But very low-level multicast support has real perks, besides the obvious performance wins ๐Ÿš€. Multicast routing in a flooding form gives the most fault-tolerant, lowest-delay delivery, and if you save it for very high priority messages, it's a solid emergency facility for high-stress network apps ๐Ÿšจ. Multicast might also help defeat traffic analysis ๐Ÿ•ต๏ธ.

Another big one is the difference between open group multicast and closed group multicast ๐ŸŽ€. In the open kind, any host can multicast to the group. In the closed kind, only members can. Closed is easier to support and works for conferencing, for example. But for more client-server structured apps, like file/database servers, computation servers, and so on as groups, open multicast is required ๐Ÿ‘‘. Research is needed for both forms. Also, security issues come up when controlling who belongs to a multicast group, and this should be handled alongside work on secure forms of routing in general ๐Ÿ”’.

3.2.4. Gateway Technologies ๐Ÿšช

Once the GN wide-area interconnects local networks, gateways are expected to become a significant performance bottleneck unless gateway performance gets major advances ๐Ÿข. Plus, lots of network management concerns push more functionality, like access control, into the gateways, which adds to their load and their need for capacity ๐Ÿ“ˆ. That raises the trade-off between general-purpose hardware and special-purpose hardware โš–๏ธ.

On the general-purpose side, it may be feasible to use a general-purpose multiprocessor built on high-end microprocessors (maybe as exotic as the GaAs MIPS ๐Ÿคฏ). Pair that with a high-speed block transfer bus, like the one proposed as part of the FutureBus standard (which can be extended to higher speeds than currently commercially planned), plus intelligent high-speed network adaptors ๐Ÿง . This would also allow direct use of hardware, operating systems, and software tools developed in other DARPA programs, like Strategic Computing ๐Ÿงฐ. It also seems to make the gateway software more portable to commercial machines as they show up in that performance range ๐Ÿ“ฒ.

The specialized hardware approach assumes that general-purpose hardware, especially the interconnection bus, can't be fast enough for the level of performance required ๐Ÿ”Œ. The emphasis would be on various interconnection network techniques. These approaches seem to need greater expense, less commercial availability, and more specialized software ๐Ÿ’ธ. They need to be critically evaluated against the general-purpose gateway hardware approach, especially if the latter uses multiple buses for fault-tolerance as well as capacity extension when nothing has failed ๐Ÿ› ๏ธ.

The same general-purpose vs. special-purpose contention shows up with operating system software ๐ŸฅŠ. Traditionally, gateways run specialized run-time executives designed specifically for the gateway and its functions. But the growing sophistication of gateways makes that less feasible ๐Ÿ“‰. It's important to look into using a standard operating system foundation on gateways, one that's known to provide the required security and reliability properties (as well as real-time performance properties) โœ….

3.2.5. VLSI and Optronics Implementations ๐Ÿ’ก

It's pretty clear gigabit communication will use fiber optics for at least the near future ๐ŸŒˆ. Without major advances in optronics that allow effective optical computers, communication has to cross the optical-electronic boundary two or more times ๐Ÿ˜ต. There are big cost, performance, reliability, and security benefits to minimizing those crossings ๐Ÿ”. (As an example of a security benefit: optics isn't prone to electronic surveillance or jamming, but electronics clearly is, so swapping an optic-electronic-optic node for a pure optic node removes that vulnerability point ๐Ÿ™ˆ.)

The benefits of better optronics are so huge that applying them here is just another motivation for an already active research area that deserves strong continued support ๐Ÿ’œ. So we're focusing on matching current (and near-term expected) optronics capabilities with network requirements ๐ŸŽฏ.

The first and maybe greatest opportunity is to achieve totally (or largely) photonic switches in the network switching nodes. That means most packets get switched without crossing the optics-electronics boundary at all โœจ. For this to be feasible, the switch must use very simple switching logic, require very little storage, and operate on packets of a significant size. The source-routed packet switches with loopback on blockage of Blazenet illustrate the type of techniques that appear required to hit this goal ๐Ÿฆ‹.

Research is required to check whether optronic implementation of switches is feasible ๐Ÿ”ฌ. It appears highly likely that networks will at some point be totally photonically switched, with an impact on networking comparable to what integrated circuits did for processors and memories ๐Ÿคฉ.

A next level of focus is optical switching in the common case in gateways. One model is a multiprocessor with an optical interconnect. Packets tied to established paths through the gateway get optically switched and processed through the interconnect. Other packets get routed to the multiprocessor, crossing into the electronics domain ๐Ÿ”€. Research is required to marry the networking requirements and technology with optronics technology, pushing the state of the art in both areas along the way ๐Ÿš€.

Since the optic-electronic boundary is here for the long haul, improvements in this area matter too. But there's already enormous commercial research activity here, particularly within the telephone companies ๐Ÿ“ž. This is another area where collaborative investigation looks way better than a new independent research effort ๐Ÿซถ.

VLSI is an established technology with active research support ๐Ÿง‘โ€๐Ÿ”ฌ. The GN effort doesn't appear to need major new initiatives in VLSI, but we should stay open to significant novel opportunities that aren't identified here ๐Ÿ‘€.

3.2.6. High-Speed Transfer Protocols ๐ŸŽ๏ธ

To hit the desired speeds, we're going to have to rethink the form of protocols ๐Ÿง .

  1. The simple idea of a stateless gateway must be replaced by a more complex model, where the gateway understands the desired function of the end point and applies suitable optimizations to the flow.
  2. If multiplexing is done in the time domain, the elements of multiplexing are probably so small that no significant processing can be performed on each one individually. They must be processed as an aggregate. This implies that the unit of multiplexing is not the same as the unit of processing.
  3. The interfaces between the structural layers of the communication system must change from a simple command/response style to a richer system that includes indications and controls.
  4. An approach must be developed that couples the memory management in the host with the structure of the transmitted data, to allow efficient transfers into host memory.

The result of rethinking all this will be a new style of communications and protocols, with a much higher degree of shared responsibility among the components (hosts, switches, gateways) ๐Ÿซง. It may have little resemblance to previous work, either in the DARPA or commercial communities ๐Ÿฆ„.


3.3. High-Speed Host Interfaces ๐Ÿš€

As networks get faster, the most significant bottleneck will turn out to be the packet processing overhead in the host ๐ŸŒ. That doesn't restrict the aggregate rates we can hit over trunks, but it prevents high data rate flows from reaching host-based applications, which blocks new applications that need high bandwidth ๐Ÿ™…. The host bottleneck is thus a serious impediment to networked use of supercomputers ๐Ÿ–ฅ๏ธ.

To build a GN, we need new ways for hosts and their high bandwidth peripherals to connect to networks. We believe the most productive path is researching the most effective ways to isolate host and LAN development paths from the GN ๐Ÿงฉ. By decoupling those paths, neither one is restricted by the momentary performance or capability bottlenecks of the other ๐Ÿ’ž. The best way to view this separation is through the notion of a network front end (NFE) ๐Ÿงš. An NFE can take electronic input data at many data rates and transform it into gigabit light data, appropriately packetized to traverse the GN ๐Ÿ’ก. It can accept inputs from many types of gateways, hosts, host peripherals, and LANs, and provide arbitration and path set-up facilities as needed ๐Ÿšฆ. Most importantly, the NFE can perform protocol arbitration to retain upward compatibility with the existing Internet protocols while letting those sophisticated network input sources run GN-specific high-throughput protocols ๐ŸŒŸ. Of course, this means we need research into high-speed NFEs so the NFE doesn't become a bottleneck ๐Ÿ˜….

3.3.1. VLSI and Optronics Implementations ๐Ÿ”ฆ

In a host interface, unless the host is optical (an unlikely prospect in the near term), the opportunities for optronic support are limited ๐Ÿ™ƒ. With a serial-to-parallel conversion on reception stepping the clock rate down by a factor of 32 (assuming a 32-bit data path on the host interface), optronic speeds aren't required in the immediate future ๐ŸŽ‰.

One exception might be encryption ๐Ÿ” The VLSI chips doing standard encryption today hit around the 10 Mbit/s range, so optronic versions of these techniques, plus techniques built specifically to take advantage of optronic capabilities, look like a seriously promising area (and the payoff would be enormous if we pull it off) โœจ.

Targeted VLSI research here doesn't look like it'll go super far, for pretty much the same reasons we already covered with high-speed switching ๐Ÿ’…. The big wins will come from work that's already well-motivated by other research (like high-performance parallelism) and has strong commercial interest ๐ŸŽ€. Again, we need to stay open to imaginative opportunities nobody predicted here, while not getting distracted by low-impact research that doesn't come with further insights attached ๐Ÿ™ˆ

3.3.2. High-Performance Transport Protocols โœจ

Okay, current transport protocols have some seriously rough problems when you're chasing max performance, especially if you want hardware doing the heavy lifting ๐Ÿ˜ญ. TCP puts its checksum in the packet header, which means the whole packet has to be formed and read completely before transmission can even begin ๐Ÿ’€. ISO TP4 is even messier, because its checksum sits in a variable part of the header at an indeterminate offset, which makes hardware implementation super hard ๐Ÿ™ˆ.

The current Internet has thrived and grown because there are TCP implementations for a whole bunch of different kinds of host computers ๐ŸŒˆ. These implementations get robust interoperability through a "least common denominator" approach to features and options ๐Ÿซถ. Some applications have shown up on the current Internet (and we can picture analogs in the GN environment) that need qualities of service the ubiquitous generic TCP doesn't generally support, so people built special purpose transport protocols for them ๐Ÿฆ‹. Examples include UDP (user datagram protocol), RDP (reliable datagram protocol), LDP (loader/debugger protocol), NETBLT (high-speed block transfer protocol), NVP (network voice protocol) and PVP (packet video protocol) ๐ŸŽ€.

Meanwhile, work is underway on a new generic transport protocol called VMTP (versatile message transaction protocol). It's meant to remedy some of TCP's deficiencies, so apps won't have to resort to special purpose protocols for some stuff ๐Ÿ‘‘. We still need research to understand how transport level protocols should be constructed for a GN so they provide adequate qualities of service and stay easy to implement ๐Ÿงš.

A new transport protocol of reasonable success can be expected to last for ten years more ๐Ÿ“…. So a new protocol should not be over optimized for current networks, and it must not ignore the functional deficiencies of current protocols ๐Ÿ’…. Those deficiencies are essential to remedy before it's even feasible to deploy current distributed systems tech for military and commercial applications ๐Ÿ’€.

Forward Error Correction (FEC) is a useful approach when the bandwidth/delay ratio of the physical medium is high, which you'd expect on transcontinental photonic links โœจ. A degenerate (super simple) form of FEC is to just transmit multiple copies of the data. That lets you trade bandwidth for delay and reliability without needing much intelligence ๐Ÿง . In fact, it's generally true that reliability, bandwidth, and delay are interrelated, and an improvement in one generally comes at the expense of the others for a given technology ๐Ÿ’œ. Research is required to find appropriate operating points in networks using transmission components that offer extremely high bandwidth with very good bit-error-rate performance ๐Ÿ”ฅ.

3.3.3. Network Adaptors โœจ

With the promised speed of networks, the future network adaptor must be viewed as a memory interconnect, tying the memory in one host to the memory in another ๐Ÿ”—. That's true at least if we want to actually realize the data rate and low latency the network makes possible at the host-to-host or process-to-process level ๐ŸŽ๏ธ. The challenge is way too big to be met by just implementing protocols in custom VLSI ๐Ÿ˜ค.

Research is required to investigate the impact of network interconnection on machine architecture, and to define and evaluate new network adaptor architectures ๐Ÿ”ฌ. Of key importance is integrating the network adaptor into the operating system, so process-to-process communications performance matches what the network offers ๐Ÿ’–. In particular, we conjecture that the transport level will be implemented largely, if not entirely, in the network adaptor ๐Ÿคฏ. That gives the host reliable memory-to-memory transfer at memory speeds, with a minimum of interrupt processing, bus overhead and packet processing ๐ŸŒธ.

Drawing an analogy to RISC technology again, maximal performance requires a well-designed and coordinated protocol, software, and hardware (network adaptor) design ๐Ÿงš. Current standard protocols are significantly flawed for hardware compatibility, which suggests we need considerable further research on high-performance protocol design ๐Ÿ˜ญ.

3.3.4. Host Operating System Software โœจ

Traditionally, communication has been an add-on to an operating system ๐Ÿ™ƒ. With the GN, the network may well become the fastest "peripheral" connected to most nodes ๐Ÿš€. High-performance process-to-process (or application to application) communication won't be achieved until the operating system is well designed for fast access to and from the network ๐Ÿ’…. For example, incorporating templates of the network packet header directly in the process descriptor may let a process initiate communications with minimal overhead ๐ŸŽ€. Similarly, memory mapping can be used to eliminate copies between data arriving from the network and that data being delivered to the applications ๐Ÿชž. With a GN, an extra copy forced by the operating system may easily double the perceived transfer time for a packet between applications ๐Ÿ˜ฑ.

Besides matching data transfer mechanisms, operating systems must be well-matched in security design to what the host interface and network support as well ๐Ÿ”. Otherwise, all but the most trivial additional security actions by the operating system in common case communication can easily eliminate the performance benefits of the GN ๐Ÿ’€. For example, if the host has to do further encryption or decryption, throughput is likely to be at least halved and latency doubled ๐Ÿ˜ญ.

Research effort is required to further refine operating systems for the level of performance the GN offers ๐Ÿ”ฌ. This effort may well be best realized by coupling existing distributed systems efforts with the GN activities, rather than starting new separate efforts ๐Ÿซถ.


3.4. Advanced Network Management Algorithms โœจ

An important emphasis for research into network management should be on decentralized approaches ๐ŸŒ. The ratio of propagation delay across the country to data rates in a GN appears to be too great to deal effectively with resource management centrally when traffic load is bursty and unstable (and if it isn't, one might argue there's no problem ๐Ÿคท). Plus, important principles of fault containment and minimal privilege for reliability and security suggest that a centralized management approach is infeasible ๐Ÿšซ. In particular, compromising the security of one portion of the network should not compromise the security of the whole network ๐Ÿ”’. Similarly, a failure or fault should affect at most a local region of the network ๐Ÿงฑ.

The challenge is clearly to provide decentralized management techniques that lead to good global behavior in the normal case and acceptable behavior in expected worst-case failures, traffic variations and security intrusions ๐Ÿ‘‘.

3.4.1. Control Flow vs. Data Flow โœจ

Network operational communications can be separated into flow of user data and flow of management/control data ๐Ÿ“ฆ. However, user data must contain some amount of control data ๐Ÿค”. One question to explore, given changes in communications and computing costs and performance, is the trade-off between these two flows โš–๏ธ. An example of a potential approach is to use data units which contain predefined path indicators ๐Ÿฆ‹. The switch can perform a simple table look-up that maps the path indicator onto the preferred outbound link, then transmits the packet immediately โšก. There is a path set-up packet which fills in the appropriate tables ๐Ÿงพ. Path set-up occurs before the first data packet flows and then, while data is flowing, to improve the routes during the lifetime of the connection ๐Ÿ”„. This concept has been discussed in the Internet engineering group under the name of soft connections ๐Ÿ’œ.

We note that separating the data flow from the control flow in the GN has security and reliability advantages as well ๐Ÿ›ก๏ธ. We could encrypt most of the packet header to provide confidentiality within the GN and to limit the ability of intruders to perform traffic analysis ๐Ÿ•ต๏ธ. And, by separating the control flow, we can encrypt all the control exchanges between switches and the host front ends, thereby offering confidentiality and integrity ๐Ÿ”. No unauthorized entity will be able to alter or examine the control traffic ๐Ÿ™…. By employing a path set-up procedure, we can assure that the GN NFE-to-NFE path is functioning and also include user-specific requirements in the route ๐Ÿ›ฃ๏ธ. For example, we could request a certain bandwidth allocation and simplify the job of the switches in handling flow control ๐ŸŽ€. We could also set up backup paths in case the output link will be busy for so many microseconds that the packet cannot be stored until the link is freed โฑ๏ธ.

3.4.2. Resource Management Algorithms โœจ

Most current networks deliver one quality of service ๐Ÿ™‚. X.25 networks deliver a reliable byte-stream ๐Ÿ“ก. Most LANs deliver a best-effort unreliable service ๐Ÿคท. There are few networks today that can support multiple types of service and allocate their resources among them ๐Ÿ’…. Indeed, for many networks, such as best-effort unreliable service, there is little management of resources at all ๐Ÿ˜ถ. The next generation of network will require a much more controlled allocation of resources ๐Ÿ‘‘.

There will be a much wider range of desired types of service, with current services such as remote procedure call mixing with new services such as video streams ๐ŸŽฌ. Unless these are separately recognized and controlled, there is little reason to believe that effective service can be delivered unless the network is very lightly loaded ๐Ÿ“‰.

In order to support multiple types of service, two things must happen, both a change from current practice ๐Ÿ”. First, the application must describe to the network what type of service is required. Second, the network must use this information to make resource allocation decisions. Both of these practices present difficulties ๐Ÿ˜ฌ.

Past experience suggests that application code is not prepared to know or specify what service it needs ๐Ÿ™ˆ. By custom, operating systems provide a virtual world, and the applications in this world are unaware of the relation between this and the reality of time and space ๐ŸŒ. Resource requests must be in real terms ๐Ÿ“. Allocation of resources in the network is difficult, because it requires that decisions be made in the network, but as network packet throughput increases, there is less time for decisions โณ.

The second conflict gets resolved by noticing that decisions have to be made on bigger chunks than the packet, which is the unit of multiplexing ๐Ÿงฉ. That means the network has to see packets as part of a sequence, not the pure datagram model we've been using ๐Ÿฆ‹. Like I said earlier in this report, research is needed to make this fancier kind of switch work without wrecking robustness ๐Ÿ˜ญ.

To let the application ask for the service it actually needs, we'll need some abstraction of service class ๐Ÿ‘‘. If it's designed cleverly, the app can describe its needs really well โœจ. Take file transfer or mail: they have two modes, bulk data transfer and remote procedure call ๐Ÿ“ฆ. The app might not know ahead of time which mode it'll be in, but if it describes both, the system can adapt by watching what's actually happening ๐Ÿงš.

Figuring out a good service specification interface takes experimenting ๐Ÿงช. And this could happen inside the current protocols, so it could start right away ๐ŸŒธ.

3.4.3. Adaptive Protocols โœจ

Network conditions can change fast and over a huge range ๐ŸŽข. That's true of the current Internet, and the GN will probably deal with it too ๐Ÿ™ƒ. Protocols that can adapt to changing circumstances would give more even, robust service than we can get now ๐Ÿ’–. For example, when error rates go up, a protocol implementation might decide to use smaller packets, which cuts down on the burden from retransmissions ๐Ÿซถ.

A protocol's environment can be described by the service it gets from the next lower layer ๐Ÿ“ถ. An implementation can adapt to changes in that service by tuning its internal mechanisms, like time-outs and retransmission strategies ๐ŸŽ›๏ธ. So if we want adaptive protocols, we have to understand how the protocol layers and the mechanisms inside them interact ๐Ÿง . People have already done some work here ๐Ÿ‘€. The SATNET measurement task force, for instance, looked at how the protocols used by SIMP, IP, and TCP interact with each other ๐Ÿ”. What we really need is a more complete characterization of the interactions at the various layer boundaries, plus protocol designs and mechanisms that allow the necessary adaptations and renegotiations ๐Ÿ’….

3.4.4. Error Recovery Mechanisms โœจ

Being large and complex, GNs will hit all kinds of faults ๐Ÿ’€. There's link or nodal failure, excessive buffer overflow caused by faulty flow and congestion control, and partial failure of the switching fabric ๐Ÿ˜ต. These failures already happen in today's networks, but they'll hit harder in GNs ๐Ÿซ . A lot of data will be "stored" in transit there, and to speed up switching, nodes will do only minimal processing on the packets passing through ๐Ÿ“ฆ. In source routing, for example, a link failure can cause the loss of every packet sent until the source hears about the change in topology ๐Ÿ˜ฑ. The longer recovery from failures takes, the bigger the performance hit users feel ๐Ÿ™ˆ.

To cut down on the effects of failures, GNs will need error recovery mechanisms ๐Ÿ› ๏ธ. The network detects failures and error conditions, reconfigures itself for the new network state, and notifies peripheral devices about the new configuration ๐Ÿ“ฃ. These protocols still have to be developed ๐Ÿง‘โ€๐Ÿ’ป. They'll respond quickly, be decentralized or distributed so fatal failures are less likely ๐ŸŒˆ, and complement the error correction mechanisms of the end-to-end protocols instead of duplicating them ๐Ÿ’œ. The two have to work in a coordinated way ๐Ÿค. So peripheral devices will need to know about the intranet recovery mechanisms and keep interacting with them, so the effect on the connections they manage stays small ๐ŸŽ€.

3.4.5. Flow Control โœจ

As networks get faster, two related problems show up ๐ŸŽ๏ธ. First, existing flow control mechanisms like windows stop working well ๐ŸชŸ. The window would have to open so wide to reach the bandwidth we want that effective flow control just can't happen ๐Ÿ˜ฉ. Second, especially on long-haul networks, the number of bits in transit at one time gets so big that most computer messages will fit into one window ๐Ÿ“š. That means traditional congestion control schemes will stop working well too ๐Ÿ’€.

What's needed is a combo of two new approaches ๐Ÿ“. For small messages, open-loop controls on flow and congestion ๐Ÿ”. Most messages computers generate today will be small, since they'll fit into one round-trip time of future networks ๐Ÿš€. For longer messages, like voice or video streams, some explicit resource commitment will be required ๐ŸŽฅ.

3.4.6. Latency Control and Real-Time Operations โœจ

There are a few distinct approaches to latency control right now โฑ๏ธ. First, some networks are physically short, more like multiprocessor buses ๐Ÿ–ฅ๏ธ. Applications in these networks are built assuming delays will be short ๐Ÿค.

Second, there are networks where physical length isn't constrained by the design, and it can differ by orders of magnitude depending on the scope of the network ๐ŸŒ. Most general purpose networks fall into this category ๐Ÿงบ. In these networks, one of two things happens. Either the application takes special steps to deal with variable latency, like echo suppression in voice networks, or the applications just aren't supported ๐Ÿ™…โ€โ™€๏ธ.

For most applications today, latency isn't an obvious issue as long as the network isn't overloaded ๐Ÿ˜Œ. Overload leads to losses and long queues, but otherwise the protocol overhead masks the variation in network latency ๐ŸŽญ. This balance will change ๐Ÿ”„. Latency from the speed of light will stay the same, but overhead will drop (it has to, if we want high performance) ๐Ÿš„. That leaves speed of light and queueing as the most critical sources of delay ๐ŸŒŸ.

This leads to a big conclusion: if queueing delay can be controlled, we can build networks with stable, controlled latency ๐ŸŽฏ. If applications need this class of service, it can be supported ๐Ÿ’–. Either the network has to be underloaded so queues don't develop at all, or a specific class of service has to be supported, where resources get allocated to stabilize the delay ๐Ÿง˜โ€โ™€๏ธ.

Even if this service is provided, applications will still see delays that vary by several orders of magnitude, depending on the physical size of the network ๐Ÿ“. Research at the application level will be needed to see how apps can be designed to cope with that variation ๐Ÿง .

3.4.7. High-Speed Internetworking and Administrational Domains โœจ

Internetworking figured out that communication services get more valuable with wider interconnection ๐Ÿ“ˆ. But it ignored management and the role of administrations ๐Ÿ™ƒ. So here's what we're seeing:

  1. The Internet is more or less unmanageable ๐Ÿ˜ฌ. You can tell from its performance, reliability, and security problems ๐Ÿซ .
  2. The Internet is being stressed by administrators building networks to match their organization instead of the geography ๐Ÿข. For example, a set of Ethernets at different company locations operates as a single Internet network, but the locations are geographically dispersed and connected by satellite or leased lines ๐Ÿ›ฐ๏ธ.

The next generation of internetworking has to focus on administration and management ๐Ÿ‘‘. Internetworking has to support cohesion inside an administration and a healthy separation between administrations ๐Ÿซง. Think of it like this: the American and Soviet embassies in Mexico City are geographically closer to each other than to their home countries, but further apart in administrational distance, including security, accounting, and so on ๐ŸŒฎ. The emerging revolution in WANs makes this even more critical ๐Ÿšจ. The amount of communication needed to exchange the state of systems is bound to increase enormously ๐Ÿ“ˆ. The potential cost of failures and security violations is frightening ๐Ÿ˜ฑ.

A promising approach seems to be high-level gateways that guard between administrations and require negotiations to set up access paths between them ๐Ÿšช. These paths get set up and labeled with agreements on authorization, security, accounting, and possible resource limits ๐Ÿท๏ธ. These administrative virtual circuits provide transparency to the physical and geographical interconnection, but they don't need to support more than datagram packet delivery ๐Ÿ“ฌ. One view is communication contracts, with high-level gateways acting as contract monitors at each end ๐Ÿ“œ. The key is focusing on controlled interadministrational connectivity, not the conventional protocol concerns ๐ŸŽฏ.

We need to focus on developing a (inter)network management architecture and the specifics of high-level gateways ๐Ÿ—๏ธ. These gateways will have to take advantage of advances in multi-processor architectures to handle the processing load ๐Ÿง . A key issue is being able to optimize communication between administrations once the contract is in place, but without losing control ๐ŸŽ›๏ธ. Related to that is allowing high-speed interconnection within a single administration, even if it's geographically dispersed ๐ŸŒ. Another issue is fault-tolerance. High-level gateways contain state information, and losing it typically disrupts communication. How does one minimize this problem? ๐Ÿค”

A key goal of these administrational gateways is failure containment ๐Ÿ›ก๏ธ. That means protecting against external problems (from outside the administration) and stopping local problems from putting liability on others ๐Ÿ™…. A particular worry is the self-organizing problems of large-scale systems that Van Jacobson observed in the Internet ๐ŸŒ€. Gateways have to damp out oscillations and control wide load swings ๐ŸŒŠ. Rate control looks like a key area to investigate, both as a basis for buffer management and congestion control and to control offered load ๐ŸŽš๏ธ.

Given the speed of new networks and the sophistication of the gateways described above, another key area to investigate is high-speed network interface adaptors ๐Ÿ”Œ.

3.4.8. Policy-Based Algorithms โœจ

Networks today usually pick routes by minimizing some measure, like delay ๐Ÿ—บ๏ธ. In the real world, though, route selection will often be constrained at the global level by policy issues, like access rights to resources and accounting and billing for usage ๐Ÿ’ธ.

It's hard for connectionless protocols like Internet to deal with policy controls ๐Ÿ˜ค. Without state in the gateway, a separate policy decision has to be made for each packet in isolation ๐Ÿ“ฆ. As networks get faster, the cost of that processing will be intolerable ๐Ÿฅต. One possible approach, discussed above, is a more sophisticated model where gateways know about the ongoing flows ๐ŸŒŠ. Another option is designing gateways that simply cache recent policy evaluations and apply them to successive packets ๐Ÿง.

Okay so routing based on policy is lowkey the hardest thing ever ๐Ÿ˜ญ๐Ÿซถ because a route only works if it's globally consistent, otherwise it can loop ๐ŸŒ€. That means every single policy decision has to be propagated globally ๐Ÿ“ฃ. And since there can be expected to be a large number of policies, all that global passing of info could easily turn into an information explosion ๐Ÿ’ฅ๐Ÿ’€.

There are at least two solutions โœจ. One is to restrict the possible classes of policy. The other is to use some form of source route, so the route consistent with some set of policies gets computed at one point only and then attached to the packet ๐ŸŽ€. Both of these have problems, no cap ๐Ÿ™ˆ. So we need a two-pronged research program, where mechanisms get proposed and, at the same time, the needed policies get defined ๐Ÿ”ฌ๐Ÿ’–.

The same trade-off shows up for accounting and billing ๐Ÿงพ. A single accounting metric, like "bytes times distance", could be proposed. That might be somewhat simple to implement, but it wouldn't permit individual billing policies, the way it's already done in parts of the telephone system ๐Ÿ“ž. The current connectionless transport architectures, like TCP/IP or the connectionless ISO configuration using TP4, don't have good tools for accounting for traffic or for restricting traffic from certain resources ๐Ÿ˜ค. Building these tools is hard in a connectionless environment, because an accounting or control facility has to deal with each packet in isolation, which means a significant processing burden as part of packet forwarding ๐Ÿฅต. That burden becomes an increasing problem as switches are expected to operate faster ๐ŸŽ๏ธ๐Ÿ’จ.

The lack of these tools is proving a significant problem for network design ๐Ÿšง. Accounting and control aren't only needed to support management requirements. They're also needed as a building block to support enforcement of things like multiple qualities of service, as discussed above ๐ŸŽ€.

Network accounting is generally considered to be simply a step that leads to billing, so it's often judged by how simple or difficult it'll be to implement ๐Ÿคทโ€โ™€๏ธ. But an accounting and billing procedure is really a mechanism for implementing a policy that's considered desirable for reasons beyond the scope of accounting per se ๐Ÿ‘‘. For example, a policy might be set up to either encourage or discourage network use while fully recovering operational cost ๐Ÿ’…. A policy of encouraging use could be implemented with a relatively high monthly attachment charge and a relatively low per-packet charge ๐ŸŒธ. A policy of discouraging use could be implemented with a low monthly charge and a high per-packet charge ๐Ÿ™…โ€โ™€๏ธ.

Network administrators have a relatively small number of variables to work with when it comes to policy objectives ๐ŸŽ›๏ธ. Still, these variables can be combined in a bunch of innovative ways โœจ. Some of the possibilities include:

  1. Classes of users (e.g., large or small institutions, for-profit or non-profit).
  2. Classes of service.
  3. Time varying (e.g., peak and off-peak).
  4. Volume (e.g., volume discounts, or volume surcharges).
  5. Access charges (e.g., per port, or port * [bandwidth of port]).
  6. Distance (e.g., circuit-miles, airline miles, number of hops).

Generally, an accounting procedure can be developed to support voluntary user cooperation with almost any single policy objective ๐Ÿฅฐ. Difficulties most often pop up when there are multiple competing policy objectives, or when there's no clear policy at all ๐Ÿ˜ฌ.

Another thing about accounting and billing procedures that needs careful thought is the cost of accumulating and processing the data that billing is based on ๐Ÿ“Š. Collecting detailed data on a per-packet basis is of particular concern ๐Ÿ“ฆ. As network circuit data rates increase, the number of instructions that must be executed on a per-packet basis can become the limiting factor in system throughput ๐Ÿ‹๏ธ. So it may be appropriate to prefer accounting and billing policies and procedures that minimize the difficulty of collecting data, even if this approach requires a compromise of other objectives ๐Ÿ’œ. Similarly, node memory required for data collection, and any network bandwidth required to transmit the data to administrative headquarters, are factors that have to be traded off against the need to process user packets ๐Ÿง ๐Ÿซถ.


3.4.9. Priority and Preemption ๐Ÿšฆ

The GN should support multiple levels of priority for traffic and the preemption of network resources for higher priority use. Network control traffic should be given the highest priority, so it can pass through the network unimpeded by congestion caused by user-level traffic ๐Ÿ›ธ. There may be additional military uses for multiple levels of priority, corresponding to the rank or level of importance of a user or the mission criticality of some particular data ๐ŸŽ–๏ธ.

The use and existence of priority levels may be different for different types of traffic ๐ŸŒˆ. For example, datagram traffic may not have multiple priority levels. Because the network's transmission speed is so high and traffic bursts may be short, it may not make sense to do any processing in the switches to deal with different priority levels ๐Ÿ™ƒ. Priority will be more important for flow- (or soft-connection-) oriented data or hard connections, in terms of permitting higher priority connections to be set up ahead of lower priority connections. Preemption will permit requests for high priority connections to reclaim network resources currently in use by lower priority traffic ๐Ÿ‘‘.

Networks such as the Wideband Satellite Network, which supports datagram and stream traffic, implement four priority levels for traffic. The highest is reserved for network control functions and the other three are for user traffic ๐Ÿ›ฐ๏ธ. The Wideband Network supports preemption of lower priority stream allocations by higher priority requests. An important part of using priority and preemption is the ability to notify users when requests for service have been denied, or when allocations have been modified or disrupted ๐Ÿ“ฌ. Such mechanisms have been implemented in the Wideband Network for streams and dynamic multicast groups โœ….

Priority and preemption mechanisms for a GN will have to be implemented in an extremely simple way so that they can take effect very quickly โšก. It is likely that they will have to be built into the hardware of the switch fabric ๐Ÿ”ฉ.


3.5. User and Network Services ๐ŸŒ

As discussed in Section 2 above, there will need to be certain services provided as part of network operation to the users (people) themselves and to the machines that connect to the network ๐Ÿค–. These services include capabilities such as white and yellow pages, which let users determine what the appropriate network identification is for other users and for network-available computing resources, and distributed fault identification and isolation ๐Ÿ”. They are needed in current networks and will continue to be required in the networks of the future. The speed of the GN will serve to accentuate this requirement, but at the same time will allow for new architectures to be put in place for such services ๐Ÿงš. For example, Ethernet speeds in the local environment have allowed for more usable services to be provided ๐Ÿ’œ.

3.5.1. Impact of High Bandwidth ๐Ÿ“š

One issue that will need to be addressed is the impact on the user of such high-bandwidth capabilities ๐Ÿ˜ตโ€๐Ÿ’ซ. Users are already becoming saturated by information in the modern information-rich environment. (Many of us receive more than 50 electronic mail messages each day, each requiring some degree of human attention.) Methods will be needed to help users cope with this ever-expanding access to data, or we'll run the risk of users turning back to the relative peace and quiet of the isolated office ๐Ÿข๐ŸŒฟ.

3.5.2. Distributed Network Directory ๐Ÿ“‡

A distributed network directory can support the user-level directory services and the lower-level name-to-address mapping services described elsewhere in this report. It can also support distributed systems and network management facilities by storing additional information about named objects. For example, the network directory might store node configurations or security levels ๐Ÿ—‚๏ธ.

Distributing the directory eases and decentralizes the administrative burdens and provides a more robust and survivable implementation ๐Ÿ›ก๏ธ.

One approach toward implementing a distributed network directory would be to base it upon the CCITT X.500/ISO DIS 9594 standard. This avoids starting from ground zero and has the advantage of facilitating interoperability with other communications networks. However, research and development will be required even if this path is chosen ๐Ÿงช.

One area in which research and development are required is the services supplied by the distributed network directory. The X.500 standard is very general and powerful, but so far specific provisions have only been made for storing information about network users and applications. As mentioned elsewhere, multilevel security is not addressed by X.500, and the approach taken toward authentication must be carefully considered in view of DoD requirements ๐Ÿ”. Also, X.500 assumes that administration of the directory will be done locally and without the need for standardization. This may not be true of GN or the larger national research network ๐Ÿค”.

The model and algorithms used by a distributed network directory are a second area of research. The model specified by X.500 must be extended into a framework that provides the necessary flexibility in terms of services, responsiveness, data management policies, and protocol layer utilization. Furthermore, the internal algorithms and mechanisms of X.500 must be extended in a number of areas. For example, they need to support redundancy of the X.500 database, internal consistency checking, fuller sharing of information about the distribution of data, and defined access-control mechanisms ๐Ÿงฉ๐Ÿ’–.


4. Avenues of Approach ๐Ÿ›ฃ๏ธ

Ongoing research and commercial activities provide an opportunity for more rapidly attacking some of the above research issues ๐Ÿƒโ€โ™€๏ธ. At the same time, there needs to be attention paid to the overall technical approach, so that multiple potential solutions can be explored and issues can be attacked in parallel ๐ŸŒŸ.

4.1. Small Prototype vs. Nationwide Network ๐Ÿš€

The central question is how far to jump, and how far the current approaches can get. That is, how far will connectionless network service get us, how far will packet switching get us, and how far do we want to go? If our goal is a Gbit/s net, then that is what we should build. Building a 100 Mbit/s network to achieve a GN is analogous to climbing a tree to get to the moon. It may get you closer, but it will never get you there ๐ŸŒ™๐Ÿ˜‚.

There are currently some network designs which can serve as the basis for a GN prototype. The next step is some work by experts in photonics and possibly high-speed electronics to explore ease of implementation ๐Ÿ”ฆ. Developing a prototype 3-5 node network at a Gbit/s data rate is realistic at this point, and would demonstrate wide-area (40 km or more) Gbit/s networking ๐ŸŽ€โœจ.

DARPA should consider installing a Gbit/s cross-country set of connected links, kinda like the NSF backbone, within 2 years โœจ. A Gbit/s link between the east and west coasts would unlock a whole new generation of (C3I), distributed computing, and parallel computing research possibilities ๐Ÿฆ‹, and it would reestablish DARPA as the premier network research funding agency in the country ๐Ÿ‘‘. This will require getting "dark" fiber from one or more of the common carriers and some collaboration with these organizations on repeaters, etc. With that collaboration, the time to a commercial network in the Gbit/s range would be substantially reduced, and the resulting nationwide GN would give the United States an enormous technical and economic advantage over countries without it ๐Ÿ’….

Showing off one fast WAN is not the whole story, though ๐Ÿ™„. We still need to study and experiment with how to connect high-bandwidth networks into a high-bandwidth internet, and the research issues above are a big part of why (iykyk ๐Ÿซถ). These experiments can run on a new generation of internet even if that means starting at lower speeds (e.g., T1 through 100 Mbit/s) ๐Ÿฃ. Just make sure the stuff you demo still works when Gbit/s shows up, ngl ๐Ÿ’œ.

4.2. Need for Parallel Efforts/Approaches โœจ

So parallel efforts are required, and there are two big reasons ๐Ÿ’–. Reason one: we need to chase different approaches (e.g., different strategies for high-bandwidth switching, different addressing techniques, etc). That's true for most research programs, but prototyping is pricey ๐Ÿ’ธ, so the decisions about which efforts get prototyped need proper review ๐Ÿง.

Reason two ๐ŸŒธ: all the different parts of the program have to happen at the same time. We literally can't wait for the high-bandwidth network to exist before we start prototyping the high-bandwidth internet ๐Ÿ˜ญ. So a phased and evolutionary approach is the move ๐Ÿฆ‹.

4.3. Collaboration with Common Carriers โœจ

Today's US computer networks basically ignore the STN (the Switched Telephone Network) ๐Ÿ™„, except they buy raw bandwidth through it. But network performance glow-ups come from better underlying media ๐Ÿ’ซ, like satellite communication, fiber optics, and photonic switching โœจ.

In the past it was "their" transmission under "our" switching ๐Ÿ™ƒ. The alternative is to use the common-carrier switching capabilities as an integral part of the networking architecture ๐Ÿงฉ. We need an objective scientific and economic look, and we need to reevaluate this question ๐Ÿ”.

Another place to team up with the carriers is network addressing ๐Ÿ“ž. Their addressing scheme ("numbering plan") has a few advantages, like proven service to 300 million users [4] ๐Ÿ’ฏ.

On the flip side ๐Ÿ™ƒ, the carriers have way fewer administrative domains (area codes) than the current plethora of locally administered local area networks in the internet system ๐ŸŒˆ.

Future networks will eventually be run by commercial communications providers, and that's pretty likely ๐Ÿข. To get the research into the marketplace, involve the potential carriers from the start ๐Ÿš€. Still, it's not clear that commercial providers (who have mostly cared about meeting the needs of 90+ percent of the user base) will vibe with the goals of this research ๐Ÿ˜ฌ. So we recommend an appropriate amalgam of academia and industry, with special attention to the potential system developers and operators ๐Ÿซถ, but we also caution that the specific and unique goals of the DARPA program must be retained ๐Ÿ”’.

The DARPA program's specific and unique goals are the whole point, so don't let them get lost in the collab shuffle ๐Ÿ‘€.

4.4. Technology Transfer โœจ

Like we said above ๐Ÿ”, we believe future networks will ultimately be managed and operated by commercial communications providers. (Heads up: that might not be the common carriers as we know them today ๐Ÿค”. It could be value-added networks that use common carrier facilities.) So to make sure the tech actually transfers, we believe you involve the potential system developers from the start ๐Ÿงš. That's why the research program would benefit from an appropriate amalgam of university and industry, with close involvement of the potential system developers and operators ๐Ÿ’œ.

4.5. Standards โœจ

The Internet program was a total slay ๐Ÿ‘‘ at influencing national and international standards. Sure, the protocols changed a bit ๐Ÿ™ƒ, but the technology and approaches CCITT and ISO used for standardizing packet-switched networks clearly had roots in the DARPA internet ๐ŸŒฑ. Still, not all tea is sweet ๐Ÿต. It's kinda hurt the research program, since the evolving standards led to pressure to adopt them in the research environment ๐Ÿ˜ค.

So yeah, it's a "catch-22" ๐Ÿ™ƒ๐ŸŽ€. We want the technology base from the research program to have maximal impact on standards activities ๐ŸŽฏ, and doing research in the context of the standards environment speeds that up โšก. But standards, by their very nature, will always lag behind the research environment ๐Ÿข.

So the only reasonable move ๐Ÿ’… seems to be an occasional "checkpointing" of the research environment ๐Ÿ“ธ, where the required conversions happen so a new plateau of standards can be used for future evolution and research. A good example is doing future mail research using X.400 and X.500 where possible ๐Ÿ“ฌ.

5. Conclusions โœจ

We really hope this doc gave you a useful compendium ๐Ÿ“š of the research issues critical to achieving the FCCSET phase III recommendations ๐ŸŽ€. These problems interact in a complex way ๐ŸŒ€. If the only goal of a new network architecture was high speed, reasonable solutions wouldn't be hard to propose ๐ŸŽ๏ธ. But if you need higher speeds while supporting multiple services, AND supporting those services across administrative boundaries so that policy concerns (e.g., access control) have to be enforced ๐Ÿ”, the interactions get complicated fr fr ๐Ÿ’€.

APPENDIX ๐Ÿงš

A. Current R and D Activities โœจ

This appendix is a little tour ๐Ÿ—บ๏ธ of ongoing R&D activities the group knew about that are relevant to achieving the GN. Sometimes there's a short abstract too. The order is random (no hidden rankings, pinky promise ๐Ÿคž), and the amount of detail doesn't say anything about significance ๐Ÿ’…. We hope these pointers help anyone who wants to chase the research issues in this report ๐Ÿซถ.

  1. Grumman (at Bethpage) ๐Ÿ›ฉ๏ธ has a three-year DARPA contract, started in January 1988, to build a 1.6 Gbit/s LAN for use on a plane or ship, or as a "building block" ๐Ÿงฑ. It's honestly raw transport capacity running on two fibers in a token-ring like mode ๐Ÿ’ซ. First milestone (after one year?) is a 100 Mbit/s demo.
  2. BBN Laboratories ๐Ÿง , as part of its current three-year DARPA Network-Oriented Systems contract, proposed design concepts for a 10-100 Gbit/s wide area network ๐ŸŒ. The work will include wavelength division multiplexing, photonic switching, self-routing packets, and protocol design โœจ.
  3. Cheriton (Stanford) is researching Blazenet ๐Ÿ”ฆ, a high-bandwidth network using photonic switching.
  4. Acampora (Bell Labs) is researching how to use wavelength division multiplexing to build a shared optical network ๐ŸŒˆ.
  5. Yeh is researching a VLSI approach to building a high-bandwidth parallel processing packet switch ๐Ÿงฉ.
  6. Bell Labs is working on a Metropolitan Area Network called "Manhattan Street Net" ๐Ÿ™๏ธ. Under Dr. Maxemchuck, this work is similar to Blazenet ๐Ÿ”. It's in the prototype stage for a small number of street intersections, and ultimately it's meant to be city-wide ๐ŸŒ†. Like Blazenet, it uses photonic switching 2 x 2 lithium niobate block switches ๐Ÿ’Ž.
  7. Ultra Network Technologies, a Silicon Valley company ๐Ÿ’ป, has a (prototype) Gbit/s fiber link that connects backplanes. This is based on the ISO-TP4 transport protocol ๐Ÿซง.
  8. Jonathan Turner at Washington University is working on a Batcher-Banyan Multicast Net ๐Ÿ•ธ๏ธ, based on the "SONET" concept, which provides 150 Mbit/s per pipe ๐Ÿšฟ.
  9. David Sincowskie at Bellcore is working with Batcher-Banyan design and has working 32x32 switches ๐ŸŽ‰.
  10. Stratacom has a commercial product that's really a T1 voice switch implemented internally by a packet switch ๐Ÿ“ฆ, where the packet is 192 bits (T1 frame). This switch can pass 10,000 packets per second ๐Ÿƒ.
  11. Stanford NAB gets 30-50 Mbit/s throughput on a 100 Mbit/s connection using Versatile Message Transaction Protocol (VMTP) [see RFC 1045] ๐Ÿ”—.
  12. The December issue of IEEE Journal on Selected Areas in Communications has tons of detail on interconnects ๐Ÿ“–.
  13. Ultranet Technology has a 480 Mbit/s connection using modified ISO TP4 ๐Ÿš„.
  14. At MIT, Dave Clark has an architecture proposal worth a look ๐Ÿ‘€.
  15. At CMU, Eric Cooper's work is relevant ๐ŸŽ“.
  16. At Protocol Engines, Inc., Greg Chesson is working on an XTP-based system ๐Ÿ› ๏ธ.
  17. Larry Landweber at Wisconsin University is doing relevant work ๐ŸŒป.
  18. Honeywell is doing relevant work for NASA ๐Ÿš€.
  19. Kung at CMU is working on a system called "Nectar" ๐Ÿงƒ, based on a STARLAN on fiber connecting dissimilar processors.
  20. Burroughs (now Unisys) has some relevant work in the IEEE 802.6 committee ๐Ÿ—‚๏ธ.
  21. Bellcore's work on "Switched Multimedia Datanet Service" (SMDS) is relevant (see the paper Dave Clark supplied) ๐Ÿ“„.
  22. FDDI-2, a scheme for making TDMA channel allocations at 200 Mbit/s ๐ŸŽš๏ธ.
  23. NRI's Kahn-Farber Proposal to NSF is a paper design for a high-bandwidth network ๐Ÿ“.
  24. Barry Goldstein's work at IBM-Yorktown ๐Ÿค“.
  25. Bell Labs S-Net, a 1280 Mbit/s prototype ๐Ÿ”ง.
  26. Fiber-LAN, owned by Bell South and SECOR, is a pre-prototype 575 Mbit/s Metro Area Net ๐ŸŒ‡.
  27. Bellcore has a chip implementation of FASTNET (1.2 Gbit/s) ๐Ÿ”Œ.
  28. Scientific Computer Systems in San Diego has a 1.4 Gbit/s prototype ๐ŸŒŠ.
  29. BBN Monarch Switch is a Space Division pre-prototype with chips being fabricated, 64 Mbit/s per path ๐Ÿ›ธ.
  30. Proteon has an 80 Mbit/s token ring ๐Ÿ’.
  31. Toronto University has a 150 Mbit/s "tree" ๐ŸŒณ, which is really a LAN.
  32. NSC Hyperchannel II is reputedly available at 250 Mbit/s ๐Ÿคซ.
  33. Tobagi at Stanford is working on EXPRESSNET, which is not commercially available ๐Ÿ”ฌ.
  34. Columbia MAGNET, 150 Mbit/s ๐Ÿงฒ.
  35. Versatile Message Transaction Protocol (VMTP) ๐Ÿ“จ.
  36. ST integrated with IP ๐Ÿงฉ.
  37. XTP (Chesson) ๐Ÿ’ซ.
  38. Stanford Transport Gateway ๐Ÿšช.
  39. X.25/X.75 ๐Ÿ“ก.
  40. Work of the Internet Activities Board ๐Ÿซถ.

B. Gigabit Working Group Members ๐Ÿงš

okay bestie, this is the cast list for the Gigabit Working Group ๐ŸŽ€ Here's who was in the room, and where each of them worked at the time. Barry Leiner is marked as the Chairman ๐Ÿ‘‘

Member                  Affiliation

Gordon Bell             Ardent Computers
Steve Blumenthal        BBN Laboratories
Vint Cerf               Corporation for National Research Initiatives
David Cheriton          Stanford University
David Clark             Massachusetts Institute of Technology
Barry Leiner (Chairman) Research Institute for Advanced Computer Science
Robert Lyons            Defense Communication Agency
Richard Metzger         Rome Air Development Center
David Mills             University of Delaware
Kevin Mills             National Bureau of Standards
Chris Perry             MITRE
Jon Postel              USC Information Sciences Institute
Nachum Shacham          SRI International
Fouad Tobagi            Stanford University

End Notes ๐Ÿ“

These are the footnotes, and they're lowkey where the receipts are ๐Ÿงพ

[1] The venue was the Workshop on Computer Networks, held 17-19 February 1987, in San Diego, CA. ๐Ÿ’…

[2] The source is "A Report to the Congress on Computer Networks to Support Research in the United States: A Study of Critical Problems and Future Options", published by the White House Office of Scientific and Technical Policy (OSTP) in November 1987. ๐Ÿ“š

[3] The report makes a distinction here. One piece is the development of a backbone network that gives gigabit capacity, which it calls the GB. The other piece is an interconnected group of high-speed networks that deliver high-bandwidth service to users, which it calls the Gigabit Network (GN). It's giving two different things that get mixed up a lot ๐Ÿซถ

[4] This one is a whole mood ๐Ÿ˜ญ The note points out that the telephone system already serves 150 million subscribers with an 11-digit address space, which is about a 1:600 ratio. Meanwhile, the address space described here is 9.6 digits, and it's already running into trouble with well under 100,000 users, which is less than a 1:30,000 ratio. That's the gap the Gigabit Working Group was worried about ๐Ÿ’€โœจ

๐Ÿ“ฆ where this page comes from