RFC 5693
Application-Layer Traffic Optimization (ALTO) Problem Statement
RFC 5693: Application-Layer Traffic Optimization (ALTO) Problem Statement 💖
Okay bestie, this is the RFC that kicks off the whole ALTO story 🦋 Here's the title block tea ☕
- Stream: Network Working Group
- Number: RFC 5693
- Category: Informational (this is a "here's some info" doc, not a standard 💅)
- Date: October 2009
- Authors: J. Seedorf (NEC) and E. Burger (Neustar Inc.) 👑
- Obsoletes / Updates: the title block doesn't list anything it obsoletes or updates 🌸
Abstract
Okay so, distributed apps like file sharing, real-time communication, and live and on-demand media streaming are literally everywhere 🌈 and they eat a lot of network resources ngl. They usually move big chunks of data over connections between nodes spread all over the Internet, and they kinda don't know much about the underlying network topology 🙈
Some apps are designed to pick a random subset of peers from a bigger set to swap data with. And if nothing is guiding those picks, or they're only working with suboptimal or local info from measurements and statistics, they often make less-than-great choices 💀
This document talks about an information-sharing service that lets apps do better-than-random peer selection ✨ That's the whole mission, bestie.
Status of This Memo
This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited 🫶
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved 💜
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License 📜
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2. State-of-the-Art . . . . . . . . . . . . . . . . . . . . . 4
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. The Problem . . . . . . . . . . . . . . . . . . . . . . . . . 7
4. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.1. File sharing . . . . . . . . . . . . . . . . . . . . . . . 8
4.2. Cache/Mirror Selection . . . . . . . . . . . . . . . . . . 8
4.3. Live Media Streaming . . . . . . . . . . . . . . . . . . . 8
4.4. Real-Time Communications . . . . . . . . . . . . . . . . . 9
4.5. Distributed Hash Tables . . . . . . . . . . . . . . . . . 9
5. Aspects of the Problem . . . . . . . . . . . . . . . . . . . . 9
5.1. Information Provided by an ALTO Service . . . . . . . . . 9
5.2. ALTO Service Providers . . . . . . . . . . . . . . . . . . 10
5.3. ALTO Service Implementation . . . . . . . . . . . . . . . 10
5.4. User Privacy . . . . . . . . . . . . . . . . . . . . . . . 10
5.5. Topology Hiding . . . . . . . . . . . . . . . . . . . . . 11
5.6. Coexistence with Caching . . . . . . . . . . . . . . . . . 11
6. Security Considerations . . . . . . . . . . . . . . . . . . . 11
7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 12
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 12
9. Informative References . . . . . . . . . . . . . . . . . . . . 131. Introduction ✨
1.1. Overview
Okay so distributed apps, both peer-to-peer (P2P) and client/server, are used for file sharing, real-time communication, and live and on-demand media streaming 🎬 and they use a significant amount of network capacity and CPU cycles in the routers [WWW.wired.fuel]. It's giving "a lot." 😭
Here's the contrast: centralized apps are one thing, but distributed apps reach for resources like files or media relays spread across the Internet, and they swap large amounts of data over connections they set up directly with the nodes that have the stuff 🧚
One big perk of highly distributed systems is that the resources they offer are often available through multiple replicas 💅 BUT, and this is the catch, apps generally don't have reliable info about the underlying network. So they have to pick among the peers holding those replicas either randomly, or based on guesses from partial observations. Sometimes that leads to suboptimal choices 🙈 For example, one peer-selection algorithm only looks at measurements taken during initial connection establishment between two peers. Since no actual data transmission has started yet, that algorithm only measures round-trip time and can't reliably figure out the real throughput between the peers. So a peer-selection algorithm that just uses round-trip time may end up picking a suboptimal set of peers 💀
A lot of today's P2P systems run on an overlay network made of direct peer connections. Those connections often don't account for the underlying network topology 🕸️ Not only can that mean suboptimal performance, but these networks can also cause congestion and some serious inefficiencies. As shown in [ACM.fear], traffic from popular P2P apps often crosses network boundaries multiple times, overloading links that are frequently subject to congestion [ACM.bottleneck]. And those transits aren't just a bad experience for the user, they can also be quite costly for the network operator 💸
Recent studies ([ACM.ispp2p], [WWW.p4p.overview], [ACM.ono]) point to a possible solution 🌟 Internet Service Providers (ISPs), network operators, or third parties can collect more reliable network information. This info includes things like topology or link capacity. Normally, that kind of info changes on a much longer time scale than the info used for congestion control on the transport layer. Giving this information to P2P apps can let them do better-than-random peer selection with respect to the underlying network topology 🗺️ As a result, it may be possible to increase application performance, reduce congestion, and decrease the overall amount of traffic across different networks. Presumably, both the applications and the network operator can benefit from this, so network operators have an incentive to provide this information (either directly, or indirectly through a third party), and applications have an incentive to use it 🫶 This document discusses issues related to an information-sharing service that enables applications to perform better-than-random peer selection.
Here's the roadmap 🗺️ Section 2 gives definitions. Section 3 introduces the problem. Section 4 describes some use cases where both P2P applications and network operators benefit from a solution to such a problem. Section 5 describes the main issues to consider when designing such a solution. Heads up: a companion document, "Application-Layer Traffic Optimization (ALTO) Requirements" [ALTO-REQS], goes into the details of these issues 📚
1.2. State-of-the-Art
Okay, these papers, [ACM.ispp2p], [PATH-SEL], and [WWW.p4p.overview], present examples of contemporary solution proposals that address the problem described in this document 🎀 Plus, these proposals have encouraging simulation and field test results. These and similar, independent, solutions all consist of two essential parts:
- a discovery mechanism that a P2P application uses to find a reliable information source, and
- a protocol that P2P applications use to query such sources in order to retrieve the information needed to perform better-than-random selection of the endpoints providing a desired resource.
It is not clear how such solutions will perform if deployed globally on the Internet. However, wide adoption is unlikely without agreement on a common solution, based upon an open standard 🌈
2. Definitions ✨
okay so this is the glossary bit, and it's giving vocab lesson 💅 These are the terms that have a special meaning in the Application-Layer Traffic Optimization (ALTO) problem, so read them like the rules of the game.
Application: a distributed communication system (e.g., file sharing) that uses the ALTO service to improve its performance or quality of experience while improving resource consumption in the underlying network infrastructure. Applications may use the P2P model to organize themselves, use the client-server model, or use a hybrid of both (i.e., a mixture between the P2P model and the client-server model). 🫶
Peer: a specific participant in an application. Colloquially, a peer refers to a participant in a P2P network or system, and this definition does not violate that assumption. If the basis of the application is the client-server or hybrid model, then the usage of the terms "client" and "server" disambiguates the peer's role. 🦋
P2P: Peer-to-Peer. 🌸
Resource: content (such as a file or a chunk of a file) or a server process (for example, to relay a media stream or perform a computation) that applications can access. In the ALTO context, a resource is often available in several equivalent replicas. In addition, different peers share these resources, often simultaneously. ✨
Resource Identifier: an application-layer identifier used to identify a resource, no matter how many replicas exist. 🎀
Resource Provider: for P2P applications, a resource provider is a specific peer that provides some resources. For client-server or hybrid applications, a provider is a server that hosts a resource. 🧚
Resource Consumer: for P2P applications, a resource consumer is a specific peer that needs to access resources. For client-server or hybrid applications, a consumer is a client that needs to access resources. 💜
Transport Address: all address information that a resource consumer needs to access the desired resource at a specific resource provider. This information usually consists of the resource provider's IP address and possibly other information, such as a transport protocol identifier or port numbers. 🔥
Overlay Network: a virtual network consisting of direct connections on top of another network and established by a group of peers. 🌈
Resource Directory: an entity that is logically separate from the resource consumer and that assists the resource consumer to identify a set of resource providers. Some P2P applications refer to the resource directory as a P2P tracker. 🍓
ALTO Service: several resource providers may be able to provide the same resource. The ALTO service gives guidance to a resource consumer and/or resource directory about which resource provider(s) to select in order to optimize the client's performance or quality of experience, while improving resource consumption in the underlying network infrastructure. 👑
ALTO Server: a logical entity that provides interfaces to the queries to the ALTO service. 🌸
ALTO Client: the logical entity that sends ALTO queries. Depending on the architecture of the application, one may embed it in the resource consumer and/or in the resource directory. 💖
ALTO Query: a message sent from an ALTO client to an ALTO server; it requests guidance from the ALTO service. 🦋
ALTO Response: a message that contains guiding information from the ALTO service as a reply to an ALTO query. ⭐
ALTO Transaction: a transaction that consists of an ALTO query and the corresponding ALTO response. 🫶
Local Traffic: traffic that stays within the network infrastructure of one Internet Service Provider (ISP). This type of traffic usually results in the least cost for the ISP. 🧚
Peering Traffic: Internet traffic exchanged by two Internet Service Providers whose networks connect directly. Apart from infrastructure and operational costs, peering traffic is often free to the ISPs, within the contract of a peering agreement. 💅
Transit Traffic: Internet traffic exchanged on the basis of economic agreements amongst Internet Service Providers (ISPs). An ISP generally pays a transit provider for the delivery of traffic flowing between its network and remote networks to which the ISP does not have a direct connection. 💸
Application Protocol: a protocol used by the application for establishing an overlay network between the peers and exchanging data on it, as well as for data exchange between peers and resource directories, if applicable. These protocols play an important role in the overall ALTO architecture. However, defining them is out of the scope of the ALTO WG. 🙈
ALTO Client Protocol: the protocol used for sending ALTO queries and ALTO replies between an ALTO client and ALTO server. 🎀
Provisioning Protocol: a protocol used for populating the ALTO server with information. 🌈
Now the diagram, which is honestly a vibe 🦋
+------+
+-----+ | Peers
+-----+ +------+ +=====| |-*-+
| |.......| |====+ +-*-*-+ *
+-----+ +------+ | * *****
Source of ALTO | *
Information Server | +-*---+
+=====| | Resource Directory
+-----+ (Tracker, proxy)
Legend:
=== ALTO client protocol
*** Application protocol (out of scope)
... Provisioning or initialization (out of scope)Figure 1: Overview of Protocol Interaction between ALTO Elements
Figure 1 shows the scope of the ALTO client protocol: peers or resource directories can use such a protocol as ALTO clients to query an ALTO server. The mapping of topological information onto an ALTO service as well as the application protocol interaction between peers and resource directories are out of scope for the ALTO client protocol. 🌸
3. The Problem 🔥
network engineers have been facing the problem of traffic optimization for a long time, and they've designed mechanisms like MPLS [RFC3031] and Diffserv [RFC3260] to deal with it. The problem these protocols address consists in finding (or setting) optimal routes (or optimal queues in routers) for packets traveling between specific source and destination addresses. Solutions are based on requirements such as low latency, high reliability, and priority. Such solutions are usually implemented at the link and network layers and tend to be almost transparent. 💜
However, distributed applications in general and, in particular, bandwidth-greedy P2P applications that are used, for example, for file sharing, cannot directly use the aforementioned techniques. By cooperating with external services that are aware of the network topology, applications could greatly improve the traffic they generate. In fact, when a P2P application needs to establish a connection, the logical target is not a stable host, but rather a resource (e.g., a file or a media relay) that can be available in multiple instances on different peers. Selection of a good host from an overlay topological proximity has a large impact on the overall traffic generated. 👑
Note that while traffic considerations are important, several other factors also play a role on the performance experienced by users of distributed applications. These include the need to avoid overloading individual nodes, fetching rare pieces of a file before those pieces are available at a multiplicity of nodes, and so on. However, better information about topological conditions does improve the overall selection algorithm on an important aspect. 🧚
Better-than-random peer selection is helpful in the initial phase of the process. Consider a P2P protocol in which a querying peer receives a list of candidate destinations where a resource resides. From this list, the peer will derive a smaller set of candidates to connect to and exchange information with. In another example, a streaming video client may be provided with a list of destinations from which it can stream content. In both cases, the use of topology information in an early stage will allow applications to improve their performance and will help ISPs make a better use of their network resources. In particular, an economic goal for ISPs is to reduce the transit traffic on interdomain links. 🌈
Addressing the Application-Layer Traffic Optimization (ALTO) problem means, on the one hand, deploying an ALTO service to provide applications with information regarding the underlying network and, on the other hand, enhancing applications in order to use such information to perform better-than-random selection of the endpoints with which they establish connections. 💖
4. Use Cases 🦋
4.1. File sharing
File-sharing applications allow users to search for content shared by other users and to download respective resources from other users. For instance, search results can consist of many instances of the same file (or chunk of a file) available from multiple sources. The goal of an ALTO solution is to help peers find the best ones according to the underlying networks. 🍓
On the application side, integration of ALTO functionalities may happen at different levels. For example, in the completely decentralized Gnutella network, selection of the best sources is totally up to the user. In systems like BitTorrent and eDonkey, central elements such as trackers or servers act as mediators. Therefore, in the former case, improvement would require modification in the applications, while in the latter it could just be implemented in some central elements. ⭐
4.2. Cache/Mirror Selection
Providers of popular content, like media and software repositories, usually resort to geographically distributed caches and mirrors for load balancing. Today, selection of the proper mirror/cache for a given user is based on inaccurate geolocation data, on proprietary network-location systems, or is often delegated to the user herself. An ALTO solution could be easily adopted to ease such a selection in an automated way. 🎀
4.3. Live Media Streaming
P2P applications for live streaming allow users to receive multimedia content produced by one source and targeted to multiple destinations, in a real-time or near-real-time way. This is particularly important for users or networks that do not support multicast. Peers often participate in the distribution of the content, acting as both receivers and senders. The goal of an ALTO solution is to help a peer to find effective communicating peers that exchange the media content. 📺
4.4. Real-Time Communications
P2P real-time communications allow users to establish direct media flows for real-time audio, video, and real-time text calls or to have text chats. In the basic case, media flows directly between the two endpoints. Unfortunately, however, a significant portion of users have limited access to the Internet due to NATs, firewalls, or proxies. Thus, other elements need to relay the media. Such media relays are distributed over the Internet with public addresses. An ALTO solution needs to help peers find the best relays. 💫
4.5. Distributed Hash Tables 🎀
okay so distributed hash tables (DHTs) are a whole class of overlay algorithms 🦋 that power lookup features in popular P2P systems, and the best part? No centralized elements anywhere 😭✨. Each peer keeps a routing table of the addresses of other peers in the same DHT, sorted by specific criteria 💅. And ngl, an ALTO solution can give DHT algorithms some seriously valuable info 🔥.
5. Aspects of the Problem 🌸
this section is giving "things you might not realize at first" 🙈 when you start poking at the problem space, so let's dive in 🧚
5.1. Information Provided by an ALTO Service 💖
the whole goal of an ALTO service is to hand applications info they can use for better-than-random peer selection 👑. there are lots of types of info that could help here, but not all of it is ALTO material 💀. info that changes super fast, like transport-layer congestion, is out of scope for ALTO 🙅♀️. that stuff is better handled by an in-band technique at the transport layer, not an out-of-band, application-layer ALTO thing 🫶. like, an ALTO solution for congestion would either be serving outdated info or get hammered with requests from apps way too often 😭. and finally, things like end-to-end delay and available bandwidth can be measured more accurately by the apps themselves 💅.
the stuff that's actually worth conveying through an out-of-band ALTO service is anything apps can't easily figure out on their own AND that changes on a much longer timescale than the instant-by-instant info used for transport congestion control ✨. examples are the operator's policies 🌈, geographical location or network proximity (like the topological distance between two peers) 🗺️, the transmission costs for sending or receiving a certain amount of data to or from a peer 💸, or how much traffic a peer's operator still allows, like with quotas or limited flat-rate pricing models 🍓.
5.2. ALTO Service Providers 🦋
at least three different kinds of entities can run ALTO services, and they're all kinda iconic 👑:
- Network operators. network operators usually have full knowledge of the network they run 🫶. they know their topology and policies inside out 💅.
- Third parties. third parties are separate from network operators, but they may have collected network info or have arrangements with operators to get it 🔥. examples include content-delivery networks like Akamai, which control wide, highly distributed infrastructures 🌐, or companies offering an ALTO service on behalf of ISPs 💜.
- User communities. user communities run distributed algorithms, for example to estimate the topology of the Internet 🧚.
5.3. ALTO Service Implementation 🎀
it's important for you to understand that there are significant user communities who expect an ALTO server to be a centralized service 🏢. and then there are other communities who expect ALTO to be a distributed service, possibly even based on or integrating with a P2P service 🌸.
so, as a result, you can reasonably expect there to be some sort of service-discovery mechanism to go along with the ALTO protocol definition 🔍✨.
5.4. User Privacy 🙈
okay so on one hand, there are data elements an ALTO client could include in its query to an ALTO server that would help make the replies more accurate 🎯. for example, if the client says what kind of application it is (like real-time communications or bulk data transfer), the server can set priorities in its replies to fit the traffic that app will generate 💖. but on the other hand, applications might consider that info private 🔒. plus, some apps may not know ahead of time what kind of request they'll be making 🤷♀️.
5.5. Topology Hiding 🦋
operators know their network topology inside and out, so they can play a big role in solving the ALTO problem 👑. but operators often consider details of that network info to be confidential 🤫.
5.6. Coexistence with Caching 🍓
caching is a way to improve the traffic apps generate, and it needs huge amounts of data transfers 💅. in some cases, caching has proven to be extremely effective at boosting user experience AND saving network resources 🔥✨.
a cache, whether it's explicit or transparent, replaces the content source 🎭. so in principle, a cache MUST use and support the same protocol as the querying peer 🫶. like, if a cache stores web content, it MUST present an HTTP interface to the web client 🌐. any cache solution for a given protocol needs to present that same protocol to the client 💜. said differently, every caching solution for a different protocol needs to implement that specific protocol 🧚. for this reason, you can only reasonably expect caching solutions for the most popular protocols, like HTTP and BitTorrent 🌈.
it is extremely important to realize that caching and ALTO are entirely orthogonal 💖. ALTO, especially if it's aware of caches, can actually direct clients to nearby caches where the user gets a much better quality of experience 🎀✨.
6. Security Considerations 💀🔐
okay this section is serious, bestie, so pay attention 🙏. this document is neither a requirements document nor a protocol specification 📄. however, we believe it's important for you to understand the areas of security and privacy that will matter for designing and implementing an ALTO solution 🛡️. also, issues like digital rights management are out of scope for ALTO, since they aren't technically enforceable at this level 🙅♀️.
some environments and use cases of ALTO may need client or server authentication before sharing sensitive info 🔑. to support those environments interoperably, the ALTO requirements document [ALTO-REQS] outlines minimum-to-implement authentication and other security requirements 📋.
applications can choose to rely on info from an ALTO server to improve peer selection 🎯. in principle, that means the ALTO service providing the info can influence how the application behaves 😳. basically, a third party (the ALTO service provider) takes an important role in a distributed system it was never involved in before 👀.
for example, if an ALTO server is deployed and run by an ISP, the P2P community might see that server as hostile 😬, because the operator could:
- use ALTO to prevent content distribution and enforce copyrights 🚫;
- redirect applications to corrupted mediators providing malicious content ☠️;
- track connections to perform content inspection or logging 👁️;
- apply policies based on criteria other than network efficiency 💸. for example, the service provider may suggest routes that are suboptimal from the user's perspective in order to avoid peering points regulated by inconvenient economic agreements 🙄.
it's important to note there is no protocol mechanism to require ALTO for P2P applications 🙅♀️. if, for some reason, ALTO fails to improve P2P application performance, ALTO won't gain popularity and the P2P community won't use it 📉.
at the time of writing, the privacy issues described in Section 5.4 are relevant for an ALTO solution 🔒. users may be reluctant to disclose sensitive info to an ALTO server 😟. operators, on the other hand, may not want to disclose info that would expose details of their interior topology 🏢. when you explore the solution space in detail, you need to consider these issues so that an ALTO protocol does not presume mandatory information disclosure, by either clients or servers 🫶💖.
7. Contributors 🎀
this document was initially edited by Enrico Marocco and Vijay Gurbani 💜. in their roles as Working Group chairs, they kept providing significant edits and input to the current authors 👑.
8. Acknowledgments 🌸
Vinay Aggarwal and the P4P working group did the research work outside the IETF 🙌. Emil Ivov, Rohan Mahy, Anthony Bryan, Stanislav Shalunov, Laird Popkin, Stefano Previdi, Reinaldo Penno, Dimitri Papadimitriou, Sebastian Kiesel, Greg DePriest, and many others gave insightful discussions, specific comments, and much needed corrections 🫶✨.
Jan Seedorf and Sebastian Kiesel are partially supported by the NAPA-WINE project (Network-Aware P2P-TV Application over Wise Networks, http://www.napa-wine.org), a research project supported by the European Commission under its 7th Framework Program (contract no. 214412) 🇪🇺. the views and conclusions in this document are those of the authors and should not be read as necessarily representing the official policies or endorsements, either expressed or implied, of the NAPA-WINE project or the European Commission 📜.
thanks in particular to Richard Yang for several reviews 💐.
9. Informative References 📚
[ACM.bottleneck]
Akella, A., Seshan, S., and A. Shaikh, "An Empirical
Evaluation of WideArea Internet Bottlenecks",
Proceedings of ACM SIGCOMM, October 2003.
[ACM.fear]
Karagiannis, T., Rodriguez, P., and K. Papagiannaki,
"Should ISPs fear Peer-Assisted Content Distribution?",
ACM USENIX IMC, Berkeley 2005.
[ACM.ispp2p]
Aggarwal, V., Feldmann, A., and C. Scheideler, "Can ISPs
and P2P systems co-operate for improved performance?",
ACM SIGCOMM Computer Communications Review (CCR), 37:3,
pp. 29-40.
[ACM.ono] Choffnes, D. and F. Bustamante, "Taming the Torrent: A
practical approach to reducing cross-ISP traffic in P2P
systems", Proceedings of ACM SIGCOMM, August 2008.
[ALTO-REQS]
Kiesel, S., Popkin, L., Previdi, S., Woundy, R., and Y.
Yang, "Application-Layer Traffic Optimization (ALTO)
Requirements", Work in Progress, April 2009.
[PATH-SEL]
Saucez, D. and B. Donnet, "The case for an informed path
selection service", Work in Progress, February 2008.
[RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
Label Switching Architecture", RFC 3031, January 2001.
[RFC3260] Grossman, D., "New Terminology and Clarifications for
Diffserv", RFC 3260, April 2002.
[WWW.p4p.overview]
Xie, H., Krishnamurthy, A., Silberschatz, A., and R. Yang,
"P4P: Explicit Communications for Cooperative Control
Between P2P and Network Providers",
<http://www.dcia.info/documents/P4P_Overview.pdf>.
[WWW.wired.fuel]
Glasner, J., "P2P Fuels Global Bandwidth Binge",
April 2005, <http://www.wired.com>.Authors' Addresses 💌
Jan Seedorf
NEC Laboratories Europe, NEC Europe Ltd.
Kurfuersten-Anlage 36
Heidelberg 69115
Germany
Phone: +49 (0) 6221 4342 221
EMail: jan.seedorf@nw.neclab.eu
URI: http://www.nw.neclab.eu
Eric W. Burger
Neustar Inc.
46000 Center Oak Plaza
Sterling, VA 20166-6579
USA
Phone:
Fax: +1 530 267 7447
EMail: eburger@standardstrack.com
URI: http://www.standardstrack.comthat's the whole RFC, bestie 🌈💖 ALTO is a whole mood and you absolutely understand it now 🧚✨
📦 where this page comes from
- ietf: rfc5693 2009-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms