RFC 5594

Report from the IETF Workshop on Peer-to-Peer (P2P) Infrastructure, May 28, 2008

📅 published July 2009, from ietf

RFC 5594 💖 Part 1 of 5

Title block tea ☕

  • RFC number: 5594
  • Title: Report from the IETF Workshop on Peer-to-Peer (P2P) Infrastructure, May 28, 2008
  • Authors: J. Peterson (NeuStar) and A. Cooper (Center for Democracy & Technology)
  • Category: Informational 🎀
  • Date: July 2009
  • Obsoletes / Updates: the title block doesn't list anything this RFC obsoletes or updates 🧚

Abstract ✨

okay so this doc is a report from a workshop the IETF held on May 28, 2008, at MIT in Cambridge, MA, USA. The Real-time Applications and Infrastructure Area Directors organized it to talk about network delay and congestion caused by all the rising P2P traffic 🔥. There were two goals: figure out the technical problems ISPs and end users were dealing with because of heavy P2P traffic, and figure out how the IETF could help. Finding the right place inside the IETF for the work, and pulling out work items that were actually doable, were the big tasks for the second goal 💅. The workshop was super well attended and it produced several work items that IETF community members have since picked up 🦋.

Status of This Memo 🌸

This memo gives information to the Internet community. It is not an Internet standard of any kind, so don't go treating it like one 🙈. Distribution of this memo is unlimited.

Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, because they spell out your rights and restrictions for this document 🫶.

Table of Contents 🧚

1. Introduction ....................................................3
2. Scoping of the Problem and Solution Spaces ......................4
3. Service Provider Perspective ....................................4
   3.1. DOCSIS Architecture and Upstream Contention ................4
   3.2. TCP Flow Fairness and Service Flows ........................5
   3.3. Service Provider Responses .................................6
4. Application Provider Perspective ................................7
5. Potential Solution Areas ........................................7
   5.1. Improving Peer Selection: Information Sharing,
        Localization, and Caches ...................................8
        5.1.1. Leveraging AS Numbers ...............................9
        5.1.2. P4P: Provider Portal for P2P Applications ...........9
        5.1.3. Multi-Layer, Tracker-Based Architecture ............10
        5.1.4. ISP-Aided Neighbor Selection .......................11
        5.1.5. Caches .............................................12
        5.1.6. Potential IETF Work ................................12
   5.2. New Approaches to Congestion Control ......................14
        5.2.1. End-to-End Congestion Control ......................15
        5.2.2. Weighted Congestion Control ........................15
   5.3. Quality of Service ........................................16
6. Applications Opening Multiple TCP Connections ..................17
7. Costs and Congestion ...........................................18
8. Next Steps .....................................................18
   8.1. Transport Issues ..........................................19
   8.2. Improved Peer Selection ...................................19
9. Security Considerations ........................................19
10. Acknowledgements ..............................................19
11. Informative References ........................................20
Appendix A.  Program Committee ....................................21
Appendix B.  Workshop Participants ................................21
Appendix C.  Workshop Agenda ......................................24
Appendix D.  Slides and Position Papers  ..........................25

1. Introduction ✨

ngl, big ISPs are running into real trouble with P2P traffic 😭. Moving static, delay-tolerant data between nodes on the Internet is a well-understood problem. The issue is that the usual transport-level fairness management is getting strained by apps built to squeeze out the best transfer rates for end users 💀. At peak times the network runs close to full capacity, so everything gets delayed, and the apps that feel it most are real-time ones like Voice over IP (VoIP) and Internet gaming 🎮. To figure out how IETF standards could help, the Real-time Applications and Infrastructure area directors ran a "P2P Infrastructure" workshop and invited subject matter experts from both the problem side and the solution side 🧚.

The workshop had two goals: understand the technical problems ISPs and end users were having because of high volumes of P2P traffic, and start figuring out how the IETF could help fix them. For that second goal, finding where in the IETF the work might happen and pulling out feasible work items were the key tasks 💖. The focus was on engineering solutions that promise some imminent benefit to the Internet as a whole. That's different from long-term research or closed proprietary stuff 🔥. Public policy has to inform this space, but debating or crafting public policy was out of scope for the workshop 🙈.

Position papers were requested in the weeks before the workshop, and a limited number of speakers were invited to present their views based on those submissions. This report summarizes everyone's contributions. The program committee and the participant list are in Appendices A and B, the agenda is in Appendix C, and Appendix D links to the presentations and the position papers submitted before the workshop 📎.

The workshop showed how much the IETF community gets that P2P and other high-volume apps affect the whole Internet 👑. Participants were excited to talk about possible standardization work that network operators, application providers, and end users would all find mutually beneficial 🫶. Two transport-related work items got serious traction: designing a protocol for very deferential end-to-end congestion control for delay-tolerant applications, and producing an informational document about the reasoning behind applications opening multiple transport connections at once and what that does 🦋. A separate area of interest came up too, focused on improving peer selection by having networks make more information available to applications 🌈. And presenters covered traditional multiple service-tier queuing approaches like Diffserv 🍓.

2. Scoping of the Problem and Solution Spaces 🌸

The Peer-to-Peer Infrastructure (P2PI) workshop was mostly born out of specific pain points that ISPs are having because of high volumes of P2P traffic 😩. Still, several participants felt the IETF should look at a more general set of problems, where P2P-related congestion might be just one example 🧚.

For instance, high-volume applications other than P2P, whether they already exist or haven't been built yet, could cause congestion problems similar to the ones P2P causes 💅. The general class of congestion problems that come from always-on, high-volume applications needs solutions that are reasonable for operators, applications, and subscribers alike ✨. Lots of attention has gone to congestion on access links, but growing traffic volumes could hit other parts of the network too 👀. Even though the workshop focused mainly on the specific causes and effects of today's P2P volumes, it will probably be useful for the IETF to think later about how to pursue solutions to these bigger problems 🔥.

Getting more data about Internet congestion might also be a helpful step before the IETF goes after solutions 📊. That data collection could focus on where in the network congestion is happening, how long and how often it lasts, what effects it has, and what its root causes are 🍓. Individual service providers were interested in sharing congestion data, but strategies for reliably and regularly getting and sharing that data at broad scale are still kinda elusive 🫠.

3. Service Provider Perspective 🎀

To help participants get a fuller picture of one specific network operator's view of P2P-induced congestion, Jason Livingood and Rich Woundy gave an overview of Comcast's network and its approach to managing P2P traffic 🌈.

3.1. DOCSIS Architecture and Upstream Contention 💜

In the Data Over Cable Service Interface Specification (DOCSIS) architecture [DOCSIS] that a lot of cable systems use, one Cable Modem Termination System (CMTS) can serve hundreds or thousands of residential cable customers 🏠. Each CMTS has multiple DOCSIS domains, and each domain typically has one downstream link and a number of upstream links 🔗. Each CMTS connects through a hybrid fiber-coaxial (HFC) network to subscribers' cable modems ⭐.

The limiting resource in this architecture is usually bandwidth, which is why bandwidth is typically the measure used for capacity planning 📈. Like with all networks, congestion shows up when instantaneous load goes over available capacity 💥.

In the upstream direction, any cable modem connected to a CMTS can request to transmit to the CMTS, and those requests get randomized to cut down on collisions 🎲. But when lots of cable modems are sending requests at once, the requests can still collide, and that causes delays 😵. DOCSIS doesn't specify a size for cable modem buffers, but buffer delays of one to four seconds have been observed with various cable modems from different vendors 🫣.

Once the CMTS grants a cable modem permission to transmit its data PDU, the modem can piggyback its next request onto that data PDU 🧃. When there's a lot of upstream traffic, piggybacking happens more often, and that sends heavy upstream users to the front of the CMTS queue, ahead of interactive applications that don't upload as much 🚀. For example, if the CMTS grants requests about every one to three milliseconds, then a cable modem sending data for a service like VoIP with a packetization delay of 20-30 milliseconds could end up in contention with another modem on the same CMTS that is constantly transmitting upstream and piggybacking each new request 📞. This may explain how heavy upstream users ultimately dominate the pipe over more interactive applications 😤. Consequentially, it is imperative that assessments of the problem space and potential solutions are mindful of the influence that specific layer-2 networks may exert on the behavior of Internet traffic, especially when thinking about easing congestion in an access network 🫶✨

3.2. TCP Flow Fairness and Service Flows 🎀

okay so, the question of how TCP flow fairness should apply to upstream requests going to the CMTS is still totally open 🫣 like, a CMTS is juggling a bunch of service flows, and each one could be one TCP flow or a whole bunch of TCP flows (or UDP, bestie 🦋). The CMTS doesn't know the source or destination IP address of a packet until that packet has already gone upstream 😭 so it can't use those addresses to enforce flow fairness. no cap, that's a real problem 💀

a single cable modem can have multiple service flows defined ✨ for example, a modem that's also a VoIP endpoint can set up a service flow just for VoIP, so that VoIP traffic skips the upstream request process to the CMTS (and so it doesn't fight for contention with the other modems 🧚). that service flow gets upstream capacity provisioned for it 💖 meanwhile the modem gets a separate service flow for best efforts traffic 🌸. some ISPs set up a flow like this for their own VoIP offerings, and others let subscribers pay extra so that particular traffic gets put into a provisioned service flow 💅

it may also be possible for an ISP to provision a flow like this on the fly, the moment it recognizes the need for one 👑 for example, Diffserv [RFC2475] bits set by the customer premises equipment could be used to classify flows 🫶

3.3. Service Provider Responses 🌈

okay so in 2005, ISP customers started complaining a lot about how delay-sensitive stuff (VoIP and gaming) was performing 😩 part of that came from the DOCSIS architecture issues we just went over. at the same time, ISPs were seeing huge growth in P2P traffic, and high levels of P2P activity were increasingly correlated with packet loss 💀

so cable ISPs have a few avenues they can pursue 💅 the newest generation of the DOCSIS specification, DOCSIS 3.0, enables faster transfer rates than most cable systems support right now ✨ the rollout of DOCSIS 3.0 will add capacity, but it will likely NOT be enough to get rid of the need for congestion management, especially in an environment where client software is designed to max out bandwidth consumption no matter how much capacity is actually available 😭

congestion management can take lots of forms 🦋 Jason and Rich explained the new protocol-agnostic approach that Comcast was trialing. before these trials, all traffic was marked as "best efforts" 🍓 during the trials, all traffic gets re-classified as "priority" ✨ when a CMTS is approaching peak congestion on a particular upstream or downstream port (the "Near Congestion State"), some subscribers will have their traffic re-classified as "best efforts" 🙈 both the threshold for deciding when a CMTS port is in Near Congestion State and the number of minutes it stays in that state are parameters being explored during the trials 🧪 to re-classify upstream traffic, a new default DOCSIS service flow is used that has the same provisioned bandwidth as the "priority" stream, but it's treated with lower priority 💜

the subscribers whose traffic gets re-marked are chosen by figuring out whether they've temporarily entered a "Long Duration Bulk Consumption State" 🌊 that state is reached by consuming a certain amount of bandwidth over a certain number of minutes (both are tweakable parameters being explored during the trials). these thresholds depend on the subscriber's service tier, so subscribers who pay for more bandwidth get higher thresholds 👑 the re-marking doesn't tell apart multiple users on the same subscriber connection, so one family member's P2P usage could drop another family member's Web browsing traffic in priority 😭 and there's currently no mechanism for users to find out that their traffic has been re-marked 🙈

ngl this is lowkey clever 💅 by temporarily lowering the priority of subscribers who've been eating bandwidth in bulk for a long time, this technique aims to keep a good experience for subscribers with burstier traffic patterns, including people using real-time apps 🫶 compared to an approach that cuts particular subscribers' bandwidth during congestion, this one takes away the ability for applications to set their own priority levels 😤 but it also avoids the negative vibes that some users link to bandwidth reductions 🌸

this approach has a ton of tweakable parameters 🎛️ a big chunk of the trial process is about finding the best settings for them, but there may also be chances to work with the research community to figure out the best way to adjust the thresholds needed to optimize the management technique 🧠

4. Application Provider Perspective 🍓

Stanislav Shalunov shared BitTorrent's take on how increased P2P traffic volumes affect things, plus possible mitigations 🦋 this section just covers the impact 🫶 his proposed solutions (which were the bulk of his talk) are addressed in the appropriate subsections of Section 5 💅

okay so as P2P usage has grown, so has end-user latency 😩 for example, a user whose uplink capacity is 250-500 Kbps and whose modem buffer has a capacity of 32-64 Kbps can easily fill that buffer (unless the modem uses Adaptive Queue Management (AQM), which is uncommon) 💀 this can cause delay on the order of seconds, with disastrous effects on application performance 😭 on a cable system with capacity shared between neighbors, one neighbor could saturate the buffer and mess with another neighbor's latency ✨ even users with dedicated bandwidth can see delays, because their own P2P traffic can saturate the link and dominate their more latency-sensitive traffic 🙈

5. Potential Solution Areas 🔥

the submissions received before the workshop covered a broad range of work on specific aspects of P2P traffic volume and other related issues 🧠 solution suggestions generally fell into one or more of three topic areas: improving peer selection, new approaches to congestion control, and quality-of-service mechanisms 💖 the workshop discussions and outcomes in each area are described below 🌈

5.1. Improving Peer Selection: Information Sharing, Localization, and Caches 🎀

peer selection is an integral factor in how efficient P2P networks are, from both the ISP and the P2P client points of view 💅 how peers get selected determines both network load and client performance ✨

the way P2P clients select peers today varies from protocol to protocol and client to client, but in general, peers are largely oblivious to routing-level and network-topology information 😭 this results in P2P topologies that are agnostic of underlay topologies and constraints 🦋

approaches to closing this gap generally involve an entity that knows about network topology, costs, or constraints (e.g., an ISP) making some of this information available to P2P clients or trackers 🫶 this information may be used to localize traffic based on some metric of locality, or to otherwise alter peer-selection decisions based on the provided network information (hereafter referred to simply as "localization") 🧚 one special case of this kind of approach would help peers find caches containing the content they seek 🌸

any change to current peer-selection algorithms comes with engineering trade-offs 💜 BitTorrent, for example, used randomized peer selection by design 🍓 choosing peers randomly out of a large selection helps average out problems among peers and allows connections to good peers that may be far away 🌈 randomized peer selection also supports "rarest first" piece selection, which lets swarms keep going even when the original seed disappears, and which spreads pieces so that more peers are likely to have pieces other peers are interested in ✨ any move away from randomized selection would have to take these factors into account 🙈

localization has the potential to improve peer selection, but the incentives for both parties in the information exchange are complicated 😵‍💫 ISPs may want to move traffic off their own networks, which could push them to give peers information that has the opposite effect of what the peers would expect 💀 peers, likewise, will want the information they receive to lead to performance improvements, otherwise they have no incentive to consult the network before selecting peers 🫠 even when both parties find the information sharing beneficial, user experiences won't necessarily be uniform, depending on the scope of the information provided and the peer's location 🌍 localization information could form one part of a peer-selection decision, but it will likely need to be balanced against other factors 💅

workshop participants discussed both current research in this area and how IETF standards work may help further the general concept of improved peer selection 🎓 those discussions are summarized below 🦋

5.1.1. Leveraging AS Numbers 👑

one simple way to potentially make peer selection more efficient would be for a peer to prefer peers within its own Autonomous System (AS) 🧚 transfers between peers in the same AS may be faster on some networks, although more data is needed to figure out how big the potential improvement is 💖 on mobile networks, for example, the usefulness of AS numbers is limited because they don't correlate to geographic location 🙈 peers may also see improvements by connecting to other peers within a specific set of ASes or IP prefixes provided by their ISPs 🫶 some ISPs may have an incentive to expose this level of granularity, since it could reduce their transit costs 💅

a case study was run on the four most popular BitTorrent torrents to see what localizing to an AS might do 📊 the swarm sizes for the torrents were 9984, 3944, 2561, and 2023, with the size distributions appearing to be polynomial 🌸 with more than 20 peers in a single AS, peers within an AS could trade only with each other, avoiding interdomain traffic 🌈 more than half (57%) of peers in the four swarms were in ASes like this 😮 so in these cases, connecting to peers within an AS could reduce transit traffic by at least 57% ✨ if the ASes have asymmetric upload and download links, though, the resulting user experience may get worse, since each peer's download speed would be limited by slower upload speeds 😭

with the largest swarm size at 9984, the probability of two peers being in the same neighborhood is too low to make localization to the neighborhood level worth it 🏘️ trying a simple localization scheme, like the AS localization described above, and checking how well it works probably makes more sense as a first step 🧠

5.1.2. P4P: Provider Portal for P2P Applications ✨

Okay bestie, so the Provider Portal for P2P Applications (P4P) project [P4P] is trying to build a framework so that providers and applications (P2P included) can actually work together 🫶 Here, "providers" can be ISPs, content distribution networks, or caching services. Each provider talks to P2P clients through a portal called an iTracker 🧚 Clients can find an iTracker through a DNS SRV record (maybe with its own brand new record type), a whois look-up, or a trusted third party.

An iTracker has different interfaces for different kinds of info the provider wants to share. The core interface lets the provider express the "virtual cost" of its intradomain or interdomain links 💅 Virtual cost can reflect whatever the provider prefers and can be based on its choice of metrics, like utilization, transit costs, or geography. It's totally up to the provider how dynamic it wants to be when updating those virtual cost numbers.

In the tests, they made two parallel swarms with about the same number of clients and similar geographical and network distributions, and both were sharing the same file 🍓 One swarm used P4P, with the ISP's network topology map fed into its iTracker. The other swarm used traditional peer selection. The swarm without P4P sent 98% of its traffic to and from peers outside the ISP, while the P4P swarm only hit 50% 😭 Download completion times for the P4P-enabled swarm also got about 20% better on average. Iconic 👑

5.1.3. Multi-Layer, Tracker-Based Architecture 🌈

The multi-layer, tracker-based P2P scheme presented at the workshop is a generic example of an architecture that shows how localization can be useful in principle ✨

In a traditional, tracker-based P2P system, trackers give clients info about seeds and peers where they can find the content they want. A multi-layered tracker setup adds extra "local" trackers that give the same info, but only for content inside their own local network scope 🦋 Client queries get re-directed from the global tracker to the right local trackers. Local trackers can also exist on multiple levels, in which case the queries get re-directed even further. This kind of architecture could also serve hybrid P2P/content delivery networks, where the global tracker acts as both a tracker and a content server, and local trackers track locally provisioned caches as well as seeds and peers 💜

One challenge here is figuring out what "local" even means for trackers, seeds, and peers 🙈 Locality could depend on traffic conditions, load balancing, static topology, policy, or some other metric. The same considerations would also be crucial for deciding where caches should go in a hybrid network.

Basically, this architecture has the abstract problem of re-directing from a global entity to a local entity. Client queries need to find their way to the right local tracker. This can happen through an off-path, explicit mechanism where local trackers register with the global tracker ahead of time, or through an on-path approach where the network proxies P2P requests. The off-path tracker format approach is preferable for performance and reliability reasons 🎀

Since the multi-layer scheme might require ISPs to help peers find optimal paths to unauthorized copies of copyrighted content, ISPs may be worried about the legal liability of taking part 😬

5.1.4. ISP-Aided Neighbor Selection 💖

ISPs know a LOT about their networks, like the bandwidth, geography, and service class of particular nodes, plus overall routing policies, OSPF and BGP metrics, and distances to peering points 🧠 The ISP-aided neighbor selection service described below tries to use that knowledge without making ISPs reveal anything that couldn't already be figured out through reverse-engineering by client applications. Smart and respectful, periodt 💅

The service is built around an "oracle" hosted by an ISP. The oracle takes a list of IP addresses from a network node, sorts the list according to its own confidential criteria, and sends the sorted list back to the node ✨ The peer ranking from the oracle can be seen as a special case of the virtual cost interface from the previous section.

This service could be used by P2P clients or trackers, or by any other application that would benefit from learning its ISP's connection preferences 🦋 The oracle could run as a web server or a UDP service at a known location (maybe similar to BIND).

For interdomain ranking, an ISP could rank its own peers first, or it could base the ranking on the AS number of the IPs in the list. Another option is for multiple ISPs to team up and have their oracles swap lists with each other 🫶

The big challenge with the oracle service is scalability 😭 If peers have to send the oracle the IP address of every peer they know, the oracle requests could get inordinately large. Plus, today's largest swarms approach 10000 peers, and when every peer is asking for a sorted list, the oracle request volume will swell. With business models that depend on P2P for content distribution growing, future swarms may be way bigger, which makes the problem even worse 💀 Possible fixes include having trackers, instead of peers, issue the oracle requests, and using other peers' sorted lists as input, rather than always using an unsorted list.

On the other hand, this approach is a big win from a legal liability perspective, because it doesn't require ISPs to know where particular content might be located or to have any role in directing peers to particular content 🙌

5.1.5. Caches 🍓

Deploying caches as peers in P2P networks came up in several proposals at the workshop 🧚 Caches can ease network load by cutting down on how much peers need to upload to each other and by localizing traffic.

There are two main concerns about P2P caches, and they're about network capacity and legal liability. For caches to be useful, they'll probably need to be large. One suggestion was that a 1 TB cache could service 30% of requests within a single AS, and a 100 TB cache could service 80% of requests 🔥 Large caches will need sizable bandwidth to avoid contention among peers. Caches wouldn't be usefully placed inside an HFC network on a cable system, for example.

The legal liability that comes with hosting a P2P cache probably lowers the incentive to do it 😬 Even under legal regimes where liability for caching is unclear, ISPs and others may see hosting a cache as too big of a legal risk to be worth it.

5.1.6. Potential IETF Work 👑

A lot of the localization work from the workshop is still in its early stages, and there are plenty of open questions about how much value localization gives for different network and overlay architectures 🤔 More data is needed to evaluate its effects on both traffic load and client performance. Understanding swarm distributions matters a lot here, because swarms with long tails may not really benefit from localization.

With all that in mind, the key task for the IETF (as the workshop identified it) is to pinpoint work items in the spaces discussed above that are incrementally beneficial 💅 Someday it might be possible to standardize entire P2P mechanisms, but as a starting point it makes more sense to single out core, manageable components for standardization. The focus should be on items that aren't so specific to one ISP or P2P network that standardizing them would be useless. Ideally, any mechanisms from this work could apply to future applications that have the same bandwidth-intensive properties as today's P2P file-sharing ✨

When looking at any of these items, it's necessary to make sure the info that networks and applications swap doesn't hurt anyone involved 🫶 Not every piece of exchanged info will be beneficial or verifiable, and that fact has to be recognized and accounted for. Solutions that leave applications or networks worse off than they are today won't get any traction, no cap 🚫

It also shouldn't be assumed that a particular party is the best one to provide a particular kind of info 🦋 For example, an ISP may not know what the connection costs are in other ISPs' networks, whereas an overlay network that gets cost info from several ISPs may have a better handle on that kind of data. Standardizing info sharing shouldn't assume who the parties doing the sharing are.

The workshop's list of potential work items is below. Workshop participants were especially interested in pushing the first three items further 💖

5.1.6.1. AS Numbers 🧚

P2P clients currently rely on IP-to-AS mapping tables when they want to determine AS numbers. Providing a standard, easier way for clients to get this info may help make peer selection more efficient on certain networks ✨

5.1.6.2. Querying for Preferred Peers 🌸

In situations where a peer or tracker can make real-time requests to a service that expresses its ISP's peering preferences, standardizing a format for requests and responses could be useful 💜 The focus would be on communicating the info, not on the criteria used to decide preferences. The info given to peers would have to be crafted so it protects the privacy of other peers and safeguards proprietary network info 🔒

5.1.6.3. Local Tracker, iTracker, Oracle, or Cache Discovery 🗺️

With trackers, iTrackers, oracles, or other mechanisms that provide info specific to a node's locality now being deployed, nodes will need a way to find these resources 🧚 One task for the IETF could be exploring a way to do discovery, potentially by leveraging an existing discovery protocol (DNS, DHCP, anycast, etc.). Depending on the resource, discovery might only need a simple look-up, or it might need a more complex determination of which resource is "closest" to the node making the request 🎀

5.1.6.4. ISP Account Usage Information 📱

When ISP subscribers are bound by network usage policies or volume-based quotas, it may be useful to have a standard way to communicate the subscriber's current usage status 🙈 This would be kind of like knowing how many minutes of cell phone airtime are left in your billing cycle. Applications could use this info to decide when and how to transfer data. One challenge in implementing such a standard would be supporting potentially limitless different ISP business models 🌀 The level of granularity an ISP can provide may also be limited by its pricing model and by how dynamic the info is expected to be 💅

5.1.6.5. Tracker Formats 🗺️

okay so a multi-layered tracker setup could be seriously helped by a standard tracker format for re-directing from a global tracker to a local tracker 🧚 ngl it's not clear yet how willing existing trackers are to chat with other trackers, but the re-direction format might have a cousin in another context. Lots of HTTP servers build their own indexes of mirror info for a similar purpose, though those aren't standardized 🌸. If the two problem spaces turn out to be similar enough, there could be room to standardize one format that covers both 💖.

5.2. New Approaches to Congestion Control 🌊

one recent informal survey presented at the workshop found that ISPs see traffic volumes from heavy users as a problem, but no single congestion management tool has been widely used yet 💀. Inside developer and research communities, the congestion issues from increased P2P traffic have sparked new thinking about congestion-control mechanisms at both the transport layer and the application layer 🔥. The subsections below explore some of these new ideas and highlight areas where IETF work may be appropriate ✨.

5.2.1. End-to-End Congestion Control 🏎️

as noted earlier, a rise in P2P usage can lead to perceptible end-user latency on the order of seconds for interactive applications 😭. One way to fix this "round-trip time (RTT) in seconds" problem would be for P2P clients to implement better congestion control that keeps the bottleneck full while yielding, so that the delay for competing traffic stays low 🫶. This has actually been implemented in BitTorrent's client by continuously sampling one-way delay (separating propagation from queuing delay) and targeting a small queuing delay value 💅. It basically approximates a scavenger service class inside an end-to-end congestion-control mechanism, forcing bulk, elastic traffic to yield to its competitors under congestion 👑.

in a similar vibe, the P4P framework supports a component that lets applications mark traffic as "bulk data" (not time sensitive) 🦋. Applications then adjust their behavior based on the feedback they get from those markings 🌈.

experimenting with standardizing these kinds of techniques, or any congestion-control framework whose design goals differ from those of TCP, may be helpful work for the IETF to pursue 🍓.

5.2.2. Weighted Congestion Control ⚖️

congestion control has typically been a protocol-level, optional, cooperative effort between the endpoints that are experiencing congestion 🤝. But if we want a long-term approach, we may need a more rigorous way for available bandwidth to be allocated by and between the hosts using a network 💜. The big idea behind weighted congestion control is to let hosts give more weight to interactive applications during times of congestion 🎀.

comparing this with Diffserv really shows off its strengths and weaknesses 👀. Unlike Diffserv, weighted congestion control could be implemented on hosts with a simple extension to socket APIs (although OSes would need to agree on it for portability) 🛠️. With weighted congestion control, control lives with the host. Even when Diffserv APIs are available, it's hard for a host to know whether the network is actually complying with its classifications 🙈. With weighted congestion control, hosts need some kind of disincentive to setting their weights at maximum levels, whereas Diffserv was never designed for individual users to employ 🚫. Both approaches rely on traffic senders to set policies, which means the congestion issues stemming from P2P use on the receiver side aren't helped by either mechanism 😤.

with Diffserv, a light user may waste their priority connecting to a heavy user on another network, which is not a problem with host-controlled weighting 🥲.

weighted congestion control is just one example of a broader set of features that make congestion-control approaches useful 🌟. These characteristics include full user control of priorities within a user's own scope, and no possibility of interpreting ISP behavior as discriminatory 🙅‍♀️. The first means ISPs should not override user decisions arbitrarily (though this doesn't rule out an ISP offering prioritization as an option) 💁. The second means the metric for decision-making needs to shut down suspicion of ISP motives 🕵️.

one metric that meets these criteria is a harm (cost) metric, where cost equals the amount of data that was not served to its destination 📉. With this metric, cost is greatest when traffic peaks are greatest 📈. It allows for a policy of not sending too much data during times of congestion, without specifying exactly how much "too much" is 🧁. The cost metric could be used for either a Diffserv approach or for weighted congestion control 🎯.

one important limit on ISPs from a congestion-control perspective is that they don't have a window into congestion on other ISPs' networks 🪟. Solving that requires a separate mechanism to express congestion across domains 🌐.

one potential avenue for the IETF or IRTF would be to set up a long-term design team to assess congestion problems in general, and the long-term effects of any proposed quick fixes 🧠. These issues aren't necessarily confined to P2P, and should be viewed in the broader context of massive increases in bandwidth use 🚀.

5.3. Quality of Service 🎟️

ISPs have tried a wide variety of short-term approaches to dealing with congestion, but several of these may not be viable in the long run 😬. For example, some ISPs found that using deep packet inspection to change the delivery characteristics of certain traffic at times of congestion is more cost effective than adding more bandwidth 💸. Over time, though, this could turn into a cat-and-mouse game where application providers keep adapting to avoid being correctly classified by Deep Packet Inspection (DPI) equipment 🐭🐱. Similarly, ISPs using traffic analysis to identify P2P traffic may find that, in the long run, the overhead needed for an effective classification scheme gets excessive 🥵.

Quality of service (QoS) arrangements may be a better fit for the long term 💫. One approach that distinguishes certain classes of traffic during times of congestion was described in Section 3.3 🔍. A standardized mechanism that may be useful for implementing QoS is Differentiated Services Code Points (DSCP) [RFC2474] 🧩.

with DSCP, devices at the edge of the network mark packets with the service level they should receive 📦. Nodes within the network don't need to remember anything about service flows, and applications don't need to request a particular service level 😌. Users effectively avoid self-interference through service classification 🫧.

although DSCP may have many uses, probably the most relevant one for the P2P congestion issue is its ability to facilitate usage-based charging 💰. User pricing agreements that charge a premium for real-time traffic and best-effort traffic could potentially shape user behavior, resulting in reduced congestion (although ISPs would need a mechanism to mitigate the risk of charging subscribers for things like unintentional malware downloads or DoS attacks) ⚠️. DSCP could also be used to limit a user's supply of high-priority bandwidth, which would have a similar effect 🎈.

equipment to support DSCP is already available 🛒. There's been some concern about a perceived lack of DSCP deployment, but it's widely used by enterprise customers, and growth has been strong thanks to uptake in VoIP at the enterprise level 📞.

DSCP still faces deployment hurdles on many networks, and the biggest barrier of all is the lack of uniform code points to be used across networks 🚧.

for example, the latest Windows Vista API marks voice traffic as CS7, which sits above the priority reserve for router traffic 🪄. To properly take advantage of this change, every switch will need to re-mark all traffic 🔁. On top of that, disparate ISPs currently have no means of verifying each other's markings, so they may be unwilling to trust the markings they receive 🥺.

6. Applications Opening Multiple TCP Connections 🧵

the workshop discussions about P2P congestion sparked a related conversation about applications (P2P or otherwise) that open multiple TCP connections 💬. When multiple users share one link, TCP flow fairness gives users with multiple open connections a larger share of the bandwidth 🥇. Since some P2P protocols use multiple open connections for a single file transfer, and users often pursue multiple transfers at once, a P2P user can end up with many more open connections at once than other users on the same link 🌀. The same goes for users of other applications that open multiple connections 🫠. A single user with multiple open connections isn't necessarily a problem on its face, but the fact that fairness is determined per flow rather than per user gives that impression 🤷‍♀️. Workshop participants thought it might be useful for the IETF to provide some information about these situations 📚.

7. Costs and Congestion 💸

workshop participants had diverging opinions about how much the cost of transferring data (as experienced by ISPs and, by extension, their subscribers) should factor into IETF thinking on P2P traffic issues 🤔.

on one hand, bandwidth costs can be significant even when you look at them apart from congestion issues 💵. Some estimates put the total cost of shipping 1 GB between $0.10 and $2 📦. The cost of transit bandwidth in markets where subscribers pay flat rates appears to have leveled off, and it may no longer be getting cheaper 📊. So it may be reasonable to expect more service providers to move to volume-based pricing (where they haven't already) as a way to control congestion and increase revenues 📈. This only works if bandwidth consumption is visible to end users, which argues for some mechanism to expose quotas and usage to applications 👁️. However, expressing cost information may be outside the technical purview of the IETF 🙃.

on the other hand, congestion can be seen simply as a manifestation of cost 🔄. An ISP that invests in capacity could be considered to be paying to relieve congestion 🏗️. Or, if subscribers are charged for congesting the network, then cost and congestion could be viewed as one and the same 🎭. The line between them may therefore be artificial ✂️.

workshop participants felt the issues highlighted here may be useful fodder for IRTF work 🧃.

8. Next Steps 🏁

the IETF community recognizes how significant both the growth in P2P traffic volumes and the network load at large are 🌍. The importance of addressing the impact of high-volume, delay-tolerant data transfer on end-user experiences was very obvious at the workshop 🙌.

at the conclusion of the workshop and in the days that followed, it became clear that the biggest areas of interest fell into two categories: transport-related issues and improved peer selection 👑✨.

8.1. Transport Issues

okay bestie, the workshop had two big transport-related work items come out of it 🚀 First, the crew wanted a standardized, delay-based, end-to-end congestion-control mechanism that P2P clients could use to stop hogging shared links and wrecking interactive apps 😭 (see Section 5.2.1 for the deets). Second, an informational document that describes what P2P apps are actually doing right now when they open multiple transport connections, plus recommended best practices for doing it well 💅 (that's Section 6).

8.2. Improved Peer Selection

the workshop people were super into the whole range of ideas from Section 5.1, the ones about networks and applications sharing info so peer selection gets smarter 🧚✨ and lowkey the best part is that it wouldn't just help P2P. Other apps that want network info could use it too 👑 The ideas on the table were pretty broad, so the first real step is picking out the most feasible bits to actually pursue 🎀

9. Security Considerations

okay this part is SERIOUS, no cap 🙈 The workshop covered a bunch of possible engineering work, and each piece has its own security stuff going on. Like, if networks start handing preference or topology info to applications, the apps might want a way to check that the info is authentic 🔐 As the IETF community starts chasing specific paths from this workshop, dealing with those security requirements is going to be absolutely crucial 💀 Do not skip this one, periodt.

10. Acknowledgements

big thank you to MIT 🏫💖, the place that hosted the workshop, and everyone at MIT and beyond who helped out with the organizing and logistics 🫶

The IETF is so grateful to the program committee (listed in Appendix A) for their time and energy: they organized the workshop, reviewed the position papers, and built an event that was actually valuable for everyone 🌈 Huge shoutout as well to the scribes, Spencer Dawkins and Alissa Cooper, who carefully recorded everything during the workshop ✍️✨

And a special thank you to all the participants (listed in Appendix B) who showed up, joined the discussions, and put in the work to make this a success 🦋 The IETF really appreciates everyone who prepared and gave presentations too 💜

11. Informative References

[DOCSIS]   CableLabs, "DOCSIS Specifications - DOCSIS 2.0 Interface",
           2008, <http://www.cablemodem.com/specifications/specifications20.html>.

[P4P]      Xie, H., Yang, Y., Krishnamurthy, A., and A. Silberschatz,
           "P4P: Provider Portal for Applications", August 2008,
           <http://uwnews.org/relatedcontent/2008/August/rc_parentID43281_thisID43282.pdf>.

[RFC2474]  Nichols, K., Blake, S., Baker, F., and D. Black,
           "Definition of the Differentiated Services Field (DS
           Field) in the IPv4 and IPv6 Headers", RFC 2474,
           December 1998.

[RFC2475]  Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
           and W. Weiss, "An Architecture for Differentiated
           Services", RFC 2475, December 1998.

Appendix A. Program Committee ✨

  • Dave Clark, MIT
  • Lars Eggert, TSV AD
  • Cullen Jennings, RAI AD
  • John Morris, Center for Democracy and Technology
  • Jon Peterson, RAI AD
  • Danny Weitzner, MIT

Appendix B. Workshop Participants 🌸

the squad 🫶 every single one of these people showed up:

  • Vinay Aggarwal, Deutsche Telekom Labs, TU Berlin
  • Marvin Ammori, Free Press
  • Loa Andersson, Acreo AB
  • Jari Arkko, Ericsson
  • Alan Arolovitch, PeerApp
  • Timothy Balcer
  • Mary Barnes, Nortel
  • Colby Barth, Cisco Systems
  • John Barlett, NetForecast
  • Salman Baset, Columbia University
  • Chris Bastian, Comcast
  • Matthew Bell, Charter Communications
  • Donald Bowman, Sandvine Inc.
  • Scott Bradner, Harvard University
  • Bob Briscoe, British Telecom
  • David Bryan, SIPeerior Technologies
  • Rex Bullinger, National Cable & Telecommunications Association
  • Gonzalo Camarillo, Ericsson
  • Mary-Luc Champel, Thomson
  • William Check, NCTA
  • Alissa Cooper, Center for Democracy and Technology
  • Patrick Crowley, Washington University
  • Leslie Daigle, Internet Society
  • Spencer Dawkins
  • John Dickinson, Bright House Networks
  • Lisa Dusseault, CommerceNet
  • Lars Eggert, Nokia Research Center
  • Joe Godas, Cablevision
  • Vernon Groves, Microsoft
  • Daniel Grunberg, Immedia Semiconductor
  • Carmen Guerrero, University Carlos III Madrid
  • Vijay Gurbani, Bell Laboratories/Alcatel-Lucent
  • William Hawkins III, ITT
  • Volker Hilt, Bell Labs, Alcatel-Lucent
  • Russell Housley, Vigil Security, LLC
  • Robert Jackson, Camiant
  • Cullen Jennings, Cisco Systems
  • Paul Jessop, RIAA
  • XingFeng Jiang, Huawei
  • Michael Kelsen, Time Warner Cable
  • Tom Klieber, Comcast
  • Eric Klinker, BitTorrent Inc.
  • Umesh Krishnaswamy
  • Gregory Lebovitz, Juniper
  • Erran Li, Bell-Labs
  • Jason Livingood, Comcast
  • Andrew Malis, Verizon
  • Enrico Marocco, Telecom Italia Lab
  • Marcin Matuszewski, Nokia
  • Danny McPherson, Arbor Networks, Inc.
  • Michael Merritt, AT&T
  • Lyle Moore, Bell Canada
  • John Morris, Center for Democracy and Technology
  • Jean-Francois Mule, Cablelabs
  • David Oran, Cisco Systems
  • Reinaldo Penno, Juniper Networks
  • Jon Peterson, NeuStar
  • Howard Pfeffer, Time Warner Cable
  • Laird Popkin, Pando Networks
  • Stefano Previdi, Cisco systems
  • Satish Putta
  • Eric Pescorla
  • Benny Rodrig, Avaya
  • Damien Saucez, UCLouvain (UCL)
  • Henning Schulzrinne, Columbia University
  • Michael Sheehan, Juniper Networks
  • Don Shulzrinne, Immedia Semiconductor
  • David Sohn, Center for Democracy and Technology
  • Martin Stiemerling, NEC
  • Clint Summers, Cox Communications
  • Robert Topolski
  • Mark Townsley, Cisco Systems
  • Yushun Wang, Microsoft
  • Hao Wang, Yale University
  • Ye Wang, Yale University
  • David Ward, Cisco
  • Nicholas Weaver, ICSI
  • Daniel Weitzner, MIT
  • Magnus Westerlund, Ericsson
  • Thomas Woo, Bell Labs
  • Steve Worona, EDUCAUSE
  • Richard Woundy, Comcast
  • Haiyong Xie
  • Richard Yang, Yale University

Appendix C. Workshop Agenda 📋

1.  Welcome/Note Well/Intro Slides
    Cullen Jennings

2.  Service Provider Perspective (Comcast)
    Rich Woundy and Jason Livingood

3.  Application Designer Perspective (BitTorrent)
    Stanislav Shalunov

4.  Lightning Talks & General Discussion
    Robb Topoloski
    Nick Weaver
    Leslie Daigle

5.  Localization and Caches
    Laird Popkin and Haiyong Xie
    Yu-Shun Wang
    Vinay Aggrawal

6.  New Approaches to Congestion
    Bob Briscoe
    Marcin Matuszewski

7.  Quality of Service
    Mary Barnes
    Henning Schulzrinne

8.  Conclusions & Wrap-Up

Appendix D. Slides and Position Papers 🦋

slides and position papers live at http://trac.tools.ietf.org/area/rai/trac/wiki/PeerToPeerInfrastructure 💻

Position papers, in all their glory:

  • Nick Weaver - The case for "Ugly Now" User Fairness
  • Paul Jessop - Position paper of the RIAA
  • Nikloaos Laotaris, Pablo Rodriguez, Laurent Massoulie - ECHOES: Edge Capacity Hosting Overlays of Nano Data Centers
  • Bruce Davie, Stefano Previdi, Jan Medved, Albert Tian - Peer Selection Guidance
  • Marie-Jose Montpetit - Community Networks: getting P2P out of prison - the next steps
  • D. Bryan, S. Dawkins, B. Lowekamp, E. Shim - Infrastructure-related Attributes of App Scenarios for P2PSIP
  • Jiang XingFeng - Analysis of the Service Discovery in DHT network
  • R. Penno - P2P Status and Requirements
  • Patrick Crowley and Shakir James - Symbiotic P2P: Resolving the conflict between ISPs and BitTorrent through mutual cooperation
  • Robb Topolski - Framing Peer to Peer File Sharing
  • M. Stiemerling, S. Niccolini, S. Kiesel, J. Seedorf - A Network Cooperative Overlay System
  • Y. Wang, S. Tan, R. Grove - Traffic Localization with Multi-Layer, Tracker-Based Peer-to-Peer Content Distribution Architecture
  • Haiyong Xie, Y. Richard Yang, Avi Silberschatz, Arvind Krishnamurthy, Laird Popkin - P4P: Provider Portal for P2P Applications
  • Michael Merritt, Doug Pasko, Laird Popkin - Network-Friendly Peer-to-Peer Services
  • Camiant (Jackson) - Camiant Submission
  • Jason Livingood, Rich Woundy - Comcast Submission
  • Benny Rodrig - Enterprise IP Networks and the P2P Traffic Load Impact
  • Ted Hardie - Peer-to-Peer traffic and "Unattended Consequences"
  • Jiang XingFeng, Ning Zong - Content Replication for Internet P2P Applications
  • Sandvine (Dundas) - Analysis of Traffic Demographics in Broadband networks
  • Sandvine (Dundas) - Traffic Management in a World with Network Neutrality
  • Stanislav Shalunov - Users want P2P, we make it work
  • R. Cuevas, A. Cuevas, I. Martinez-Yelmo, C. Guerrero - Internet scale mobility service: a case study on building a DHT based service for ISPs
  • M. Barnes, B. McCormick - Peer to Peer Infrastructure Considerations
  • Henning Schulzrinne - Encouraging Bandwidth Efficiency for Peer-to-Peer Applications
  • Damien Saucez, Benoit Donnet, Olivier Bonaventure, Dimitri Papdimitriou - Towards an Open Path Selection Architecture
  • Eric Rescorla - Notes on P2P Blocking and Evasion
  • Vinay Aggrawal, Anja Feldmann - ISP-Aided Neighbor Selection in P2P Systems
  • Enrico Marocco, Vijay K. Gurbani, Volker Hilt, Ivica Rimac, Marco Tomsu - Peer-to-Peer Infrastructure: A Survey of Research on the Application-Layer Traffic Optimization Problem and the Need for Layer Cooperation
  • Tony Moncaster, Bob Briscoe, Louise Burness - Is There a Problem With Peer-to-peer Traffic?
  • David Sohn, Alissa Cooper - Peer-to-Peer Infrastructure Considerations
  • Bob Briscoe, Lou Burness, Tony Moncaster, Phil Eardley - Solving this traffic management problem... and the next, and the next
  • Hannes Tschofenig, Marcin Matuszewski - Dealing with P2P Traffic in an Operator Network
  • Jean-Francois Mule - CableLabs submission
  • Alan Arolovitch - Peer-to-peer infrastructure: Case for cooperative P2P caching
  • Leslie Daigle - Defining Success: Questions for the Future of the Internet and Bandwidth-Intensive Activities
  • William Check, Rex Bullinger -- NCTA Position Paper
  • Jari Arkko - Incentives and Deployment Considerations for P2PI Solutions

Authors' Addresses 💌

Jon Peterson
NeuStar
USA

EMail: jon.peterson@neustar.biz


Alissa Cooper
Center for Democracy & Technology
1634 Eye St. NW, Suite 1100
Washington, DC  20006
USA

EMail: acooper@cdt.org
📦 where this page comes from