RFC 6897

Multipath TCP (MPTCP) Application Interface Considerations

📅 published March 2013, from ietf

Part 1 of 7 💖

Abstract ✨

Okay so MPTCP (Multipath TCP) adds the ability to use multiple paths for a regular TCP session 🦋 It's designed to be totally backward compatible with apps, but the data transport is different from regular TCP, and there are a bunch of extra degrees of freedom apps might want to exploit 👀 This doc sums up the impact MPTCP may have on apps, like changes in performance 📈 It also talks about compatibility issues when MPTCP is used with apps that aren't MPTCP-aware 🙈 And finally, it describes a basic application interface that's a simple extension of TCP's interface for MPTCP-aware apps 💅

Document Header 🎀

This is RFC 6897, titled Multipath TCP (MPTCP) Application Interface Considerations, from the IETF (Internet Engineering Task Force) 🌈 It was written by M. Scharf of Alcatel-Lucent Bell Labs and A. Ford of Cisco, and it's dated March 2013 🍓 The category is Informational and the ISSN is 2070-1721 ✨ The header doesn't list any RFCs that it obsoletes or updates.

Status of This Memo 🧚

This doc is not an Internet Standards Track specification. It's published just for informational purposes 💜

It's a product of the IETF, so it represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). Not every document the IESG approves is a candidate for any level of Internet Standard, so see Section 2 of RFC 5741 for that 🫶

You can find info about the current status of this doc, any errata, and how to send feedback at http://www.rfc-editor.org/info/rfc6897 🔥

Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved 💅

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, because they describe your rights and restrictions regarding this document 👑 Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License 🦋

Table of Contents 📚

1. Introduction ....................................................3
2. Terminology .....................................................4
3. Comparison of MPTCP and Regular TCP .............................5
   3.1. Effect on Performance ......................................5
        3.1.1. Throughput ..........................................5
        3.1.2. Delay ...............................................6
        3.1.3. Resilience ..........................................7
   3.2. Potential Problems .........................................8
        3.2.1. Impact of Middleboxes ...............................8
        3.2.2. Dealing with Multiple Addresses inside
               Applications ........................................9
        3.2.3. Security Implications ..............................10
4. Operation of MPTCP with Legacy Applications ....................10
   4.1. Overview of the MPTCP Network Stack .......................10
   4.2. Address Issues ............................................11
        4.2.1. Specification of Addresses by Applications .........11
        4.2.2. Querying of Addresses by Applications ..............12
   4.3. MPTCP Connection Management ...............................13
        4.3.1. Reaction to Close Call by Application ..............13
        4.3.2. Other Connection Management Functions ..............13
   4.4. Socket Option Issues ......................................13
        4.4.1. General Guideline ..................................13
        4.4.2. Disabling of the Nagle Algorithm ...................13
        4.4.3. Buffer Sizing ......................................14
        4.4.4. Other Socket Options ...............................14
   4.5. Default Enabling of MPTCP .................................14
   4.6. Summary of Advice to Application Developers ...............15

5. Basic API for MPTCP-Aware Applications .........................15
   5.1. Design Considerations .....................................15
   5.2. Requirements on the Basic MPTCP API .......................16
   5.3. Sockets Interface Extensions by the Basic MPTCP API .......17
        5.3.1. Overview ...........................................17
        5.3.2. Enabling and Disabling of MPTCP ....................19
        5.3.3. Binding MPTCP to Specified Addresses ...............19
        5.3.4. Querying the MPTCP Subflow Addresses ...............20
        5.3.5. Getting a Unique Connection Identifier .............20
6. Other Compatibility Issues .....................................21
   6.1. Usage of TLS over MPTCP ...................................21
   6.2. Usage of the SCTP Sockets API .............................21
   6.3. Incompatibilities with Other Multihoming Solutions ........21
   6.4. Interactions with DNS .....................................22
7. Security Considerations ........................................22
8. Conclusion .....................................................23
9. Acknowledgments ................................................23
10. References ....................................................24
   10.1. Normative References .....................................24
   10.2. Informative References ...................................24
Appendix A. Requirements on a Future Advanced MPTCP API ...........26
  A.1. Design Considerations ......................................26
  A.2. MPTCP Usage Scenarios and Application Requirements .........27
  A.3. Potential Requirements on an Advanced MPTCP API ............29
  A.4. Integration with the SCTP Sockets API ......................30

1. Introduction 🦋

Okay so MPTCP lets a regular TCP session use multiple paths at once 🛣️ The reasons for this extension include better throughput, better overall resource utilization, and resilience to network failure 💪 Those motivations get discussed, along with the high-level design decisions, in the multipath TCP architecture [4] ✨ MPTCP [5] gives you the same reliable, in-order, byte-stream transport as TCP, and it's designed to be backward compatible with both applications and the network layer 🫶 It requires support inside the network stack of both endpoints, so both sides need it 🔥

This doc goes through the effects MPTCP may have on apps, like performance changes compared to regular TCP 📈 Next, it defines how MPTCP and apps that don't know about the multipath transport work together 🙈 MPTCP is designed to be usable without any app changes, but some compatibility issues still have to be considered 🌈 Then, this memo specifies a basic Application Programming Interface (API) for MPTCP-aware apps 💅 The API here is an extension of the regular TCP API, so an MPTCP-aware app gets the same level of control and access to info about an MPTCP connection that it would get with the standard TCP API on a regular TCP connection 👑

The de facto standard API for TCP/IP apps is the "sockets" interface [8] 🧚 This doc gives an abstract definition of MPTCP-specific extensions to that interface. These are operations an app can use to get or set extra MPTCP-specific info on a socket, so it has the same level of info and control over MPTCP as it does with regular TCP 💜 It's up to the apps, high-level programming languages, or libraries to decide whether to use these optional extensions. For instance, an app may want to turn MPTCP on or off for certain data transfers, or limit its use to certain interfaces 🍓 The abstract spec follows the Portable Operating System Interface (POSIX) standard [8] as much as possible 🎀

An advanced API for MPTCP is outside the scope of this document. Such an API could offer more fine-grained control over multipath transport functions and policies 🔮 The appendix has a brief, non-compulsory list of potential features for that kind of advanced API 🌸

There can be interactions or incompatibilities between MPTCP and other APIs or sockets interface extensions, which get discussed later in this doc 😭 Some network stack implementations, especially on mobile devices, have centralized connection managers or other higher-level APIs to deal with multi-interface issues, as surveyed in [15] 📱 Their interaction with MPTCP is outside the scope of this document 🙅‍♀️

The target readers of this doc are app developers whose software may benefit a lot from MPTCP 💖 This doc also gives developers of MPTCP the info they need to implement the API in a TCP/IP network stack 🛠️

2. Terminology 📖

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [3] 💅

This doc uses the MPTCP terminology introduced in [5] ✨

When it comes to the API towards apps, there are three terms to know 👀

  • Legacy API: The interface towards TCP that apps use right now. This doc explains the effect of MPTCP on such apps, plus the resulting issues 🦕
  • Basic API: A simple extension of TCP's interface for apps that are aware of MPTCP. This doc describes it abstractly. It gives access to multipath address info and a level of control that's equivalent to regular TCP 🎀
  • Advanced API: An API that offers more fine-grained control over the behavior of MPTCP. Its spec is outside the scope of this document 🔮

3. Comparison of MPTCP and Regular TCP 🦋

This section talks about the effect of MPTCP on performance as an app sees it, compared with what you'd expect from regular TCP 📊

3.1. Effect on Performance 🚀

Okay, this is a big one. One of the key goals of adding multipath capability to TCP is to improve a transport connection's performance by spreading load over separate subflows across potentially disjoint paths 🌈 It's also an explicit goal of MPTCP that it provides a connection that performs at least as well as one using single-path TCP 💜 A matching congestion control algorithm is described in [7] 🫶 The next sections sum up the performance effect of MPTCP as an app sees it 👑

3.1.1. Throughput ✨

Okay bestie, the most obvious glow-up MPTCP gives you is more throughput 🚀, because it can pool more than one path (where available) between two endpoints. This usually gives an application as much bandwidth or more 💅, although there can be exceptions, e.g., due to differences in the congestion control dynamics. Like, if a new subflow just started, the short-term throughput can be smaller than the theoretical optimum 🥲. If the flows share bottlenecks, the congestion control algorithms will in most cases make sure the load is evenly spread between regular and multipath TCP sessions, so no end user gets worse performance than if everyone were using single-path TCP 🫶. There are some known corner cases where an upgrade to MPTCP can affect other users [21].

This performance boost also means an MPTCP session could reach throughput greater than the capacity of a single interface on the device 🤯. If any applications make assumptions about interfaces due to throughput, they must take this into account (although an MPTCP implementation must always respect an application's request for a particular interface).

MPTCP can add and remove subflows as paths change availability, so connection bandwidth could vary more and change more often 🌊. Apps that adapt to available bandwidth (such as video and audio streaming) may need to adjust some of their assumptions to handle this most effectively 📺.

Carrying MPTCP signaling info comes with a small overhead 🥱. So using MPTCP instead of a single TCP connection results in a smaller goodput. Also, if multiple subflows share the same bottleneck, this overhead slightly reduces the capacity available for data transport. Yet this reduction will be negligible in many usage scenarios, and the protocol has optimizations in its design so the overhead stays minimal ✨.

3.1.2. Delay ⏱️

The benefits of MPTCP for throughput and resilience may come at some cost to data delivery delay and delay jitter 😬.

If the delays on the constituent subflows of an MPTCP connection differ, the jitter an application perceives may look higher, since the data gets spread across the subflows. Although MPTCP ensures in-order delivery to the application, delivery could be more bursty than usual with single-path TCP, especially on highly asymmetric paths 🌪️.

Applications with high real-time requirements might be affected by this 📞. One possible remedy is to disable MPTCP for such jitter-sensitive applications, either by using the basic API defined in this document or by other means, like system policies 🙅‍♀️.

The actual delay and jitter of data transport over MPTCP depend on the scheduling and congestion control algorithms used for sending data, plus the heuristics used to establish and shut down subflows 🧠. A sender can implement strategies to minimize the delay jitter seen by applications, but this requires an accurate estimation of the path characteristics. If the scheduling decisions are suboptimal, or if assumptions about the path characteristics turn out to be wrong, delay jitter may increase and affect delay-sensitive applications 😵. In general, for a delay-sensitive application, it would be desirable to select an appropriate congestion control algorithm for its traffic needs 🎯.

Alternatively, MPTCP could be used in high-reliability rather than high-throughput modes, such as by mirroring traffic on subflows, or by only using additional subflows for hot standby 🛟. These traffic scheduling methods would not cause delay variation in the same way. These additional modes, and the selection of alternative scheduling algorithms, would need to be indicated by an advanced API. The specification of that API requires further analysis and is outside the scope of this document 📝.

If data transport on one subflow fails, the retransmissions inside MPTCP could affect the delivery delay to the application 🙃. Yet without MPTCP, that data or the whole connection might have been lost, and other reliability mechanisms (e.g., application-level recovery) would likely have an even larger delay impact 💀.

Applications that make round-trip time (RTT) estimates at the application level may have some issues 🤔. Whilst the average delay calculated will be accurate, whether this is useful depends on what the application needs it for. If a new application wishes to derive such information, it should consider how multiple subflows may affect its measurements, and how it may wish to respond. In such a case, an application may wish to express its scheduling preferences, as described later in this document 💅.

3.1.3. Resilience 🛡️

Another performance improvement from MPTCP is better resilience 💪. Using multiple subflows at once means that if one fails, all traffic moves to the remaining subflow(s), and any lost packets can be retransmitted on those subflows.

As one special case, MPTCP can be used with only one active subflow at a given point in time. In that case, resilience compared to single-path TCP is still improved 🌈. MPTCP also supports make-before-break and break-before-make handovers between subflows. In both cases, the MPTCP connection can survive an unavailability or change of an IP address (e.g., due to shutdown of an interface or handover). MPTCP closes or resets the MPTCP connection separately from the individual subflows, as described in [5] 🦋.

Subflow failure may be caused by issues within the network, which an application would be unaware of, or by interface failure on the node 👻.

An application may, under certain circumstances, be in a position to be aware of such failure (e.g., by radio signal strength, or simply an interface enabled flag), so it must not make assumptions about an MPTCP flow's stability based on this. An MPTCP implementation must never override an application's request for a given interface, however, so the cases where this issue may apply are limited 🙈.

3.2. Potential Problems 🚧

3.2.1. Impact of Middleboxes 📦

MPTCP has been designed to pass through the majority of middleboxes 🫡. Empirical evidence suggests that new TCP options can successfully be used on most paths in the Internet [22]. Nevertheless, some middleboxes may still refuse to pass MPTCP messages due to the presence of TCP options, or they may strip TCP options. If this is the case, MPTCP falls back to regular TCP. Although this won't create a problem for the application (its communication will be set up either way), there may be additional, and honestly user-perceivable, delay while the first handshake fails 😩. So an alternative approach could be to try both MPTCP and regular TCP connection attempts at the same time, and respond to whichever replies first, in a fashion similar to the "Happy Eyeballs" mechanism for IPv6 [16]. One could also apply a shorter timeout on the MPTCP attempt, and thus reduce the setup delay if fallback to regular TCP is needed 🏃‍♀️.

An MPTCP implementation can learn the rate of MPTCP connection attempt successes or failures to particular hosts or networks, and on particular interfaces, and could therefore learn heuristics for when and when not to use MPTCP 🧩. A detailed discussion of the various fallback mechanisms, for failures occurring at different points in the connection, is presented in [5]. It must be emphasized that all such heuristics could also fail, and learning can be difficult in certain environments, e.g., if the host is mobile 📱.

There may also be middleboxes that transparently change the length of content. If such middleboxes are present, MPTCP's reassembly of the byte stream in the receiver is difficult. Still, MPTCP can detect such middleboxes and then fall back to regular TCP 🔍. An overview of the impact of middleboxes is presented in [4], and MPTCP's mechanisms to work around these issues are presented and discussed in [5].

MPTCP can also have other unexpected implications. For instance, intrusion detection systems could be triggered 🚨. A full analysis of MPTCP's impact on such middleboxes is for further study after deployment experiments 🧪.

3.2.2. Dealing with Multiple Addresses inside Applications 🏠

In regular TCP, there is a one-to-one mapping of the sockets interface to a flow through a network. Since MPTCP can make use of multiple subflows, applications cannot implicitly rely on this one-to-one mapping any more 😤.

Whilst this doesn't matter for most applications, some applications may need to adapt to the presence of multiple addresses, because their implicit assumptions are outdated 🕰️. In this section, selected examples of the resulting issues are discussed. Whether such implicit assumptions matter is an application-level decision, and this document only provides general guidance and a basic API to retrieve relevant information 🤓.

A few applications require the transport to be along a single path. They can disable the use of MPTCP, as described later in this document 🛣️. Examples include monitoring tools that want to measure the available bandwidth on a path, or routing protocols such as BGP that require the use of a specific link.

Certain applications store the IP addresses of TCP connections, e.g., by logging mechanisms. Such logging mechanisms will continue to work with MPTCP, but two important aspects have to be mentioned 📚. First, if the application is not aware of MPTCP, it will use the existing interface to the network stack. This implies that an MPTCP-unaware application will track the IP addresses of the first subflow only, and IP addresses used by follow-up subflows will be ignored 👀. Second, an MPTCP-aware application can use the basic API described in this document to monitor the IP addresses of all subflows, e.g., for logging mechanisms. If an MPTCP connection uses several subflows, this will possibly imply that data structures have to be adapted, and that the amount of data that has to be logged and stored per connection will increase 💾.

An MPTCP implementation may choose to maintain an MPTCP connection even if the IP address of the original subflow is no longer allocated to a host, depending on the policy concerning the first subflow (fate-sharing; see Section 4.2.2). In this case, the IP address exposed to an MPTCP-unaware application can differ from the addresses actually being used by MPTCP. It is even possible that the IP address gets assigned to another host during the lifetime of an MPTCP connection 😳. As further discussed below, this could be an issue if the IP addresses are exchanged by applications, e.g., inside the application protocol. This issue can be addressed by enabling fate-sharing, at the cost of resilience, because the MPTCP connection then cannot close the initial subflow 🫣.

3.2.3. Security Implications ✨

okay so bestie, letting multiple IP addresses share one MPTCP connection can open up some extra security holes 🙈 like attackers could try to hijack connections 💀. BUT the protocol design of MPTCP is built to keep that risk low 🫶. An attacker sitting on one of the paths can cause harm, sure, but that's hardly an additional security risk compared to single-path TCP, which is already vulnerable to man-in-the-middle attacks 😭. A detailed threat analysis of MPTCP is published in [6] if you want the deep dive 🔍.

Impact on Transport Layer Security (TLS) is discussed in Section 6.1 🔐.

4. Operation of MPTCP with Legacy Applications 🎀

4.1. Overview of the MPTCP Network Stack 🧚

MPTCP is an extension of TCP, but it's designed to be backward compatible for legacy (MPTCP-unaware) applications 💖. TCP talks to other parts of the network stack through different interfaces 🌸. The de facto standard API between TCP and applications is the sockets interface 💅. Here's where MPTCP sits in the stack, shown in Figure 1 ✨:

                 +-------------------------------+
                 |           Application         |
                 +-------------------------------+
                        ^                  |
              ~~~~~~~~~~|~Sockets Interface|~~~~~~~~~
                        |                  v
                 +-------------------------------+
                 |             MPTCP             |
                 + - - - - - - - + - - - - - - - +
                 | Subflow (TCP) | Subflow (TCP) |
                 +-------------------------------+
                 |       IP      |      IP       |
                 +-------------------------------+

                  Figure 1: MPTCP Protocol Stack

In general, MPTCP can affect every interface that assumes a TCP connection is tied to a single IP address and TCP port pair 🦋. It can also affect anything that assumes one socket endpoint, one network interface, or one given path through the network 👑.

This means there are two classes of applications 👯‍♀️:

  • Legacy applications: These applications are unaware of MPTCP and use the existing API towards TCP without any changes 💅. This is the default case 🌈.
  • MPTCP-aware applications: These applications indicate support for an enhanced MPTCP interface ⭐. This document specifies a minimum set of API extensions for such applications 🍓.

In the following sections, we're looking at how much MPTCP affects legacy applications that use the existing sockets API 🧐. The existing sockets API means applications deal with data structures that store, amongst other things, the IP addresses and TCP port numbers of a TCP connection 📦. A design goal of MPTCP is that legacy applications can keep using the established sockets API without any changes 🥹. BUT, in MPTCP there's a one-to-many mapping between the socket endpoint and the subflows 😳. That has several subtle implications for legacy applications that use sockets API functions 🙃.

4.2. Address Issues 🏡

4.2.1. Specification of Addresses by Applications 📍

During binding, an application can either select a specific address or bind to INADDR_ANY 🎯. On some systems, other socket options (e.g., SO_BINDTODEVICE) can also bind to a specific interface 🔌. If an application uses a specific address or binds to a specific interface, then MPTCP MUST respect this and not interfere with the application's choices 🙅‍♀️. Binding to a specific address or interface means the application isn't aware of MPTCP, so it will disable the use of MPTCP on that connection 🚫. An application that wants to bind to a specific set of addresses with MPTCP must use multipath-aware calls to do it (as described in Section 5.3.3) 🛠️.

If an application binds to INADDR_ANY, it's assumed the application doesn't care which addresses are used locally 🤷‍♀️. In this case, a local policy MAY allow MPTCP to automatically set up multiple subflows on that connection 🌊.

The basic sockets API of MPTCP-aware applications lets them express further preferences in an MPTCP-compatible way (e.g., binding to a subset of interfaces only) 💜.

4.2.2. Querying of Addresses by Applications 🔍

Applications can use the getpeername() or getsockname() functions to retrieve the IP address of the peer or of the local socket 📡. These functions can be used for all sorts of things, including security mechanisms, geo-location, or interface checks 🗺️. The sockets API was designed on the assumption that a socket uses just one address 💭. Since that address is visible to the application, the application may assume the information these functions return stays the same for the lifetime of a connection 🧸. BUT, in MPTCP, unlike in TCP, a connection maps one-to-many onto subflows, and subflows can be added and removed while the connection keeps existing 😵‍💫. Since the subflow addresses can change, MPTCP cannot expose addresses through getpeername() or getsockname() that are both valid and constant for the connection's lifetime 💔.

This problem is handled as follows 📝: If used by a legacy application, the MPTCP stack MUST always return the addresses and port numbers of the first subflow of an MPTCP connection, in all circumstances, even if that particular subflow is no longer in use 🥺.

Because the addresses may not be valid anymore once the first subflow is closed, the MPTCP stack MAY close the whole MPTCP connection when the first subflow is closed (i.e., fate-sharing between the initial subflow and the MPTCP connection as a whole) 🤝. This fate-sharing avoids reusing the pair of IP addresses and ports while an MPTCP connection is still in progress 🛡️. The catch is that it reduces the usefulness of MPTCP if the IP addresses of the first subflow aren't available any more (e.g., mobility events) 📉. Whether to close the whole MPTCP connection by default SHOULD be controlled by a local policy 🎛️. Further experiments are needed to look into its implications 🧪.

The functions getpeername() and getsockname() SHOULD also always return the addresses of the first subflow if the socket is used by an MPTCP-aware application 💁‍♀️. This keeps things consistent with MPTCP-unaware applications, and also with the Stream Control Transmission Protocol (SCTP) 🌟. Instead of getpeername() or getsockname(), MPTCP-aware applications can use the new API calls described in Section 5.3 to retrieve the full list of address pairs for the subflows in use 🙌.

4.3. MPTCP Connection Management 🎀

4.3.1. Reaction to Close Call by Application 👋

As described in [5], MPTCP distinguishes between closing subflows (by TCP FIN) and closing the whole MPTCP connection (by Data FIN) 🧠.

When an application closes a socket, e.g., by calling the close() function, this means the application has no more data to send, just like with single-path TCP 💌. MPTCP will then close the MPTCP connection via Data FIN messages 🎬. This is completely transparent to the application 🫧.

To sum up, the semantics of the close() interface for applications are unchanged compared to TCP 💯.

4.3.2. Other Connection Management Functions 🧩

In general, an MPTCP connection is maintained separately from individual subflows 🪢. MPTCP therefore has internal mechanisms to establish, close, or reset the MPTCP connection [5] 🔧. These mechanisms provide functions equivalent to single-path TCP and can be mapped accordingly 🗂️. So these MPTCP internals don't affect the application interface 🫶.

4.4. Socket Option Issues ⚙️

4.4.1. General Guideline 📚

The existing sockets API includes options that change how sockets and their underlying communications protocols behave 🎚️. Various socket options exist at the socket, TCP, and IP levels 🪜. The value of an option can usually be set with the setsockopt() system function, and the getsockopt() function gets information 📬. In general, the existing sockets interface functions can't configure each MPTCP subflow individually 😬. To stay backward compatible, existing APIs SHOULD therefore apply to all subflows within one connection, as far as possible 🧵.

4.4.2. Disabling of the Nagle Algorithm 🍼

One commonly used TCP socket option (TCP_NODELAY) disables the Nagle algorithm as described in [2] 🏃‍♀️. This option is also specified in the POSIX standard [8] 📜. Applications can use this option with MPTCP in exactly the same way 💕. It then SHOULD disable the Nagle algorithm for the MPTCP connection, i.e., all subflows 🚀.

In addition, the MPTCP protocol instance MAY use a different path scheduler algorithm if TCP_NODELAY is present 🗓️. For instance, it could use an algorithm optimized for latency-sensitive traffic (for instance, only transmitting on one path) ⚡. Specific algorithms are outside the scope of this document 🙈.

4.4.3. Buffer Sizing 📦

Applications can explicitly configure send and receive buffer sizes through the sockets API (SO_SNDBUF, SO_RCVBUF) 🪣. These socket options can also be used with MPTCP, and then they affect the buffer size of the MPTCP connection 🌀. BUT when defining buffer sizes, application programmers should take into account that transport over several subflows needs a certain amount of buffer for resequencing in the receiver 🧮. MPTCP may also need more storage space in the sender, especially if retransmissions are sent over more than one path 📈. Also, very small send buffers may stop MPTCP from scheduling data efficiently across different subflows 😵. So it doesn't make sense to use MPTCP with small send or receive buffers 🚫.

An MPTCP implementation MAY set a lower bound for send and receive buffers, and treat a small buffer size request as an implicit request not to use MPTCP 🫠.

4.4.4. Other Socket Options 🧃

TCP features the ability to send "Urgent" data, but its use is not recommended in general, and specifically not with MPTCP [4] 🚨.

Some network stacks may provide additional implementation-specific socket options or interfaces that affect TCP's behavior 🧪. In such cases, implementers must ensure these options don't interfere with the MPTCP interface 🫡.

4.5. Default Enabling of MPTCP 🌈

Whether a network stack automatically enables MPTCP for sockets, even without any explicit sign of MPTCP awareness from the application, is up to a local policy at the end system 🖥️. This choice may even be under the user's control through system preferences ⚙️.

Enabling MPTCP, whether by the application or by system defaults, does not necessarily mean MPTCP will always be used 🙃. Both endpoints must support MPTCP, and there must be multiple addresses at at least one endpoint, for MPTCP to be used 📍. Even when those requirements are met, MPTCP may not be used right away on a connection ⏳. It may make sense to bring multiple paths into operation only after a given period of time, or if the connection is saturated 🌊✨.

4.6. Summary of Advice to Application Developers 💅

  • Using the default MPTCP configuration: okay bestie, like TCP, MPTCP is designed to be efficient and robust in the default configuration ✨. Application developers should not explicitly configure TCP (or MPTCP) features unless this is really needed 🫶.
  • Socket buffer dimensioning: multipath transport needs larger buffers in the receiver for resequencing, as we already covered 🧠. Applications should use reasonable buffer sizes (such as the operating system default values) in order to fully benefit from MPTCP. A full discussion of buffer sizing issues is given in [5] 📚.
  • Facilitating stack-internal heuristics: the path management and data scheduling by MPTCP is handled by stack-internal algorithms that may implicitly try to self-optimize their behavior according to assumed application needs 🤖. For instance, an MPTCP implementation may use heuristics to determine whether an application requires delay-sensitive or bulk data transport, using, for instance, port numbers, the TCP_NODELAY socket options, or the application's read/write patterns as input parameters 👀. An application developer can help these heuristics out by avoiding atypical interface use cases 💖. For instance, for long bulk data transfers, it does not make sense to enable the TCP_NODELAY socket option, nor is it reasonable to use many small socket send() calls each with small amounts of data only 🙈.

5. Basic API for MPTCP-Aware Applications ✨

5.1. Design Considerations 🎀

While applications can use MPTCP with the unmodified sockets API, multipath transport comes with lots of degrees of freedom 🌈. MPTCP manages the data transport over different subflows automatically. By default, this is totally transparent to the application, but an application could use an additional API to interface with the MPTCP layer and control important aspects of the MPTCP implementation's behavior 🧚.

This document describes a basic MPTCP API. The API contains a minimum set of functions that provide an equivalent level of control and information as exists for regular TCP. It maintains backward compatibility with legacy applications 💜.

An advanced MPTCP API is outside the scope of this document. The basic API does not allow a sender or a receiver to express preferences about the management of paths or the scheduling of data, even if this can have a significant performance impact and if an MPTCP implementation could benefit from additional guidance by applications 😭. A list of potential further API extensions is provided in the appendix. The specification of such an advanced API is for further study and may partly be implementation-specific 🔮.

MPTCP mainly affects the sending of data. But a receiver may also have preferences about data transfer choices, and it may have performance requirements as well. Yet, configuring such preferences is outside the scope of the basic API 🙃.

5.2. Requirements on the Basic MPTCP API 📋

Because the sockets interface is so important, there are several fundamental design objectives for the basic interface between MPTCP and applications 👑:

  • Consistency with existing sockets APIs must be maintained as far as possible. To support the huge base of applications using the original API, a legacy application must be able to keep using standard sockets interface functions when run on a system supporting MPTCP. Also, MPTCP-aware applications should be able to access the socket without any major changes 🫶.
  • Sockets API extensions must be minimized and independent of an implementation 🧼.
  • The interface should handle both IPv4 and IPv6 🌸.

Here's the list of core requirements for the basic API 🔥:

REQ1: Turn on/off MPTCP: An application should be able to request to turn on or turn off the usage of MPTCP. This means that an application should be able to explicitly request the use of MPTCP if this is possible. Applications should also be able to request not to enable MPTCP and to use regular TCP transport instead. This can be implicit in many cases, since MPTCP must be disabled by the use of binding to a specific address. MPTCP may also be enabled if an application uses a dedicated multipath address family (such as AF_MULTIPATH [20]) ✨.

REQ2: An application should be able to restrict MPTCP to binding to a given set of addresses 📍.

REQ3: An application should be able to obtain information on the pairs of addresses used by the MPTCP subflows 🗺️.

REQ4: An application should be able to extract a unique identifier for the connection (per endpoint) 🆔.

REQ1 is the most important one, because some applications could benefit a lot from MPTCP, but there are also cases where it hardly makes sense 🤷‍♀️. The existing sockets API already has similar mechanisms to enable or disable advanced TCP features. REQ2 matches up with binding addresses using the bind() socket call, or explicit device bindings with a SO_BINDTODEVICE option 🔗. REQ3 makes sure there's an equivalent to getpeername() or getsockname() that can handle more than one subflow 💅. Finally, the application should be able to retrieve a unique connection identifier (local to the endpoint it's running on) for the MPTCP connection 🦋. This replaces the (address, port) pair as the connection identifier in single-path TCP, which is no longer static in MPTCP 🥹.

An application can keep using getpeername() or getsockname() alongside the basic MPTCP API. As already explained, both functions return the corresponding addresses of the first subflow 🌟.

5.3. Sockets Interface Extensions by the Basic MPTCP API 🧩

5.3.1. Overview 👀

The abstract, basic MPTCP API is a set of new values associated with an MPTCP socket 🎁. These values can be used to change properties of an MPTCP connection or to retrieve information. They could be accessed through new symbols on existing calls such as setsockopt() and getsockopt(), or they could be implemented as entirely new function calls. This implementation decision is out of scope for this document 🙌. The following list presents symbolic names for these MPTCP socket settings:

  • TCP_MULTIPATH_ENABLE: Enable/disable MPTCP
  • TCP_MULTIPATH_ADD: Bind MPTCP to a set of given local addresses, or add a set of new local addresses to an existing MPTCP connection
  • TCP_MULTIPATH_REMOVE: Remove a local address from an MPTCP connection
  • TCP_MULTIPATH_SUBFLOWS: Get the pairs of addresses currently used by the MPTCP subflows
  • TCP_MULTIPATH_CONNID: Get the local connection identifier for this MPTCP connection

Table 1 is a list of the abstract socket operations for the basic configuration of MPTCP 📊. The first column gives the symbolic name of the operation. The second and third columns show whether the operation provides values to be read ("Get") or takes values to configure ("Set"). The fourth column lists the type of data associated with the operation. The data types are listed for information only. In addition to IP addresses, an application MAY also indicate TCP port numbers, as detailed below 🍓.

+------------------------+-----+-----+------------------------------+
| Name                   | Get | Set |           Data type          |
+------------------------+-----+-----+------------------------------+
| TCP_MULTIPATH_ENABLE   |  o  |  o  |           boolean            |
| TCP_MULTIPATH_ADD      |     |  o  |      list of addresses       |
|                        |     |     |         (and ports)          |
| TCP_MULTIPATH_REMOVE   |     |  o  |      list of addresses       |
|                        |     |     |         (and ports)          |
| TCP_MULTIPATH_SUBFLOWS |  o  |     |  list of pairs of addresses  |
|                        |     |     |         (and ports)          |
| TCP_MULTIPATH_CONNID   |  o  |     |           integer            |
+------------------------+-----+-----+------------------------------+

                     Table 1: MPTCP Socket Operations

There are restrictions on when these new socket operations can be used 🚦:

  • TCP_MULTIPATH_ENABLE: This value should only be set before the establishment of a TCP connection. Its value should only be read after the establishment of a connection 🔒.
  • TCP_MULTIPATH_ADD: This operation can be applied both before connection setup and during a connection. If used before, it controls the local addresses that an MPTCP connection can use. In the latter case, it allows MPTCP to use an additional local address, if there has been a restriction before connection setup 🏠.
  • TCP_MULTIPATH_REMOVE: This operation can be applied both before connection setup and during a connection. In both cases, it removes an address from the list of local addresses that may be used by subflows 👋.
  • TCP_MULTIPATH_SUBFLOWS: This value is read-only and can only be used after connection setup 📖.
  • TCP_MULTIPATH_CONNID: This value is read-only and should only be used after connection setup 🪪.

5.3.2. Enabling and Disabling of MPTCP 🔛

An application can explicitly indicate multipath capability by setting TCP_MULTIPATH_ENABLE to the value "true". In this case, the MPTCP implementation SHOULD try to negotiate MPTCP for that connection. Note that multipath transport won't necessarily be enabled, because it requires support at both end systems, no middleboxes on the path that would block any additional signaling, and at least one endpoint with multiple addresses 🧐.

Building on the backward compatibility specified in Section 4.2.1, if an application enables MPTCP but binds to a specific address or interface, MPTCP MUST be enabled, but MPTCP MUST respect the application's choice and only use addresses that are explicitly provided by the application 🎯. Note that an application could use the legacy bindings and then expand on them using TCP_MULTIPATH_ADD. Also note that more than one local address can be initially available to MPTCP in this case, if the application has bound to a specific interface with multiple addresses 🌷.

An application can disable MPTCP by setting TCP_MULTIPATH_ENABLE to a value of "false". In that case, MPTCP MUST NOT be used on that connection 🚫.

After connection establishment, an application can get the value of TCP_MULTIPATH_ENABLE. A value of "false" then means lack of MPTCP support. A value of "true" means that MPTCP is supported 💖.

5.3.3. Binding MPTCP to Specified Addresses 🎀

okay bestie, so before the connection even gets set up, an app can use TCP_MULTIPATH_ADD to say "hey, these are the local IP addresses MPTCP is allowed to bind to" 💅. You pass in a list of addresses in a matching data structure. And because of that, this call also decides which addresses can get advertised to the peer through MPTCP signaling 🦋.

If an app binds to a specific address or interface, it doesn't have to use TCP_MULTIPATH_ADD for that address too 🫶. But here's the key part: in that case MPTCP MUST only use the explicitly specified addresses, as Section 5.3.2 explains 🔥.

An app MAY also give a TCP port number. If it does, MPTCP MUST attempt to bind to that port 🌸. That port MAY be different from the one the existing subflows are using ✨. If the app doesn't give a port, the MPTCP implementation picks one automatically, and it is RECOMMENDED that the same port gets used across all subflows 💜.

You can also use this operation to change the address list while an MPTCP connection is already running 🧚. In that case it adds more local addresses the connection can use, and those addresses can be signaled to the peer. Just know that this is only a hint, and an MPTCP implementation MAY select only a subset of the addresses 🍓.

TCP_MULTIPATH_REMOVE lets you take a local address (or a bunch of them) out of an MPTCP connection 🚪. MPTCP MUST close any subflows that were using the address being removed, and it MUST signal the removal to the peer 💀. If other paths are available from the supplied address list but MPTCP isn't using them yet, an MPTCP implementation SHOULD establish alternative subflows before it removes the address 🌈.

Also, remember that these operations SHOULD support both IPv4 and IPv6 addresses, even in the same call 👑.

5.3.4. Querying the MPTCP Subflow Addresses 🔍

An app can ask which addresses the currently established subflows in an MPTCP connection are using, through the read-only TCP_MULTIPATH_SUBFLOWS operation 💖.

What you get back is a list of pairs of tuples, where each tuple has an IP address and a TCP port number 🌷. In each pair, the first tuple is the local IP address and local TCP port, and the second is the remote IP address and remote TCP port that the subflow uses 🫧. The list MUST only include established subflows 🙈. And both addresses in each pair MUST be either IPv4 or IPv6, never a mix ✨.

5.3.5. Getting a Unique Connection Identifier 🪪

Sometimes an app wants a unique ID for its connection, kind of like the (address, port) pair you'd use with regular TCP 💅. For that, it can query TCP_MULTIPATH_CONNID to get a local connection identifier for the MPTCP connection 🦋.

This SHOULD be an integer number, and it SHOULD be locally unique. For example, the MPTCP token works 🎀.

6. Other Compatibility Issues 🧩

6.1. Usage of TLS over MPTCP 🔐

Transport Layer Security (TLS) [17] can run on top of MPTCP's basic API 🫶. Here's the catch: when TLS checks any addresses MPTCP uses against names or addresses in X.509 certificates [18] [19], it MUST only compare them with the address MPTCP used to start the initial subflow, as presented to TLS 🙈. The addresses on later subflows don't need to be checked against any TLS certificate info 💅. If you want finer-grained control, you'd need an advanced API or proactive subflow management through the basic API ✨.

6.2. Usage of the SCTP Sockets API 🧃

For multihoming, a bunch of sockets API extensions have been defined for SCTP [13] 🌸. Since MPTCP also does multipath transport from and to multihomed end systems, some of those interface function calls actually apply to MPTCP too, just in a similar way 🦋.

API developers might want to mash SCTP and MPTCP calls together so the app gets one consistent interface 🫶. BUT it has to be said: the transport service MPTCP gives you is different from SCTP's, which is why not every SCTP API function maps straight over to MPTCP 😭. Also, a network stack that does MPTCP doesn't necessarily support SCTP or its sockets interface extensions 🌈. So the basic MPTCP API only defines extra socket options, which are a backward-compatible extension of TCP's application interface 💜. Integrating with the SCTP API is outside the scope of the basic API 🧚.

6.3. Incompatibilities with Other Multihoming Solutions 🚧

MPTCP can clash with some related sockets API extensions 💀. A multihoming shim layer conflicts with multipath transport like MPTCP or SCTP [11], so be careful that MPTCP doesn't step on the toes of the overlapping features in other APIs 👑:

  • SHIM API [11]: This API specifies sockets API extensions for the multihoming shim layer.
  • HIP API [12]: The Host Identity Protocol (HIP) also results in a new API.
  • API for Mobile IPv6 [10]: For Mobile IPv6, a significantly extended sockets API exists as well (in addition to API extensions for IPv6 [9]).

To avoid conflicts, multiaddressed MPTCP SHOULD NOT be enabled if a network stack uses SHIM6, HIP, or Mobile IPv6 🙅‍♀️. Apps also shouldn't try to use both the MPTCP API and another multihoming or mobility layer API at the same time 🫠.

That said, some MPTCP functionality, like congestion control, could maybe be used in a SHIM6 or HIP environment 🔮. Whether that works is for further study 📚.

6.4. Interactions with DNS 🌐

In multihomed or multiaddressed environments there are a bunch of issues that aren't specific to MPTCP but still need to be considered 🙃. They're summarized in [14] 📝.

DNS can be involved too 🧠. An app is expected to loop over the list of addresses returned from a call like getaddrinfo() 💅. But MPTCP itself MUST NOT make any assumptions that multiple A or AAAA records from the same DNS query point to the same host 🚫. That's because multiple addresses might point to multiple servers for load-balancing 😭.

7. Security Considerations 🔒

This document first defines how the standard TCP/IP API behaves for MPTCP-unaware apps 📱. In general, turning on MPTCP has some security implications for apps. Those are introduced in Section 5.3.3, and the threats are detailed further in [6] 🕵️‍♀️. The MPTCP protocol spec [5] defines several mechanisms to protect MPTCP against those attacks 🛡️.

The syntax and semantics of the API for MPTCP-unaware apps don't change 🙌. But the assumptions non-MPTCP-aware apps might make about the data the backward-compatible API returns are discussed in Section 4.2.2 👀. System admins may want to turn off MPTCP for certain apps that signal addresses, or that make security decisions (like opening firewall holes) based on responses to those queries 🚨.

The basic MPTCP API for MPTCP-aware apps also defines functions that give an equivalent level of control and information as regular TCP does 💜. This document doesn't mandate one specific implementation of the basic MPTCP API 🧩. The implementation should be designed so it doesn't affect memory management assumptions in existing code 🧠. Implementors should keep in mind that the data structures will be more complex than standard TCP's, for example when multiple subflow addresses have to be stored 📦. When handling those structures, be careful not to add security vulnerabilities to apps 😱.

The new functions that add and remove local addresses from an MPTCP connection (TCP_MULTIPATH_ADD and TCP_MULTIPATH_REMOVE) don't add security threats as long as the MPTCP stack verifies that the addresses the app provides really are available as source addresses for subflows ✅.

HOWEVER, apps should use TCP_MULTIPATH_ADD with care, since new subflows might get established to those addresses 🫣. It could also cause some form of information leakage, because MPTCP might advertise those addresses to the other connection endpoint, which could then learn IP addresses of interfaces that aren't otherwise visible 🕳️.

Using different addresses should not be assumed to mean using different paths, especially for security purposes ⚠️.

MPTCP-aware apps should also be careful when querying and using info about the addresses subflows use (TCP_MULTIPATH_SUBFLOWS) 🧐. Since MPTCP can open and close subflows dynamically, a list of addresses queried once can get outdated during the lifetime of an MPTCP connection 🥀. The list might then contain invalid entries, meaning addresses that aren't used anymore or that might not even be assigned to that host anymore 😵. Apps that want to make sure MPTCP only uses a certain set of addresses should explicitly bind to those addresses 🔐.

One specific example is running TLS on top of MPTCP. You can find guidance on that in Section 6.1 🔑.

8. Conclusion 🎬

This document looks at MPTCP's implications and its performance impact on apps 📊. It also specifies a basic MPTCP API 🛠️. For legacy apps, the existing sockets API is guaranteed to keep working 🙏. MPTCP-aware apps can use the basic MPTCP API, which gives them some control over the transport layer, equivalent to regular TCP 💖.

9. Acknowledgments 🫶

The authors sincerely thank the following people for their helpful comments and reviews of the document: Philip Eardley, Lavkesh Lahngir, John Leslie, Costin Raiciu, Michael Tuexen, and Javier Ubillos 🌟.

Michael Scharf is supported by the German-Lab project (http://www.german-lab.de/) funded by the German Federal Ministry of Education and Research (BMBF). Alan Ford was previously supported by Roke Manor Research and by Trilogy (http://www.trilogy-project.org/), a research project (ICT-216372) partially funded by the European Community under its Seventh Framework Program 🇪🇺✨.

10. References 📚

10.1. Normative References 📖

These are the references the spec leans on, so here they are in full 🧾:

[1]   Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
      September 1981.

[2]   Braden, R., "Requirements for Internet Hosts - Communication
      Layers", STD 3, RFC 1122, October 1989.

[3]   Bradner, S., "Key words for use in RFCs to Indicate Requirement
      Levels", BCP 14, RFC 2119, March 1997.

[4]   Ford, A., Raiciu, C., Handley, M., Barre, S., and J. Iyengar,
      "Architectural Guidelines for Multipath TCP Development",
      RFC 6182, March 2011.

[5]   Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, "TCP
      Extensions for Multipath Operation with Multiple Addresses",
      RFC 6824, January 2013.

[6]   Bagnulo, M., "Threat Analysis for TCP Extensions for Multipath
      Operation with Multiple Addresses", RFC 6181, March 2011.

[7]   Raiciu, C., Handley, M., and D. Wischik, "Coupled Congestion
      Control for Multipath Transport Protocols", RFC 6356,
      October 2011.

[8]   "IEEE Standard for Information Technology -- Portable Operating
      System Interface (POSIX) Base Specifications, Issue 7", IEEE
      Std. 1003.1-2008, 2008.

10.2. Informative References 📚

Okay bestie, these are the extra-credit reads that back up the whole spec 🤓✨

[9] Stevens, W., Thomas, M., Nordmark, E., and T. Jinmei, "Advanced Sockets Application Program Interface (API) for IPv6", RFC 3542, May 2003.

[10] Chakrabarti, S. and E. Nordmark, "Extension to Sockets API for Mobile IPv6", RFC 4584, July 2006.

[11] Komu, M., Bagnulo, M., Slavov, K., and S. Sugimoto, "Sockets Application Program Interface (API) for Multihoming Shim", RFC 6316, July 2011.

[12] Komu, M. and T. Henderson, "Basic Socket Interface Extensions for the Host Identity Protocol (HIP)", RFC 6317, July 2011.

[13] Stewart, R., Tuexen, M., Poon, K., Lei, P., and V. Yasevich, "Sockets API Extensions for the Stream Control Transmission Protocol (SCTP)", RFC 6458, December 2011.

[14] Blanchet, M. and P. Seite, "Multiple Interfaces and Provisioning Domains Problem Statement", RFC 6418, November 2011.

[15] Wasserman, M. and P. Seite, "Current Practices for Multiple-Interface Hosts", RFC 6419, November 2011.

[16] Wing, D. and A. Yourtchenko, "Happy Eyeballs: Success with Dual-Stack Hosts", RFC 6555, April 2012.

[17] Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.2", RFC 5246, August 2008.

[18] Cooper, D., Santesson, S., Farrell, S., Boeyen, S., Housley, R., and W. Polk, "Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile", RFC 5280, May 2008.

[19] Saint-Andre, P. and J. Hodges, "Representation and Verification of Domain-Based Application Service Identity within Internet Public Key Infrastructure Using X.509 (PKIX) Certificates in the Context of Transport Layer Security (TLS)", RFC 6125, March 2011.

[20] Sarolahti, P., "Multi-address Interface in the Socket API", Work in Progress, March 2010.

[21] Khalili, R., Gast, N., Popovic, M., and J. Le Boudec, "Performance Issues with MPTCP", Work in Progress, February 2013.

[22] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A., Handley, M., and H. Tokuda, "Is it Still Possible to Extend TCP?", Proc. ACM Internet Measurement Conference (IMC), November 2011.

Appendix A. Requirements on a Future Advanced MPTCP API 🦋

A.1. Design Considerations 🧠

Okay so multipath transport gives you SO many degrees of freedom 🌈 The basic MPTCP API only defines a minimum set of API extensions for the interface between the MPTCP layer and applications, and honestly? It doesn't give you much control over how the MPTCP implementation behaves 🙈 A future, advanced API could cover more MPTCP features and give you way more control 💅

Apps that use TCP can have different requirements on the transport layer. Developers are super used to how regular TCP acts, but the new opportunities MPTCP opens up could let the service get optimized even further ✨ An advanced API could let MPTCP-aware apps specify preferences and control certain aspects of behavior, on top of the simple control the basic interface gives you. It could also tackle stuff that's completely out of scope for the basic API, like whether a receiving app could influence the sending policy 👀 Better integration with TLS is another relevant goal (cf. Section 6.1) that needs more work 🔐

Also, an advanced MPTCP API could be part of a brand new overall interface between the network stack and apps, one that deals with other issues too, like the split between identifiers and locators 🧚 An API that doesn't use IP addresses (and instead uses, e.g., the connectbyname() function) would be useful for tons of purposes, MPTCP or not 💖

It's also been suggested that a separate address family called AF_MULTIPATH [20] be used. This separate address family could be used to swap multiple addresses between an application and the standard sockets API, but it would be a much more fundamental change compared to the basic API described in this document 🤯

This appendix lists potential usage scenarios and requirements for the advanced API. The specification and implementation of a corresponding API are outside the scope of this document 📝

A.2. MPTCP Usage Scenarios and Application Requirements 🎀

There are different MPTCP usage scenarios, bestie 🫶 An app that wants to send bulk data will want MPTCP to give it a high-throughput service right away, by creating and maxing out utilization of all the available subflows. This is the default MPTCP use case 🚀

But at the other extreme, there are apps that are super interactive but only need a small amount of throughput. These are best served by low latency and jitter stability 🌸 In that situation, it'd be preferable for the traffic to use only the lowest-latency subflow (assuming it has enough capacity), maybe with one or two extra subflows for resilience and recovery 🛟 The tricky part is that the delay on a path can fluctuate a lot, and just always picking the path with the smallest delay might lead to instability 😭

The choice between bulk data transport and latency-sensitive transport affects the scheduler, in terms of whether traffic should, by default, go on one subflow or across several 🍓 Even if the total bandwidth needed is less than what's available on one individual path, it's still desirable to spread the load to reduce stress on potential bottlenecks, and this is why this method should be the default for bulk data transport 💅 However, that may not be optimal for apps that need latency/jitter stability 🥺

For the latter option, a further question comes up: should additional subflows be used whenever the primary subflow is overloaded, or only when the primary path fails (hot standby)? Basically, is latency stability or bandwidth more important to the app? 🤔 This gives you two different options. First, there's the single path that can overflow into an additional subflow. Second, there's the single path with hot standby, where an app may want an alternative backup subflow to improve resilience 🛡️ If data delivery on the first subflow fails, the data transport could immediately continue on the second subflow, which is otherwise idle 🌙

Yet another complication is the potential MPTCP brings for changes in available bandwidth as the number of available subflows changes 🌊 That kind of bandwidth jitter can be confusing for some apps, like video or audio streaming, that dynamically adapt codecs based on available bandwidth 🎧 Those apps may prefer MPTCP to try to provide a consistent bandwidth as far as possible, and avoid maxing out the use of all subflows 🎬

A further, mostly orthogonal question is whether data should be duplicated over the different subflows, especially if there's spare capacity 👯 This could improve both the timeliness and reliability of data delivery ⭐

So in summary, there are at least three possible performance objectives for multipath transport:

  1. High bandwidth
  2. Low latency and jitter stability
  3. High reliability

These aren't necessarily disjoint, since there are also broadband interactive apps that need both high-speed bulk data traffic and low latency and jitter 🌈

In an advanced API, apps could give the MPTCP implementation high-level guidance about these performance requirements, like which requirement is the most important 👑 The MPTCP stack would then use internal mechanisms to fulfill this abstract indication of desired service, as far as possible 🧚‍♀️ This would affect how data (including retransmissions) gets assigned to existing subflows (e.g., 'use all in parallel', 'use as overflow', 'hot standby', 'duplicate traffic'), as well as decisions about when to set up additional subflows, and to which addresses 🗺️ In both cases, different policies can exist, and these can be expected to be implementation-specific 🧩

So an advanced API could provide a mechanism for how apps specify their high-level requirements in an implementation-independent way. One possibility would be to select one "application profile" out of a number of choices that characterize typical apps 📋 Still, since apps today don't have to tell TCP about their communication requirements, it requires further study to figure out whether such an approach would be realistic 🤷‍♀️

Of course, independent of an advanced API, some of this functionality could also be partly achieved by MPTCP-internal heuristics that infer some app preferences, e.g., from existing socket options, such as TCP_NODELAY 🪄 Whether that would be reliable, and indeed appropriate, is for further study 🔍

A.3. Potential Requirements on an Advanced MPTCP API 🎀

okay so this part is basically a wishlist 🧚 the doc says it's for information only, so none of this is locked in. it's just a list of what a fancier MPTCP API could do 💖

REQ5: An application should be able to establish MPTCP connections without using IP addresses as locators. 🌸

REQ6: An application should be able to get usage info and stats about all subflows (like the ratio of traffic sent over each one) 📊

REQ7: An application should be able to request a change in how many subflows are in use, which means adding or removing them 🔁 and it gets even more precise: asking to set up a specific subflow to a given destination, or asking to terminate one particular subflow that already exists 🎯

REQ8: An application should be able to tell the MPTCP implementation about its high-level performance needs, like with a profile 💅

REQ9: An application should be able to describe the communication it's about to do: how much data it expects to send, how long the connection will last, or how fast data will show up. sometimes apps can forecast this stuff, and if they can, that info could be an extra input for the implementation's heuristics, like deciding when to set up more subflows 🧠

REQ10: An application should be able to control when subflows get set up and torn down automatically 🚦 this would mean picking between different path manager heuristics, like "try as soon as possible" or "wait until there's a bunch of data" ⏳

REQ11: An application should be able to set preferred subflows or subflow usage policies 👑 this would mean choosing between different multipath scheduler configurations. for example, an app might want some subflows to act as backup only 🛟

REQ12: An application should be able to control its level of redundancy by saying whether segments should go out over more than one path in parallel 🪞

REQ13: An application should be able to control whether the MPTCP connection and the initial subflow share fate, ngl this is the kind of thing that could overwrite system policies 🔒

REQ14: An application should be able to register for callbacks so it's told about changes to subflows on an MPTCP connection 📣 this "push" style would let the app make timely logging and config changes when needed, instead of constantly polling for info 😤✨

An advanced API that covers all of this would let developers configure MPTCP way more specifically 🎀 it could also stop the internal, implicit heuristics from making suboptimal choices 💅 BUT, and this is a big but, it's honestly unclear whether all of these would really help applications, since they go way beyond what the existing API for regular TCP gives you 🤷‍♀️

A subset of these functions might also get implemented system-wide or through other config mechanisms 🌈 those implementation details are left for further study 📚

A.4. Integration with the SCTP Sockets API 🦋

the advanced API might also integrate with or use the SCTP sockets API 🧚 these SCTP functions have functionality similar to the basic MPTCP API:

  • sctp_bindx()
  • sctp_connectx()
  • sctp_getladdrs()
  • sctp_getpaddrs()
  • sctp_freeladdrs()
  • sctp_freepaddrs()

their syntax and semantics are described in [13] 📖

a potential goal for the advanced API is to give the application a consistent MPTCP and SCTP interface 💖 that's left for further study too 🫶

Authors' Addresses ✨

Michael Scharf
Alcatel-Lucent Bell Labs
Lorenzstrasse 10
70435 Stuttgart
Germany

EMail: michael.scharf@alcatel-lucent.com


Alan Ford
Cisco
Ruscombe Business Park
Ruscombe, Berkshire  RG10 9NN
UK

EMail: alanford@cisco.com
📦 where this page comes from