RFC 3926

FLUTE - File Delivery over Unidirectional Transport

๐Ÿ“… published October 2004, from ietf

FLUTE - File Delivery over Unidirectional Transport โœจ

okay bestie, RFC 3926 is the FLUTE spec ๐Ÿฆ‹ It's dated October 2004 and it's an Experimental doc, so it's giving "still in the lab" energy ๐Ÿงช

The title block says it was written by T. Paila (Nokia), M. Luby (Digital Fountain), R. Lehtonen (TeliaSonera), V. Roca (INRIA Rhone-Alpes), and R. Walsh (Nokia) for the Network Working Group ๐Ÿ’– It's Experimental, and the header doesn't list anything it obsoletes or updates. Very clean, very no-drama ๐Ÿ’…

Here's the full ToC, copied exactly:

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  2
       1.1.  Applicability Statement  . . . . . . . . . . . . . . . .  3
             1.1.1.  The Target Application Space . . . . . . . . . .  3
             1.1.2.  The Target Scale . . . . . . . . . . . . . . . .  4
             1.1.3.  Intended Environments  . . . . . . . . . . . . .  4
             1.1.4.  Weaknesses . . . . . . . . . . . . . . . . . . .  4
   2.  Conventions used in this Document. . . . . . . . . . . . . . .  5
   3.  File delivery  . . . . . . . . . . . . . . . . . . . . . . . .  5
       3.1.  File delivery session  . . . . . . . . . . . . . . . . .  6
       3.2.  File Delivery Table. . . . . . . . . . . . . . . . . . .  8
       3.3.  Dynamics of FDT Instances within file delivery session .  9
       3.4.  Structure of FDT Instance packets. . . . . . . . . . . . 11

             3.4.1.  Format of FDT Instance Header  . . . . . . . . . 12
             3.4.2.  Syntax of FDT Instance . . . . . . . . . . . . . 13
             3.4.3.  Content Encoding of FDT Instance . . . . . . . . 16
       3.5.  Multiplexing of files within a file delivery session . . 17
   4.  Channels, congestion control and timing  . . . . . . . . . . . 18
   5.  Delivering FEC Object Transmission Information . . . . . . . . 19
       5.1.  Use of EXT_FTI for delivery of FEC Object Transmission
             Information. . . . . . . . . . . . . . . . . . . . . . . 20
             5.1.1.  General EXT_FTI format . . . . . . . . . . . . . 20
             5.1.2.  FEC Encoding ID specific formats for EXT_FTI . . 21
       5.2.  Use of FDT for delivery of FEC Object Transmission
             Information. . . . . . . . . . . . . . . . . . . . . . . 25
   6.  Describing file delivery sessions. . . . . . . . . . . . . . . 25
   7.  Security Considerations  . . . . . . . . . . . . . . . . . . . 26
   8.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 29
   9.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 29
       Normative References . . . . . . . . . . . . . . . . . . . . . 29
       Informative References . . . . . . . . . . . . . . . . . . . . 30
   A.  Receiver operation (informative) . . . . . . . . . . . . . . . 32
   B.  Example of FDT Instance (informative). . . . . . . . . . . . . 33
       Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 34
       Full Copyright Statement . . . . . . . . . . . . . . . . . . . 35

Status of this Memo ๐ŸŒธ

This memo defines an Experimental Protocol for the Internet community. It does not specify an Internet standard of any kind. Discussion and suggestions for improvement are requested. Distribution of this memo is unlimited.

Copyright (C) The Internet Society (2004).

Abstract ๐Ÿฆ‹

FLUTE is a protocol for unidirectional delivery of files over the Internet, and it's especially suited to multicast networks. It's built on Asynchronous Layered Coding, the base protocol designed for massively scalable multicast distribution. ๐Ÿงš


1. Introduction ๐Ÿ’–

So this doc defines FLUTE version 1, a protocol for unidirectional file delivery over the Internet. It's built on Asynchronous Layered Coding (ALC), version 1 [2], which is the base protocol made for massively scalable multicast distribution. ALC only handles transport of arbitrary binary objects, and for file delivery that's not enough ngl. The end systems also need to know what those objects actually are ๐Ÿ“‚

So FLUTE is the technique that signals and maps file properties onto ALC concepts, so receivers can assign those parameters to the objects they get. That's why "file" in this doc means an "object" as discussed in ALC. Even though this spec uses multicast addressing a lot as an example, the techniques work for unicast addressing too ๐ŸŒˆ

This doc is a specific transport application of ALC, and it adds:

  • Definition of a file delivery session built on top of ALC, including transport details and timing constraints.
  • In-band signalling of the transport parameters of the ALC session.
  • In-band signalling of the properties of delivered files.
  • Details associated with the multiplexing of multiple files within a session.

Here's the layout ๐Ÿ‘‘ Section 3 starts by defining the file delivery session, then introduces the File Delivery Table, which is the core of the whole spec. It also covers multiplexing transport objects inside a session. Section 4 is about congestion control and channels with FLUTE. Section 5 explains how the Forward Error Correction (FEC) Object Transmission Information gets delivered inside a session. Section 6 lists the required parameters for describing file delivery sessions in the general case. Section 7 covers security considerations for FLUTE file delivery. Last but not least, there are two informative appendices. Appendix A describes an envisioned receiver operation for the receiver of the session, and Appendix B gives an example of a File Delivery Table ๐Ÿ“

Statement of Intent ๐Ÿซถ

This memo contains part of the definitions needed to fully specify a Reliable Multicast Transport protocol under RFC2357. Per RFC2357, using any reliable multicast protocol on the Internet requires an adequate congestion control scheme.

While we wait for such a scheme to be available, or for an existing one to be proven adequate, the Reliable Multicast Transport working group (RMT) publishes this Request for Comments in the "Experimental" category ๐Ÿงช

It's the RMT's intent to re-submit this specification as an IETF Proposed Standard as soon as that condition is met. ๐Ÿ”ฅ

1.1. Applicability Statement ๐ŸŒธ

1.1.1. The Target Application Space ๐Ÿ’…

FLUTE works for delivering large and small files to many hosts, using delivery sessions that last several seconds or more. For example, FLUTE could deliver big software updates to lots of hosts at once ๐Ÿš€ It could also carry continuous but segmented data, like time-lined text for subtitling. It can potentially use its layering inheritance from ALC and LCT to scale the richness of the session to the congestion status of the network. It's also suitable for basic transport of metadata, like SDP [12] files that let user applications access multimedia sessions ๐ŸŽฌ

1.1.2. The Target Scale ๐ŸŒˆ

Massive scalability is a primary design goal for FLUTE. IP multicast is inherently massively scalable, but the best-effort service it provides doesn't include session management, congestion control, or reliability. FLUTE adds all of that using ALC and IP multicast, without giving up any of the inherent scalability of IP multicast ๐Ÿ’–

1.1.3. Intended Environments ๐Ÿงš

All of the environmental requirements and considerations that apply to the ALC building block [2], and to any additional building blocks that FLUTE uses, also apply to FLUTE.

FLUTE works with both multicast and unicast delivery, but it's primary application is unidirectional multicast file delivery. FLUTE requires connectivity between a sender and receivers, but it does not require connectivity from receivers back to a sender. It inherently works with all types of networks, including LANs, WANs, Intranets, the Internet, asymmetric networks, wireless networks, and satellite networks ๐Ÿ›ฐ๏ธ

FLUTE is compatible with both IPv4 and IPv6, since no part of the packet is IP version specific. It works with both multicast models: Any-Source Multicast (ASM) [13] and Source-Specific Multicast (SSM) [15] โœจ

FLUTE is applicable for Internet use, with a suitable congestion control building block, and also for provisioned or controlled systems, like delivery over wireless broadcast radio systems ๐Ÿ“ป

1.1.4. Weaknesses ๐Ÿ™ˆ

Some networks don't play nice with some of the congestion control protocols that could be used with FLUTE. For a satellite or wireless network in particular, receivers may have no way to effectively reduce their reception rate, since there may be a fixed transmission rate allocated to the session.

FLUTE provides reliability using the FEC building block, which reduces the error rate applications see. However, FLUTE doesn't provide a way for senders to verify the reception success of receivers, and specifying such a method is outside the scope of this document ๐Ÿ’€

2. Conventions used in this Document ๐ŸŽ€

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [1] ๐Ÿ’œ

The terms "object" and "transport object" are consistent with the definitions in ALC [2] and LCT [3]. The terms "file" and "source object" are pseudonyms for "object" ๐Ÿฆ‹

3. File delivery ๐Ÿ“ฆโœจ

okay so bestie, Asynchronous Layered Coding [2] (ALC) is a protocol built for delivering arbitrary binary objects ๐Ÿฆ‹. It's especially suited for massively scalable, unidirectional, multicast distribution, which is giving huge broadcast energy ๐Ÿ“ก๐Ÿ’–. ALC is the basic transport for FLUTE, so FLUTE inherits ALC's requirements. no cap, it's a whole family tree ๐ŸŒณ๐Ÿ‘‘

This spec is all about delivering files ๐ŸŽ€. The core idea is to define how the properties of each file get carried in-band, right alongside the file itself. iconic, right? ๐Ÿ’…

Let's use an example so it clicks. Imagine a 5200 byte file at "http://www.example.com/docs/file.txt" ๐Ÿงš. The properties the file delivery protocol needs to carry are:

  • Identifier of the file, written as a URI ๐Ÿ”—. This identifier may be globally unique, and it may also tell you where the file lives. In our example: "http://www.example.com/docs/file.txt".
  • File name, which you can usually figure out from the URI ๐Ÿ—‚๏ธ. In our example: "file.txt".
  • File type, written as a MIME media type. You can usually guess this from the file extension too ๐Ÿ“. In our example: "text/plain". Okay, here's the tea ๐Ÿซ–: if an explicit MIME type is given separately from the extension and it doesn't match what the extension says, then the explicitly provided value MUST be used as the MIME type. Explicit wins, periodt ๐Ÿ’….
  • File size, in bytes ๐Ÿ“. In our example: "5200". If the file is content encoded, this is the size before encoding, so the number reflects the original file ๐Ÿ™ˆ.
  • Content encoding of the file, within transport ๐Ÿ—œ๏ธ. In our example, the file could be encoded with ZLIB [10]. In that case the transport object carrying the file probably won't be the same size as the file, which is totally normal ๐Ÿ™ƒ. The transport object size gets delivered to receivers as part of the FLUTE protocol ๐Ÿ“ฌ.
  • Security properties of the file, like digital signatures and message digests ๐Ÿ”. For example, you could use S/MIME [18] as the content encoding type for files with this authentication wrapper, and you could use XML-DSIG [19] to digitally sign an FDT Instance โœ๏ธ. Very secure, very slay ๐Ÿ›ก๏ธโœจ

3.1. File delivery session ๐ŸŒธ

ALC is a protocol instantiation of the Layered Coding Transport building block (LCT) [3], so ALC inherits LCT's session concept ๐Ÿซถ. In this document, we use the term ALC/LCT session to mean the interchangeable terms ALC session and LCT session. Same vibe, different name ๐Ÿ’œ.

An ALC/LCT session is a set of logically grouped ALC/LCT channels, all tied to a single sender who sends packets with ALC/LCT headers for one or more objects ๐Ÿ“ฆ. An ALC/LCT channel is defined by the combination of a sender and an address that the sender associates with the channel ๐Ÿ“. A receiver joins a channel to start getting the data packets the sender sends to it, and leaves a channel to stop getting them ๐Ÿšชโœจ.

One of the fields in the ALC/LCT header is the Transport Session Identifier (TSI) ๐Ÿท๏ธ. The TSI is scoped by the source IP address, and the (source IP address, TSI) pair uniquely identifies a session. So the receiver uses that pair, carried in every packet, to figure out which session each packet came from ๐Ÿ•ต๏ธโ€โ™€๏ธ๐Ÿ’–. If multiple objects are carried in one session, the Transport Object Identifier (TOI) field in the ALC/LCT header tells you which object the data in the packet came from ๐Ÿงฉ. Each object gets a unique TOI within the scope of a session, no duplicates, bestie ๐Ÿ™…โ€โ™€๏ธ.

Now for the messy real-world stuff ๐Ÿ˜ญ. If the sender doesn't have a permanent IP address that receivers can reach, but the packets receivers get carry a temporary IP address for the sender, then the TSI is scoped by that temporary IP address for the duration of the session ๐ŸŒซ๏ธ. For example, the sender might sit behind a Network Address Translation (NAT) device that temporarily assigns an address that receivers can reach, and then the TSI is scoped by that temporary address, the one that shows up in the packets the receiver gets ๐Ÿ . Another example: the sender might send its original packets over IPv6, but part of the network isn't IPv6 capable, so an IPv6 to IPv4 translator changes the packets' IP address to a different IPv4 address ๐Ÿ”„. Receivers on the IPv4 side get packets carrying that IPv4 address, so their TSI is scoped by it ๐Ÿซ . How receivers learn the sender's IP address used for scoping, whether it's permanent or temporary, is outside the scope of this document ๐Ÿšง.

When FLUTE is used for file delivery over ALC, these rules apply ๐Ÿ“‹:

  • The ALC/LCT session is called a file delivery session ๐ŸŽ€.
  • The ALC/LCT concept of 'object' means either a 'file' or a 'File Delivery Table Instance' (section 3.2) ๐Ÿ“š.
  • The TOI field MUST be included in ALC packets sent within a FLUTE session. The exception: ALC packets sent in a FLUTE session with the Close Session (A) flag set to 1 (signaling the end of the session) that contain no payload (carrying no information for any file or FDT) SHALL NOT carry the TOI ๐Ÿ›‘. For the LCT definition of the Close Session flag, see Section 5.1 of RFC 3451 [3], and for an example of its use within an ALC packet, see Section 4.2 of RFC 3450 [2] ๐Ÿ”.
  • The TOI value '0' is reserved for delivering File Delivery Table Instances. Each File Delivery Table Instance is uniquely identified by an FDT Instance ID ๐Ÿ†”.
  • Each file in a file delivery session MUST be associated with a TOI (>0) within the scope of that session ๐Ÿ™Œ.
  • Information carried in the headers and payload of a packet is scoped by the source IP address and the TSI. Information specific to an object in those headers and payload is further scoped by the TOI for file objects, and further scoped by both the TOI and the FDT Instance ID for FDT Instance objects ๐Ÿง ๐Ÿ’….

3.2. File Delivery Table ๐Ÿ—‚๏ธ๐Ÿ’–

The File Delivery Table (FDT) is how you describe the various attributes of files being delivered within the file delivery session ๐Ÿ“œ. The lists below are examples of those attributes, and they aren't meant to be mutually exclusive or exhaustive, so think of them as a starter pack ๐ŸŽ.

Attributes related to delivering a file ๐Ÿšš:

  • TOI value that represents the file
  • FEC Object Transmission Information (including the FEC Encoding ID and, if relevant, the FEC Instance ID)
  • Size of the transport object carrying the file
  • Aggregate rate of sending packets to all channels

Attributes related to the file itself ๐Ÿ“„:

  • Name, Identification and Location of the file (specified by the URI)
  • MIME media type of the file
  • Size of the file
  • Encoding of the file
  • Message digest of the file

Some of these attributes MUST be included in a file's file description entry, and others are optional, as defined in section 3.4.2 ๐Ÿ“โœจ.

Logically, the FDT is a set of file description entries for the files to be delivered in the session ๐Ÿงพ. Each file description entry MUST include the TOI for the file it describes and the URI identifying the file ๐Ÿ”—. The TOI rides along in each ALC/LCT data packet while the file is being delivered, so the TOI in the file description entry is how the receiver figures out which data packets belong to which file ๐Ÿ•ต๏ธโ€โ™€๏ธ. Each file description entry may also contain one or more descriptors that map the attributes above onto the file ๐Ÿ—บ๏ธ.

Each file delivery session MUST have an FDT that is local to that session ๐Ÿก. The FDT MUST provide a file description entry mapped to a TOI for each file that appears within the session ๐Ÿ’…. An object delivered within the ALC session but not described in the FDT is not considered a 'file' belonging to the file delivery session ๐Ÿ™…โ€โ™€๏ธ. How to handle these unmapped TOIs (TOIs not resolved by the FDT) is out of scope for this specification ๐Ÿšง.

Within the file delivery session, the FDT is delivered as FDT Instances ๐ŸŽ. An FDT Instance contains one or more file description entries of the FDT. Any FDT Instance can be equal to, a subset of, a superset of, or complementary to any other FDT Instance ๐Ÿ”„. A given FDT Instance may be repeated several times during a session, even after later FDT Instances (with higher FDT Instance ID numbers) have been sent ๐Ÿ”. Each FDT Instance contains at least one file description entry and at most the complete FDT of the file delivery session ๐Ÿ“ฆ.

A receiver of the file delivery session keeps an FDT database of received file description entries ๐Ÿง . The receiver maintains that database, for example, as it receives FDT Instances ๐Ÿ“ฅ. So at any moment, the contents of the FDT database represent the receiver's current view of the session's FDT ๐Ÿ‘€. Since each receiver behaves independently of the others, it SHOULD NOT be assumed that the FDT database contents are the same for all receivers of a given file delivery session ๐Ÿซฃ.

Since the FDT database is an abstract concept, its structure and how it's maintained are left to individual implementations, so they're out of scope for this specification ๐Ÿ› ๏ธ๐Ÿ’œ.

3.3. Dynamics of FDT Instances within file delivery session ๐Ÿ’–

okay so, here's how FDT Instances behave while a file delivery session is going ๐Ÿงš these are the rules:

  • For every file delivered within a file delivery session there MUST be a file description entry included in at least one FDT Instance sent within the session. A file description entry contains at a minimum the mapping between the TOI and the URI. ๐Ÿ“
  • An FDT Instance MAY appear in any part of the file delivery session, and packets for an FDT Instance MAY be interleaved with packets for other files or other FDT Instances within a session. ๐ŸŒ€
  • The TOI value of '0' MUST be reserved for delivery of FDT Instances. The use of other TOI values for FDT Instances is outside the scope of this specification. ๐Ÿ”’
  • An FDT Instance is identified by the use of a new fixed length LCT Header Extension EXT_FDT (defined later in this section). Each FDT Instance is uniquely identified within the file delivery session by its FDT Instance ID. Any ALC/LCT packet carrying an FDT Instance (indicated by TOI = 0) MUST include EXT_FDT. ๐Ÿท๏ธ
  • It is RECOMMENDED that the FDT Instance that contains the file description entry for a file is sent prior to the sending of the described file within a file delivery session. ๐Ÿš€
  • Within a file delivery session, any TOI > 0 MAY be described more than once. For example, a previous FDT Instance 0 describes the TOI of value '3'. Now, subsequent FDT Instances can either keep TOI '3' unmodified in the table, not include it, or complement the description. However, subsequent FDT Instances MUST NOT change the parameters already described for a specific TOI. ๐Ÿ”
  • An FDT Instance is valid until its expiration time. The expiration time is expressed within the FDT Instance payload as a 32 bit data field. The value of the data field represents the 32 most significant bits of a 64 bit Network Time Protocol (NTP) [5] time value. These 32 bits provide an unsigned integer representing the time in seconds relative to 0 hours 1 January 1900. Handling of wraparound of the 32 bit time is outside the scope of NTP and FLUTE. โฐ
  • The receiver SHOULD NOT use a received FDT Instance to interpret packets received beyond the expiration time of the FDT Instance. ๐Ÿ™ˆ
  • A sender MUST use an expiry time in the future upon creation of an FDT Instance, relative to its Sender Current Time (SCT). โณ
  • Any FEC Encoding ID MAY be used for the sending of FDT Instances. The default is to use FEC Encoding ID 0 for the sending of FDT Instances. (Note that since FEC Encoding ID 0 is the default for FLUTE, this implies that Source Block Number and Encoding Symbol ID lengths both default to 16 bits each.) ๐Ÿงฉ

Generally, a receiver needs to receive an FDT Instance describing a file before it is able to recover the file itself. In this sense, FDT Instances are of higher priority than files. Thus, it is RECOMMENDED that FDT Instances describing a file be sent with at least as much reliability within a session (more often or with more FEC protection) as the files they describe. In particular, if FDT Instances are longer than one packet payload in length, it is RECOMMENDED that an FEC code that provides protection against loss be used for delivering FDT Instances. How often the description of a file is sent in an FDT Instance, or how much FEC protection is provided for each FDT Instance (if the FDT Instance is longer than one packet payload), is dependent on the particular application and outside the scope of this document. ๐Ÿ‘‘

3.4. Structure of FDT Instance packets ๐ŸŽ€

FDT Instances are carried in ALC packets with TOI = 0 and with an additional REQUIRED LCT Header extension called the FDT Instance Header. The FDT Instance Header (EXT_FDT) contains the FDT Instance ID that uniquely identifies FDT Instances within a file delivery session. The FDT Instance Header is placed in the same way as any other LCT extension header. There MAY be other LCT extension headers in use. ๐Ÿง

The LCT extension headers are followed by the FEC Payload ID, and finally the Encoding Symbols for the FDT Instance, which contains one or more file description entries. A FDT Instance MAY span several ALC packets. The number of ALC packets is a function of the file attributes associated with the FDT Instance. The FDT Instance Header is carried in each ALC packet carrying the FDT Instance. The FDT Instance Header is identical for all ALC/LCT packets for a particular FDT Instance. ๐Ÿ“ฆ

The overall format of ALC/LCT packets carrying an FDT Instance is depicted in Figure 1 below. All integer fields are carried in "big-endian" or "network order" format, that is, most significant byte (octet) first. As defined in [2], all ALC/LCT packets are sent using UDP. ๐ŸŒŠ

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         UDP header                            |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                Default LCT header (with TOI = 0)              |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          LCT header extensions (EXT_FDT, EXT_FTI, etc.)       |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       FEC Payload ID                          |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|              Encoding Symbol(s) for FDT Instance              |
|                           ...                                 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 1 - Overall FDT Packet ๐ŸŒˆ

3.4.1. Format of FDT Instance Header ๐Ÿฆ‹

FDT Instance Header (EXT_FDT) is a new fixed length, ALC PI specific LCT header extension [3]. The Header Extension Type (HET) for the extension is 192. Its format is defined below: ๐Ÿซถ

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   HET = 192   |   V   |          FDT Instance ID              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Version of FLUTE (V), 4 bits:

This document specifies FLUTE version 1. So, in any ALC packet that carries an FDT Instance and that belongs to the file delivery session as specified in this specification, this field MUST be set to '1'. ๐Ÿ”ข

FDT Instance ID, 20 bits:

For each file delivery session, the numbering of FDT Instances starts from '0' and is incremented by one for each subsequent FDT Instance. After reaching the maximum value (2^20-1), the numbering starts again from '0'. When wraparound from 2^20-1 to 0 occurs, 0 is considered higher than 2^20-1. A new FDT Instance reusing a previous FDT Instance ID number, due to wraparound, may not implicitly expire the previous FDT Instance with the same ID. It would be reasonable for FLUTE Senders to only construct and deliver FDT Instances with wraparound IDs after the previous FDT Instance using the same ID has expired. However, mandatory receiver behavior for handling FDT Instance ID wraparound and other special situations (for example, missing FDT Instance IDs resulting in larger increments than one) is outside the scope of this specification and left to individual implementations of FLUTE. ๐Ÿ”„

3.4.2. Syntax of FDT Instance โœจ

okay so the FDT Instance is where all the file description entries live ๐Ÿ—‚๏ธ๐Ÿ’– they're the ones doing the mapping job from 3.2 above, so this part is kinda the heart of it ๐Ÿซถ

The FDT Instance is an XML structure with one root element, FDT-Instance. The FDT-Instance element MUST contain an Expires attribute, which tells you the expiry time of the FDT Instance โฐ. The FDT-Instance element MAY also contain the Complete attribute (boolean). When that's TRUE, it's basically saying "bestie, no new data is coming in future FDT Instances in this session" ๐Ÿ™…โ€โ™€๏ธ. So either FDT Instances with higher ID numbers won't be used, or if they are used, they'll only give the exact same file parameters that are already in this one and the previous ones. For example, you could use this to hand over a complete list of files for an entire FLUTE session (a "complete FDT") ๐Ÿ“œโœจ

The FDT-Instance element MAY contain attributes that give common parameters for all the files in an FDT Instance ๐ŸŽ€. These attributes MAY also be provided for individual files in the File element. If the same attribute shows up in both the FDT-Instance and the File elements, the value in the File element wins ๐Ÿ‘‘

For every file being declared in a given FDT Instance, there's a single file description entry in that FDT Instance. Each entry is a File element, and it's a child element of the FDT Instance structure ๐Ÿฆ‹

The attributes of the File element represent the attributes given to the file being delivered in the file delivery session. The XML attribute name matches a MIME field name, and the XML attribute value matches the value of that MIME field body. Each File element MUST contain at least two attributes, TOI and Content-Location. TOI MUST be assigned a valid TOI value as described in section 3.3 above. Content-Location MUST be assigned a valid URI as defined in [6] ๐ŸŒธ

Besides those mandatory attributes, the FDT-Instance element and the File element MAY contain other attributes. These are the ones that get called out specifically:

  • Where the MIME type is described, the attribute Content-Type MUST be used for the purpose defined in [6] ๐Ÿ’…
  • Where the length is described, the attribute Content-Length MUST be used for the purpose defined in [6]. The transfer length is defined as the length of the object transported, in bytes. It's often super important to tell receivers the transfer length, because the source block structure has to be known for the FEC decoder to recover the source blocks of the file, and the transfer length is often needed to properly work out that source block structure ๐Ÿงฉ. There will generally be a difference between the length of the original file and the transfer length if content encoding is applied to the file before transport, which is why the Content-Encoding attribute exists. If the file is not content encoded before transport (so Content-Encoding isn't used), the transfer length is the length of the original file, and Content-Length is also the transfer length. But if the file is content encoded before transport (so Content-Encoding is used), e.g. compression applied before transport to cut down the number of bytes that need to be sent, then the transfer length is generally different from the length of the original file. In that case, the attribute Transfer-Length MAY be used to carry the transfer length ๐Ÿ“ฆ
  • Where the content encoding scheme is described, the attribute Content-Encoding MUST be used for the purpose defined in [6] ๐ŸŽ
  • Where the MD5 message digest is described, the attribute Content-MD5 MUST be used for the purpose defined in [6] ๐Ÿ”
  • The FEC Object Transmission Information attributes described in section 5.2 ๐Ÿ› ๏ธ

Here's the XML Schema [8][9] for the FDT Instance ๐Ÿค“๐Ÿ’œ

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:fl="http://www.example.com/flute"
           elementFormDefault:xs="qualified"
           targetNamespace:xs="http://www.example.com/flute">
 <xs:element name="FDT-Instance">
  <xs:complexType>
   <xs:sequence>

    <xs:element name="File" maxOccurs="unbounded">
     <xs:complexType>
      <xs:attribute name="Content-Location"
                    type="xs:anyURI"
                    use="required"/>
      <xs:attribute name="TOI"
                    type="xs:positiveInteger"
                    use="required"/>
      <xs:attribute name="Content-Length"
                    type="xs:unsignedLong"
                    use="optional"/>
      <xs:attribute name="Transfer-Length"
                    type="xs:unsignedLong"
                    use="optional"/>
      <xs:attribute name="Content-Type"
                    type="xs:string"
                    use="optional"/>
      <xs:attribute name="Content-Encoding"
                    type="xs:string"
                    use="optional"/>
      <xs:attribute name="Content-MD5"
                    type="xs:base64Binary"
                    use="optional"/>
      <xs:attribute name="FEC-OTI-FEC-Encoding-ID"
                    type="xs:unsignedLong"
                    use="optional"/>
      <xs:attribute name="FEC-OTI-FEC-Instance-ID"
                    type="xs:unsignedLong"
                    use="optional"/>
      <xs:attribute name="FEC-OTI-Maximum-Source-Block-Length"
                    type="xs:unsignedLong"
                    use="optional"/>
      <xs:attribute name="FEC-OTI-Encoding-Symbol-Length"
                    type="xs:unsignedLong"
                    use="optional"/>
      <xs:attribute name="FEC-OTI-Max-Number-of-Encoding-Symbols"
                    type="xs:unsignedLong"
                    use="optional"/>
      <xs:anyAttribute processContents="skip"/>
     </xs:complexType>
    </xs:element>
   </xs:sequence>

   <xs:attribute name="Expires"
                 type="xs:string"
                 use="required"/>
   <xs:attribute name="Complete"
                 type="xs:boolean"
                 use="optional"/>
   <xs:attribute name="Content-Type"
                 type="xs:string"
                 use="optional"/>
   <xs:attribute name="Content-Encoding"
                 type="xs:string"
                 use="optional"/>
   <xs:attribute name="FEC-OTI-FEC-Encoding-ID"
                 type="xs:unsignedLong"
                 use="optional"/>
   <xs:attribute name="FEC-OTI-FEC-Instance-ID"
                 type="xs:unsignedLong"
                 use="optional"/>
   <xs:attribute name="FEC-OTI-Maximum-Source-Block-Length"
                 type="xs:unsignedLong"
                 use="optional"/>
   <xs:attribute name="FEC-OTI-Encoding-Symbol-Length"
                 type="xs:unsignedLong"
                 use="optional"/>
   <xs:attribute name="FEC-OTI-Max-Number-of-Encoding-Symbols"
                 type="xs:unsignedLong"
                 use="optional"/>
   <xs:anyAttribute processContents="skip"/>
  </xs:complexType>
 </xs:element>
</xs:schema>

Any valid FDT Instance must use the above XML Schema. This is how FDT stays extensible, so it can support private attributes within the file description entries ๐Ÿงš Those could be things like attributes about delivering the file (timing, packet transmission rate, and so on) โฑ๏ธ

If the basic FDT XML Schema gets extended with new descriptors, the ones for a single file MUST be placed within the attributes of the File element. The ones that apply to all files in the current FDT Instance MUST be placed within the FDT-Instance element. It is RECOMMENDED that the new descriptors applied in the FDT are in the format of MIME fields, and that they're defined either in the HTTP/1.1 specification [6] or in another well-known specification ๐Ÿ’ซ


3.4.3. Content Encoding of FDT Instance ๐Ÿช„

The FDT Instance itself MAY be content encoded, for example compressed ๐Ÿ—œ๏ธ This specification defines the FDT Instance Content Encoding Header (EXT_CENC). EXT_CENC is a new fixed-length, ALC PI specific LCT header extension [3], and its Header Extension Type (HET) is 193. If the FDT Instance is content encoded, the EXT_CENC MUST be used to signal the content encoding type. In that case, the EXT_CENC header extension MUST be used in all ALC packets carrying the same FDT Instance ID. That also means that whenever EXT_CENC is used, it MUST be used together with a proper FDT Instance Header (EXT_FDT). Within a file delivery session, FDT Instances that are not content encoded and FDT Instances that are content encoded MAY both appear ๐ŸŒˆ If content encoding isn't used for a given FDT Instance, the EXT_CENC MUST NOT be used in any packet carrying that FDT Instance. Here's the format of EXT_CENC ๐Ÿ‘€

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   HET = 193   |     CENC      |          Reserved             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Content Encoding Algorithm (CENC), 8 bits:

This field signals the content encoding algorithm used in the FDT Instance payload. How this field is defined is outside the scope of this specification. Applicable content encoding algorithms include, for example, ZLIB [10], DEFLATE [16] and GZIP [17] ๐Ÿฌ

Reserved, 16 bits:

This field MUST be set to all '0' ๐Ÿซง


3.5. Multiplexing of files within a file delivery session ๐ŸŽข

The delivered files are carried as transport objects (identified with TOIs) in the file delivery session ๐Ÿ“ฌ All of these objects, including the FDT Instances, MAY be multiplexed in any order and in parallel with each other within a session. That means packets for one file MAY be interleaved with packets for other files or other FDT Instances within a session ๐Ÿ“

Multiple FDT Instances MAY be delivered in a single session using TOI = 0. In this case, it is RECOMMENDED that the sending of a previous FDT Instance SHOULD end before the sending of the next FDT Instance starts. However, due to unexpected network conditions, packets for the FDT Instances MAY be interleaved ๐ŸŒช๏ธ A receiver can still tell which FDT Instance a packet is about, because each FDT Instance is uniquely identified by its FDT Instance ID, which is carried in the EXT_FDT headers ๐Ÿ”๐Ÿ’…

4. Channels, congestion control and timing โœจ

okay so ALC/LCT has this whole concept of channels and congestion control ๐ŸŒŠ and FLUTE is meant to be used in four different scenarios, bestie:

  • (a) Use a single channel and a single-rate congestion control protocol. ๐ŸŽ€
  • (b) Use multiple channels and a multiple-rate congestion control protocol. In this case the FDT Instances MAY be delivered on more than one channel. ๐Ÿฆ‹
  • (c) Use a single channel without congestion control supplied by ALC, but only when in a controlled network environment where flow/congestion control is being provided by other means. ๐Ÿซถ
  • (d) Use multiple channels without congestion control supplied by ALC, but only when in a controlled network environment where flow/congestion control is being provided by other means. In this case the FDT Instances MAY be delivered on more than one channel. ๐Ÿ’…

When there's just one channel, like in (a) and (c), the idea of "prior" and "after" is super intuitive for when objects get delivered, you know? ๐Ÿ’–

But once you've got multiple channels, like in (b) and (d), it gets messy ๐Ÿ˜ญ You can't just say one object was delivered "prior" to another. An object can start going out on one or more channels before a second object even starts, but the extra channels/layers can finish the second object before the first one is done. That's totally fine when objects go one after another on a single channel. So if your FLUTE app has a mandatory or critical need for the first transport object to finish "prior" to the second one, it is RECOMMENDED that you only use a single channel for the file delivery session. ๐Ÿ”ฅ

Also, if you're using multiple channels, a receiver who joins at a low reception rate will only be joined to the lower layers of the session ๐Ÿข And FDT Instances are way more important than files, because a file's reception depends on getting the FDT Instance that describes it ๐Ÿ‘‘ So the following is RECOMMENDED:

  1. The layers that carry packets for FDT Instances SHOULD NOT be biased towards the layers that lower rate receivers aren't joined to. For example, it's fine to put all the packets for an FDT Instance into the lowest layer (as long as that layer carries enough packets to deliver the FDT to higher rate receivers in a reasonable amount of time), but it's not okay to put all of them into the higher layers that only high rate receivers will get ๐Ÿ™ˆ
  2. If FDT Instances are generally longer than one Encoding Symbol and some of their packets go to layers that lower rate receivers don't receive, an FEC Encoding other than FEC Encoding ID 0 SHOULD be used to deliver FDT Instances. The reason is that even with zero packet loss in the network, a lower rate receiver won't receive every packet sent for an FDT Instance ๐ŸŒธ

5. Delivering FEC Object Transmission Information ๐Ÿงš

FLUTE borrows the FEC building block [4] from ALC ๐Ÿค When you use FLUTE for file delivery over ALC, the FEC Object Transmission Information MUST be delivered in-band within the file delivery session. There are two methods specified for this: the ALC-specific LCT extension header EXT_FTI [2], and FDT.

The receiver of a file delivery session MUST support delivering FEC Object Transmission Information using EXT_FTI for the FDT Instances carried using TOI value 0. For TOI values other than 0, the receiver MUST support both methods: EXT_FTI and FDT. ๐Ÿ’œ

The FEC Object Transmission Information that gets delivered to receivers MUST be exactly the same whether it's sent using EXT_FTI or FDT (or both) ๐ŸŽฏ Section 5.1 describes the required FEC Object Transmission Information that MUST be delivered for the various FEC Encoding IDs, and it also covers delivery using EXT_FTI. Section 5.2 covers delivery using FDT.

The FEC Object Transmission Information for a given TOI might be available from several sources. When that happens, it is RECOMMENDED that the receiver of the file delivery session prioritizes the sources like this, in order of decreasing priority:

  1. FEC Object Transmission Information that is available in EXT_FTI.
  2. FEC Object Transmission Information that is available in the FDT.

5.1. Use of EXT_FTI for delivery of FEC Object Transmission Information ๐Ÿ’Œ

As specified in [2], the EXT_FTI header extension is meant to carry the FEC Object Transmission Information for an object in-band. Individual implementations get to decide how often, and in which ALC packets, the EXT_FTI header extension gets included ๐Ÿคทโ€โ™€๏ธ In environments with a higher packet loss rate, EXT_FTI might need to show up more often than in environments with low error probability. One thing that's not optional: the EXT_FTI MUST be included in at least one sent ALC packet for each FDT Instance ๐Ÿ”ฅ

The ALC specification doesn't define the format or processing of the EXT_FTI header extension, so the next sections spell out EXT_FTI as it's used in FLUTE. In FLUTE, the FEC Encoding ID (8 bits) is carried in the Codepoint field of the ALC/LCT header ๐ŸŽ€

5.1.1. General EXT_FTI format ๐ŸŒˆ

The general EXT_FTI format covers the structure and the attributes of FEC Object Transmission Information that apply to any FEC Encoding ID, so it's the base for everything else ๐Ÿงฑ

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   HET = 64    |     HEL       |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                       Transfer Length                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   FEC Instance ID             | FEC Enc. ID Specific Format   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Header Extension Type (HET), 8 bits: it's just 64, as defined in [2] ๐Ÿซถ

Header Extension Length (HEL), 8 bits: the length of the whole Header Extension field, counted in multiples of 32-bit words. This length includes the FEC Encoding ID specific format part too ๐Ÿ“

Transfer Length, 48 bits: the length, in bytes, of the transport object that carries the file. (It's the same as the file length if the file isn't content encoded.) ๐Ÿ“ฆ

FEC Instance ID, optional, 16 bits: this field is used for the FEC Instance ID. It's only present if the FEC Encoding ID is in the range 128-255. When the FEC Encoding ID is in the range 0-127, this field is set to 0 ๐Ÿงฉ

FEC Encoding ID Specific Format: different FEC encoding schemes need different sets of encoding parameters ๐Ÿ’โ€โ™€๏ธ So the structure and length of this field depend on the FEC Encoding ID. The next sections define the structure of this field for FEC Encoding ID numbers 0, 128, 129, and 130.

5.1.2. FEC Encoding ID specific formats for EXT_FTI ๐Ÿฆ‹

5.1.2.1. FEC Encoding IDs 0, 128, and 130 ๐Ÿ’–

FEC Encoding ID 0 is 'Compact No-Code FEC' (Fully-Specified) [7]. FEC Encoding ID 128 is 'Small Block, Large Block and Expandable FEC' (Under-Specified) [4]. FEC Encoding ID 130 is 'Compact FEC' (Under-Specified) [7]. For these FEC Encoding IDs, the FEC Encoding ID specific format of EXT_FTI looks like this ๐ŸŒธ

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  General EXT_FTI format       |    Encoding Symbol Length     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                  Maximum Source Block Length                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Encoding Symbol Length, 16 bits: the length of an Encoding Symbol, in bytes.

All Encoding Symbols of a transport object MUST be equal to this length, with one optional exception: the last source symbol of the last source block, so redundant padding isn't mandatory in that last symbol ๐Ÿ™Œ That last source symbol MUST be logically padded out with zeroes when another Encoding Symbol gets computed from it, so the sender and receiver interpret that Encoding Symbol value the same way. The padding doesn't actually have to be sent along with the data of the last source symbol ๐Ÿคซ

Maximum Source Block Length, 32 bits: the maximum number of source symbols per source block ๐Ÿงฎ

This EXT_FTI spec requires that both the sender and receivers know an algorithm for working out the size of all the source blocks of the transport object carrying the file identified by the TOI (or within the FDT Instance identified by the TOI and the FDT Instance ID). The algorithm SHOULD be the same for all files that use the same FEC Encoding ID within a session ๐ŸŽ€

Section 5.1.2.3 describes an algorithm that is RECOMMENDED for this use โœจ

For FEC Encoding IDs 0, 128 and 130, this algorithm is the only well-known way the receiver can figure out the length of each source block. So it does two big jobs: (a) it tells the receiver the length of each particular source block while packets for that block are coming in, which is essential to all of these FEC schemes; and (b) it gives the receiver the source block structure right away, so the receiver knows where to save recovered source blocks from the very start of data packet reception for the file. That second part is an optimization, but it's essential for some implementations ๐Ÿ”ฅ๐Ÿ˜ญ

5.1.2.2. FEC Encoding ID 129 โœจ

Okay bestie, FEC Encoding ID 129 is Small Block Systematic FEC (Under-Specified) ๐Ÿฆ‹. Its FEC Encoding ID specific format of EXT_FTI looks like this:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      General EXT_FTI format       |    Encoding Symbol Length     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Maximum Source Block Length  | Max. Num. of Encoding Symbols |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Encoding Symbol Length, 16 bits: this is the length of an Encoding Symbol in bytes. ๐Ÿ’–

All Encoding Symbols of a transport object MUST be this length, with one optional exception: the last source symbol of the last source block, so redundant padding isn't mandatory in that symbol ๐ŸŒธ. That last source symbol MUST be logically padded out with zeroes whenever another Encoding Symbol is computed from it, so the sender and receiver interpret that Encoding Symbol value the same way โœจ. The padding doesn't actually need to be sent along with the data of the last source symbol ๐Ÿซถ.

Maximum Source Block Length, 16 bits: the maximum number of source symbols per source block ๐Ÿ“.

Maximum Number of Encoding Symbols, 16 bits: the maximum number of Encoding Symbols that can be generated for a source block ๐Ÿ‘‘.

This EXT_FTI specification needs both the sender and the receivers to know an algorithm for figuring out the size of every source block in the transport object that carries the file identified by the TOI (or within the FDT Instance identified by the TOI and the FDT Instance ID) ๐Ÿ’…. The algorithm SHOULD be the same for all files that use the same FEC Encoding ID within a session ๐Ÿงš.

Section 5.1.2.3 describes an algorithm that is RECOMMENDED for this use ๐Ÿ’œ. For FEC Encoding ID 129, the FEC Payload ID in each data packet already carries the source block length for the source block that goes with the Encoding Symbol in that packet ๐Ÿ”ฅ. So technically you could just use the source block lengths sitting in the FEC Payload ID of the data packets ๐Ÿ™ˆ. The Section 5.1.2.3 algorithm is still handy, though, because it lets the receiver work out the source block structure right at the start of receiving data packets for the file โœจ. If you do use that algorithm, the source block lengths in the data packets MUST agree with what the algorithm calculates ๐Ÿ’€.

5.1.2.3. Algorithm for Computing Source Block Structure ๐Ÿงฎ

This algorithm builds a source block structure where every source block is as close to the same length as possible ๐ŸŽ€. A first number of source blocks are all the same larger length, and the remaining second number of source blocks are all the same smaller length ๐ŸŒˆ. The total number of source blocks (N), the first number of source blocks (I), the second number of source blocks (N-I), the larger length (A_large) and the smaller length (A_small) get calculated like this:

   Input:
      B -- Maximum Source Block Length, i.e., the maximum number of
           source symbols per source block
      L -- Transfer Length in bytes
      E -- Encoding Symbol Length in bytes

   Output:
      N -- The number of source blocks into which the transport
           object is partitioned.

      The number and lengths of source symbols in each of the N
      source blocks.

   Algorithm:
   (a) The number of source symbols in the transport object is
       computed as T = L/E rounded up to the nearest integer.
   (b) The transport object is partitioned into N source blocks,
       where N = T/B rounded up to the nearest integer
   (c) The average length of a source block, A = T/N
       (this may be non-integer)
   (d) A_large = A rounded up to the nearest integer
       (it will always be the case that the value of A_large is at
       most B)
   (e) A_small = A rounded down to the nearest integer
       (if A is an integer A_small = A_large,
       and otherwise A_small = A_large - 1)
   (f) The fractional part of A, A_fraction = A - A_small
   (g) I = A_fraction * N
       (I is an integer between 0 and N-1)
   (h) Each of the first I source blocks consists of A_large source
       symbols, each source symbol is E bytes in length.  Each of the
       remaining N-I source blocks consist of A_small source symbols,
       each source symbol is E bytes in length except that the last
       source symbol of the last source block is L-(((L-1)/E) rounded
       down to the nearest integer)*E bytes in length.
Heads up: this algorithm doesn't have to be done with floating point maths ๐Ÿ™ˆ. Integer arithmetic works too, and it helps you dodge potential floating point rounding errors ๐Ÿ’–.

5.2. Use of FDT for delivery of FEC Object Transmission Information ๐Ÿ“ฆ

The FDT delivers FEC Object Transmission Information for each file using an appropriate attribute, either inside the "FDT-Instance" element or inside the "File" element of the FDT structure ๐ŸŒธ. For future FEC Encoding IDs, if the attributes listed below can't capture the FEC Object Transmission Information, then additional new attributes MAY be used ๐Ÿ”ฎ. Here's how the existing attributes line up:

  • "Transfer-Length" is semantically equivalent with the field "Transfer Length" of EXT_FTI ๐Ÿ’œ.
  • "FEC-OTI-FEC-Encoding-ID" is semantically equivalent with the field "FEC Encoding ID" as carried in the Codepoint field of the ALC/LCT header ๐ŸŽ€.
  • "FEC-OTI-FEC-Instance-ID" is semantically equivalent with the field "FEC Instance ID" of EXT_FTI โœจ.
  • "FEC-OTI-Maximum-Source-Block-Length" is semantically equivalent with the field "Maximum Source Block Length" of EXT_FTI for FEC Encoding IDs 0, 128 and 130, and semantically equivalent with the field "Maximum Source Block Length" of EXT_FTI for FEC Encoding ID 129 ๐Ÿ“.
  • "FEC-OTI-Encoding-Symbol-Length" is semantically equivalent with the field "Encoding Symbol Length" of EXT_FTI for FEC Encoding IDs 0, 128, 129 and 130 ๐ŸŒˆ.
  • "FEC-OTI-Max-Number-of-Encoding-Symbols" is semantically equivalent with the field "Maximum Number of Encoding Symbols" of EXT_FTI for FEC Encoding ID 129 ๐Ÿ‘‘.

6. Describing file delivery sessions ๐ŸŽง

To start receiving a file delivery session, the receiver needs to know the transport parameters tied to that session ๐Ÿฆ‹. Working out those parameters and kicking off reception is the entry point, and from that moment on the receiver's operation falls into this specification's scope ๐Ÿ’…. According to [2], the transport parameters of an ALC/LCT session that the receiver needs to know are:

  • The source IP address;
  • The number of channels in the session;
  • The destination IP address and port number for each channel in the session;
  • The Transport Session Identifier (TSI) of the session;
  • An indication that the session is a FLUTE session. Demultiplexing objects on reception is implicit in any use of FLUTE, and this covers the ALC requirement of saying whether or not a session carries packets for more than one object (all FLUTE sessions carry packets for more than one object) ๐Ÿซถ.

Optionally, the following parameters MAY be associated with the session (Note, the list is not exhaustive) โœจ:

  • The start time and end time of the session;
  • FEC Encoding ID and FEC Instance ID when the default FEC Encoding ID 0 is not used for the delivery of FDT;
  • Content Encoding format if optional content encoding of FDT Instance is used, e.g., compression;
  • Some information that tells receiver, in the first place, that the session contains files that are of interest ๐Ÿ’–.

It's envisioned that these parameters would be described using some session description syntax (such as SDP [12] or XML based) and stored in a file that the receiver grabs before the FLUTE session begins, through some transport protocol (such as Session Announcement Protocol [11], email, HTTP [6], SIP [22], manual pre-configuration, etc.) ๐Ÿ“ฌ. But how the receiver finds those parameters is out of scope for this document, same as it is for LCT and ALC ๐Ÿ™ˆ. In particular, this specification doesn't mandate or exclude any mechanism ๐ŸŒธ.

7. Security Considerations โœจ

okay bestie, this section is SERIOUS but also kinda iconic, so let's get into it ๐Ÿ”’๐Ÿ’…. The security considerations from ALC [2], LCT [3] and FEC [4] also apply to FLUTE ๐Ÿซถ. And on top of that, any security considerations for a congestion control building block you pair with FLUTE also apply to FLUTE. so it's like a whole security family tree ๐ŸŒณโœจ

Okay now the scary part ๐Ÿ˜ญ. Because FLUTE uses FEC, it's especially vulnerable to denial-of-service attacks where an attacker sends forged packets to the session. These can stop a receiver from successfully rebuilding the FDT or file, or make it rebuild something wrong, which is a whole mess ๐Ÿ’€. ALC has the same problem, and FLUTE is hit extra hard because MANY receivers can get the exact same forged packet. A sneaky attacker can spoof file packets and get a file recovered incorrectly ๐Ÿ™ˆ.

Even worse, a malicious forger can spoof FDT Instance packets with wrong FDT-Instance fields. There are a bunch of ways this can go wrong. For example, the attacker could change the Content-Location field of TOI 'n' so it points at a system file or a user config file. Then TOI 'n' could carry a Trojan Horse or some other virus ๐Ÿ‘พ. So it is STRONGLY RECOMMENDED that the FLUTE delivery service at the receiver does NOT have write access to system files or directories, or any other critical areas ๐Ÿ”. Also, as described for MIME [20][21], you should be super careful about any MIME types that could trigger remote execution of actions in the recipient's environment ๐Ÿšจ. And note that RFC 1521 [21] covers important security issues for this environment, even though RFC 2048 [20] has obsoleted its protocol ๐Ÿ“š.

Another example: a forger could generate a bad Content-MD5 sum, so receivers reject the file and mark it as corrupted. The Content-Encoding can also be changed, which stops receivers from handling the file properly. These examples show that the FDT information is critical to the FLUTE delivery service ๐Ÿ‘‘.

At the application level, it is RECOMMENDED that an integrity check on the entire received object be done once the object is rebuilt, to make sure it matches what was sent. This matters especially for objects that are FDT Instances ๐Ÿงš. For strong cryptographic integrity protection, a digital signature the receiver can verify SHOULD be used for this application level check ๐Ÿ”. But here's the tea โ˜•: if even ONE corrupted or forged packet gets used to rebuild the object, the object will probably come out wrong. The integrity check will then fail, and in that case the inaccurately rebuilt object SHOULD be discarded. So accepting a single forged packet can be an effective denial of service attack for distributing objects, but an integrity check at least stops people from accidentally using a wrong object ๐Ÿ™Œ. The details of an application level integrity check on the received object are outside the scope of this document, so that's a different story ๐Ÿ“–.

At the packet level, it is RECOMMENDED that packet level authentication be used to make sure each received packet is authentic and uncorrupted, carrying FEC data for the object from the specified sender ๐Ÿฆ‹. This one is the real MVP because corrupt or forged packets can be thrown out one by one, and the good authenticated packets can still rebuild the object accurately ๐Ÿ’–. So the effect of a denial of service attack that injects forged packets scales only with the number of forged packets, NOT with the object size ๐Ÿ“ˆ. There's no IETF standard yet for multicast packet level authentication, but TESLA [14] is a known multicast packet authentication scheme that would work ๐ŸŒˆ.

Packet level authentication also gives some protection against denial of service attacks on multiple rate congestion control, not just inaccurate objects ๐Ÿšฆ. Attackers can inject forged packets with wrong congestion control info into the multicast stream. That can hurt network elements and receivers downstream of the attack, and much less significantly the rest of the network and other receivers ๐Ÿ˜ฌ. So it is also RECOMMENDED that packet level authentication be used to guard against these attacks. TESLA [14] can help limit the damage to some extent too. The catch is that with TESLA, a receiver can only tell whether a packet is authentic several seconds after receiving it, so an attack on the congestion control protocol can work for several seconds before the receiver can react and slow down its session reception rate โณ.

Reverse Path Forwarding checks SHOULD be turned on in all network routers and switches along the path from the sender to the receivers, to limit the chance of a bad agent injecting forged packets into the multicast tree data path ๐ŸŒฒ.

A receiver with a wrong or corrupted implementation of the multiple rate congestion control building block can mess with the health of the network between it and the sender. It can also affect the reception rates of other receivers joined to the session ๐Ÿ˜ฑ. So it is RECOMMENDED that receivers be required to identify themselves as legitimate BEFORE they receive the Session Description needed to join the session ๐Ÿชช. How they identify themselves is outside the scope of this document, bestie ๐Ÿ™.

Another FLUTE vulnerability is receivers getting an incorrect Session Description for the session ๐Ÿ™ƒ. That could mean legitimate receivers with the wrong Session Description can't receive the session content, or receivers accidentally try to receive at a much higher rate than they can handle, which disrupts traffic in parts of the network ๐ŸŒช๏ธ. To avoid this, it is RECOMMENDED that measures be taken to stop receivers from accepting incorrect Session Descriptions. For example, use source authentication so receivers only accept legitimate Session Descriptions from authorized senders ๐Ÿ›ก๏ธ. How this is done is outside the scope of this document ๐Ÿ’ซ.


8. IANA Considerations ๐Ÿงพ

No information in this specification is directly subject to IANA registration ๐Ÿ™…โ€โ™€๏ธ. However, building block components used by ALC may bring in additional IANA considerations. In particular, the FEC building block used by FLUTE DOES require IANA registration of the FEC codec used ๐Ÿ“.

9. Acknowledgements ๐ŸŽ€

The following people contributed to this specification: Brian Adamson, Mark Handley, Esa Jalonen, Roger Kermode, Juha-Pekka Luoma, Jani Peltotalo, Sami Peltotalo, Topi Pohjolainen, and Lorenzo Vicisano. The authors would like to thank all the contributors for their valuable work reviewing and giving feedback on this specification ๐Ÿซถ๐Ÿ’œ.

Normative References ๐Ÿ“š

  • [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [2] Luby, M., Gemmell, J., Vicisano, L., Rizzo, L., and J. Crowcroft, "Asynchronous Layered Coding (ALC) Protocol Instantiation", RFC 3450, December 2002.
  • [3] Luby, M., Gemmell, J., Vicisano, L., Rizzo, L., Handley, M., and J. Crowcroft, "Layered Coding Transport (LCT) Building Block", RFC 3451, December 2002.
  • [4] Luby, M., Vicisano, L., Gemmell, J., Rizzo, L., Handley, M., and J. Crowcroft, "Forward Error Correction (FEC) Building Block", RFC 3452, December 2002.
  • [5] Mills, D., "Network Time Protocol (Version 3) Specification, Implementation", RFC 1305, March 1992.
  • [6] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
  • [7] Luby, M. and L. Vicisano, "Compact Forward Error Correction (FEC) Schemes", RFC 3695, February 2004.
  • [8] Thompson, H., Beech, D., Maloney, M. and N. Mendelsohn, "XML Schema Part 1: Structures", W3C Recommendation, May 2001.
  • [9] Biron, P. and A. Malhotra, "XML Schema Part 2: Datatypes", W3C Recommendation, May 2001.

Informative References ๐ŸŒธ

  • [10] Deutsch, P. and J-L. Gailly, "ZLIB Compressed Data Format Specification version 3.3", RFC 1950, May 1996.
  • [11] Handley, M., Perkins, C., and E. Whelan, "Session Announcement Protocol", RFC 2974, October 2000.
  • [12] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998.
  • [13] Deering, S., "Host extensions for IP multicasting", STD 5, RFC 1112, August 1989.
  • [14] Perrig, A., Canetti, R., Song, D., and J. Tygar, "Efficient and Secure Source Authentication for Multicast, Network and Distributed System Security Symposium, NDSS 2001, pp. 35-46.", February 2001.
  • [15] Holbrook, H., "A Channel Model for Multicast, Ph.D. Dissertation, Stanford University, Department of Computer Science, Stanford, California", August 2001.
  • [16] Deutsch, P., "DEFLATE Compressed Data Format Specification version 1.3", RFC 1951, May 1996.
  • [17] Deutsch, P., "GZIP file format specification version 4.3", RFC 1952, May 1996.
  • [18] Ramsdell, B., "Secure/Multipurpose Internet Mail Extensions (S/MIME) Version 3.1 Message Specification", RFC 3851, July 2004.
  • [19] Eastlake, D., Reagle, J., and D. Solo, "(Extensible Markup Language) XML-Signature Syntax and Processing", RFC 3275, March 2002.
  • [20] Freed, N., Klensin, J., and J. Postel, "Multipurpose Internet Mail Extensions (MIME) Part Four: Registration Procedures", RFC 2048, November 1996.
  • [21] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text", RFC 1521, November 1996.
  • [22] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: session initiation protocol", RFC 3261, June 2002.

Appendix A. Receiver operation (informative) ๐Ÿงš

okay so this section is a friendly little walkthrough of how a receiver might run a file delivery session. It's a rough sequence, not a strict state-by-state spec, so think of it as a vibe check for the whole flow ๐Ÿ’…

  1. First, the receiver grabs the description of the file delivery session, which is identified by the pair (source IP address, Transport Session Identifier) ๐ŸŽ€ It also collects the destination IP addresses and their matching ports for that session.
  2. Next, the receiver joins the channels so it can catch the packets for the session โœจ It may schedule this join using the timing info in the session description, if there is one ๐Ÿ•ฐ๏ธ
  3. Now it's receiving ALC/LCT packets for the session ๐Ÿ“ฆ It checks that each packet matches the declared Transport Session Identifier. If a packet doesn't match, it gets silently discarded ๐Ÿ™ˆ bye bye, not you.
  4. While packets are coming in, the receiver sorts them by their TOI and saves the useful bits in the right spot so it can rebuild the matching file ๐Ÿ—‚๏ธ Multiple files can be rebuilt at the same time, which is so efficient it's giving multitasking queen ๐Ÿ‘‘
  5. Then the receiver recovers an object ๐ŸŒธ An object is ready once it has an appropriate set of packets carrying Encoding Symbols for that transport object. What counts as "appropriate" depends on the FEC Encoding ID, the FEC Instance ID, and other info in the FEC Object Transmission Information ๐Ÿง 
  6. If the recovered object is an FDT Instance with FDT Instance ID 'N', the receiver parses the payload of instance 'N' and updates its FDT database to match ๐Ÿ“– It spots FDT Instances inside a session by the EXT_FDT header extension. Any object delivered with EXT_FDT is an FDT Instance, and each one is uniquely identified by its FDT Instance ID. Also, TOI '0' is exclusively reserved for FDT delivery, so don't try to use it for anything else ๐Ÿšซ
  7. If the recovered object is a file instead, the receiver looks up the FDT database for its properties and assigns the file those properties ๐Ÿ’– It also double-checks that the received content length matches the database entry. Optionally, if an MD5 checksum was used, the receiver checks that the calculated MD5 matches the one in the FDT database ๐Ÿ”
  8. Okay, now the messy stuff ๐Ÿ˜ญ What the receiver does with imperfectly received files (missing data, mismatching digests, and so on) is outside the scope of this specification. If a file gets recovered before its file description entry shows up, one possible move is to wait until an FDT Instance arrives that includes the missing properties โณ
  9. Finally, if the file delivery session end time hasn't been reached yet, the receiver goes back to step 3 ๐Ÿ” Otherwise, it's over, and that's a wrap ๐ŸŽฌ

Appendix B. Example of FDT Instance (informative) ๐Ÿ“

Here's a cute example of what an FDT Instance can look like in XML. Two files, one with a full set of properties and one with a little extra spice ๐ŸŒˆ

<?xml version="1.0" encoding="UTF-8"?>
<FDT-Instance xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:fl="http://www.example.com/flute"
xsi:schemaLocation="http://www.example.com/flute-fdt.xsd"
Expires="2890842807">
        <File
           Content-Location="http://www.example.com/menu/tracklist.html"
           TOI="1"
           Content-Type="text/html"/>
        <File
           Content-Location="http://www.example.com/tracks/track1.mp3"
           TOI="2"
           Content-Length="6100"
           Content-Type="audio/mp3"
           Content-Encoding="gzip"
           Content-MD5="+VP5IrWploFkZWc11iLDdA=="
           Some-Private-Extension-Tag="abc123"/>
</FDT-Instance>

Authors' Addresses ๐Ÿ’Œ

The people behind this one, with their contact details exactly as listed:

Toni Paila
Nokia
Itamerenkatu 11-13
Helsinki  FIN-00180
Finland

EMail: toni.paila@nokia.com


Michael Luby
Digital Fountain
39141 Civic Center Dr.
Suite 300
Fremont, CA  94538
USA

EMail: luby@digitalfountain.com


Rami Lehtonen
TeliaSonera
Hatanpaan valtatie 18
Tampere  FIN-33100
Finland

EMail: rami.lehtonen@teliasonera.com


Vincent Roca
INRIA Rhone-Alpes
655, av. de l'Europe
Montbonnot
St Ismier cedex  38334
France

EMail: vincent.roca@inrialpes.fr


Rod Walsh
Nokia
Visiokatu 1
Tampere  FIN-33720
Finland

EMail: rod.walsh@nokia.com

Copyright (C) The Internet Society (2004).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

Heads up, this is the legal fine print ๐Ÿ™ˆ The document and the information in it are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/S HE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property ๐Ÿฆ‹

The IETF takes no position on the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology in this document. It also takes no position on whether any license under those rights might or might not be available, and it doesn't claim to have made any independent effort to find such rights. Info on the IETF's procedures for rights in IETF Documents can be found in BCP 78 and BCP 79 ๐Ÿ“š

Copies of IPR disclosures made to the IETF Secretariat, and any assurances of licenses to be made available, can be obtained from the IETF's online IPR repository at http://www.ietf.org/ipr. The same goes for the result of any attempt to get a general license or permission for implementers or users of this specification to use those proprietary rights ๐Ÿ”

The IETF also invites any interested party to bring to its attention any copyrights, patents, patent applications, or other proprietary rights that may cover technology required to implement this standard. Please send that info to the IETF at ietf-ipr@ietf.org ๐Ÿ’œ


Acknowledgement ๐Ÿซถ

Funding for the RFC Editor function is currently provided by the Internet Society. Thank you, Internet Society, for keeping the lights on โœจ

๐Ÿ“ฆ where this page comes from