RFC 2728
The Transmission of IP Over the Vertical Blanking Interval of a Television Signal
Network Working Group RFC 2728 💖
okay bestie, let's get into this one 🎀 RFC 2728 is titled The Transmission of IP Over the Vertical Blanking Interval of a Television Signal 📺✨
Here's the title block, the important bits:
- Network Working Group 🌸
- Request for Comments: 2728 🦋
- Authors: R. Panabaker (Microsoft), S. Wegerif (Philips Semiconductors), and D. Zigmond (WebTV Networks) 💅
- Category: Standards Track 👑
- Date: November 1999 🍓
- The title block doesn't list anything it obsoletes or updates.
Status of this Memo ✨
This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited 🫶
Copyright Notice 💜
Copyright (C) The Internet Society (1999). All Rights Reserved. 🌈
1. Abstract ✨
okay so the abstract is giving "here's the whole plan" 💅 This document describes a method for broadcasting IP data in a unidirectional manner using the vertical blanking interval of television signals. It includes a description for compressing IP headers on unidirectional networks, a framing protocol identical to SLIP, a forward error correction scheme, and the NABTS byte structures 🦋
2. Introduction 🎀
so this RFC proposes several protocols for sending IP datagrams using the Vertical Blanking Interval (VBI) of a television signal 📺 The VBI is the part of the TV signal you never actually see, but it can carry point-to-multipoint IP data services, which would relieve congestion and traffic on the usual Internet access networks 🫶 Wherever possible, these protocols use existing RFC standards and non-standards 💖
Traditionally, point-to-point connections (TCP/IP) have been used even for broadcast-type data 😭 Things like news feeds, stock quotes, newsgroups, weather reports, and the like are typically sent over and over to individual clients, instead of being broadcast once to the big crowd of users who want them 🙈
IP is quickly becoming the go-to way to distribute one-to-many data on intranets and the Internet ✨ With cheap PC hardware for receiving TV signals (plus the broadcast data streams riding along with them) coming soon, a defined standard for sending data over traditional broadcast networks is basically imperative 🔥 The lack of standards in this area, plus the cost of hardware, has kept traditional broadcast networks from becoming effective data deliverers to the home and office 💀
This document describes sending IP using the North American Basic Teletext Standard (NABTS), a recognized, industry-supported way of transporting data on the VBI 🧚 NABTS is traditionally used on 525-line television systems such as NTSC 📺 Another byte structure, WST, is traditionally used on 625-line systems such as PAL and SECAM 🌸 These generalizations have exceptions, so countries should be treated on an individual basis 🙈 These existing television system standards will let the TV and Internet communities provide inexpensive broadcast data services 💅 A set of existing protocols will sit on top of the specific FEC for NABTS, including a serial stream framing protocol similar to SLIP (RFC 1055 [Romkey 1988]) and a compression technique for unidirectional UDP/IP headers 🦋
The protocols in this document are meant for unidirectional delivery of IP datagrams using the VBI 🫶 So no return channel is described, or for that matter possible, in the VBI 🌈
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 💜
3. Proposed protocol stack 🧚
the protocol stack shows the layers used to transmit VBI data 🦋 Each layer has no knowledge of the data it encapsulates, so it's abstracted from the other layers 💅 At the link layer, NABTS defines the modulation scheme used to transport data on the VBI 📺 At the network layer, IP handles moving data to the appropriate clients 🫶 At the transport layer, UDP determines the flow of data to the appropriate processes and applications ✨
+-------------------+
| |
| Application |
| |
+-------------------+
| | )
| UDP | )
| | )
+-------------------+ (-- IP
| | )
| IP | )
| | )
+-------------------+
| SLIP-style |
| encapsulation |
| |
+-------------------+
| FEC |
|-------------------|
| NABTS |
| |
+---------+---------+
| |
| NTSC/other |
| |
+-------------------+
|
|
| cable, off-air, etc.
+--------<----<----<--------These protocols can be described with a bottom-up component model, using NABTS carried over NTSC modulation as the example 🧩
Video signal --> NABTS --> FEC --> serial data stream --> Framing
protocol --> compressed UDP/IP headers --> application dataso here's how to read that diagram 👀 TV signals carry NABTS packets, which contain a Forward Error Correction (FEC) protocol, modulated onto them 📺 The data in these sequential, ordered packets forms a serial data stream, and a framing protocol marks where IP packets are, with compressed headers, carrying application data 💖
The structure of these components and protocols is described in the following subsections 🌈
3.1. VBI 📺
The characteristics and definition of the VBI depend on the television system in use, whether that's NTSC, PAL, SECAM, or something else 🦋 For more info on TV standards worldwide, see ref [12] 💅
3.1.1. 525 line systems 🍓
A 525-line television frame is made of two fields of 262.5 horizontal scan lines each 📺 The first 21 lines of each field aren't part of the visible picture, and they're collectively called the Vertical Blanking Interval (VBI) ✨
Of those 21 lines, the first 9 are used while the cathode ray gets repositioned to the top of the screen 🙈 The remaining lines are available for data transport 🫶
There are 12 possible VBI lines, broadcast 60 times a second (each field 30 times a second) 🔥 In some countries, Line 21 is reserved for transporting closed captioning data (Ref.[11]) 💀 In that case there are 11 possible VBI lines, some or all of which could be used for IP transport 🧚 It should be noted that some of these lines are sometimes used for existing, proprietary, data and testing services 👑 So IP delivery becomes one more data service using a subset or all of these lines 💜
3.1.2. 625 Line Systems 🌸
A 625-line television frame is made of two fields of 312.5 horizontal scan lines each 📺 The first few lines of each field are used while the cathode ray gets repositioned to the top of the screen 🦋 The lines available for data insertion are 6-22 in the first field and 319-335 in the second field ✨
So there are 17 possible VBI lines, broadcast 50 times a second (each field 25 times a second), and some or all of them could be used for IP transport 💖 It should be noted that some of these lines are sometimes used for existing, proprietary, data and testing services 😭 IP therefore becomes one more data service using a subset or all of these lines 🌈
3.2. NABTS 🌸
okay so NABTS (the North American Basic Teletext Standard) is defined in the Electronic Industry Association's EIA-516, Ref. [2], and ITU.R BT.653-2, system C, Ref. [13]. it's the industry-accepted way of modulating data onto the VBI, usually of an NTSC signal. this section walks through the NABTS packet format as it's used to transport IP 💖
bestie, only a subset of the NABTS standard gets used, which is totally normal in NABTS implementations 🫶 if you want more on the standard and how it's implemented, EIA-516 has all the details ✨
the NABTS packet is a 36-byte structure encoded onto one horizontal scan line of a television signal, and it looks like this 👀
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| clock sync | byte sync | packet addr. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| packet address (cont.) | cont. index |PcktStructFlags|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| user data (26 bytes) |
: :
: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| | FEC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+the two-byte Clock Synchronization and one-byte Byte Synchronization sit at the start of every scan line that carries a NABTS packet. their job is to sync the decoding sampling rate and byte timing ⏰
the three-byte Packet Address field is Hamming encoded (as specified in EIA-516) and gives 4 data bits per byte, so there are 4096 possible packet addresses 🦋 these addresses tell apart services that come from the same source. the receiver needs that to work out which packets are related and part of the same service. NABTS packet addresses distinguish different data services, possibly inserted at different points of the transmission system, and most likely totally unrelated. Section 4 of this document covers Packet Addresses in detail 📖
the one-byte Continuity Index field is a Hamming encoded byte that goes up by one for each subsequent packet of a given Packet Address. the value counts from 0 to 15, increasing by one every time a data packet is transmitted. this is how the decoder can tell whether packets got lost in transmission 🧩
the Packet Structure field is also a Hamming encoded byte, and it describes the structure of the rest of the packet 🎀 the least significant bit is always "0" in this implementation. the second least significant bit says whether the Data Block is full: "0" means it's full of useful data, and "1" means some or all of it is filler data. the two most significant bits give the length of the suffix of the Data Block. in this implementation that's either 2 or 28 bytes (10 for a 2-byte FEC suffix, 11 for a 28-byte FEC suffix). that suffix is what powers the forward error correction in the next section 💅 here's the table of possible Packet Structure values:
Data Packet, no filler D0
Data Packet, with filler 8C
FEC Packet A1the Data Block field is 26 bytes. zero to 26 of those bytes are useful data (part of an IP packet or SLIP frame) and the rest is filler. data is byte-ordered least significant bit first. filler is marked by Ox15 followed by as many OxEA as it takes to fill the Data Block field. once you drop the filler, the sequential data blocks form an asynchronous serial stream of data 🌊
these NABTS packets get modulated onto the television signal one after another, on any combination of lines 📺
3.3. FEC 🔥
because VBI data only travels one way, Forward Error Correction (FEC) is needed so the receiver can trust the data 🛡️ the type of FEC described here, and in the appendix of this document, for NABTS has been around for years and is hugely popular with the broadcast industry. it can correct single-byte errors and single- and double-byte erasures in the data block and suffix of a NABTS packet. in a NABTS system the FEC algorithm chops a serial stream of data into 364-byte "bundles". the data is laid out as 14 packets of 26 bytes each. a function runs over the 26 bytes of each packet to get the two suffix bytes, and those (plus a header) complete the NABTS packet, (8+26+2) 🧮
for every 14 packets in the bundle, two extra packets get tacked on that carry only FEC data, and each of those holds 28 bytes of FEC data 💜 the first packet in the bundle has a Continuity Index of 0, and the two FEC-only packets at the end have Continuity Index values of 14 and 15. to get this data, you first write the packets into a table of 16 rows and 28 columns 🗂️
the same expression as above gets applied to the columns of the table, with the suffix now sitting in the bytes at the base of the columns, in rows 15 and 16. with NABTS headers on each of those rows, you end up with a bundle of 16 NABTS packets, ready to be modulated onto lines of the television signal in order 🎬
on the receiving end, these formulae can check the data with very high accuracy ✨ if single bit errors, double bit errors, single-byte errors, or single- and double-byte erasures show up in any row or column (including a whole packet that got lost), they can be corrected with the algorithms in the appendix. how well the correcting works depends on the particular implementation on the receiver 🤞
3.4. Framing 🧚
a framing protocol identical to SLIP is proposed for wrapping up the packets described in the following section, which keeps this data separate from the lower protocol layers 🎁 it uses two special characters: END (0xc0) and ESC (0xdb). to send a packet, the host sends the packet and then the END character. if a data byte in the packet has the same code as END, it gets replaced by a two-byte sequence of ESC (0xdb) and 0xdc. if a data byte has the same code as ESC, it gets replaced by a two-byte sequence of ESC (0xdb) and 0xdd instead. SLIP implementations are everywhere, so see RFC 1055 [Romkey 1988] for more detail 📚
+--------------+--+------------+--+--+---------+--+
| packet |c0| packet |db|dd| |c0|
+--------------+--+------------+--+--+---------+--+
END ESC ENDthe packet framed this way shall be formatted according to its schema type, which is identified by the schema field. the schema field starts every packet 🪄
+-----------+---------------------------------------------+
| schema | packet (formatted according to schema) |
| 1 byte | ?? bytes (schema dependant length) |
+-----------+---------------------------------------------+when the most significant bit in this field is set to "1", the field stretches to two bytes, which allows for 32768 additional schemas 🌈
+-----------+---------------------------------------------+
| extended | packet (formatted according to schema) |
| schema | ?? bytes (schema dependant length) |
| 2 bytes | |
+-----------+---------------------------------------------+section 3.5 describes one such schema, for sending IP. it's the only schema specified by this document. additional schemas may be proposed for other packet types or other compression schemes, as described in section 7 💜
3.4.1 Maximum Transmission Unit Size 🎀
the maximum length of an uncompressed IP packet, or Maximum-Transmission Unit (MTU) size, is 1500 octets 📏 packets larger than 1500 octets MUST be fragmented before transmission, and the client VBI interface MUST be able to receive full 1500 octet packet transmissions 🫶
3.5. IP Header Compression Scheme 🎀
okay so the one-byte scheme is basically the rulebook for how the IP packet itself gets encoded 📦. bandwidth is literally precious, so we wanna squeeze out every drop of efficiency possible 💅. one move is OPTIONAL compression of the UDP/IP header, which is kinda a cousin of the TCP/IP header compression Van Jacobson came up with (RFC 1144) 🦋. it's not an exact copy though, because the link only goes one way (unidirectional transmission, bestie 🙈). so this doc proposes one scheme for compressing UDP/IP version 4, and it gets the value 0x00. any future encapsulation schemes SHOULD use a unique value in this field ✨.
only schema 0x00 is defined in this document, and this schema MUST be supported by all receivers 🫶. in schema 0x00, the IP packet can show up in one of two forms. the first is full, uncompressed headers:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| group | uncompressed IP header (20 bytes) |
+-+-+-+-+-+-+-+-+ +
| |
: .... :
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | uncompressed UDP header (8 bytes) |
+-+-+-+-+-+-+-+-+ +
| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | payload (<1472 bytes) |
+-+-+-+-+-+-+-+-+ +
| |
: .... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+the first byte in the 0x00 scheme is the Compression Key 🔑. it's one byte: the high bit says whether the header got compressed, and the other seven bits tell you which compression group it belongs to 💜.
if the high bit of the Compression Key is set to zero, no compression happens and the full header gets sent, like in the diagram above. the client VBI interface SHOULD store the most recent uncompressed header for a given group value so it can rebuild later compressed headers 🧠. packets with identical group bits are assumed to have identical UDP/IP headers for all UDP and IP fields, except the IP identification and UDP checksum fields 🌸.
group values MAY be recycled following 60 seconds of nonuse by the preceding UDP/IP session (no uncompressed packets sent), or by sending packets with uncompressed headers for the 60-second duration following the last packet in the preceding UDP/IP session. furthermore, the first packet sent following 60 seconds of nonuse, or compressed header packets only use, MUST use an uncompressed header 🙌. client VBI interfaces SHOULD disregard compressed packets received 60 or more seconds after the last uncompressed packet using a given group address. this avoids any incorrectly decompressed packets due to group number reuse, and it limits the outage from a lost uncompressed packet to 60 seconds 😭.
if the high bit of the Compression Key is set to one, a compressed version of the UDP/IP header is sent. the client VBI interface MUST then combine the compressed header with the stored uncompressed header of the same group and recreate a full packet ✨. for compressed packets, the only portions of the UDP/IP header sent are the IP identification and UDP checksum fields:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| group | IP identification | UDP checksum|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|UDP cksm (cont)| payload (<1472 bytes) |
+-+-+-+-+-+-+-+-+ +
: .... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+all datagrams belonging to a multi fragment IP packet SHALL be sent with full headers, in the uncompressed header format. so only packets that have not been fragmented can be sent with the most significant bit of the compression key set to "1" 🚫.
a 32-bit CRC has also been added to the end of every packet in this scheme to ensure data integrity 🛡️. it's performed over the entire packet, including schema byte, compression key, and either compressed or uncompressed headers. it uses the same algorithm as the MPEG-2 transport stream (ISO/IEC 13818-1). the generator polynomial is:
1 + D + D2 + D4 + D5 + D7 + D8 + D10 + D11 + D12 + D16 + D22 + D23 +
D26 + D32as in the ISO/IEC 13818-1 specification, the initial state of the sum is 0xFFFFFFFF. this is not the same algorithm used by Ethernet. this CRC provides a final check for damaged datagrams that span FEC bundles or were not properly corrected by FEC 🎯.
4. Addressing Considerations 🏷️
the addressing of IP packets SHOULD adhere to existing standards in this area. including an appropriate source address is needed so the receiving client can tell sources apart (and thus services) if multiple hosts are sharing an insertion point and NABTS packet address 👀.
NABTS packet addressing is not regulated or standardized, and it requires care to make sure unrelated services on the same channel aren't broadcasting with the same packet address. this could happen because there are multiple possible NABTS insertion sites, including show production, network redistribution, regional operator, and local operator 📺.
traditionally, the marketplace has recognized this concern and made amicable arrangements for distributing these addresses for each channel 🤝.
5. IANA Considerations 📋
IANA will register new schemas on a "First Come First Served" basis [RFC 2434]. anyone can register a scheme, as long as they provide a point of contact and a brief description. the scheme number will be assigned by IANA. registrants are encouraged to publish complete specifications for new schemas (preferably as standards-track RFCs), but this is NOT REQUIRED 💖.
6. Security Considerations 🔐
as with any broadcast network, there are security issues due to the accessibility of data. it's assumed that the responsibility for securing data lies in other protocol layers, including the IP Security (IPSEC) protocol suite, Transport Layer Security (TLS) protocols, as well as application layer protocols appropriate for a broadcast-only network.
broadcast means the data is out there for anyone who can receive it, so this is seriously big deal territory 😬🚨
7. Conclusions 🎉
this document provides a method for broadcasting Internet data over a television signal for reception by client devices. with an appropriate broadcast content model, this will become an attractive method of providing data services to end users. by using existing standards and a layered protocol approach, this document has also provided a model for data transmission on unidirectional and broadcast networks 🌈.
8. Acknowledgements 💜
the description of the FEC in Appendix A is taken from a document prepared by Trevor Dee of Norpak Corporation. Dean Blackketter of WebTV Networks, Inc., edited the final draft of this document 🙏.
9. References 📚
[1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[2] Deering, S., "Host Extensions for IP Multicasting", STD 5, RFC 1112, August 1989.
[3] EIA-516, "Joint EIA/CVCC Recommended Practice for Teletext: North American Basic Teletext Specification (NABTS)" Washington: Electronic Industries Association, c1988
[4] International Telecommunications Union Recommendation. ITU-R BT.470-5 (02/98) "Conventional TV Systems"
[5] International Telecommunications Union Recommendation. ITU.R BT.653-2, system C
[6] Jack, Keith. "Video Demystified: A Handbook for the Digital Engineer, Second Edition," San Diego: HighText Pub. c1996.
[7] Jacobson, V., "Compressing TCP/IP Headers for Low-Speed Serial Links", RFC 1144, February 1990.
[8] Mortimer, Brian. "An Error-correction system for the Teletext Transmission in the Case of Transparent Data" c1989 Department of Mathematics and Statistics, Carleton University, Ottawa Canada
[9] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 2434, October 1998.
[10] Norpak Corporation "TTX71x Programming Reference Manual", c1996, Kanata, Ontario, Canada
[11] Norpak Corporation, "TES3 EIA-516 NABTS Data Broadcast Encoder Software User's Manual." c1996, Kanata, Ontario, Canada
[12] Norpak Corporation, "TES3/GES3 Hardware Manual" c1996, Kanata, Ontario, Canada
[13] Pretzel, Oliver. "Correcting Codes and Finite Fields: Student Edition" OUP, c1996
[14] Rorabaugh, C. Britton. "Error Coding Cookbook" McGraw Hill, c1996
[15] Romkey, J., "A Nonstandard for Transmission of IP Datagrams Over Serial Lines: SLIP", STD 47, RFC 1055, June 1988.
[16] Recommended Practice for Line 21 Data Service (ANSI/EIA-608-94) (Sept., 1994)
[17] Stevens, W. Richard. "TCP/IP Illustrated, Volume 1,: The Protocols" Reading: Addison-Wesley Publishing Company, c1994.
10. Acronyms 🔤
FEC - Forward Error Correction
IP - Internet Protocol
NABTS - North American Basic Teletext Standard
NTSC - National Television Standards Committee
NTSC-J - NTSC-Japanese
PAL - Phase Alternation Line
RFC - Request for Comments
SECAM - Sequentiel Couleur Avec Memoire
(sequential color with memory)
SLIP - Serial Line Internet Protocol
TCP - Transmission Control Protocol
UDP - User Datagram Protocol
VBI - Vertical Blanking Interval
WST - World System Teletext11. Editors' Addresses and Contacts 💌
Ruston Panabaker, co-editor
Microsoft
One Microsoft Way Redmond, WA 98052
EMail: rustonp@microsoft.com
Simon Wegerif, co-editor
Philips Semiconductors
811 E. Arques Avenue
M/S 52, P.O. Box 3409 Sunnyvale, CA 94088-3409
EMail: Simon.Wegerif@sv.sc.philips.com
Dan Zigmond, WG Chair
WebTV Networks
One Microsoft Way Redmond, WA 98052
EMail: djz@corp.webtv.net12. Appendix A: Forward Error Correction Specification 🧚
this FEC is optimized for data carried in the VBI of a television signal. teletext has been in use for many years, and data transmission errors have been sorted into three main types:
- randomly distributed single bit errors
- loss of lines of NABTS data
- burst errors
the quantity and distribution of these errors is highly variable and depends on many factors. the FEC is designed to fix all these types of errors 🎀✨
12.1. Mathematics used in the FEC 🧮
okay so Galois fields are the foundation of this whole FEC algorithm, bestie 💖 Rather than explain fields in general (snooze 😴), we're zooming straight into the exact one this algorithm uses ✨
The field is GF(2^8) with a primitive element alpha of 00011101. It's a set of 256 elements, plus the operations of "addition", "subtraction", "division", and "multiplication" on those elements. Each element is an 8-bit binary number 🎀
Here, "addition" and "subtraction" are the same operation: both are the XOR of two elements. So the "sum" of 00011011 and 00000111 is 00011100. Iconic, no notes 💅
Division of two elements is long division where the subtractions get swapped for XOR. Multiplication is standard long multiplication, except the final addition stage is replaced with XOR. Easy peasy 🦋
All arithmetic in the following FEC is done modulo 100011101. So after you multiply two numbers, you replace the result with its remainder when divided by 100011101. There are 256 values in this field (256 possible remainders), which are the 8-bit numbers. It is very important to remember that when we write A*B = C, we more accurately imply modulo(A*B) = C. 🚨
Multiplication and division are, ngl, super time consuming to do by hand. The good news is that elements of the Galois Field share two important properties with real numbers 🌈
A*B = POWERalpha(LOGalpha(A) + LOGalpha(B))
A/B = POWERalpha(LOGalpha(A) - LOGalpha(B))The Galois Field is limited to 256 entries, so the power and log tables are limited to 256 entries too. The addition and subtraction shown is standard, so the result must be modulo alpha. Written as a "C" expression, it's giving:
A*B = apower[alog[A] + alog[B]]
A/B = apower[255 + alog[A] - alog[B]]You may notice that alog[A] + alog[B] can be greater than 255, and therefore a modulo operation should be performed. This is not necessary if the power table is extended to 512 entries by repeating the table. The same logic is true for division as shown. The power and log tables are calculated once using the long multiplication shown above, so you only pay the cost one time 🥹✨
12.2. Calculating FEC bytes 🔢
The FEC algorithm splits a serial stream of data into "bundles". These are arranged as 16 packets of 28 bytes when the correction bytes are included. So the bundle has 16 horizontal codewords interleaved with 28 vertical codewords. It's a whole grid situation 🧚
Two sums are calculated for a codeword, S0 and S1. S0 is the sum of all bytes of the codeword, each multiplied by alpha to the power of its index in the codeword. S1 is the sum of all bytes of the codeword, each multiplied by alpha to the power of three times its index in the codeword. In "C", the sum calculations would look like:
Sum0 = 0;
Sum1 = 0;
For (i = 0;i < m;i++)
{
if (codeword[i] != 0)
{
Sum0 = sum0 ^ power[i + alog[codeword[i]]];
Sum1 = sum1 ^ power[3*i + alog[codeword[i]]];
}
}Note that the codeword order is different from the packet order 👀 Codeword positions 0 and 1 are the suffix bytes at the end of a packet horizontally, or at the end of a column vertically. Main character energy 👑
When calculating the two FEC bytes, the summation above must produce two sums of zero. All codewords except 0 and 1 are know (sic) so the sums for the known codewords can be calculated. Let's call these values tot0 and tot1 🫶
Sum0 = tot0^power[0+alog[codeword[0]]]^power[1+alog[codeword[1]]]
Sum1 = tot1^power[0+alog[codeword[0]]]^power[3+alog[codeword[1]]]This gives us two equations with two unknowns, which we can solve 🔥
codeword[1] = power[255+alog[tot0^tot1]-alog[power[1]^power[3]]]
codeword[0] = tot0^power[alog[codeword[1]]+alog[power[1]]]12.3. Correcting Errors 🩹
This section describes the procedure for detecting and correcting errors using the FEC data calculated above. Upon reception, we begin by rebuilding the bundle. This is perhaps the most important part of the procedure, because if the bundle is not built correctly, it cannot possibly correct any errors 😭 The continuity index is used to determine the order of the packets and whether any packets are missing (not captured by the hardware).
The recommendation, when building the bundle, is to convert it from packet order to codeword order. This simplifies the codeword calculations. You do this by taking the last byte of a packet and making it the second byte of the codeword, and taking the second-last byte of a packet and making it the first byte of a codeword. Also, the packet with continuity index 15 becomes codeword position one, and the packet with continuity index 14 becomes codeword position zero. Very specific, very important 🎯
The same FEC is used regardless of the number of bytes in the codeword. So let's think of the codewords as an array codeword[vert][hor], where vert is 16 packets and hor is 28. Each byte in the array is protected by both a horizontal and a vertical codeword, so it's double-covered, bestie 🛡️✨
For each of the codewords, the sums must be calculated. If both the sums for a codeword are zero, then no errors have been detected for that codeword. Otherwise, an error has been detected, and further steps need to be taken to see if the error can be corrected. In "C", the horizontal summation would look like:
for (i = 0; i < 16; i++)
{
sum0 = 0;
sum1 = 0;
for (j = 0;j < hor;j++)
{
if (codeword[i][j] != 0)
{
sum0 = sum0 ^ power[j + alog[codeword[i][j]];
sum1 = sum1 ^ power[3*j + alog[codeword[i][j]];
}
}
if ((sum0 != 0) || (sum1 != 0))
{
Try Correcting Packet
}
}Similarly, the vertical one looks like:
for (j = 0;i < hor;i++)
{
sum0 = 0;
sum1 = 0;
for (i = 0;i < 16;i++)
{
if (codeword[i][j] != 0)
{
sum0 = sum0 ^ power[i + alog[codeword[i][j]];
sum1 = sum1 ^ power[3*i + alog[codeword[i][j]];
}
}
if ((sum0 != 0) || (sum1 != 0))
{
Try Correcting Column
}
}12.4. Correction Schemes 🎀
This FEC provides four possible corrections, and honestly they're all kind of a glow-up:
- Single bit correction in codeword. All single bit errors.
- Double bit correction in a codeword. Most two-bit errors.
- Single byte correction in a codeword. All single-byte errors.
- Packet replacement. One or two missing packets from a bundle.
12.4.1. Single Bit Correction 💫
When correcting a single bit in a codeword, the byte and bit position must be calculated. The equations are:
Byte = 1/2LOGalpha(S1/S0)
Bit = 8LOGalpha(S0/POWERalpha(Byte))In "C", this is written:
Byte = alog[power[255 + alog[sum1] - alog[sum0]]];
if (Byte & 1)
Byte = Byte + 255;
Byte = Byte >> 1;
Bit = alog[power[255 + alog[sum0] - Byte]] << 3;
while (Bit > 255)
Bit = Bit - 255;The error is correctable if Byte is less than the number of bytes in the codeword and Bit is less than eight. For this math to be valid, both sum0 and sum1 must be non-zero. The codeword is corrected by:
codeword[Byte] = codeword[Byte] ^ (1 << Bit);12.4.2. Double Bit Correction 🧩
Double bit correction is way more complex than single bit correction, and there are two reasons. First, not all double bit errors are deterministic. That is, two different bit patterns can generate the same sums. Second, the solution is iterative. To find two bit errors, you assume one bit is in error and then solve for the second error as a single bit error. It's a bit of a puzzle, but a fun one 🙈
The procedure is to iteratively move through the bits of the codeword, changing each bit's state. The new sums are calculated for the modified codeword. Then the single bit calculation above determines whether this is the correct solution. If not, the bit is restored and the next bit is tried.
For a long codeword, this can involve many calculations. However, tricks can speed up the process. For example, the vertical sums give a strong indication of which bytes are in error horizontally. Bits in other bytes need not be tried. Smart, not hard 🧠💅
12.4.3. Single Byte Correction 🍓
For single byte correction, the byte position and bits to correct must be calculated. The equations are:
Byte = 1/2*LOGalpha(S1/S0)
Mask = S0/POWERalpha[Byte]Notice that the byte position is the same calculation as for single bit correction. The mask will allow more than one bit in the byte to be corrected. In "C", the mask calculation looks like:
Mask = power[255 + alog[sum0] - Byte]Both sum0 and sum1 must be non-zero for the calculations to be valid. The Byte value must be less than the codeword length, but Mask can be any value. This corrects the byte in the codeword by:
Codeword[Byte] = Codeword[Byte] ^ Mask12.4.4. Packet Replacement 📦
If a packet is missing, as determined by the continuity index, then its byte position is known and does not need to be calculated. The formula for single packet replacement is therefore the same as for the Mask calculation of single byte correction. Instead of XORing an existing byte with the Mask, the Mask replaces the missing codeword position:
Codeword[Byte] = MaskWhen two packets are missing, both the codeword positions are known by the continuity index. This again gives two equations with two unknowns, which is solved to give the following equations:
Mask2 = POWERalpha(2*Byte1)*S0+S1
POWERalpha(2*Byte1+Byte2) +POWERalpha(3*BYTE2)
Mask1 = S0 + Mask2*POWERalpha(Byte2)/POWERalpha(BYTE1)In "C", these equations are written:
if (sum0 == 0)
{
if (sum1 == 0)
mask2 = 0;
else
mask2=power[255+alog[sum1]-alog[power[byte2+2*byte1]
^power[3*Byte2]]];
}
else
{
if ((a=sum1^power[alog[sum0]+2*byte1]) == 0)
mask2 = 0;
else
mask2 =
power[255+alog[a]-alog[power[byte2+2*byte1]^power[3*byte2]]];
}
if (mask2 = 0)
{
if (sum0 == 0)
mask1 = 0;
else
mask1 = power[255+alog[sum0]-byte1];
}
else
{
if ((a=sum0^power[alog[mask2] + byte2]) == 0)
mask1 = 0;
else
mask1 = power[255+alog[a]-byte1];
}Notice that, in the code above, care is taken to check for zero values 🫧 The missing codeword position can be fixed by:
codeword[byte1] = mask1;
codeword[byte2] = mask2;12.5. FEC Performance Considerations ⚡
okay so this part is about how you actually use the error correction, not just what each fix does 🧠 The section above shows how to fix the different kinds of errors, but it doesn't tell you how to chain those fixes together to correct a whole bundle. There are lots of ways to do it, and the best one depends on stuff like:
- The amount of processing power you have 💻
- The number of packets per VBI you need to process 📦
- The type of hardware capturing the data 📡
- The delivery path of the VBI 🛤️
- How the code is implemented 🧑💻
bestie, the minimum recommended approach is to do single bit or single byte correction for one pass in each direction, then packet replacement if it makes sense 🫶 You're also allowed to do more than one pass in each direction. The idea is that an error the first pass missed might get fixed on the second pass, since correcting in the other direction already cleared some of the errors out of the way ✨ it's giving teamwork 💜
Now here's the thing you have to keep in your head: the code can be in several possible states 👀
- It says the codeword is correct, and it actually is ✅
- It says the codeword is correct, but it isn't. This is a detection failure 💀
- It says the codeword is incorrect but can't fix it. This is a detection 🙈
- It says the codeword is incorrect and fixes it correctly. This is a correction 🎉
- It says the codeword is incorrect but fixes the wrong bits. This is a false correction 😭
These categories overlap, which is so annoying 🥲 For example, a pair of sums might say you have a double bit error and a byte error at the same time. At the code level, you have no way of knowing which one is right and which one is a false correction. Honestly, neither might be right if both are false corrections 💅
BUT if you know something about the kinds of errors your delivery channel makes, you can seriously level up your efficiency 🔥 If errors are randomly spread out, like on a weak terrestrial broadcast, then single and double bit correction are stronger than single byte correction 🌈
13. Appendix B: Architecture 🏗️
okay the architecture this document is about breaks down into three areas: insertion, the distribution network, and the receiving client 🧩
Inserting IP data onto the television signal can happen at any point in the delivery system 📺 A VBI encoder usually takes in a video signal plus an asynchronous serial stream of bytes that form framed IP packets. It then packetizes that data onto a chosen set of lines using NABTS and an FEC. That composite signal gets modulated with other channels and then broadcast onto the distribution network. Operators further down the distribution chain can add their own data to other unused lines too 🫧
The distribution networks include coax plant, off-air, and analog satellite systems. They're mostly unidirectional broadcast networks 📡 They have to provide a signal to noise ratio that's good enough for the FEC to recover lost data, otherwise broadcasting the data won't work well at all 😤
The receiving client needs to be able to do a bunch of things 💖 It has to tune, do NABTS waveform sampling where appropriate, filter on NABTS addresses where appropriate, run forward error correction, unframe the data, verify the CRC, and decompress the UDP/IP header if it's compressed. The best part? All of this can run in PC software and cheap off-the-shelf hardware 🤑 no cap, that's so accessible ✨
14. Appendix C: Scope of proposed protocols 🌸
the protocols in this document are for sending IP packets. Their scope could stretch to other applications outside this area though 🦋 Lots of these protocols could run on any unidirectional network.
The unidirectional framing protocol encapsulates IP datagrams on the serial stream. The UDP/IP header compression cuts down on overhead when you transmit, which saves bandwidth 🧚 Together, these two protocols could be widely used on different unidirectional broadcast networks or modulation schemes to move any kind of packet data efficiently. New versions of Internet protocols could be supported too, which gives us a standardized way to transport data 👑
15. Full Copyright Statement ©️
Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement 🙏
Funding for the RFC Editor function is currently provided by the Internet Society. 💜
📦 where this page comes from
- ietf: rfc2728 1999-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms