RFC 6108

Comcast's Web Notification System Design

๐Ÿ“… published February 2011, from ietf

Comcast's Web Notification System Design ๐ŸŽ€

okay bestie, RFC 6108 is an Independent Submission and it's Informational ๐Ÿงš It was written by C. Chung, A. Kasyanov, J. Livingood, and N. Mody (all from Comcast), plus B. Van Lieu (unaffiliated). It came out in February 2011 and has ISSN 2070-1721. There are no obsoletes or updates listed, so it's not replacing or changing anything else ๐Ÿ’…

Abstract โœจ

okay so the goal here is to describe a way of sending critical end-user notifications to web browsers, and Comcast (an ISP) actually deployed it ๐Ÿฆ‹ It's used for near-immediate heads-ups, like warning customers that their traffic looks like malware or a virus infection ๐Ÿ˜ญ Other proprietary systems can do this too, but they use Deep Packet Inspection (DPI) technology. This doc describes a system that does not rely on DPI. It's built on open IETF standards and open source apps, and it's giving main character energy ๐Ÿ‘‘

Status of This Memo ๐Ÿ’œ

This document is not an Internet Standards Track specification, so it's published just for info ๐ŸŒธ

This is a contribution to the RFC Series, independent of any other RFC stream. The RFC Editor picked to publish it at their discretion and makes no statement about its value for implementation or deployment. Documents approved for publication by the RFC Editor are not a candidate for any level of Internet Standard; see Section 2 of RFC 5741.

Info about the current status, errata, and how to give feedback is at http://www.rfc-editor.org/info/rfc6108. ๐Ÿ“

Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these docs carefully, because they describe your rights and restrictions with respect to this document ๐Ÿ™ˆ

Table of Contents ๐Ÿงพ

1. Introduction ....................................................3
2. High-Level Design of the System .................................3
3. Design Requirements .............................................3
   3.1. General Requirements .......................................4
   3.2. Web Proxy Requirements .....................................6
   3.3. ICAP Server Requirements ...................................7
   3.4. Messaging Service Requirements .............................8
4. Implementation Details ..........................................8
   4.1. Functional Components Described, as Implemented ............9
   4.2. Functional Diagram, as Implemented ........................10
5. High-Level Communication Flow, as Implemented ..................11
6. Communication between Web Proxy and ICAP Server, as
   Implemented ....................................................12
7. End-to-End Web Notification Flow, as Implemented ...............13
   7.1. Step-by-Step Description of the End-to-End Web
        Notification Flow .........................................14
   7.2. Diagram of the End-to-End Web Notification Flow ...........15
8. Example HTTP Headers and JavaScript for a Web Notification .....16
9. Deployment Considerations ......................................18
10. Security Considerations .......................................19
11. Debating the Necessity of Such a Critical Notification
    System ........................................................19
12. Suggesting a Walled Garden as an Alternative ..................20
13. Intended Next Steps ...........................................21
14. Acknowledgements ..............................................21
15. References ....................................................21
   15.1. Normative References .....................................21
   15.2. Informative References ...................................23

1. Introduction ๐Ÿฆ‹

okay so Internet Service Providers (ISPs) need a system that can talk to customers almost instantly, to deliver critical service notices, like warnings about a likely malware infection ๐Ÿ˜ญ The web browser is the most popular client software people use, so it's basically the perfect vehicle for these notifications ๐Ÿ’– This doc describes a system Comcast (a broadband ISP) deployed to send near-immediate notifications to web browsers โœจ

While evaluating options, the authors found that most commercially available systems used Deep Packet Inspection (DPI) tech. A DPI-based system would totally work, but Comcast and other ISPs are trying to avoid widespread DPI deployment and use, and are looking for alternatives ๐Ÿ” So Comcast wanted a system based on open standards and non-proprietary software that didn't require DPI ๐Ÿซถ The system described here is specific to the Data-Over-Cable Service Interface Specifications (DOCSIS, [CableLabs_DOCSIS]) networks used by most cable-based broadband ISPs. But the concepts in this document can generally be applied to lots of other network types if those ISPs want alternatives to DPI ๐ŸŒธ

2. High-Level Design of the System ๐Ÿงš

The web notification system design is based on the Internet Content Adaptation Protocol (ICAP) [RFC3507] ๐ŸŽ€ It uses open source apps: the Squid web proxy, the GreasySpoon ICAP server, and Apache Tomcat. ICAP is an existing IETF protocol that allows for message transformation or adaptation ๐Ÿ‘‘ An ICAP client passes a HyperText Transport Protocol (HTTP, [RFC2616]) response to an ICAP server for content adaption. The ICAP server then responds to the client with the HTTP response containing the notification message, using the "respmod" method defined in Section 3.2 of [RFC3507] ๐Ÿ’…

3. Design Requirements ๐Ÿ’œ

This section covers all of the key requirements Comcast considered when designing this system โœจ It's here to show the important design choices they made to avoid using DPI, among other things ๐Ÿ™ˆ Each requirement comes with an "Additional Background" paragraph that gives extra info, context, or other useful explanation ๐ŸŒˆ

3.1. General Requirements ๐Ÿ’…

R3.1.1. Must Only Be Used for Critical Service Notifications ๐Ÿšจ

Additional Background: okay so the system must only send critical notifications, not trivial ones. The big example, and the main reason this whole system exists, is telling a user that their computer is infected with malware, that their security is at severe risk and/or already compromised, and that they should take immediate, corrective action NOW. ๐Ÿ˜ญ

R3.1.2. Must Use TCP Port 80 ๐ŸŒ

Additional Background: the system must send notifications over TCP port 80, which is the well-known port for HTTP traffic. Most customers use a web browser as their main app, so this was deemed the best way to get them an immediate, critical notification. It's giving efficient and iconic ๐Ÿฆ‹

R3.1.3. Must Support Block Listing ๐Ÿšซ

Additional Background: it's unlikely, but the HyperText Markup Language (HTML, [RFC2854]) or JavaScript [RFC4329] used for notifications could cause problems when someone visits a particular website. So the system must be able to use a block list of website Uniform Resource Identifiers (URIs, [RFC3986]) or Fully Qualified Domain Names (FQDNs, Section 5.1 of [RFC1035]) that conflict with it. That way it simply doesn't send notifications for those sites, which keeps errors and weird results to a minimum. ๐Ÿงš

Also, even though a lot of development and testing went into making sure this system doesn't act unexpectedly, and standard ICAP (which has been in use for many years) is used, it is critical that if it ever does misbehave, there must be a way to rapidly exempt specific URIs or FQDNs. Safety net, bestie ๐Ÿซถ

R3.1.4. Must Not Cause Problems with Instant Messaging (IM) Clients Using TCP Port 80 ๐Ÿ’ฌ

Additional Background: some IM clients use TCP port 80 in their communications, often as a backup when the standard, well-known ports don't work. Other IM clients use TCP port 80 by default, and some are even based in a web browser. So this system must not conflict with, or cause unexpected results for, IM clients (or any other client types) that use TCP port 80. Not me causing drama with the chat apps ๐Ÿ’€

R3.1.5. Must Handle Pre-Existing Active TCP Sessions Gracefully ๐ŸŒธ

Additional Background: since the web notification system may temporarily re-route TCP port 80 traffic to deliver a critical notification, previously established TCP port 80 sessions must not be disrupted while they're being routed to the proxy layer. The critical notification also happens at a well-defined point in time, so it's logical to assume an end user may already have an active TCP port 80 session that's still running when the notification goes out. Those connections must not be reset, and they must be handled gracefully. Main character energy: nobody gets their session yanked away ๐Ÿ’–

R3.1.6. Must Not Use TCP Resets ๐Ÿ™…โ€โ™€๏ธ

Additional Background: TCP resets have been widely criticized, both across the Internet community in general and in [RFC3360]. Comcast, for example, got called out for using TCP resets while running a DPI-based network management system. So TCP resets must not be used as a function of this system. Absolutely not, we don't do that here ๐Ÿ’…

R3.1.7. Must Be Non-Disruptive ๐ŸŽ€

Additional Background: the web notification system must not disrupt the end-user experience, for example by causing significant client errors. Nobody wants their browsing to suddenly break ngl ๐Ÿ™ˆ

R3.1.8. User Notification Acknowledgement Must Stop Further Immediate Notifications โœ‹

Additional Background: once a user acknowledges a critical notification, the notification should immediately stop. Otherwise, the user may think the system is stuck in an error state, and they may not believe the critical notification is real. It's also pretty likely they'll be annoyed that the system ignored their acknowledgement, and honestly, same ๐Ÿ˜ค

R3.1.9. Non-Modification of Content Should Be Maintained ๐Ÿ“œ

Additional Background: the system should not significantly alter the content of the HTTP response from any website the user is visiting. Hands off the website's stuff, periodt โœจ

R3.1.10. Must Handle Unexpected Content Gracefully ๐Ÿงฉ

Additional Background: sometimes developers and implementers assume only a narrow range of inputs will ever show up, and that the designers thought of all of them ahead of time. The authors think that's a bad assumption. Instead, unexpected or even malformed inputs should be expected. So the system must gracefully and transparently handle traffic that is unexpected, even though there will be cases where it can't provide a critical web notification because of this. Widely varying content should be expected, and all of that unexpected traffic must be handled without giving the user visible errors or weird results. Plan for chaos, bestie ๐ŸŒˆ

R3.1.11. Web Content Must Not Be Cached ๐Ÿ”’

Additional Background: keeping users' privacy safe is important. So content that flows through the system, or that the system incidentally sees, must not be cached. Privacy is not optional ๐Ÿซถ

R3.1.12. Advertising Replacement or Insertion Must Not Be Performed Under ANY Circumstances ๐Ÿšซ๐Ÿ“ข

Additional Background: the system must not be used to replace any advertising a website provides, or to insert advertising into websites. That covers pages that already have ad space and pages that don't have any ads at all. This is a huge concern for end users, privacy advocates, and the wider Internet community, so it has to be crystal clear that this system will never be used for those purposes. No ads, no exceptions, iykyk ๐Ÿ‘‘

3.2. Web Proxy Requirements ๐Ÿฆ‹

R3.2.1. Open Source Software Must Be Used ๐ŸŒป

Additional Background: the system must use an open source web proxy server. (As noted in Section 2 and Section 4.1, Squid has been chosen.) It's possible to use any web proxy, but open source means others can easily access openly available documentation for the software, along with the other benefits people usually credit to open source. Transparency is so attractive ๐Ÿ’œ

R3.2.2. ICAP Client Should Be Integrated ๐Ÿงƒ

Additional Background: the web proxy server should have an integrated ICAP client, which makes the design and implementation simpler. Fewer moving parts, less stress ๐Ÿ“

R3.2.3. Access Control Must Be Implemented ๐Ÿ”

Additional Background: access to the proxy must be limited exclusively to the IP addresses of the users who are meant to get notifications, and only for limited periods of time. Also, because a Session Management Broker (SMB) is used, as described in Section 4.1 below, the proxy must restrict access only to the address of the SMB. Strict guest list, no plus-ones ๐Ÿ’…

3.3. ICAP Server Requirements ๐Ÿงš

R3.3.1. Must Provide ICAP Response Support ๐Ÿ”

Additional Background: the system must support response adaptation, in accordance with [RFC3507]. An ICAP client passes a HyperText Transport Protocol (HTTP, [RFC2616]) response to an ICAP server for content adaptation. The ICAP server then responds back to the client with the HTTP response containing the notification message, using the "respmod" method defined in Section 3.2 of [RFC3507]. Pass it, adapt it, send it back โœจ

R3.3.2. Must Provide Consistency of Critical Notifications ๐Ÿชž

Additional Background: the system must be able to consistently provide a specific notification. For example, if a critical alert telling a user they're infected with malware is wanted, that notification should look the same for every user and not vary. Same vibe for everyone, no mixed signals ๐Ÿ’–

R3.3.3. Must Support Multiple Notification Types ๐ŸŒˆ

Additional Background: the initial and only critical notification this system sends is meant to alert users about a malware infection. But malware is a rapidly and constantly evolving threat, so the system must be able to evolve and provide different types of critical notifications. For example, if malware starts splitting into several categories with very different implications for end users, it may become desirable to send a notification narrowly tailored to each category. Adaptable era ๐Ÿฆ„

R3.3.4. Must Support Notification to Multiple Users Simultaneously ๐Ÿ‘ฏโ€โ™€๏ธ

Additional Background: the system must be able to serve notifications to different users at the same time. For example, if 100 users have been infected with malware and critically need to hear about it, the system must be able to notify several users at a time, or all of them at once, rather than just one user at a time. Nobody gets left on read ๐Ÿ”ฅ

3.4. Messaging Service Requirements โœจ

R3.4.1. A Messaging Service Must Be Used ๐Ÿ’–

okay so the Messaging Service (see Section 4.1 below) caches the notifications for each specific user ๐Ÿงš that means the notification messages get stored by the system and don't have to be fetched again every single time one is needed. it's giving efficiency. because of that, the system can scale up to notify multiple users at once, which is what the earlier requirement "Must Support Notification to Multiple Users Simultaneously" is asking for ๐Ÿ’…

R3.4.2. Must Process Acknowledgements on a Timely Basis ๐Ÿซถ

the Messaging Service must process notification acknowledgements from end users quickly ๐Ÿƒโ€โ™€๏ธ this ties back to the earlier requirement "User Notification Acknowledgement Must Stop Further Immediate Notifications" so once someone acks, the spam stops (no cap) ๐Ÿ™…โ€โ™€๏ธ

R3.4.3. Must Ensure Notification Targeting Accuracy ๐ŸŽฏ

the Messaging Service must make sure notifications actually reach the people they're meant for ๐Ÿ’Œ for example, say the system wants to send a critical notification to User A and User B but NOT User C. then User C must not be sent a notification. no sneaky extras, periodt ๐Ÿ’€

R3.4.4. Should Keep Notification Records for Customer Support Purposes ๐Ÿ“

the Messaging Service should keep some kind of record that a notification was sent to a user ๐Ÿ—‚๏ธ this matters when someone calls customer support. picture this: a user sees the critical notification about a malware infection, and then they call Comcast's Customer Security Assurance team in the customer service org ๐Ÿ“ž the Customer Security Assurance rep should be able to confirm that the user really did get the malware notification while helping them clean up ๐Ÿงน that's the whole point of the record, bestie ๐ŸŒธ


4. Implementation Details ๐Ÿ“

this section lays out the core functional components of the system, the way they're actually implemented ๐Ÿ› ๏ธ then it shows them in a diagram so you can see how everything links up and relates to each other ๐Ÿฆ‹

4.1. Functional Components Described, as Implemented ๐ŸŒˆ

this section describes, honestly and transparently, the software (S) packages the system uses and all the details of how it works ๐Ÿ” the authors know there could be several alternative software choices for each component ngl ๐Ÿคทโ€โ™€๏ธ the point here is just to document the choices that were actually made and deployed ๐Ÿš€

  • S4.1.1. Web Proxy: the system uses Squid Proxy, an open source web proxy app that's widely used and has an integrated ICAP client ๐Ÿฆ‘
  • S4.1.2. ICAP Server: the system uses GreasySpoon, an open source app ๐Ÿฅ„ the ICAP server pulls the notifications from the Messaging Service cache when content adaptation is needed ๐Ÿ’ซ
  • S4.1.3. Customer Database: the Customer Database holds the info the system needs to send a critical notification to a given user ๐Ÿ—ƒ๏ธ it can also track which users were notified and which are still pending ๐Ÿ“‹
  • S4.1.4. Messaging Service: the system uses Apache Tomcat, an open source app ๐Ÿฑ it's a process engine that grabs specific web notification messages from a catalog of possible notifications ๐Ÿ“š right now only one notification is in use, the malware infection one, but as noted in Section 3.3 the system may eventually need multiple notifications (the requirement is "Must Support Multiple Notification Types") ๐Ÿงฉ when a notification for a specific user isn't in the cache, the process fetches it from the Customer Database and fills the cache for a set period of time โณ
  • S4.1.5. Session Management Broker (SMB): a Load Balancer (LB) with a customized layer 7 inspection policy tells HTTP and non-HTTP traffic apart on TCP port 80 ๐Ÿšฆ this is how the system meets the requirements in Section 3 above ๐Ÿ’… the LB is from A10 Networks ๐Ÿ”ฅ the SMB works as a full stateful TCP proxy and can forward packets from existing TCP sessions that aren't in its internal session table (this covers the requirement "Must Handle Pre-Existing Active TCP Sessions Gracefully") ๐Ÿซง new HTTP sessions get load balanced to the web proxy layer, either transparently or using source Network Address Translation (NAT [RFC3022]) from the SMB ๐ŸŒท

Non-HTTP traffic for established TCP sessions that aren't in the SMB session table just gets forwarded to its destination transparently through the TCP proxy layer ๐Ÿฆ‹ (again, same requirement: "Must Handle Pre-Existing Active TCP Sessions Gracefully") โœจ

4.2. Functional Diagram, as Implemented ๐Ÿ—บ๏ธ

   +--------+        +------------+        +----------+
   |  ICAP  | <----> | Messaging  | <----> | Customer |
   | Server |        |  Service   |        | Database |
   +--------+        +------------+        +----------+
     ^
     |                +----------+
     |                |          |
     |      +-------> | Internet | <-------+
     |      |         |          |         |
     |      |         +----------+         |
     |      |              ^               |
     v      v              |               |
   +----------+            v               v
   |+--------+|        +-------+       +--------+
   ||  ICAP  || <----> |  SMB  | <---> | Access |
   || Client ||        +-------+       | Router |
   |+--------+|                        +--------+
   || SQUID  ||                            ^
   || Proxy  ||                            |
   |+--------+|                            v
   +----------+                       +----------+
                                      |  CMTS*   |
                                      +----------+
                                          ^
                                          |
                                          v
                                       +------+
                                       |  PC  |
                                       +------+

    * A Cable Modem Termination System (CMTS)
      is an access network element.

         Figure 1: Web Notification System - Functional Components

5. High-Level Communication Flow, as Implemented ๐ŸŽ€

so in Section 4 we met the functional components, and Figure 1 showed how they connect ๐Ÿ”— this section walks through the high-level communication (C) flow of a transaction, so you can see how everything works together in action ๐ŸŽฌ there's a lot more detail in later sections, so stay tuned ๐Ÿ‘€

  • C5.1. Setup of Differentiated Services (Diffserv): using Diffserv [RFC2474] [RFC2475] [RFC2597] [RFC3140] [RFC3246] [RFC3260] [RFC4594], set a policy that sends TCP port 80 traffic to the web notification system's web proxy ๐Ÿ›ฃ๏ธ
  • C5.2. Session Management: TCP port 80 packets get routed to a Session Management Broker (SMB), which tells HTTP from non-HTTP traffic and new sessions from existing ones ๐Ÿš HTTP packets are forwarded to the web proxy by the SMB ๐ŸŒ non-HTTP packets, like instant messaging (IM) traffic, get forwarded to a TCP proxy layer for routing to their destination, or the SMB acts as a full TCP proxy and forwards those non-HTTP packets to the destination itself ๐Ÿ’ฌ pre-established TCP sessions on port 80 are identified by the SMB and forwarded with no impact ๐ŸŒ™
  • C5.3. Web Proxy Forwards Request: the web proxy forwards the HTTP request on to the destination site, a web server, just like a web proxy normally would ๐Ÿ“จ
  • C5.4. On Response, Send Message to ICAP Server: once the HTTP response comes back from the destination server, the web proxy sends a message to the ICAP server for the web notification ๐Ÿ“ฌ
  • C5.5. Messaging Service: the Messaging Service should respond with the appropriate notification content, or a null response if no notification is cached ๐Ÿซง
  • C5.6. ICAP Server Responds: the ICAP server responds and hands the web proxy the appropriate content for the web notification ๐ŸŽ
  • C5.7. Web Proxy Sends Response: the web proxy then forwards the HTTP response containing the web notification to the client's web browser ๐ŸŒธ
  • C5.8. User Response: the user sees the critical web notification and clicks an appropriate option, such as OK/acknowledged, snooze/remind me later, etc. ๐Ÿ‘†
  • C5.9. More Information: depending on the notification, the user may get more info ๐Ÿ’ก for example, as noted previously, the system was designed to send critical notifications about malware infections ๐Ÿฆ  so in a malware case, the user may be told to go to a malware remediation web page with directions on how to try removing the malware and how to help secure hosts against future infections ๐Ÿ›ก๏ธ
  • C5.10. Turn Down Diffserv: once the notification transaction is complete, remove any special Diffserv settings ๐Ÿงน

6. Communication between Web Proxy and ICAP Server, as Implemented ๐Ÿฆ„

the web proxy and ICAP server are the critical components of this system ๐Ÿ‘‘ this section shows the communication between those two components ๐Ÿ’œ

   +------------+
   |  www URI   |
   +------------+
      ^      |
   (2)|      |(3)
      |      v
     +--------+     (4)     +--------+     (4)     +--------+
     |        |------------>|        |------------>|        |
     |        |     (5)     |        |     (5)     |        |
     | Proxy  |<------------|  ICAP  |<------------|  ICAP  |
     | Module |     (6)     | Client |     (6)     | Server |
     |        |------------>|        |------------>|        |
     |        |     (7)     |        |     (7)     |        |
     |        |<------------|        |<------------|        |
     +--------+             +--------+             +--------+
      ^      |
   (1)|      |(8)
      |      v
   +------------+              (9)             +------------+
   |            |----------------------------->|            |
   |  Browser   |              (10)            | Web Server |
   |            |<-----------------------------|            |
   +------------+                              +------------+

   (1) - HTTP GET (TCP 80)
   (2) - Proxy HTTP GET (TCP 80)
   (3) - HTTP 200 OK w/ Response
   (4) - ICAP RESPMOD
   (5) - ICAP 200 OK
   (6) - TCP Stream - Encapsulate Header
   (7) - ICAP 200 OK Insert Message
   (8) - HTTP 200 OK w/ Response + Message Frame
   (9) - HTTP GET for Message
   (10) - HTTP 200 w/ Message Content

         Figure 2: Communication between Web Proxy and ICAP Server

7. End-to-End Web Notification Flow, as Implemented โœจ

okay bestie, this section walks through one full end-to-end notification, step by step, so you can see how the whole system actually moves together ๐Ÿฆ‹๐Ÿ’–

7.1. Step-by-Step Description of the End-to-End Web Notification Flow

Policy-Based Routing ๐Ÿ›ฃ๏ธ

  1. TCP port 80 packets from the user who needs to be notified get routed to the web proxy via policy-based routing ๐Ÿ’…
  2. Packets get forwarded to the Session Management Broker, which sets up a session with the web proxy and routes the packets to it ๐ŸŒธ

Web Proxy ๐ŸŽ€

  1. The user's HTTP request gets directed to the web proxy โœจ
  2. The web proxy receives the HTTP traffic and pulls content from the requested website ๐ŸŒ
  3. The web proxy gets the response and forwards it to the ICAP server for response adaptation ๐Ÿ”
  4. The ICAP server checks the HTTP content to figure out whether the notification message can be inserted ๐Ÿ”
  5. The ICAP server starts a request to the Messaging Service cache process with the IP address of the user ๐Ÿ“ฎ
  6. If a notification message for the user exists, the appropriate notification is cached on the Messaging Service. Then the Messaging Service sends the appropriate notification content back to the ICAP server ๐Ÿ’Œ
  7. Once the notification message comes out of the Messaging Service cache, the ICAP server may insert it into the HTTP response body, without changing or modifying the original content of the HTTP response ๐Ÿง
  8. The ICAP server then sends the response back to the web proxy, which forwards the HTTP response back to the browser ๐Ÿš€
  9. If the user's IP address isn't found or isn't provisioned for a notification message, the ICAP server should return a "204 No modifications needed" response to the ICAP client, as defined in Section 4.3.3 of [RFC3507]. So the user won't get any web notification message at all ๐Ÿ™ˆ
  10. The user sees the web notification and clicks an appropriate option, like OK/acknowledged or snooze/remind me later, etc. ๐Ÿ‘‘

7.2. Diagram of the End-to-End Web Notification Flow

The two figures below show the communications flow from the web browser, through the web notification system ๐Ÿงš

Figure 3 shows what happens when a notification request can't be inserted, because the notification type for the user's IP address isn't cached in the Messaging Service ๐Ÿ’€

                            ICAP     ICAP    Message          Customer
         Browser   Proxy   Client   Server   Service  Internet    DB
           |  HTTP  |         |         |        |        |        |
           |  GET   | Proxy   |         |        |        |        |
           +------->| Request |         |        |        |        |
           |        +---------|---------|--------|------->|        |
           |        |         |         |        | 200 OK |        |
           |        |<--------|---------|--------|--------+        |
           |        | ICAP    |         |        |        |        |
           |        | RESPMOD | ICAP    |        |        |        |
           |        +-------->| RESPMOD | Check  |        |        |
           |        |         +-------->| Cache  |        |        |
           |        |         |         | for IP |        |        |
           |        |         |         | Match  |        |        |
           |        |         |         +------->|        |        |
           |        |         |         | Cache  |        |        |
           |        |         |         | Miss   |        |        |
           |        |         |         |<-------+ Request|        |
           |        |         | 204 No  |        | Type   |        |
           |        |         | Modif.  |        +--------|------->|
           |        |         | Needed  |        |        |        |
           |        | No      |<--------+        |        | Type   |
           |        | Insert  |         |        |        |Returned|
           | 200 OK |<--------+         |        |<-------|--------+
           | w/o    |         |         |        |        |        |
           | Insert |         |         |        |        |        |
           |<-------+         |         |        |        |        |
           |        |         |         |        |        |        |

Figure 3: End-to-End Web Notification Flow - with Cache Miss

Figure 4 shows what happens when a notification request for the user's IP address is cached in the Messaging Service ๐Ÿ’œ

                            ICAP     ICAP    Message          Customer
         Browser   Proxy   Client   Server   Service  Internet    DB
           |  HTTP  |         |         |        |        |        |
           |  GET   | Proxy   |         |        |        |        |
           +------->| Request |         |        |        |        |
           |        +---------|---------|--------|------->|        |
           |        |         |         |        | 200 OK |        |
           |        |<--------|---------|--------|--------+        |
           |        | ICAP    |         |        |        |        |
           |        | RESPMOD | ICAP    |        |        |        |
           |        +-------->| RESPMOD | Check  |        |        |
           |        |         +-------->| Cache  |        |        |
           |        |         |         | for IP |        |        |
           |        |         |         | Match  |        |        |
           |        |         |         +------->|        |        |
           |        |         |         | Cache  |        |        |
           |        |         |         | Hit    |        |        |
           |        |         | Insert  |<-------+        |        |
           |        | Return  | Type    |        |        |        |
           |        | 200 OK  |<--------+        |        |        |
           |        | with    |         |        |        |        |
           |        | Insert  |         |        |        |        |
           | 200 OK |<--------+         |        |        |        |
           | w/     |         |         |        |        |        |
           | Notify |         |         |        |        |        |
           |<-------+         |         |        |        |        |
           |        |         |         |        |        |        |

Figure 4: End-to-End Web Notification Flow - with Cache Hit


8. Example HTTP Headers and JavaScript for a Web Notification ๐Ÿ“

This figure shows an example of a normal HTTP GET request from the user's web browser to www.example.com, a web server on the Internet ๐ŸŒˆ

------------------------------------------------------------------------
1.  HTTP GET Request to www.example.com
------------------------------------------------------------------------
http://www.example.com/

GET / HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14)
        Gecko/20080404 Firefox/2.0.0.14
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Pragma: no-cache
------------------------------------------------------------------------

Figure 5: Example HTTP Headers for a Web Notification - HTTP GET

In this next figure, the traffic goes through the web proxy, which talks to the ICAP server and returns the response from www.example.com. In this case, that response is a 200 OK, with the desired notification message inserted ๐ŸŽ€

------------------------------------------------------------------------
2.  Response from www.example.com via PROXY
------------------------------------------------------------------------
HTTP/1.x 200 OK
Date: Thu, 08 May 2008 16:26:29 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Tue, 15 Nov 2005 13:24:10 GMT
Etag: "b80f4-1b6-80bfd280"
Accept-Ranges: bytes
Content-Length: 438
Connection: close
Content-Type: text/html; charset=UTF-8
Age: 18
X-Cache: HIT from localhost.localdomain
Via: 1.0 localhost.localdomain (squid/3.0.STABLE5)
Proxy-Connection: keep-alive
------------------------------------------------------------------------

Figure 6: Example HTTP Headers for a Web Notification - HTTP Response

The figure below shows an example of the web notification content inserted in the 200 OK response. In this example, it's JavaScript code ๐Ÿงšโ€โ™€๏ธ

------------------------------------------------------------------------
3.  Example of JavaScript containing Notification Insertion
------------------------------------------------------------------------
<!--all elements used in a notification should have cascading style
sheet (css) properties defined to avoid unwanted inheritance from
parent page-->

<style type="text/css">
#example {
  position: absolute; left: 100px; top: 50px;
  z-index: 9999999; height: auto; width: 550px;
  padding: 10px;
  border: solid 2px black;
  background-color:#FDD017;
  opacity: 0.8; filter: alpha(opacity = 80);
}
</style>

<script language="javascript" type="text/javascript">
// ensure that content is not part of an iframe
if (self.location == top.location) {
  // this is a floating div with 80% transparency
  document.write('<div id="example" name="example">');
  document.write('<h2>IMPORTANT MESSAGE</h2>');
  document.write('<p>Lorem ipsum dolor sit amet, consecteteur ');
  document.write('adipisicing elit, sed do eiusmod tempor ');
  document.write('incididunt ut labore et dolore magna aliqua. ');
  document.write('Ut enim ad minim veniam, quis nostrud ');
  document.write('exercitation ullamco laboris nisi ut aliquip ex ');
  document.write('ea commodo consequat.');
  document.write('</div>');
}</script>
------------------------------------------------------------------------

Figure 7: Example JavaScript Used in a Web Notification


9. Deployment Considerations ๐ŸŒท

The parts of the web notification system should be spread out across the network and kept close to end users. This keeps routing performance and the user's browsing experience looking excellent โœจ Also, a HTTP-aware load balancer should be used in each datacenter where servers live, so traffic can be spread across N+1 servers and the system can be scaled up easily ๐Ÿ“ˆ

10. Security Considerations ๐Ÿ”

This is a seriously critical web notification system. It was built to give end user customers another way to be told that their computer has been infected with malware, and that's a big deal ๐Ÿ˜ญ

Depending on the exact text of the notification, users could fear that it's some kind of phishing attack ๐Ÿ˜จ So care has been taken with the text and any links inside the web notification itself. For example, if the notification text changes over time, it may be best to provide a general URI or a telephone number ๐Ÿ“ž On the flip side, the notification must not ask for login credentials, and it must not ask a user to follow a link to change their password, because those are super common phishing techniques ๐Ÿ™…โ€โ™€๏ธ

Finally, care should be taken to make people confident that the web notification is valid and comes from a trusted party, and/or that the user has an alternate way to check whether it's legit ๐Ÿซถ One alternate way to validate the notification may be to call customer support (in this example, Comcast's Customer Security Assurance team). This ties back to a key requirement, specifically "Should Keep Notification Records for Customer Support Purposes", in Section 3.4 ๐Ÿง 

11. Debating the Necessity of Such a Critical Notification System ๐Ÿซถ

okay bestie, let's talk about the spicy question. some community members might ask if it's EVER okay to change Internet content to send a critical service notice about malware infection. even in the smallest way, even if it's openly documented, even if it's thoroughly tested, and even if the motivations are good. ๐Ÿค”

anyone with those concerns should know this document is by no means the first to propose this, especially as a tactic against a security problem. it basically builds on earlier IETF work, like [RFC3507]. those folks should also study the many organizations using ICAP and the many software systems that have implemented ICAP. ๐Ÿ“š

also, please review Section 1. it explains that this is a common feature of DPI systems, made by DPI vendors and many, if not most, major networking equipment vendors. the authors are honestly trying to AVOID the need for widespread, ubiquitous deployment of DPI ๐Ÿ™…โ€โ™€๏ธ using open source software and open protocols. they're also motivated to transparently describe how the system works, what it IS intended to do, what it IS NOT intended to do, and the purposes for which it WILL NOT be used. โœจ

the authors also believe it's important for ISPs to transparently disclose network management techniques and systems, and to have a venue to do so, which is what happened here. ๐ŸŽ€ they think the IETF and the wider Internet community should encourage and positively reinforce those disclosures. publishing a document like this for reference and comment could motivate other ISPs to be just as open and to engage with the IETF and other organizations in the Internet community. not publishing would do the opposite: less disclosure, more DPI, and less ISP participation in the critical technical bodies that make up parts of the Internet community. ๐Ÿ˜ฌ

and it's critical that everyone recognizes the good motivations of ISPs like Comcast. malware is a massive, ongoing problem and a huge threat to the security of average Internet users. it's now a multi-billion-dollar underground economy running on identity theft, financial fraud, spam, and other criminal activity. ๐Ÿ’€ a critical notification system like this is only necessary because host-based security hasn't been successful at defending against and preventing malware infection. so ISPs like Comcast are being urged by their customers, by security and/or privacy organizations, and by governmental organizations to help solve this massive problem, since so many other tactics have been unsuccessful. ๐ŸŒˆ

for example, Howard Schmidt, the Special Advisory for Cyber Security to President Obama of the United States of America, said in 2005:

"As attacks on home-based and unsecured networks become as prevalent as those against large organizations, the need for ISPs to do everything they can to make security easier for their subscribers is critical for the preservation of our nation's information backbone. Additionally, there is tremendous potential to grow further the use of broadband around the world; and making safety and security part of an ISP's core offering will enable the end user to fully experience the rich and robust benefits broadband provides".

12. Suggesting a Walled Garden as an Alternative ๐Ÿฐ

okay so a "walled garden" is an environment that controls the information and services a subscriber can use, and what network access permissions they get. ๐ŸŒธ placing a user in a walled garden is another approach ISPs may take to notify users, and other documents and community efforts are exploring it as a possible alternative. so web notifications should be considered one of many possible notification methods that merit documentation. ๐Ÿฆ‹

BUT, it's giving challenges ๐Ÿ˜ญ. a walled-garden approach can pose challenges and may, in some cases, be considered disruptive to end users. for example, a user could be playing a game online on a dedicated, Internet-connected game console, which would likely stop working once they're placed in the walled garden. or the user might be on a telephone call using some kind of Voice Over IP (VoIP) device, which would also likely stop working in the walled garden. in both cases, the user isn't using a web browser, so they wouldn't have a way to figure out why their service seemingly stopped working. ๐Ÿ™ˆ


13. Intended Next Steps ๐Ÿงš

real talk: at the time of this writing, no existing IETF working group is focused on malware infection and related issues. ๐Ÿ’” so there wasn't a definite venue for this document, and it was submitted to the Independent Submissions Editor as an independent submission.

documenting and disclosing this system is already good for the Internet community, but publishing it has other perks too. members of the community, including IETF members, get a stable document to refer to if the community ever starts new work on malware, security, and critical notification to end users. ๐Ÿ”ฅ and in the true spirit of a Request for Comment, it's hoped that other members of the community will be motivated to propose alternative systems or other improvements. ๐Ÿ‘‘


14. Acknowledgements ๐Ÿ’œ

the authors want to thank Alissa Cooper for her review and comments on the document, and Nevil Brownlee for his excellent feedback. they also thank the others who reviewed the document. ๐Ÿซถโœจ


15. References ๐Ÿ“š

15.1. Normative References ๐Ÿ’–

   [RFC1035]  Mockapetris, P., "Domain names - implementation and
              specification", STD 13, RFC 1035, November 1987.

   [RFC2474]  Nichols, K., Blake, S., Baker, F., and D. Black,
              "Definition of the Differentiated Services Field (DS
              Field) in the IPv4 and IPv6 Headers", RFC 2474,
              December 1998.

   [RFC2475]  Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
              and W. Weiss, "An Architecture for Differentiated
              Services", RFC 2475, December 1998.

   [RFC2597]  Heinanen, J., Baker, F., Weiss, W., and J. Wroclawski,
              "Assured Forwarding PHB Group", RFC 2597, June 1999.

   [RFC2616]  Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
              Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
              Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.

   [RFC2854]  Connolly, D. and L. Masinter, "The 'text/html' Media
              Type", RFC 2854, June 2000.

   [RFC3022]  Srisuresh, P. and K. Egevang, "Traditional IP Network
              Address Translator (Traditional NAT)", RFC 3022,
              January 2001.

   [RFC3140]  Black, D., Brim, S., Carpenter, B., and F. Le Faucheur,
              "Per Hop Behavior Identification Codes", RFC 3140,
              June 2001.

   [RFC3246]  Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec,
              J., Courtney, W., Davari, S., Firoiu, V., and D.
              Stiliadis, "An Expedited Forwarding PHB (Per-Hop
              Behavior)", RFC 3246, March 2002.

   [RFC3260]  Grossman, D., "New Terminology and Clarifications for
              Diffserv", RFC 3260, April 2002.

   [RFC3507]  Elson, J. and A. Cerpa, "Internet Content Adaptation
              Protocol (ICAP)", RFC 3507, April 2003.

   [RFC3986]  Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
              Resource Identifier (URI): Generic Syntax", STD 66,
              RFC 3986, January 2005.

   [RFC4329]  Hoehrmann, B., "Scripting Media Types", RFC 4329,
              April 2006.

   [RFC4594]  Babiarz, J., Chan, K., and F. Baker, "Configuration
              Guidelines for DiffServ Service Classes", RFC 4594,
              August 2006.

15.2. Informative References ๐ŸŒˆ

   [CableLabs_DOCSIS]
              CableLabs, "Data-Over-Cable Service Interface
              Specifications", CableLabs Specifications, Various DOCSIS
              Reference Documents, <http://www.cablelabs.com/
              specifications/archives/docsis.html>.

   [RFC3360]  Floyd, S., "Inappropriate TCP Resets Considered Harmful",
              BCP 60, RFC 3360, August 2002.

Authors' Addresses ๐ŸŽ€

   Chae Chung
   Comcast Cable Communications
   One Comcast Center
   1701 John F. Kennedy Boulevard
   Philadelphia, PA  19103
   US
   EMail: chae_chung@cable.comcast.com
   URI:   http://www.comcast.com


   Alex Kasyanov
   Comcast Cable Communications
   One Comcast Center
   1701 John F. Kennedy Boulevard
   Philadelphia, PA  19103
   US
   EMail: alexander_kasyanov@cable.comcast.com
   URI:   http://www.comcast.com


   Jason Livingood
   Comcast Cable Communications
   One Comcast Center
   1701 John F. Kennedy Boulevard
   Philadelphia, PA  19103
   US
   EMail: jason_livingood@cable.comcast.com
   URI:   http://www.comcast.com


   Nirmal Mody
   Comcast Cable Communications
   One Comcast Center
   1701 John F. Kennedy Boulevard
   Philadelphia, PA  19103
   US
   EMail: nirmal_mody@cable.comcast.com
   URI:   http://www.comcast.com


   Brian Van Lieu
   Unaffiliated
   Bethlehem, PA  18018
   US
   EMail: brian@vanlieu.net
๐Ÿ“ฆ where this page comes from