RFC 6561

RFC 6561: Recommendations for the Remediation of Bots in ISP Networks ✨

Part 1 of 7 🎀

Okay bestie, let's start with the header info 💅. This is RFC 6561, and it's an Informational doc from the IETF (Internet Engineering Task Force). The authors are J. Livingood, N. Mody, and M. O'Reirdan, all from Comcast. It was published in March 2012, and its ISSN is 2070-1721. It doesn't obsolete or update any other RFCs, since it has no Obsoletes or Updates lines. Main character energy, no cap 🦋

Abstract 🌸

Okay so this doc is all about how ISPs can use different remediation techniques to deal with malicious bot infestations on their subscribers' computers 💻. Users with infected machines are at risk of losing personal data and become way more vulnerable to online fraud 😭. Those computers can also end up, without anyone meaning for it to happen, as part of an online crime network, a spam network, and/or a phishing network. They can even get used in a distributed denial-of-service attack 💀. Cleaning up and remediating bot installs makes it harder for botnets to operate, and could lower the amount of online crime on the Internet in general and/or on a particular ISP's network 🔥.

Status of This Memo 🧚

This document is not an Internet Standards Track specification; it is published for informational purposes.

It's a product of the IETF, so it reflects the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). Not every doc the IESG approves is up for any level of Internet Standard, so check out Section 2 of RFC 5741 if you're curious 🤓

If you want the current status, any errata, or a way to send feedback, head to http://www.rfc-editor.org/info/rfc6561 💜

Copyright Notice 🌈

Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they're in effect on the date of publication. Please actually read these, because they spell out your rights and restrictions for this document 📜. Any Code Components pulled out of this document must include the Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License.

Table of Contents 📖

   1. Introduction ....................................................3
      1.1. Key Terminology ............................................3
           1.1.1. Malicious Bots, or Bots .............................3
           1.1.2. Bot Networks, or Botnets ............................4
           1.1.3. Host ................................................5
           1.1.4. Malware .............................................5
           1.1.5. Fast Flux ...........................................5
   2. Problem Statement ...............................................6
   3. Important Notice of Limitations and Scope .......................7
   4. Detection of Bots ...............................................8
   5. Notification to Internet Users .................................12
      5.1. Email Notification ........................................13
      5.2. Telephone Call Notification ...............................13
      5.3. Postal Mail Notification ..................................14
      5.4. Walled Garden Notification ................................14
      5.5. Instant Message Notification ..............................16
      5.6. Short Message Service (SMS) Notification ..................16
      5.7. Web Browser Notification ..................................17
      5.8. Considerations for Notification to Public Network
           Locations .................................................18
      5.9. Considerations for Notification to Network
           Locations Using a Shared IP Address .......................18
      5.10. Notification and End User Expertise ......................19
   6. Remediation of Hosts Infected with a Bot .......................19
      6.1. Guided Remediation Process ................................21
      6.2. Professionally Assisted Remediation Process ...............22
   7. Failure or Refusal to Remediate ................................23
   8. Sharing of Data from the User to the ISP .......................23
   9. Security Considerations ........................................23
   10. Privacy Considerations ........................................24
   11. Acknowledgements ..............................................24
   12. Informative References ........................................26
   Appendix A.  Examples of Third-Party Malware Lists ................28

1. Introduction 🎀

This is the part where the doc says what it's all about 👑. Basically, it gives recommendations on how ISPs can use various remediation techniques to manage the effects of malicious bot infestations on their subscribers' computers. Internet users with infected computers are exposed to things like losing personal data and getting more susceptible to online fraud. Those computers can also become inadvertent participants in, or components of, an online crime network, a spam network, and/or a phishing network, and they can be used as part of a distributed denial-of-service attack. Mitigating these effects and remediating bot installations makes it more difficult for botnets to operate, and could reduce the level of online crime on the Internet in general and/or on a particular ISP's network 🔥

1.1. Key Terminology 🦋

This section defines the key terms used in this document. Read this first, bestie, because the rest of the doc relies on it 🧠

1.1.1. Malicious Bots, or Bots 🤖

A malicious or potentially malicious bot (short for "robot", and from here on just called a "bot") is a program installed on a system so that the system can automatically, or semi-automatically, perform a task or a set of tasks. This usually happens under the command and control of a remote administrator, known as the "bot master". Bots are also called "zombies" 🧟. They may have been installed sneakily, without the user fully understanding what the bot will do once it's on there. Sometimes it's unknowing, like tagging along with some other software install, or it's done under false pretenses, or it happens in a bunch of other possible ways 😬

It's important to note that there are "good" bots ✨. Good bots often show up in places like gaming and Internet Relay Chat (IRC) [RFC1459], where a continual, interactive presence can be a requirement for taking part in the games. Since good bots are doing useful, lawful, and non-disruptive stuff, there's no reason for a provider to monitor for them or alert users to their presence 🙌

Even so, there may be good or harmless bots out there, but for the purposes of this document, all mention of bots shall assume that the bots involved are malicious or potentially malicious in nature. Malicious bots shall generally be assumed to have been deployed without the permission or conscious understanding of a particular Internet user. So without the user knowing, bots can turn the user's device into a platform for malicious activities 💀. That category also explicitly includes potentially malicious bots, which may look neutral at first but could just be sitting around waiting for remote instructions to transform or start doing bad stuff. In general, installing a malicious bot without the user's knowledge and consent is considered unlawful in most regions, and malicious bots usually get up to unlawful or otherwise seriously disruptive activities 🚨

1.1.2. Bot Networks, or Botnets 🕸️

A "bot network", or "botnet", is a coordinated network of bots that can act on instructions generated remotely. Its malicious activities either focus on the information on the local machine or act to provide services for remote machines. Bots are super customizable, so they can be programmed to do a whole lot of things 😈. The major malicious activities include, but are not limited to, identity theft, spam, spim (spam over Instant Messaging (IM)), spit (spam over Internet telephony), email address harvesting, distributed denial-of-service (DDoS) attacks, key-logging, fraudulent DNS pharming (redirection), hosting proxy services, fast flux (see Section 1.1.5) hosting, hosting of illegal content, use in man-in-the-middle attacks, and click fraud 🍓

Infection vectors (infection pathways) include un-patched operating systems, software vulnerabilities (which include so-called zero-day vulnerabilities where no patch yet exists), weak or non-existent passwords, malicious web sites, un-patched browsers, malware, vulnerable helper applications, inherently insecure protocols, protocols implemented without security features switched on, and social engineering techniques to gain access to the user's computer 😭. Detecting and destroying bots is an ongoing issue, and it's a constant battle between the Internet security community and network security engineers on one side and bot developers on the other ⚔️

Bots used to talk over IRC, and they were easy to shut down once the command and control server was found and deactivated 🙃. Newer command and control methods have evolved, and the ones bot masters use now make bots way more resistant to deactivation. With peer-to-peer (P2P) architectures and their protocols, the use of HTTP and other resilient communication protocols, and the widespread adoption of encryption, bots are a lot harder to identify and isolate from normal network usage. So people are leaning harder on anomaly detection and behavioral analysis, both locally and remotely, to find bots 🔍

1.1.3. Host 💻

In this document, the host or computer of an end user means a computing device that connects to the Internet. That covers devices users actually use, like personal computers (including laptops, desktops, and netbooks), mobile phones, smart phones, home gateway devices, and other end user computing devices that are connected, or can connect, to the public Internet and/or private IP networks 📱

More and more household systems and devices have embedded hosts that are connected, or can connect, to the public Internet and/or private IP networks too. But these devices may not be under the interactive control of the Internet user, which can be the case with various smart home and smart grid devices 🏠⚡

1.1.4. Malware 🦠

Malware is short for "malicious software" 🙈. In this case, malicious bots count as a subset of malware. Other forms of malware could include viruses and other similar types of software. Internet users can sometimes get their hosts infected with malware, which may include a bot or cause a bot to install itself, just by accidentally visiting a specific web site, downloading a file, or doing other stuff online 😵‍💫

Other times, Internet-connected hosts get infected with malware through externally initiated malicious activities, like exploiting vulnerabilities or brute force guessing of access credentials 🔑

1.1.5. Fast Flux ✨

okay so DNS fast fluxing 🦋 is when a domain gets bound in DNS using A records pointing at multiple IP addresses, and every one of those records has a super short Time-to-Live (TTL) value attached 😭. The result? The domain keeps resolving to different IP addresses over a short stretch of time. it's giving shapeshifter energy 🧚

DNS fast flux is typically used alongside proxies that usually run on compromised user hosts 💀. Those proxies route the web requests along to the real host, which is the one actually serving the data people are looking for. So the real host becomes way harder to detect, and the backend or hidden site stays up for as long as possible. iconic behaviour, honestly 💅

---

2. Problem Statement 🎀

Hosts used by Internet users, which in this case are customers of an Internet Service Provider (ISP), can get infected with malware that may contain and/or install one or more bots on a host 🦠. And ngl, they can be a major problem for an ISP for a bunch of reasons (not to mention the problems they create for the users themselves 😿). First off, these bots can be used to send spam, and in some cases the volumes are honestly massive [Spamalytics] 📨. That spam can cost ISPs extra in wasted network, server, and/or personnel resources, plus a whole lot of other potential costs and side effects 💸. It can also hurt the reputation of the ISP, their customers, and the email reputation of the IP address space the ISP uses (often just called "IP reputation") 📉. Here's the tea: IP space that's been dragged by bad reputation may keep carrying that bad rep even when it's used for totally innocent stuff after the space gets reassigned 🥲.

And it gets worse 😤. These bots can act as platforms for directing, participating in, or otherwise conducting attacks on critical Internet infrastructure [Threat-Report] 🚨. Bots are frequently part of coordinated DDoS attacks for criminal, political, or other motivations [Gh0st][Dragon][DDoS] ⚔️. For example, bots have been used to attack Internet resources and infrastructure ranging from web sites to email servers and DNS servers, plus the critical Internet infrastructure of entire countries [Estonia][Combat-Zone] 🌍. The motivations behind these coordinated DDoS attacks can range from criminal extortion attempts all the way to online protesting and nationalistic fervor [Whiz-Kid] 🔥. DDoS attacks may also come from simple personal vendettas, or from people who just want a cheap thrill at someone else's expense 🙄.

There is good evidence that bots are being used in corporate environments for corporate espionage, including exfiltration of corporate financial data and intellectual property 🕵️‍♀️. This also extends to the possibility of bots being used for state-sponsored purposes like espionage 🌐.

While any computing device can be infected with bots, the majority of bot infections hit the personal computers used by Internet end users 💻. Because ISPs provide IP connectivity (among many other services) to Internet users, they're in a unique position to attempt to detect and observe botnets operating in their networks 👀. ISPs may also be in a unique position to notify their customers of actual, potential, or likely infection by bots or other infection 📬.

From the end user's side, being told they might have an infected computer on their network is seriously important info 🫶. Once they know, they can remove the bots, resolve any problems that may come from the infection, and protect themselves against future threats 🛡️. It's important to notify users that they may be infected because bots can eat up vast amounts of local computing and network resources, enable theft of personal information (including personal financial information), enable the host to be used for criminal activities (which may make the Internet user legally culpable), and destroy or leave the host in an unrecoverable state via "kill switch" bot technologies 💀💀.

So the intent of this document is to give guidance to ISPs and other organizations on remediating hosts infected with bots 🧹. The goal is to reduce the size of botnets and minimize the potential harm bots can inflict on Internet infrastructure in general, as well as on individual Internet users 🌸. Efforts by ISPs and other organizations can, over time, shrink the pool of bot-infected hosts on the Internet, which in turn could mean smaller botnets with less capability for disruption 📉✨.

The potential mitigation of bots happens through a process of detection, notification to Internet users, and remediation of bot infections, using a variety of tools described later in this document 🧰.

---

3. Important Notice of Limitations and Scope 🚧

Heads up, bestie: the techniques in this document in no way guarantee the remediation of all bots 🙈.

Bot removal can be a task that requires specialized knowledge, skills, and tools, so it may be beyond the ability of average users 🥴. Attempts at bot removal may often be unsuccessful, or only partially successful, leaving the user's system unstable and unsatisfactory, or even still infected 😩. Attempts at removal can also cause side effects ranging from a loss of data to partial or complete loss of system usability 💔.

In general, the only way a user can be sure they've removed some of today's increasingly sophisticated malware is by "nuking-and-paving" the system 💣. That means reformatting the drive, reinstalling the operating system and applications (including all patches) from scratch, and then restoring user files from a known clean backup 💾. However, the introduction of persistent memory-based malware may mean that, in some cases, this isn't enough, and it may prove to be more than any end user can reasonably be expected to handle [BIOS] 😵. Experienced users would have to re-flash or re-image persistent memory sections or components of their hosts to remove persistent memory-based malware 🔧. And in some cases, not even nuking-and-paving will solve the problem, which calls for hard drive replacement and/or complete replacement of the host 🥀.

Devices with embedded operating systems, such as video gaming consoles and smart home appliances, will most likely be beyond a user's capability to fix on their own, so they could require vendor-specific advice, updates, and tools 🎮🏠. Still, in some cases, these devices have a function or switch that lets the user reset them to a factory default configuration, which may sometimes help the user remediate the infection ✨. Care should be taken when giving remediation advice to Internet users, given the increasingly wide array of computing devices that can be, or could be, infected by bots in the future 🌈.

This document is not intended to address issues relating to the prevention of bots on an end user device. That's out of scope for this document 🙅‍♀️💜.

4. Detection of Bots ✨

okay bestie, first things first 🦋: an ISP has to figure out that an Internet user is infected, or likely infected, with a bot. (We're assuming the user is their customer or is otherwise connected to the ISP's network.) The ISP should try to detect bots using methods, processes, and tools that keep the personally identifiable information (PII) of their customers private 🔒. The ISP should not block legitimate traffic during bot detection 🙅‍♀️, and should instead use detection methods, tools, and processes that are non-disruptive and transparent to Internet users and end user applications. The vibe is "don't break anything while you're checking" 💅.

Detection methods, tools, and processes can include analysis of specific network and/or application traffic flows (like traffic to an email server), analysis of aggregate network and/or application traffic data, data feeds from other ISPs and organizations (like lists of the ISP's IP addresses that have been reported for sending spam), feedback from the ISP's customers or other Internet users, and a whole bunch of other possibilities ✨. In practice, it has proven effective to confirm a bot infection by combining multiple bot detection data points. This helps corroborate information of varying dependability or consistency, and it helps avoid or minimize false positive identification of hosts 🎀. Detection should also, where possible and feasible, try to classify the specific bot infection type. The goal is to confirm it's malicious in nature, estimate the variety and severity of threats it may pose (like a spam bot, a key-logging bot, a file distribution bot, etc.), and figure out potential methods for eventual remediation 🧚. However, botnet management is super dynamic, and criminals are motivated by quick financial rewards, so botnets frequently update or change their core capabilities 😭. That means a botnet the ISP initially detected and classified as made up of one particular type of bot needs to be continuously monitored and tracked, so the ISP can correctly identify the threat the botnet poses at any particular point in time 👑.

Detection is also time sensitive ⏰. If complex analysis is required and multiple confirmations are needed to verify that a bot is really present, the bot may cause some damage (to either the infected host or a remotely targeted system) before it can be stopped 💀. So an ISP needs to balance the desire or need to definitively classify and/or confirm a bot's presence, which may take an extended period of time, with the ability to predict the likelihood of a bot in a very short period of time 🌸. Those determinations must have a relatively low false positive rate in order to maintain the trust of users 💜. This "definitive-versus-likely" challenge is difficult. When in doubt, ISPs should err on the side of caution by communicating that a bot infection has taken place 🫶. This also means Internet users may benefit from installing client-based security software on their host. That software can enable rapid heuristically based detection of bot activity, such as detecting a bot as it starts to communicate with other botnets and execute commands 🔥. Any bot detection system should also be capable of adapting, either through manual intervention or automatically, so it can cope with a rapidly evolving threat 🌈.

As noted above, detection methods, tools, and processes should ensure that the privacy of customers' personally identifiable information (PII) is maintained 🔐. That protection for PII should also extend to third parties processing data on behalf of ISPs. Bot detection methods, tools, and processes are similar to the spam and virus defenses an ISP deploys for the benefit of its customers (and may be directly related to those defenses). Even so, attempts to detect bots should take into account the ISP's need to take care that any PII collected or incidentally detected is properly protected 🙈. This matters because, just as spam defenses may involve scanning the content of email messages (which may contain PII), bot defenses may similarly come into incidental contact with PII 📬. The definition of PII varies from one jurisdiction to the next, so proper care should be taken to ensure that any actions taken comply with legislation and good practice in the jurisdiction where the PII is gathered 🌍. Finally, depending on the geographic region an ISP operates in, certain bot detection methods may need to be included in relevant terms of service documents or other documents that are available to that ISP's customers 📄.

There are several bot detection methods, tools, and processes an ISP may choose to use, as listed below ✅. It's important to note that the technical solutions available are relatively immature and are likely to change over time, evolving rapidly in the coming years 🚀. While these items are described in relation to ISPs, they may also be applicable to organizations operating other networks, such as campus networks and enterprise networks 🏫.

a. Where it is not legally proscribed and is an accepted industry practice in a particular market region, an ISP may in some manner "scan" its IP space to detect un-patched or otherwise vulnerable hosts, or to detect signs of infection. This may give the ISP the opportunity to easily identify Internet users who appear to be already infected or who are at great risk of being infected with a bot. ISPs should note that some types of port scanning may leave network services in a hung state or render them unusable due to common frailties, and that many modern firewall and host-based intrusion detection implementations may alert the Internet user to the scan. As a result, the scan may be interpreted as a malicious attack against the host. Vulnerability scanning has a higher probability of leaving accessible network services and applications in a damaged state, and it will often result in a higher probability of detection by the Internet user and subsequent interpretation as a targeted attack 😬. Depending on the vulnerability an ISP is scanning for, some automated methods of vulnerability checking may result in data being altered or created afresh on the Internet user's host, which can be a problem in many legal environments ⚖️. It should also be noted that, due to the prevalence of Network Address Translation devices, Port Address Translation devices, and/or firewall devices in user networks, network-based vulnerability scanning may be of limited value. So while we note that this is one technique that may be used, it is unlikely to be particularly effective and has problematic side effects, which leads the authors to recommend against the use of this particular method 🙅‍♀️.

b. An ISP may also communicate and share selected data, via feedback loops or other mechanisms, with various third parties 🤝. Feedback loops are consistently formatted feeds of real-time (or nearly real-time) abuse reports offered by threat data clearinghouses, security alert organizations, other ISPs, and other organizations. The formats for feedback loops include those defined in both the Abuse Reporting Format (ARF) [RFC5965] and the Incident Object Description Exchange Format (IODEF) [RFC5070]. The data may include, but is not limited to, IP addresses of hosts that appear to be either definitely or probably infected, IP addresses, domain names, or fully qualified domain names (FQDNs) known to host malware and/or be involved in the command and control of botnets, recently tested or discovered techniques for detecting or remediating bot infections, new threat vectors, and other relevant information 📡. A few good examples of data sharing are noted in Appendix A 💡.

c. An ISP may use Netflow [RFC3954] or other similar passive network monitoring to identify network anomalies that may be indicative of botnet attacks or bot communications 🕵️‍♀️. For example, an ISP may be able to identify compromised hosts by spotting traffic destined to IP addresses associated with the command and control of botnets, or destined to the combination of an IP address and a control port associated with a command and control network (sometimes command and control traffic comes from a host that also has legitimate traffic). In addition, bots may be identified when a remote host is under a DDoS attack, because hosts participating in the attack will likely be infected by a bot. This can often be observed at network borders, although ISPs should beware of source IP address spoofing techniques that may be used to avoid or confuse detection 🎭.

d. An ISP may use DNS-based techniques to perform detection 🔍. For example, a given classified bot may be known to query a specific list of domain names at specific times or on specific dates. In the example of the so-called "Conficker" bot (see [Conficker]), detection is often done by matching DNS queries to a well-known list of domains associated with malware. In many cases, such lists are distributed by or shared through third parties, such as threat data clearinghouses 🗂️.

e. Because hosts infected by bots are frequently used to send spam or participate in DDoS attacks, the ISP servicing those hosts will normally receive complaints about the malicious network traffic 📨. Those complaints may be sent to role accounts specified in RFC 2142 [RFC2142], such as abuse@, or to other relevant addresses, such as abuse or security addresses the site specifies as part of its WHOIS (or other) contact data 📇.

f. ISPs may also discover likely bot-infected hosts located on other networks 👀. So when it's legally permissible in a particular market region, it may be worthwhile for ISPs to share information about those compromised hosts with the relevant remote network operator, security researchers, and blocklist operators 🌐.

g. ISPs may operate or subscribe to services that provide "sinkholing" or "honeynet" capabilities 🍯. This may enable the ISP to obtain near-real-time lists of bot-infected hosts as they attempt to join a larger botnet or propagate to other hosts on a network 🪤.

h. ISP industry associations should examine the possibility of collating statistics from ISP members, so they can provide good statistics about bot infections based on real ISP data 📊.

i. An Intrusion Detection System (IDS) can be a useful tool to actually help identify the malware 🔬. An IDS tool such as Snort (an open source IDS platform; see [Snort]) can be placed in a walled garden and used to analyze end user traffic to confirm the malware type. This will help with remediation of the infected device 🛠️.

5. Notification to Internet Users ✨

okay so, once an ISP has detected a bot (or the strong likelihood of one), they should totally go tell the Internet user that they might have a bot problem 🚨 an ISP should pick the best method or methods for reaching their customers, and that depends on a bunch of stuff: what the ISP can technically do, how their network is built, money, server resources, org resources, how many infected hosts they're seeing at any given time, and how bad the threats could be. Notification can use one or more of the methods in the subsections below, plus other methods that aren't listed here 💅

none of these methods is guaranteed to work 100% of the time, and each one has its own limitations 😭 sometimes an ISP will find that a combo of two or more methods works best, and that lowers the chance that malware blocks the notification. so the authors recommend using multiple notification methods 🫶 and notification is time sensitive, bestie. if the user doesn't get or look at the notification quickly, a bot could launch an attack, exploit the user, or cause other harm 💀 if it's possible, an ISP should set up a preferred way to communicate when the subscriber first signs up for service 🎀

also, as part of notifying people, ISPs should keep a record of which IP addresses were assigned to which subscribers, for long enough that common bot detection tech can accurately link an infected IP address to a subscriber 🔍 but the record should only be kept for as long as it's needed to support bot detection, and no longer, to protect the subscriber's privacy 🦋

one big thing to remember: notifications need to be resistant to spoofing 🙅‍♀️ the goal is to protect, as reasonably as possible, against legit notifications getting faked, or against people using fake notifications to launch more attacks on malware victims, or even to deliver more malware. yikes 😬

the end user should be able to pick their preferred notification method by opting in to that method 💖 it is recommended that the end user should not be allowed to opt out of notification entirely.

when users get notified, an ISP should try to share as much info as possible about which bot detection methods they use at the ISP 🕵️‍♀️ but only as long as it doesn't tip off the people creating or deploying the bots so they can dodge detection. balance is everything 🧚

5.1. Email Notification

email is a super common way for ISPs to notify people 💌 but it has a drawback: it's not guaranteed to get read in a reasonable time frame, or at all 😩 the user might be using a different primary email address than the one they gave the ISP. some ISPs don't even give out email accounts as part of their internet bundle, and they don't really need or have a way to ask for or keep the primary email addresses of their users 🙈

another possibility is that the user, their email client, or their email servers decide the notification is spam 🗑️ which could delete it or stick it in a folder the user doesn't check often or on time. bot masters have also been known to impersonate the ISP or a trusted sender and send fake emails to users 🎭 this social engineering trick often leads to new bot infestations, which is so messy. and finally, if the user's email credentials get compromised, a hacker or a bot could just log into the account and delete the notification before the user ever reads it 😤

5.2. Telephone Call Notification

a phone call can be really effective in particularly high-risk situations 📞 but it might not be feasible, because making a huge number of calls costs a lot in time, money, org resources, server resources, or other ways of measuring it 💸 there's also no guarantee the user will pick up 📵 and if the infected device is the thing answering the phone, the bot on that host might be able to disconnect, divert, or otherwise mess with an incoming call 👾 users can also think it's a telemarketing call and not welcome it, or just not accept it at all 🙄 and finally, even if an ISP rep actually gets to talk to a user, that user is very likely to lack the technical expertise to understand the threat or deal with it effectively 🥺

5.3. Postal Mail Notification

postal mail is probably the least popular and least effective way to communicate 📭 it takes forever to prep, forever to deliver, and it costs money for printing, paper, and postage 💀

5.4. Walled Garden Notification

a walled garden is another approach ISPs can take to notify users 🌸 it's an environment that controls which information and services a subscriber can use, and which network access permissions they get 🧱 a walled garden can be anywhere from strict to leaky. in a strict walled garden, access to most of the Internet is usually limited by the ISP 🔒 a leaky walled garden, on the other hand, lets the user access all Internet resources except the ones deemed malicious, and it makes sure they can still reach the ones that can be used to notify users about infections 💦

walled gardens work well because the ISP can notify the user and block all communication between the bot and its command and control channel at the same time 🙌 in lots of cases the user is almost guaranteed to see the notification and take any remediation steps 💯 but this approach can cause other challenges, bestie. for example, the user might not be using a host with a web browser, or a browser might not be running, or they could be using an app that uses ports redirected to the walled garden 🎮 one example: someone is playing an online game on a dedicated, Internet-connected game console 🕹️ another: the user might be on a phone call or waiting for a call on a Voice over IP (VoIP) device 📱 because of this, the ISP may feel like it needs a potentially long white list of domains that aren't subject to the usual walled garden restrictions, which could be a seriously onerous task operationally 😩

for these reasons a leaky walled garden makes more sense, but it has its own set of drawbacks 🫠 the ISP has to assume the user will eventually use a web browser to acknowledge the notification, because otherwise the user stays stuck in the walled garden and doesn't even know it 😵 if the only purpose of the leaky walled garden is to notify users about the bot infection, it isn't ideal, since notification is time sensitive, and the user might not get the notification until they request the targeted service or resource 😬 that means the bot can potentially do more damage. also, the ISP has to figure out which services or resources to restrict for notification purposes 🧐 this doesn't have to be resource specific and can be time based or policy based. for example, notification could happen on a timed basis, like every 10 minutes for all HTTP requests, or the notification could show up for one in every five HTTP requests ⏰

the ISP has a few options for deciding when to let the user out of the walled garden 🚪 one approach is to let the user decide when to exit 💁‍♀️ this is suggested when the main purpose of the walled garden is just to notify users and give remediation info, especially since notification doesn't guarantee successful remediation 🧹 it could also happen that, for whatever reason, the user decides they can't take the time to clean their host right now, and that other online stuff they want to get back to matters more 😅 exiting the walled garden might also involve a process to verify that it's actually the user asking to leave, and not the bot 🤖

once the user acknowledges the notification, they can either remediate and leave the walled garden, or leave without fixing the problem 🤷‍♀️ another approach is a stricter policy: the user has to clean the host before they're allowed out of the walled garden 🧼 this might not be technically feasible, though, depending on the type of bot, what obfuscation techniques the bot uses, and a bunch of other factors 🌀 so the ISP may also need tools that scan the infected host (like a virus scan, not a port scan) to figure out whether it's still infected, or rely on the user's judgment that the bot has been disabled or removed 🔎 one challenge here is that a user might have multiple hosts sharing a single IP address, like through a common home gateway device that does Network Address Translation (NAT) 🏠 in that case, the ISP may need to figure out, from user feedback or other means, that all affected hosts have been remediated, which may or may not be technically feasible 🫣

finally, when a walled garden is used, a list of well-known addresses for both operating system vendors and security vendors should be created and maintained in a white list that lets access to those sites through 📋 this matters a lot because end users who are looking for operating system and application patches need to be able to reach them from inside the walled garden 🩹 it is recommended that walled gardens be seriously considered as a notification method, since they're easy to implement and have been proven effective at getting end user attention 📣🌈

5.5. Instant Message Notification 💬

Okay so IM gives the ISP a simple way to reach the user, and it's honestly kind of iconic for that. If the ISP provides IM service and the user subscribes to it, the user can be notified easily ✨. IM-based notification can be a cost-effective way to communicate with users automatically from an IM alert system, or manually through the ISP's support staff. Ideally, the ISP should allow the user to register their IM identity in an ISP account management system and grant permission to be contacted this way. And if the IM service provider supports off-line messaging, the user can be notified whether or not they're currently logged into the IM system 🫶.

BUT there are drawbacks, bestie 😭. There's a high probability that a subscriber may read the message as spim and just ignore it 💀. Also, not every user uses IM, and the user may not have given their IM identity to the ISP, so some alternative means have to be used. Even if a user does have an IM address, they might not be signed on when the notification is attempted. There may also be a privacy concern for users when an IM notification has to travel over a third-party network and/or IM service 🙈. So if this method gets used, the notification should be discreet and must not include any PII in the notification itself. No PII in the message, periodt 🔒.

---

5.6. Short Message Service (SMS) Notification 📱

SMS lets the ISP send a brief description of the problem to notify the user, usually on a mobile device like a mobile phone or smart phone 📲. Ideally, the ISP should allow the user to register their mobile number and/or SMS address in an ISP account management system and grant permission to be contacted this way. The main perk of SMS is that people are used to getting texts and are likely to actually read them 👀. However, users may not act on the notification right away if they aren't in front of their host when it arrives.

One downside is that ISPs may have to follow up with an alternate means of notification if not all of the necessary information fits in one message, because of the character limits on individual messages (typically 140 characters) 📏. Another downside is the cost. The ISP has to either build its own SMS gateway to interface with the various wireless network service providers, or use a third-party SMS clearinghouse (relay) to notify users. In both cases, an ISP may rack up fees related to SMS notifications, depending on the method used 💸. An extra downside is that SMS messages sent to a user may result in a charge to the user from their wireless provider, depending on the plan they subscribe to and the country they live in. And a minor one: it's possible to notify the wrong user if the intended user changes their mobile number but forgets to update it with the ISP 🤦‍♀️.

There are several other drawbacks with this method too. There's a high probability that a subscriber may read the message as spam and ignore it. Also, not every user uses SMS, and/or the user may not have provided their SMS address or mobile number to the ISP. Even when a user does have one, their device may not be powered on or otherwise available on a wireless network when the notification is attempted. There may also be a privacy concern for users when an SMS notification has to go over a third-party network and/or SMS clearinghouse. So if this method gets used, the notification should be discreet and not include any PII in the notification itself 🫣.

---

5.7. Web Browser Notification 🌐

Near-real-time notification to the user's web browser is another technique that may be used to notify the user [RFC6108], though how such a system might operate is outside the scope of this document 🤷‍♀️. It could have a comparative advantage over a walled garden notification, because it doesn't restrict traffic to a specified list of destinations the way a walled garden would, by definition 🙅‍♀️. However, just like with a walled garden notification, there's no guarantee that a user is making use of a web browser at any given time. Though honestly, such a system could definitely provide a notification whenever that browser is eventually used 🦋. Compared to a walled garden, a web browser notification is probably preferred from the perspective of Internet users, since it doesn't risk disrupting non-web sessions, such as online games, VoIP calls, etc. (as noted in Section 5.4) 🎮📞.

There are alternative methods of web browser notification offered commercially by a number of vendors. Many of the techniques used are proprietary, and it's not within the scope of this document to describe how they're implemented. These techniques have been successfully implemented at several ISPs 🏆.

It should be noted that web notification is only intended to notify devices running a web browser.

---

5.8. Considerations for Notification to Public Network Locations 🚉

Delivering a notification to a location that provides a shared public network, like a train station, public square, coffee shop, or similar spot, may be of low value 📉. Why? The users connecting to these networks are typically highly transient and generally not known to site or network administrators. For example, a system may detect that a host on such a network has a bot, but by the time a notification is generated, that user has already departed from the network and moved elsewhere 🏃‍♀️💨.

---

5.9. Considerations for Notification to Network Locations Using a Shared IP Address 🏢

Delivering a notification to a location that accesses the Internet routed through one or more shared public IP addresses may be of low value too, since it can be quite difficult to tell users apart when providing a notification 😵‍💫. For example, on a business network of 500 users, all sharing one public IP address, it may be sub-optimal to send a notification to all 500 users if you only need one specific user to be notified and take action. As a result, such networks may find value in establishing a localized bot detection and notification system, just as they're likely to establish other localized systems for security, file sharing, email, and so on 🔐.

However, should an ISP implement some form of notification to such networks, it may be better to simply send notifications to a designated network administrator at the site. In that case, the local network administrator may like to receive additional information in the notification, such as a date and timestamp, the source port of the infected system, and malicious sites and ports that may have been visited 🗂️.

---

5.10. Notification and End User Expertise 🧠

The ultimate effectiveness of any of the aforementioned forms of notification is heavily dependent on both the expertise of the end user and the wording of the notification 💅. For example, while a user may receive and acknowledge a notification, they may lack the technical expertise to understand the threat or deal with it effectively. So it's important that notifications use clear, easily understood language, so the majority of users (who are non-technical) can actually get it 🙌. On top of that, a notification should provide easily understood guidance on how to remediate a threat, as described in Section 6, potentially with one path for technical users and another for non-technical users 🛠️✨.

6. Remediation of Hosts Infected with a Bot ✨

Okay bestie, this section is all about remediation, which just means getting a host cleaned up: removing the bot, disabling it, or otherwise making it totally harmless 💅 By this point the ISP has already spotted the bot, told the user one of their hosts is infected, and might now offer some recommended ways to clean things up 🧹 The generally recommended vibe is that the ISP should hand the user the right tools and education so they can remediate it themselves 🫶 Removing a bot is risky and genuinely tricky, so this is the move, ngl.

For example, the ISP might make a special website full of security-focused content just for this purpose 🌸 It should be a well-publicized security website that a user with a bot infection can be sent to for remediation. The site should clearly explain why the user was notified, and it may include an explanation of what bots are and the threats they pose. There should be a clear explanation of the steps the user should take to try cleaning their host, plus info on how users can keep the host free of future infections. The site should also have a guided process that takes non-technical users through remediation, step by step, in a way that's easy to understand 💖

When it comes to explaining what bots are and the threats they pose, something simple like this may do the trick:

What is a bot?  A bot is a piece of software, generally installed
on your machine without your knowledge, which either sends spam or
tries to steal your personal information.  They can be very
difficult to spot, though you may have noticed that your computer
is running much more slowly than usual or you may notice regular
disk activity even when you are not doing anything.  Ignoring this
problem is risky to you and your personal information.  Thus, bots
need to be removed to protect your data and your personal
information.

Many bots are designed to be super stealthy 🙈 Because of that, there may be a need to make sure the Internet user really understands how big the threat is, even though the bot is so sneaky.

It's also important to note that it may not be immediately obvious to the Internet user which devices have been infected with a particular bot 🤔 This may be because of the user's home network setup. A home network can include several hosts, and when a home gateway that performs Network Address Translation (NAT) is used to share a single public IP address, any of those devices can be infected with a bot. So the consequence for an ISP is that remediation advice may not be immediately actionable for the Internet user, since they may need to do some extra investigating inside their own home network 🕵️‍♀️

An added complication is that the user may have a bot infection on a device such as a video console, multimedia system, appliance, or other end user computing device that doesn't have a typical desktop computing interface 🎮 So the ISP should, where possible, be diligent about identifying and communicating the specific nature of the infected device, and then provide further appropriate remediation advice. If the ISP can't pin down the device or identify its type, then it should make it clear to the user that any initial advice is generic, and that further advice can be given (or is available) once the type of infected device is known 🦋

There are a bunch of online forums out there that give security-related support to end users 💜 These forums are staffed by volunteers and often focus on using a common tool set to help end users remediate hosts infected with malware. It may be advantageous for ISPs to build a relationship with one or more forums, perhaps by offering free hosting or other forms of sponsorship 🌈

It's also important to keep in mind that not all users will be technically adept, as noted in Section 5.10. So it may be more effective to offer a range of remediation options. This may include a very detailed "do it yourself" approach for experts, a simpler guided process for the average user, and even assisted remediation as described in Section 6.2 🍓

---

6.1. Guided Remediation Process

Minimally, the Guided Remediation Process should include the following goals, with options and/or recommendations for achieving them:

1. Back up personal files. For example:

Before you start, make sure to back up all of your important
data.  (You should do this on a regular basis anyway.)  You
can back up your files manually or using a system backup
software utility, which may be part of your Operating System
(OS).  You can back up your files to a USB Thumb Drive (aka
USB Key), a writable CD/DVD-ROM, an external hard drive, a
network file server, or an Internet-based backup service.

It may be advisable to suggest that the user backup is performed onto separate backup media or devices if they suspect bot infection 💾

2. Download OS patches and Anti-Virus (A/V) software updates. For example, links could be provided to Microsoft Windows updates, Apple Mac OS updates, or other major operating systems that are relevant to users and their devices 🛡️
3. Configure the host to automatically install updates for the OS, A/V, and other common web browsers such as Microsoft Internet Explorer, Mozilla Firefox, Apple Safari, Opera, and Google Chrome ⭐
4. Get professional assistance if they are unable to remove the bots themselves. If purchasing professional assistance, then the user should be encouraged to predetermine how much they are willing to pay for that help. For example, if the host being remediated is old and can easily be replaced with a new, faster, larger, and more reliable system for a certain cost, then it makes no sense to spend more than that cost to fix the old host. On the other hand, if the customer has a brand-new host, it might make perfect sense to spend the money to attempt to remediate it 💰
5. To continue, regardless of whether the user or a knowledgeable technical assistant is working on remediating the host, the first task should be to determine which of multiple potentially infected machines may be the one that needs attention (in the common case of multiple hosts in a home network). Sometimes, as in cases where there is only a single directly attached host, or the user has been noticing problems with one of their hosts, this can be easy. Other times, it may be more difficult, especially if there are no clues as to which host is infected. If the user is behind a home gateway/router, then the first task may be to ascertain which of the machines is infected. In some cases, the user may have to check all machines to identify the infected one 🔍
6. ISPs may also look at offering a CD/DVD with remediation processes and software in the event that a host is so badly infected as to be unable to communicate over the Internet 💿
7. User surveys to solicit feedback on whether the notification and remediation process is effective and what recommended changes could be made in order to improve the ease, understandability, and effectiveness of the remediation process 📝
8. If the user is interested in reporting the host's bot infection to an applicable law enforcement authority, then the host effectively becomes a cyber "crime scene", and the infection should not be mitigated unless or until law enforcement has collected the necessary evidence. For individuals in this situation, the ISP may wish to provide links to local, state, federal, or other relevant computer crime offices. (Note: Some "minor" incidents, even if highly traumatic to the user, may not be sufficiently serious for law enforcement to commit some of their limited resources to an investigation.) In addition, individual regions may have other, specialized computer crime organizations to which these incidents can be reported. For example, in the United States, that organization is the Internet Crime Complaint Center, at http://www.ic3.gov. 🚨
9. Users may also be interested in links to security expert forums, where other users can assist them 🫂

---

6.2. Professionally Assisted Remediation Process

It should be acknowledged that, based on the current state of remediation tools and the technical abilities of end users, many users may be unable to remediate on their own 🥺 As a result, it is recommended that users have the option for professional assistance. This may entail online or telephone assistance for remediation, as well as working face to face with a professional who has training and expertise in the removal of malware. It should be made clear at the time of offering this service that it is intended for those who do not have the skills or confidence to attempt remediation, and that it is not intended as an up-sell by the ISP 🙅‍♀️

---

7. Failure or Refusal to Remediate

ISP systems should track the bot infection history of hosts so they can detect when users consistently fail to remediate or refuse to take any steps to remediate 📊 In such cases, ISPs may need to consider taking additional steps to protect their network, other users and hosts on that network, and other networks. Such steps may include a progression of actions up to and including account termination. Refusal to remediate can be viewed as a business issue, and as such, no technical recommendation is possible 💼

---

8. Sharing of Data from the User to the ISP

As an additional consideration, it may be useful to create a process by which users could choose, at their option and with their express consent, to share data regarding their bot infections with their ISP and/or another authorized third party 🔐 Such third parties may include governmental entities that aggregate threat data, such as the Internet Crime Complaint Center referred to earlier in this document, academic institutions, and/or security researchers. While in many cases the information shared with the user's ISP or designated third parties will only be used for aggregated statistical analysis, it is also possible that certain research needs may be best met with more detailed data. Thus, any such data sharing from a user to the ISP or authorized third party may contain some type of personally identifiable information, either by design or inadvertently. As a result, any such data sharing should be enabled on an opt-in basis, where users review and approve of the data being shared and the parties with which it is to be shared, unless the ISP is already required to share such data in order to comply with local laws and applicable regulations 🫶✨

9. Security Considerations ✨

okay bestie, this one is SERIOUS 😭🔒. This document goes deep on the security risks of botnets, and it's been baked into every section above because it's that important. 🫶 The risks come in two flavours. First, the malicious bot infections themselves can enable identity theft, theft of authentication credentials, and a host getting used without its owner knowing it's helping with a DDoS attack, among many other messy outcomes 💀. Second, there are risks tied to the specific ways we tell Internet users about a notification 📨. Bot networks and bot infections are extremely serious security stuff, so please read this document carefully, periodt. 💅

Now for notifications from Section 5 💌. Care should be taken to assure users that a notification came from a trustworthy site and/or party 🛡️, because that makes it way harder for phishers and other malicious folks using social engineering tactics to fake it. If that's not possible, then the user should have some level of trust that the notification is valid, and/or have some way to verify it through another mechanism or step 🔍. Main character energy is double-checking 🦋.

10. Privacy Considerations 🌸

This document gives a high-level tour of the activities ISPs should be sensitive about 👀, meaning the places where collecting or communicating PII might be possible. Also, when ISPs send notifications to end users (see Section 5) 📬, those notifications should NOT... wait, let me be precise here: those notifications should not include PII 🙅‍♀️. Keep it clean, bestie 🧼.

As noted in Section 8 🎀, any sharing of data from the user to the ISP and/or authorized third parties should happen on an opt-in basis ✅. The ISP and/or authorized third parties should also clearly state what data will be shared and who it will be shared with 📋. Full transparency, no sneaky stuff 🫣.

Lastly, as mentioned in other sections ⏳, some legal jurisdictions have requirements about how long subscriber-related or other data can be kept. An ISP operating in one of those places should be aware of the rules and comply with them ⚖️. Legal stuff is not the fun part, but it's the part that keeps everyone out of trouble 💜.

11. Acknowledgements 🙌

The authors want to give a huge shout-out 📣 to these people and groups for doing a detailed review of this document and/or giving comments and feedback that made it better and helped it evolve 🌈:

• Mark Baugher
• Richard Bennett
• James Butler
• Vint Cerf
• Alissa Cooper
• Jonathan Curtis
• Jeff Chan
• Roland Dobbins
• Dave Farber
• Stephen Farrell
• Eliot Gillum
• Joel Halpern
• Joel Jaeggli
• Scott Keoseyan
• Murray S. Kucherawy
• The Messaging Anti-Abuse Working Group (MAAWG)
• Jose Nazario
• Gunter Ollmann
• David Reed
• Roger Safian
• Donald Smith
• Joe Stewart
• Forrest Swick
• Sean Turner
• Robb Topolski
• Maxim Weinstein
• Eric Ziegast

Icons, all of them 🫶✨.

12. Informative References 📚

Here are the reading-list receipts 🧾🤓. Every single one, with all of its details:

[BIOS]     Sacco, A. and A. Ortega, "Persistent BIOS Infection",
           March 2009, <http://www.coresecurity.com/files/
           attachments/Persistent_BIOS_Infection_CanSecWest09.pdf>.

[Combat-Zone]
           Alshech, E., "Cyberspace as a Combat Zone: The Phenomenon
           of Electronic Jihad", February 2007, <http://
           www.memrijttm.org/content/en/report.htm?report=1822>.

[Conficker]
           Porras, P., Saidi, H., and V. Yegneswaran, "An Analysis of
           Conficker's Logic and Rendezvous Points", March 2009,
           <http://mtc.sri.com/Conficker/>.

[DDoS]     Saafan, A., "Distributed Denial of Service Attacks:
           Explanation, Classification and Suggested Solutions",
           March 2009, <www.exploit-db.com/download_pdf/14738/>.

[Dragon]   Nagaraja, S. and R. Anderson, "The snooping dragon:
           social-malware surveillance of the Tibetan movement",
           March 2009,
           <http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-746.pdf>.

[Estonia]  Evron, G., "Battling Botnets and Online Mobs: Estonia's
           Defense Efforts during the Internet War", 2008, <http://
           journal.georgetown.edu/wp-content/uploads/9.1-Evron.pdf>.

[Gh0st]    Vallentin, M., Whiteaker, J., and Y. Ben-David, "The Gh0st
           in the Shell: Network Security in the Himalayas",
           February 2010, <http://www.infowar-monitor.net/wp-content/
           uploads/2010/02/cs294-28-paper.pdf>.

[RFC1459]  Oikarinen, J. and D. Reed, "Internet Relay Chat Protocol",
           RFC 1459, May 1993.

[RFC2142]  Crocker, D., "MAILBOX NAMES FOR COMMON SERVICES, ROLES AND
           FUNCTIONS", RFC 2142, May 1997.

[RFC3954]  Claise, B., "Cisco Systems NetFlow Services Export Version
           9", RFC 3954, October 2004.

[RFC5070]  Danyliw, R., Meijer, J., and Y. Demchenko, "The Incident
           Object Description Exchange Format", RFC 5070,
           December 2007.

[RFC5965]  Shafranovich, Y., Levine, J., and M. Kucherawy, "An
           Extensible Format for Email Feedback Reports", RFC 5965,
           August 2010.

[RFC6108]  Chung, C., Kasyanov, A., Livingood, J., Mody, N., and B.
           Van Lieu, "Comcast's Web Notification System Design",
           RFC 6108, February 2011.

[Snort]    Roesch, M., "Snort Home Page", March 2009,
           <http://www.snort.org/>.

[Spamalytics]
           Kanich, C., Kreibich, C., Levchenko, K., Enright, B.,
           Voelker, G., Paxson, V., and S. Savage, "Spamalytics: An
           Empirical Analysis of Spam Marketing Conversion",
           October 2008, <http://www.icir.org/christian/publications/
           2008-ccs-spamalytics.pdf>.

[Threat-Report]
           Ahamad, M., Amster, D., Barret, M., Cross, T., Heron, G.,
           Jackson, D., King, J., Lee, W., Naraine, R., Ollman, G.,
           Ramsey, J., Schmidt, H., and P. Traynor, "Emerging Cyber
           Threats Report for 2009: Data, Mobility and Questions of
           Responsibility will Drive Cyber Threats in 2009 and
           Beyond", October 2008, <http://smartech.gatech.edu/
           bitstream/1853/26301/1/CyberThreatsReport2009.pdf>.

[Whiz-Kid] Berinato, S., "Case Study: How a Bookmaker and a Whiz Kid
           Took On a DDOS-based Online Extortion Attack", May 2005,
           <http://www.csoonline.com/article/220336/How_a_Bookmaker_and_a_Whiz_Kid_Took_On_a_DDOS_based_Online_Extortion_Attack>.

Appendix A. Examples of Third-Party Malware Lists 🧚

As mentioned in Section 4 🍓, there are loads of potential third parties that might be willing to share lists of infected hosts 🤝. This list is just for example purposes 💁‍♀️. It is not meant to be exclusive or exhaustive, and it can change over time 🔄.

• Arbor - Atlas, see http://atlas.arbor.net/
• Internet Systems Consortium - Secure Information Exchange (SIE), see https://sie.isc.org/
• Microsoft - Smart Network Data Services (SNDS), see https://postmaster.live.com/snds/
• SANS Institute / Internet Storm Center - DShield Distributed Intrusion Detection System, see http://www.dshield.org/about.html
• ShadowServer Foundation, see http://www.shadowserver.org/
• Spamhaus - Policy Block List (PBL), see http://www.spamhaus.org/pbl/
• Spamhaus - Exploits Block List (XBL), see http://www.spamhaus.org/xbl/
• Team Cymru - Community Services, see http://www.team-cymru.org/

Authors' Addresses 💌

The authors can be reached here 📮:

Jason Livingood
Comcast Cable Communications
One Comcast Center
1701 John F. Kennedy Boulevard
Philadelphia, PA  19103
USA

EMail: jason_livingood@cable.comcast.com
URI:   http://www.comcast.com


Nirmal Mody
Comcast Cable Communications
One Comcast Center
1701 John F. Kennedy Boulevard
Philadelphia, PA  19103
USA

EMail: nirmal_mody@cable.comcast.com
URI:   http://www.comcast.com


Mike O'Reirdan
Comcast Cable Communications
One Comcast Center
1701 John F. Kennedy Boulevard
Philadelphia, PA  19103
USA

EMail: michael_oreirdan@cable.comcast.com
URI:   http://www.comcast.com

That's the whole RFC, bestie! 🌈🎀 Thanks for hanging out with me through all seven parts 🫶💖