RFC 9075

Report from the IAB COVID-19 Network Impacts Workshop 2020

๐Ÿ“… published July 2021, from ietf

RFC 9075 ๐Ÿ“š๐Ÿฆ‹ Report from the IAB COVID-19 Network Impacts Workshop 2020

okay bestie, this one is the IAB (Internet Architecture Board) writing up their COVID-19 network impacts workshop ๐ŸŒธ It's RFC 9075, filed as Informational ๐Ÿ’– and published in July 2021 โœจ The authors are J. Arkko, S. Farrell, M. Kรผhlewind, and C. Perkins ๐Ÿซถ The ISSN is 2070-1721 ๐ŸŽ€ It doesn't list anything it obsoletes or updates ๐Ÿ’…

Abstract โœจ

so the COVID-19 pandemic shook up how people used the Internet ๐Ÿ˜ญ Quarantines and work-from-home happened, and that totally changed Internet traffic ๐ŸŒˆ

The IAB ran a workshop about these network impacts on November 9-13, 2020 ๐ŸŽ€ It brought together researchers, network operators, network management experts, and Internet technologists to swap experiences ๐Ÿ’œ Because travel and contact restrictions were still a thing, it was held online ๐Ÿ’ป

note that this is a report on what happened at the workshop ๐Ÿฆ‹ The views in it belong to the workshop participants, and they don't necessarily match IAB views and positions ๐Ÿ’…

Status of This Memo ๐ŸŒธ

this one is not an Internet Standards Track specification ๐Ÿ™ˆ It's published just for information ๐Ÿ“–

The IAB put this out, and it's a record of stuff the IAB thought was worth keeping forever ๐Ÿ’– It represents the consensus of the Internet Architecture Board (IAB) โœจ Docs the IAB approves for publication are not candidates for any level of Internet Standard, so see Section 2 of RFC 7841 for the details ๐Ÿงš

for the current status, any errata, and how to give feedback, check https://www.rfc-editor.org/info/rfc9075 ๐Ÿ”ฅ

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. ๐Ÿ’œ

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they were on the publication date ๐Ÿ“œ Please read these carefully, because they spell out your rights and restrictions for this document ๐Ÿซถ

Table of Contents ๐Ÿงš

1.  Introduction
2.  Scope
3.  Workshop Topics and Discussion
  3.1.  Measurement-Based Observations on Network Traffic Dynamics
    3.1.1.  Overall Traffic Growth
    3.1.2.  Changes in Application Use
    3.1.3.  Mobile Networks and Mobility
    3.1.4.  A Deeper Look at Interconnections
    3.1.5.  Cloud Platforms
    3.1.6.  Last-Mile Congestion
    3.1.7.  User Behavior
  3.2.  Operational Practices and Architectural Considerations
    3.2.1.  Digital Divide
    3.2.2.  Applications
    3.2.3.  Observability
    3.2.4.  Security
    3.2.5.  Discussion
  3.3.  Conclusions
4.  Feedback on Meeting Format
5.  Position Papers
6.  Program Committee
7.  Informative References
Appendix A.  Workshop Participants
IAB Members at the Time of Approval
Acknowledgments
Authors' Addresses

1. Introduction ๐ŸŒˆ

so the IAB ran a workshop on how COVID-19 hit the network, from November 9-13, 2020 ๐Ÿ’ป It was for researchers, operators, network management experts, and Internet technologists to share what they went through ๐Ÿ’– Since travel and contact restrictions were still going, it was online ๐Ÿซถ

okay so COVID-19 changed what users were doing, and that changed Internet traffic ๐Ÿ˜ญ The shifts were fast and huge, especially when quarantine and work-from-home kicked in ๐Ÿ”ฅ Traffic changed in volume and location, and the mix of apps changed too ๐Ÿฆ‹ It also shifted security practices and the attack patterns that went after the new attack surface created by working from home during a global crisis ๐Ÿ’€

a call for position papers went out in July 2020 ๐Ÿ“ฎ In total, 15 position papers came in from 33 authors ๐ŸŽ€ Section 5 lists them โœจ There were also other kinds of contributions and pointers to existing work ๐Ÿงš Quite a few position papers referred to parallel work being published at measurement-related academic conferences ๐Ÿ’œ

invitations went out based on those position papers and other expressions of interest ๐ŸŒธ The workshop conference calls had 46 participants, all listed in Appendix A ๐Ÿ‘‘

the workshop ran over one week with three sessions: i) measurements and observations, ii) operational and security issues, and iii) future considerations and conclusions ๐ŸŒŸ Those sessions were on Monday, Wednesday, and Friday, which was a lowkey perfect side effect ๐Ÿ™ˆ The gaps between sessions were great for mailing list discussion and building extra workshop material ๐Ÿ’…

2. Scope ๐ŸŽ€

the COVID-19 pandemic absolutely wrecked people's lives and economies around the globe ๐Ÿ˜ญ It also hit networking hard ๐ŸŒ With tons of people working from home or otherwise depending on the network for daily life, traffic volume surged ๐Ÿ”ฅ ISPs and operators reported 20% or more traffic growth in a matter of weeks ๐Ÿ“ˆ IXP traffic was rising too ๐ŸŒŠ Most kinds of traffic went up, and conversational multimedia traffic grew by more than 200% in some cases ๐Ÿคฏ On some conferencing platforms, user time spent on conferencing rose by an order of magnitude ๐Ÿ’€

honestly the Internet handled this pretty well overall โœจ It wasn't perfect, since there were outages, video quality dips, and other issues ๐Ÿ™ˆ Still, it's iconic to watch how the tech, operators, and service providers responded to big changes in traffic patterns ๐Ÿ‘‘

understanding what actually happened to Internet traffic is interesting on its own ๐Ÿฆ‹ How it affected user experience or what the services were meant to do is just as interesting ๐Ÿ’– So measurements and reports on 2020 Internet traffic are valuable ๐Ÿ“Š It would also be fascinating to know what network management and capacity expansion actions got taken in general ๐Ÿงš Anecdotes suggest that Internet and service providers tracked how their services were used and often adjusted them for the new traffic patterns ๐ŸŒธ That ranged from dynamic allocation of computing resources to way more complex changes ๐Ÿ’…

the impacts of this crisis are also a chance to understand how traffic shifts and growth affect networking more broadly ๐ŸŒˆ That helps us prep for future situations, crises or otherwise, and lets us adjust the technology so it's even better at handling change ๐Ÿซถ

the scope of the workshop, based on the call for contributions, included:

  • measurements of traffic changes, user experience and problems, service performance, and other relevant aspects ๐Ÿ“Š
  • discussion about the behind-the-scenes network management and expansion activities ๐ŸŽ€
  • sharing experiences in the fields of general Internet connectivity, conferencing, media/entertainment, and Internet infrastructure ๐Ÿ“
  • lessons learned on preparedness and operations ๐Ÿงš
  • lessons learned on Internet technology and architecture ๐Ÿ’œ

3. Workshop Topics and Discussion ๐Ÿฆ‹

3.1. Measurement-Based Observations on Network Traffic Dynamics ๐Ÿ“Š

the workshop kicked off with measurements ๐Ÿ’– A big chunk of the submitted papers presented and discussed measurement data, and those gave us a solid base for understanding the situation from lots of angles across different kinds of networks ๐ŸŒŸ

the pandemic changed Internet traffic in different ways for different networks ๐ŸŒˆ Still, every network saw some kind of change ๐Ÿ”ฅ That could be less traffic, more traffic, a shift in weekday and weekend daily patterns, or a change in traffic classes ๐Ÿ’… Traffic volume, directionality ratios, and where traffic came from and went to were radically different from before COVID-19 ๐Ÿ’€

at a high level, traffic from home networks went up a lot ๐Ÿ  Mobile networks showed different trends ๐Ÿ“ฑ Traffic either went up as well, for instance in places where residential ISP services are less common, or it went down because people were moving around less ๐Ÿซถ That mobile decrease was kind of the opposite of what was seen on residential ISPs ๐Ÿ™ˆ

diurnal congestion at interconnect points, and in certain last-mile networks, was reported, mainly in March โš ๏ธ Still, no persistent congestion was observed โœจ A downward trend in download throughput to certain cloud regions was also measured, which can probably be explained by increased use of cloud services โ˜๏ธ That's another sign that the scaling of shared resources in the Internet is working reasonably well enough to handle even bigger traffic changes, like the first nearly global lockdown of the COVID-19 pandemic ๐Ÿ‘‘

3.1.1. Overall Traffic Growth

okay so the pandemic absolutely fast-forwarded how much data the world sends ๐Ÿ’ปโœจ. The workshop [Feldmann2020] looked at measurements from one ISP, three IXPs, a metropolitan educational network, and a mobile operator, and right at the start they noticed the network handled everything surprisingly well ๐Ÿซถ. Traffic growth spiked hard in March and April, and the Internet still held up. After the lockdown kicked in during March, the ISP and all three IXPs saw traffic go up by 15-20%. Normally that much growth would stretch across a whole year, but it happened in just a few weeks ๐Ÿ˜ญ. Iconic, but also a LOT.

At DE-CIX Frankfurt, the world's largest Internet Exchange Point in terms of data throughput, 2020 brought the biggest jump in peak traffic within a single year since the IXP was founded in 1995 ๐Ÿ”ฅ. Meanwhile, mobile traffic slightly receded ๐Ÿ“ฑ. In access networks, upstream traffic grew faster than downstream, which makes sense because everyone was suddenly on videoconferencing and remote work and school apps ๐ŸŽ€.

Most of the increase happened outside the old peak hours ๐ŸŒธ. Before the first COVID-19 lockdowns, the main time of use was weekday evenings. Since March, usage has been spread more evenly across the day. The growth mostly landed outside the previous peak times (think daytime, while people were working from home). For the first time ever, weekday utilization looked like weekend utilization, which is kind of wild ๐Ÿ™ˆ. The extra volume could be absorbed either by using existing reserve capacity or by quickly switching on additional bandwidth. That's one big reason the Internet coped so well during the first lockdown period ๐Ÿ’….

Some lockdowns were lifted or relaxed around May 2020 ๐ŸŒˆ. As people were allowed back out for some daily activities, traffic dropped at the IXPs and the ISP, just as expected. Mobile traffic, though, started growing again.

3.1.2. Changes in Application Use

The mix of data traffic changed a lot once the pandemic began. Videoconferencing and virtual private networks (VPNs) for accessing company resources from home went up sharply ๐Ÿ’œ. In ISP and IXP networks, traffic tied to web conferencing, video, and gaming jumped in March 2020 [Feldmann2020], because people really wanted tools like Zoom or Microsoft Teams. The relative traffic share of many "essential" apps like VPN and conferencing tools went up by more than 200% ๐Ÿš€.

People were also spending way more hours at home, so they watched more videos and played more games, which pumped up entertainment traffic ๐ŸŽฎ. Meanwhile, other traffic classes dropped a lot, like education, social media, and, for some periods, content delivery networks (CDNs) ๐Ÿฅ€. In April and June, web conferencing was still high compared to pre-pandemic times, while CDN and social media traffic dipped a little. Lots of people were still working from home, but restrictions had eased, so in-person socializing likely went up and online socializing likely went down ๐Ÿฆ‹.

3.1.2.1. Example Campus Networks

University campuses felt it too, especially because remote teaching had to happen ๐ŸŽ“. The Politecnico di Torino (Italy) rolled out its own in-house remote teaching solution, and outgoing traffic grew by 2.5 times, driven by more than 600 daily online classes ๐Ÿ“ˆ. Incoming traffic went the other way and dropped by a factor of 10, because in-person activity had stopped completely. The measurements showed this change did not lead to noticeable performance impairments, and students in remote regions of Italy didn't see significantly poor performance either โœจ. Outgoing traffic also went up because of other remote working tools, like collaboration platforms, VPNs, and remote desktops ๐Ÿ’ป.

Similar changes showed up when measuring REDIMadrid [Feldmann2020], a European educational and research network connecting 16 independent universities and research centers in the Madrid metropolitan region ๐Ÿ“. Traffic volume on working days dropped by up to 55% during the pandemic. Just like in the ISP and IXP networks, working days and weekend days started looking more alike in terms of total traffic ๐Ÿซถ. The hourly patterns showed a traffic increase between 9 pm and 7 am ๐ŸŒ™. That could be because users were working at more unusual times, but it could also potentially be caused by overseas students (mainly from Latin America and East Asia, based on the Autonomous System (AS) numbers those connections came from) who were accessing university network resources from their home countries ๐ŸŒ.

Since users of the academic network (students and research staff) had to leave campus because of lockdown measures, the ingress and egress ratio changed drastically too ๐Ÿ . Before the lockdown, incoming traffic was way bigger than outgoing traffic. After it, the ratio became much more balanced. This shift makes sense once you think about remote work. Users were mostly connecting to the network to access resources, which is why outgoing traffic went up. On top of that, external (Internet-based) resources that used to be reached from the educational network during work were now reached from people's homes instead ๐Ÿ’….

3.1.3. Mobile Networks and Mobility

Mobile network data usage seemed to decline after localized lockdown measures were imposed, since those reduced typical levels of mobility and roaming ๐Ÿ“ต.

[Lutu2020] measured the cellular network of O2 UK to see how changes in people's mobility affected traffic patterns ๐Ÿ“ถ. By analyzing cellular network signaling information about users' device mobility, they observed a 50% decrease in mobility (according to different mobility metrics) across the UK during the lockdown period ๐Ÿ˜ฎ. There was no correlation between that reduction and the number of confirmed COVID-19 cases, so only the enforced government order was effective at significantly reducing mobility. The reduction was also bigger in densely populated urban areas than in rural ones. For London specifically, the mobile network data suggested that approximately 10% of residents temporarily relocated during the lockdown ๐Ÿ™๏ธ.

These mobility changes had immediate effects on cellular traffic patterns ๐ŸŒŠ. The downlink data traffic volume, aggregated for all bearers (including conversational voice), decreased by up to 25% across the entire UK during the lockdown period. This matches the countrywide drop in mobility, and it probably happened because people leaned more on residential broadband to run download-heavy apps like video streaming ๐Ÿ“บ. Radio cell load fell by approximately 15% across the UK after the stay-at-home order was enacted, which backs up the drop in cellular connectivity usage ๐Ÿ“‰.

Uplink data traffic volume, on the other hand, barely changed (between -7% and +1.5%) during lockdown ๐Ÿค. This was mainly thanks to the increase in 4G voice traffic (Voice over LTE (VoLTE)) across the UK, which peaked at 150% after the lockdown compared to the national median value before the pandemic. That made up for the drop in uplink data traffic ๐Ÿ’œ.

Finally, mobility changes affected network usage differently across geodemographic area clusters ๐Ÿ—บ๏ธ. In densely populated urban areas, mobile network usage (downlink and uplink traffic volume, radio load, and active users) dropped much more significantly than in rural areas. For London, this was likely because the central districts have lots of seasonal residents (like tourists) and business and commercial areas ๐Ÿข.

3.1.4. A Deeper Look at Interconnections

Traffic at points of network interconnection went up noticeably, but most operators reacted quickly by adding capacity [Feldmann2020] ๐Ÿ”Œ. The amount of increase varied. Some networks that hosted popular apps like videoconferencing saw traffic growth of several hundred to several thousand percent ๐Ÿคฏ. At the IXP level, port utilization went up, which is mostly explained by higher traffic demand from residential users ๐Ÿก.

Measurements of interconnection links at major US ISPs, done by the Center for Applied Internet Data Analysis (CAIDA) and the Massachusetts Institute of Technology (MIT), found some evidence of diurnal congestion around March 2020 [Clark2020]. But most of that congestion disappeared within a few weeks, which suggests operators really did take steps to add capacity or otherwise mitigate the congestion ๐Ÿฆธโ€โ™€๏ธ.

3.1.5. Cloud Platforms

Cloud infrastructure played a key role in supporting bandwidth-intensive videoconferencing and remote learning tools, so people could practice social distancing during COVID-19 โ˜๏ธ๐Ÿ’–. Network congestion between cloud platforms and access networks could affect the quality of experience of these cloud-based apps ๐Ÿ˜ฌ. CAIDA used web-based speed test servers to take download and upload throughput measurements from virtual machines in public cloud platforms to various access ISPs in the United States [Mok2020] ๐Ÿงš.

The key findings were:

  • Persistent congestion events were not widely observed between cloud platforms and these networks, especially for large-scale ISPs. Still, we could observe large diurnal download throughput variations in peak hours from some locations to the cloud ๐Ÿ“Š.
  • There was evidence of persistent congestion in the egress direction to regional ISPs serving suburban areas in the US ๐Ÿ˜๏ธ. Their users could have suffered from poor video streaming or file download performance from the cloud ๐Ÿ˜ญ.
  • The macroscopic analysis over 3 months (June-August 2020) revealed downward trends in download throughput from ISPs and educational networks to certain cloud regions ๐Ÿ“‰. We believe that increased use of the cloud during the pandemic could be one of the factors that contributed to the decreased performance ๐Ÿ’œ.

3.1.6. Last-Mile Congestion โœจ

okay so the last mile is literally the main character of broadband ๐Ÿ’… like, if the last mile is bad, your whole experience is bad, no cap. A recent Internet Measurement Conference (IMC '20) paper by Fontugne et al. looked at last-mile latency using traceroute data from RIPE Atlas probes in 646 ASes, hunting for recurring performance drops [Fontugne2020-1]. In normal times, the probes saw persistent last-mile congestion in only 10% of ASes ๐Ÿซ , but during the COVID-19 outbreak they recorded 55% more congested ASes ๐Ÿ˜ญ. The stay-at-home measures hit hardest in networks with a huge number of users and in certain parts of the world ๐ŸŒ. Japan was the most impacted country in their study, specifically the Nippon Telegraph and Telephone (NTT) Corporation Open Computer Network (OCN), and they saw similar patterns across several other Japanese networks too, including Internet Initiative Japan (IIJ) (AS2497) ๐Ÿ‡ฏ๐Ÿ‡ต.

Then it gets better ๐ŸŒˆ. From mid-2020 onward, performance was actually better than before the pandemic. In Japan, that was partly thanks to upgrades originally planned for the Tokyo Olympics ๐Ÿ…, and more broadly, network operators were scrambling to cope with these exceptional circumstances and honestly serving ๐Ÿ‘‘. The pandemic showed that the Internet's adaptive design and its proficient community can keep things running through unprecedented events ๐Ÿซถ. Plus, based on the many research and operational reports published lately, the pandemic seems to be shaping a more resilient Internet, and as Nietzsche put it, "What does not kill me makes me stronger" ๐Ÿ’ชโœจ. Iconic behavior, honestly.

3.1.7. User Behavior

the type of traffic users needed also shifted in 2020 ๐Ÿฆ‹. Upstream traffic went up because of videoconferencing, remote schooling, and similar apps. The National Cable & Telecommunications Association (NCTA) and Comcast reported that downstream traffic grew 20%, but upstream grew by as much as 30-37% [NCTA2020] [Comcast2020]. Vodafone said upstream traffic doubled, growing by 100%, in some markets [Vodafone2020] ๐Ÿคฏ.

Ericsson's ConsumerLab surveyed users about their usage and experiences during the crisis ๐Ÿ“‹. Some of the key findings in [ConsumerlabReport2020] were:

  • 9 in 10 users increased their Internet activities, and time spent connected went up ๐Ÿ“ฑ. Also, 1 in 5 started brand-new online activities ๐ŸŒŸ. Many people in the older generation felt helped by video calling ๐Ÿ’•, parents felt their children's education was helped ๐ŸŽ’, and so on.
  • Network performance was generally found satisfactory ๐Ÿ™Œ. 6 in 10 were very satisfied with fixed broadband, and 3 in 4 felt mobile broadband was the same or better than before the crisis ๐Ÿ“ถ. Consumers said resilience and quality of service were the most important responsibilities for network operators ๐Ÿ’–.
  • Smartphone app usage changed a lot ๐Ÿ“ฒ. The fastest growth was in apps for COVID-19 tracking and information, remote working, e-learning, wellness, education, remote health consultation, and social shared experiences ๐Ÿง˜. The biggest drops were in travel and booking, ride hailing, location, and parking apps ๐Ÿš—๐Ÿ’€.

Some of these behaviors are probably permanent changes [ConsumerlabReport2020] ๐Ÿ”ฎ. Video calls and other new services, especially among the older generation, are likely to have a long-lasting effect ๐Ÿ“ž. Surveys across various organizations also point to a likely long-term increase in the number of people interested in remote work [WorkplaceAnalytics2020] [McKinsey2020] ๐Ÿ ๐Ÿ’œ.

3.2. Operational Practices and Architectural Considerations

the second and third days of the workshop were all about open discussions on operational and architectural issues that were arising ๐Ÿ—ฃ๏ธ. They also covered conclusions that could be drawn from earlier discussions and other issues raised in the position papers ๐Ÿ“.

3.2.1. Digital Divide

Measurements from Fastly confirmed that Internet traffic volume rose fast in multiple countries ๐Ÿ“ˆ while COVID cases were climbing and lockdown policies were kicking in ๐Ÿ˜ท. Download speeds dropped too, but way less dramatically than the jump in overall bandwidth usage ๐Ÿ™ƒ. School closures caused a massive traffic spike in many regions, and other public policy announcements triggered big traffic shifts ๐Ÿ”€. So the takeaway is that governments should coordinate with operators, giving them time to make preemptive operational changes in some cases ๐Ÿค. Bestie, that's just good planning โœจ.

Measurements from the US showed that download rates correlate with income levels ๐Ÿ’ธ. But here's the twist ๐Ÿ‘€: download rates in the lowest income zip codes went up as the pandemic went on, closing the gap with higher income areas. One possible reason in the data is that some ISPs, such as Comcast and Cox, increased speeds for users on certain lower-cost plans and in certain areas ๐Ÿš€. This suggests network capacity was available, and the income-download correlation wasn't necessarily caused by differences in the deployed infrastructure across regions ๐Ÿ˜๏ธ. It was noted, though, that certain access link technologies give more flexibility than others in this regard ๐Ÿงต.

3.2.2. Applications

Web conferencing systems (e.g., Microsoft Teams, Zoom, Webex) saw incredible growth ๐Ÿš€, with overnight traffic increases of 15-20% in response to public policy changes like lockdowns. That required significant and rapid changes in infrastructure provisioning ๐Ÿ› ๏ธ.

Major video providers (YouTube, etc.) reduced their bandwidth by 25% in some regions ๐Ÿ“‰. It was suggested this had a huge impact on videoconferencing quality until networks could scale to handle the full bit rate ๐Ÿ˜ฌ. Other operators of some other services saw limited impact ๐Ÿคทโ€โ™€๏ธ.

Updates to popular games have a significant impact on network load ๐ŸŽฎ. Some discussions took place between ISPs, CDNs, and the gaming industry about possibly coordinating various high-bandwidth update events, similar to what was done for entertainment and video download speeds ๐Ÿ—“๏ธ. There was an apparently difficult interplay between bulk downloads and interactive real-time applications, potentially due to buffer bloat and queuing delays ๐Ÿฅด.

It was noted that operators have experience with rapid Internet traffic growth ๐Ÿง . New applications with exponential growth aren't that unusual on the network, and the lockdown traffic spike wasn't that unprecedented for many. Plenty of operators already have tools and mechanisms to deal with this ๐Ÿงฐ. The challenge is making sure that knowledge gets shared ๐Ÿ“ฃ.

Following these observations, traffic prioritization was discussed, starting from Differentiated Services Code Point (DSCP) marking ๐Ÿท๏ธ. The question was whether a minimal priority-marking scheme would have helped during the pandemic, for example by allowing marking of less-than-best-effort traffic ๐Ÿค”. The discussion quickly turned into a broader QoS and observability conversation, and it also touched on the effects of increased encryption ๐Ÿ”. The group, unsurprisingly, couldn't resolve all the different perspectives and interests involved ๐Ÿ˜…, but the discussion showed that progress was made ๐ŸŒธ.

3.2.3. Observability

there's a clear contrast in experience here ๐Ÿ‘€. Many operators reported few problems in terms of metrics like measured download bandwidth ๐Ÿ“Š, while videoconferencing apps had significant usability problems running on those same networks ๐Ÿ˜ตโ€๐Ÿ’ซ. The interaction between application providers and network providers worked very smoothly to resolve these issues, backed by strong personal contacts and relationships ๐Ÿ’ž. But it seems clear that the metrics many operators use to understand network performance don't fully capture the impact on certain applications, and that's an observability gap ๐Ÿ•ณ๏ธ. So, do we need more tools to figure out the various impacts on user experience? ๐Ÿง

These types of applications use surprisingly large amounts of Forward Error Correction (FEC) ๐Ÿงฉ. Applications hide lots of loss to keep the user experience good ๐Ÿ™ˆ, which makes problems harder to observe. The network can be behaving poorly while the experience is still good enough ๐Ÿ˜ถโ€๐ŸŒซ๏ธ. Resiliency measures can improve user experience but also hide severe problems ๐Ÿซฅ. There may be a missing feedback loop between application developers and operators ๐Ÿ”.

It's clear that application providers and operators find it difficult to isolate problems. Is the problem the local Wi-Fi, the access network, the cloud network, or something else? ๐Ÿ“ถโ˜๏ธ Metrics from access points would help, but in general, lacking observability into the network as a whole is a real concern when debugging performance issues ๐Ÿ”.

Further, it can be difficult to route problem reports to the person who can actually fix them, especially if the reported information has to be shared across multiple networks in the Internet ๐Ÿ“จ. COVID-enhanced cooperation made it easier to debug problems, and lines of communication really matter ๐Ÿ’ฌ๐Ÿซถ.

3.2.4. Security ๐Ÿ”

okay so the COVID-19 era gave security people a LOT to deal with, and it comes down to two big problems: (1) bad actors spinning up new scams fast using COVID-19 as bait, and (2) a massive increase in attack surface when everyone suddenly started working from home ๐Ÿ˜ญ

During 2020, a lot of people moved to home working, and the way they used the network changed too. IT departments rolled out new equipment super fast and used VPNs for the very first time, while others had existing setups suddenly under way more load. As VPN tech got more popular and more people used it, it became a juicier target ๐ŸŽฏ. One Advanced Persistent Threat group (APT29) got initial footholds by using recently published exploits against a range of VPN software [Kirsty2020]. Not okay, but iconic for the attackers, I guess ๐Ÿ’€

Of all the scams the United Kingdom National Cyber Security Centre (UK NCSC) detected that pretended to come from the UK Government, more were about COVID-19 than anything else ๐Ÿ™ƒ. There are also other reports of a strong rise in phishing, fraud, and scams related to COVID [Kirsty2020]. Overall cybercrime levels haven't gone up based on the data so far, but there was definitely a shift in what attackers were doing. Both the NCSC and the Department of Homeland Security Cybersecurity and Infrastructure Security Agency (DHS CISA) saw more COVID-19 themes being used by malicious cyber actors as bait ๐Ÿชค. Attackers sent COVID-19 scams and phishing emails to individuals, small and medium businesses, large organizations, and organizations involved in national and international COVID-19 responses (healthcare bodies, pharmaceutical companies, academia, and medical research organizations) ๐Ÿฅ. New targets, like organizations working on COVID-19 vaccine development, got hit with VPN exploits, which is a super clear reminder of what can happen when infrastructure is vulnerable ๐Ÿšจ.

It's unclear how to detect and counter these attacks at scale, and that's the tricky part ๐Ÿฅฒ. Approaches like using Indicators of Compromise and crowdsourced flagging of suspicious emails were found to work well against COVID-19-related scams [Kirsty2020]. Observing the DNS to detect malicious use is widespread and effective too โœจ. DNS over HTTPS has privacy benefits, but the way it's deployed right now can bypass these existing protective DNS measures ๐Ÿ‘€.

It was also noted that when everyone moves their jobs online, not understanding security becomes a much bigger problem ๐Ÿ˜ฌ. Is it reasonable to expect every Internet user to get password training? Or is there a deeper problem with a technical solution? Modern advice is to use a layered approach to security defenses, and user education is just one of those layers ๐Ÿง….

Communication platforms like Zoom aren't new, and lots of people have used them for years ๐Ÿ“น. But once COVID-19 pushed more organizations and individuals to these tools, they became an attractive target because usage went up. This led to more malicious cyber actor activity, like hijacking online meetings that weren't secured with passwords, or using unpatched software as an attack vector ๐Ÿšช. So how can new or existing measures protect users from attacks aimed at the next vulnerable service? ๐Ÿค”

Overall, there may have been fewer security challenges than expected when lots of people suddenly started working from home ๐Ÿ™Œ. But attacker agility, the importance of robust and scalable defense mechanisms, and some existing security problems and challenges may have become even more obvious and intense with more use of Internet-based services. This was especially true in a pandemic and in times of uncertainty, when users can be more vulnerable to social engineering techniques and attacks ๐Ÿ’œ.

3.2.5. Discussion ๐Ÿ’ฌ

There's a concern that we're missing observability for the network as a whole ๐Ÿ‘๏ธ. Each application provider and operator has their own little lens, and nobody has the big-picture view of the network ๐Ÿ”.

How much of a safety margin do we need? ๐Ÿคทโ€โ™€๏ธ Some of the resiliency comes from not running the network too close to its limit. That leaves room for traffic to shift and gives the network headroom to cope ๐ŸŒŠ. The best-effort nature of the network may help here. Running the network closer to its limits usually improves performance, but highly optimized networks may be less robust ๐ŸŽ๏ธ.

Finally, it was observed that we get what we measure ๐Ÿ“Š. There may be an argument for operators to shift their measurement focus away from pure capacity and toward Quality of Experience (QoE) or resilience instead โœจ. The Internet is critical infrastructure, and people are finally realizing that ๐ŸŒ. We should treat this as a wake-up call to improve resilience, both in protocol design and operational practice, and not necessarily optimize for absolute performance or quality of experience ๐Ÿซถ.

3.3. Conclusions ๐ŸŽ€

There's a wealth of data about how the Internet performed during the COVID-19 crisis ๐Ÿ“š. The main takeaway from the various measurements is that pretty large shifts happened ๐Ÿ“ˆ. And those shifts weren't just people swapping one app for another. They actually changed traffic flows and directions, and in many cases caused a significant traffic increase ๐Ÿš€. Early reports also seem to show that the shifts went relatively smoothly from the point of view of overall consumer experience ๐Ÿ˜Œ.

An important but not-so-visible reason things ran smoothly is that many people and organizations were highly motivated to make sure the user experience stayed good ๐Ÿ’ช. A lot of collaboration happened behind the scenes, problems got fixed, many providers significantly increased their capacity, and so on ๐Ÿ› ๏ธ.

On the security front, COVID-19 showed how agile malicious actors can be when user Internet usage shifts, and how much potential disruption and damage they can cause ๐Ÿ˜ฑ. It also showed how agile defenders can be when they have the tools and information they need to protect users and networks. It showcased the power of Indicators of Compromise when defenders around the world work together on the same problem ๐Ÿค.

In general, the Internet also seems well suited to adapting to new situations, at least within some bounds ๐Ÿฆ‹. The Internet is designed for flexibility and extensibility, rather than being optimized for today's particular traffic types. That makes it possible to use it for many applications and in many deployment situations, and to make changes as needed ๐Ÿงฉ. This generality shows up in many parts of the overall system, from basic Internet technology to browsers, from name servers to content delivery networks and cloud platforms ๐ŸŒ. When usage changes, what's usually needed is different services, maybe some reallocation of resources, and then continuing application of existing security defenses. Fundamental technology or hardware changes aren't needed ๐Ÿ™…โ€โ™€๏ธ.

That said, it doesn't mean no improvements are needed ๐Ÿ‘‡:

  • We need a better understanding of the health of the Internet ๐Ÿฉบ. The critical role the Internet plays in our lives means its health needs to get significant attention going forward. Understanding how well networks work isn't just a technical matter. It's also crucially important to the people and economies of the societies using it. So projects and research that monitor Internet and service performance on a broad scale, across different networks, are important ๐Ÿ“ก.
  • We need to maintain defensive mechanisms for times of crisis ๐Ÿ›ก๏ธ. Malicious cyber actors keep adjusting their tactics to take advantage of new situations, and the COVID-19 pandemic was no exception. Malicious actors used the strong appetite for COVID-19-related information as an opportunity to deliver malware and ransomware and to steal user credentials ๐Ÿ˜ค. With the shift to working from home, more users vulnerable to attack, and IT departments often overwhelmed by rolling out new infrastructure and devices, sharing Indicators of Compromise (IoC) was a vital part of the response to COVID-19-related scams and attacks ๐Ÿซก.
  • We need to ensure that broadband is available to all and that Internet services equally serve different groups ๐ŸŒˆ. The pandemic showed how the effects of the digital divide can be amplified during a crisis, and it highlighted the importance of equitable Internet access ๐Ÿ’–.
  • We need to keep working on all the other improvements that are seen as necessary anyway, like further security improvements and the ability for networks and applications to collaborate better ๐Ÿ”ง.
  • We need to make sure informal collaboration between the different parties involved in operating the network continues, and gets stronger, to keep operational resilience going ๐Ÿค—.

4. Feedback on Meeting Format ๐Ÿ–ฅ๏ธ

While there are often virtual participants in IAB workshops, the IAB had no experience running workshops entirely virtually ๐Ÿซฃ.

Feedback on this event format was largely positive though ๐Ÿฅฐ. It was especially useful that the three sessions were scheduled on Monday, Wednesday, and Friday. That meant the time in between could be used for mailing list discussion and for putting together additional workshop material ๐Ÿ“. The positive feedback was likely at least partly because many workshop participants already knew one another from previous face-to-face events, mostly IETF meetings ๐Ÿ‘ฏ.

The process for sending invitations to the workshop should be improved for next time, though, because a few invitations were initially lost ๐Ÿ“ญ. In a virtual meeting, it may make more sense to invite not just one person but all coauthors of a paper, for instance ๐Ÿ’Œ. At least for this workshop, we didn't seem to suffer from having too many participants, and in many cases some participants may not be able to attend a particular session on certain days ๐ŸŒธ.

5. Position Papers ๐Ÿ“šโœจ

Okay so these position papers came in, and they're listed in alphabetical order bc organization is everything ๐Ÿ’…

  • Afanasyev, A., Wang, L., Yeh, E., Zhang, B., and Zhang, L.: Identifying the Disease from the Symptoms: Lessons for Networking in the COVID-19 Era [Afxanasyev2020]
  • Arkko, J.: Observations on Network User Behaviour During COVID-19 [Arkko2020]
  • Bronzino, F., Culley, E., Feamster, N., Liu, S., Livingood, J., and Schmitt, P.: IAB COVID-19 Workshop: Interconnection Changes in the United States [Bronzino2020]
  • Campling, A. and Lazanski, D.: Will the Internet Still Be Resilient During the Next Black Swan Event? [Campling2020]
  • Cho, K.: On the COVID-19 Impact to broadband traffic in Japan [Cho2020]
  • Clark, D.: Measurement of congestion on ISP interconnection links [Clark2020]
  • Favale, T., Soro, F., Trevisan, M., Drago, I., and Mellia, M.: Campus traffic and e-Learning during COVID-19 pandemic [Favale2020]
  • Feldmann, A., Gasser, O., Lichtblau, F., Pujol, E., Poese, I., Dietzel, C., Wagner, D., Wichtlhuber, M., Tapiador, J., Vallina-Rodriguez, N., Hohlfeld, O., and Smaragdakis, G.: A view of Internet Traffic Shifts at ISP and IXPs during the COVID-19 Pandemic [Feldmann2020]
  • Fontugne, R., Shah, A., and Cho, K.: The Impact of COVID-19 on Last-mile Latency [Fontugne2020]
  • Gillmor, D.: Vaccines, Privacy, Software Updates, and Trust [Gillmor2020]
  • Gu, Y. and Li, Z.: Covid 19 Impact on China ISP's Network Traffic Pattern and Solution Discussion [Gu2020]
  • Jennings, C. and Kozanian, P.: WebEx Scaling During Covid [Jennings2020]
  • Lutu, A., Perino, D., Bagnulo, M., Frias-Martinez, E., and Khangosstar, J.: A Characterization of the COVID-19 Pandemic Impact on a Mobile Network Operator Traffic [Lutu2020]
  • Mok, R., and claffy, kc: Measuring the impact of COVID-19 on cloud network performance [Mok2020]
  • Paine, K.: IAB COVID-19 Network Impacts [Kirsty2020]

6. Program Committee ๐ŸŽ€

The workshop program committee squad was Jari Arkko, Stephen Farrell, Cullen Jennings, Colin Perkins, Ben Campbell, and Mirja Kรผhlewind. Main character lineup ๐Ÿ‘‘

7. Informative References ๐Ÿงš

Here's the whole reference stash, all in one place ๐Ÿ“

Appendix A. Workshop Participants ๐ŸŒธ

Here's the alphabetical roll call of everyone who showed up to the workshop ๐Ÿฆ‹

  • Jari Arkko (Ericsson/IAB)
  • Ben Campbell (Independent/IAB)
  • Andrew Campling (419 Consulting)
  • Kenjiro Cho (IIJ)
  • kc claffy (CAIDA)
  • David Clark (MIT CSAIL)
  • Chris Dietzel (DE-CIX)
  • Idilio Drago (University of Turin)
  • Stephen Farrell (Trinity College Dublin/IAB)
  • Nick Feamster (University of Chicago)
  • Anja Feldmann (Max Planck Institute for Informatics)
  • Romain Fontugne (IIJ Research Lab)
  • Oliver Gasser (Max Planck Institute for Informatics)
  • Daniel Kahn Gillmor (ACLU)
  • Yunan Gu (Huawei)
  • Oliver Hohlfeld (Brandenburg University of Technology (BTU))
  • Jana Iyengar (Fastly)
  • Cullen Jennings (Cisco/IAB)
  • Mirja Kรผhlewind (Ericsson/IAB)
  • Dominique Lazanski
  • Zhenbin Li (Huawei/IAB)
  • Franziska Lichtblau (Max Planck Institute for Informatics)
  • Jason Livingood (Comcast)
  • Andra Lutu (Telefonica Research)
  • Vesna Manojlovic (RIPE NCC)
  • Rรผdiger Martin (EC)
  • Larry Masinter (Retired)
  • Matt Matthis (Google)
  • Jared Mauch (Akamai/IAB)
  • Deep Medhi (NSF)
  • Marco Mellia (Politecnico di Torino)
  • Ricky Mok (CAIDA)
  • Karen O'Donoghue (Internet Society)
  • Kirsty Paine (NCSC)
  • Diego Perino (Telefonica Research)
  • Colin Perkins (University of Glasgow/IRTF/IAB)
  • Enric Pujol (Benocs)
  • Anant Shah (Verizon Media Platform)
  • Francesca Soro (Politecnico di Torino)
  • Brian Trammell (Google)
  • Martino Trevisan
  • Georgios Tselentis (European Commission)
  • Lan Wang (University of Memphis)
  • Rob Wilton (Cisco)
  • Jiankang Yao (CNNIC)
  • Lixia Zhang (UCLA)

IAB Members at the Time of Approval ๐Ÿ‘‘

These were the Internet Architecture Board members when this document got approved for publication ๐Ÿ’–

  • Jari Arkko
  • Deborah Brungard
  • Ben Campbell
  • Lars Eggert
  • Wes Hardaker
  • Cullen Jennings
  • Mirja Kรผhlewind
  • Zhenbin Li
  • Jared Mauch
  • Tommy Pauly
  • David Schinazi
  • Russ White
  • Jiankang Yao

Acknowledgments ๐Ÿซถ

The authors want to thank the workshop participants, the members of the IAB, the program committee, the folks on the architecture discussion list for the interesting conversations, and Cindy Morgan for handling the practical arrangements. ๐Ÿ’œ

Extra special thanks go to the participants who also contributed to this report. Romain Fontugne provided text based on his blog post at https://eng-blog.iij.ad.jp/archives/7722. Ricky Mok wrote text on cloud platforms. Martino Trevisan wrote text on campus networks. David Clark covered congestion measurements at interconnects. Oliver Hohlfeld wrote the text on traffic growth, changes in traffic shifts, campus networks, and interconnections. Andra Lutu handled mobile networks. Kirsty Paine wrote text on security impacts. And Jason Livingood gets a huge thank you for his review and additions ๐ŸŒˆ

Authors' Addresses ๐Ÿ’Œ

  • Jari Arkko, Ericsson, Email: jari.arkko@ericsson.com
  • Stephen Farrell, Trinity College Dublin, Email: stephen.farrell@cs.tcd.ie
  • Mirja Kรผhlewind, Ericsson, Email: mirja.kuehlewind@ericsson.com
  • Colin Perkins, University of Glasgow, Email: csp@csperkins.org
๐Ÿ“ฆ where this page comes from