RFC 7768
Port Management to Reduce Logging in Large-Scale NATs
RFC 7768: Port Management to Reduce Logging in Large-Scale NATs 🎀
okay bestie, let's get into it 💅 First, the title block facts:
- Stream: Independent Submission 🦋
- Category: Informational (ISSN 2070-1721) 🌸
- Date: January 2016 📅
- Authors: T. Tsou (Philips Lighting), W. Li (China Telecom), T. Taylor and J. Huang (Huawei Technologies) 👑
- Obsoletes / Updates: the title block doesn't list any RFCs that this one obsoletes or updates 🧚
Abstract ✨
so the vibe is: IPv6 transition plans need big shared NATs (like AFTR and NAT64) to stretch the limited IPv4 addresses until the transition is finished 🌈 people have been going back and forth on how to share ports between subscribers who all sit behind the same IPv4 address, and one big part of that debate is that operators need to log who got which transport address. the argument is that handing out individual ports dynamically means way too many logs 💀 this document shows a way to get dynamic port sharing while keeping log volumes low 🫶
Status of This Memo 🌸
this one is not an Internet Standards Track spec. it's published just for info 💖
it's a contribution to the RFC Series, and it's independent of any other RFC stream. the RFC Editor picked to publish it at their discretion and makes no claim about whether it's good for implementing or deploying 🙈 documents the RFC Editor approves for publication aren't candidates for any level of Internet Standard. see Section 2 of RFC 5741 for that 💅
for the current status, any errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc7768 🦋
Copyright Notice 💜
copyright (c) 2016 IETF Trust and the persons identified as the document authors. all rights reserved 🎀
this doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the publication date. read those carefully, because they spell out your rights and restrictions for this doc 📜
Table of Contents 🧚
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. A Suggested Solution . . . . . . . . . . . . . . . . . . . . 3
3. Issues Of Traceability . . . . . . . . . . . . . . . . . . . 4
4. Other Considerations . . . . . . . . . . . . . . . . . . . . 5
5. Security Considerations . . . . . . . . . . . . . . . . . . . 6
6. Informative References . . . . . . . . . . . . . . . . . . . 7
Appendix A. Configure Server Software to Log Source Port . . . . 9
A.1. Apache . . . . . . . . . . . . . . . . . . . . . . . . . 9
A.2. Postfix . . . . . . . . . . . . . . . . . . . . . . . . . 9
A.3. Sendmail . . . . . . . . . . . . . . . . . . . . . . . . 9
A.4. sshd . . . . . . . . . . . . . . . . . . . . . . . . . . 10
A.5. Cyrus IMAP and UW IMAP . . . . . . . . . . . . . . . . . 10
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 111. Introduction 🌸
during the IPv6 transition period, some large-scale NAT devices might show up, like Dual-Stack Lite (DS-Lite), Address Family Transition Router (AFTR), and NAT64 💖 when a NAT device has to set up a brand new connection for an internal address behind it, it has to create a new mapping entry for that connection. the entry holds the source IP address, the source port or ICMP identifier, the converted source IP address, the converted source port, the protocol (TCP/UDP), and so on 🧚
due to legislation and law enforcement requirements, it's sometimes necessary to log these mappings for a while, like 6 months 😭 the mapping info is super privacy sensitive, so if possible, it should be deleted as soon as it can be 🔒 some high-performance NAT devices might need to create a huge number of new sessions per second. if every mapping entry gets logged, the log traffic could hit tens of megabytes per second or more, and that's a real problem for generating, transmitting, and storing logs 💀
according to a test discussed in "Analysis of NAT64 Port Allocation Methods for Shared IPv4 Addresses" [ALLOC-METHODS], in a network with 20,000 subscribers over a 60-day period, the raw log size can reach 42.5 TB if it's based on per-session logging. if it's based on port blocks instead, the log size is 40.6 GB 🤯 compression can help before the logs get stored, but the log size is still big 📦
[RFC6888], REQ-13 suggests "maximize port utilization" and REQ-14 suggests "minimize log volume" 🎀 the catch is that it's hard to get both. there's a trade-off between how efficiently ports get used and how fast log records get generated ⚖️ it's giving tension, honestly 🙈
2. A Suggested Solution 🦋
this doc proposes a solution that lets port ranges be shared dynamically between users while keeping the number of logs that have to be generated as low as possible ✨ the short version: ports get allocated to users in blocks. logs are only generated when blocks are allocated or deallocated. this gives you the traceability you need while cutting log generation by a factor equal to the block size, compared to fully dynamic port allocation 🔥
here's how it works in more detail 💅 when the user sends out their first packet, a port resource pool is allocated for them. for example, ports 2001~2300 of a public IP address get assigned to the user's resource pool. only one log should be generated for this port block. when the NAT needs to set up a new mapping entry for the user, it can use a port from that user's pool plus the matching public IP address 🫶 if the user needs more port resources, the NAT can allocate another port block to their pool, like ports 3501~3800. again, just one log needs to be generated for that block 🌈
[RFC6431] takes this idea further by allocating non-contiguous sets of ports with a pseudorandom function 🎲 scattering the allocated ports like this gives a modest barrier against port guessing attacks 🛡️ Section 5 talks more about randomization 💜
now say a given internal address has been assigned more than one port block. the sessions using ports inside a block will start and end at different times 🕰️ if no ports in some block get used for a configurable amount of time, the NAT can remove that block from the resource pool allocated to the internal address and make it available for other users 🔁 in theory, logging deallocations of port blocks isn't necessary, because the ports in a deallocated block won't be used again until the block gets reallocated. still, the deallocation may be logged when it happens, which adds robustness for troubleshooting and other procedures 🧚
deallocation comes with a few headaches in practice 😩 the first one is picking the timeout value for the block. if idle timers are applied to the individual mappings (sessions) inside the block, and those follow the recommendations for NAT behavior for the protocol in question, then the extra time you'd configure as a guard for the whole block doesn't need to be more than a few minutes. in that case, the block timer is just a slightly more conservative extension of the individual session idle timers 🫧 if instead a single idle timer is used for the whole block, it has to follow the recommendations for the protocol that block's ports are associated with. for example, REQ-5 of [RFC5382] requires an idle timer expiry duration of at least 2 hours and 4 minutes for TCP ⏰
the next issue with port block deallocation is the clash between wanting to randomize port allocation and wanting to make unused resources available to other internal addresses 🥺 as mentioned above, ideally port selection happens across the entire set of blocks allocated to the internal address. but taken to its fullest extent, that policy minimizes the chance that all ports in any given block stay idle long enough to be released 😭
so the alternative is this: when choosing which block to pick a port from, the NAT should leave out of its range of choice the block that has been idle the longest, unless no ports are available in any of the other blocks 🫶 the phrase "block that has been idle the longest" refers to the block where the time since the last packet was observed in any of its sessions, in either direction, is earlier than the corresponding time in any of the other blocks assigned to that internal address 🕵️♀️
3. Issues Of Traceability 🕵️♀️
Section 12 of [RFC6269] is a great discussion of the traceability issue, so go read it 📚 full traceability with the NAT-logging practices proposed here requires that the remote destination record the source port of a request along with the source address (and presumably the protocol, if it isn't implicit) 💅 the logs at each end also have to be timestamped, and the clocks have to be synchronized within a certain degree of accuracy ⏱️ this is one reason for the guard timing on block release: it raises the amount of clock skew between the two ends that you can tolerate 🧠
the ability to configure various server apps to record source ports has been looked into, and here's what came back 🔍
- Source-port recording can be configured in Apache, Postfix, sendmail, and sshd. check the Appendix for the configuration guide 🎀
- Source-port recording isn't supported by IIS, Cyrus IMAP, or UW IMAP 🙃 but it shouldn't be too hard to get Cyrus IMAP and UW IMAP to support it by modifying the source code 🧑💻
wherever source-port logging can be enabled, this memo strongly urges operators to turn it on 🫶 similarly, intrusion detection systems should capture the source port as well as the source address of suspect packets 🚨
in some cases [RFC6269], a server may not record the source port of a connection at all. to allow traceability here, the NAT device needs to record the destination IP address of a connection instead. but as [RFC6269] points out, that's an incomplete solution to traceability, because multiple users of the same shared public IP address may access the service at the same time 😭 from this document's point of view, in those situations the game is lost, so to speak, and port allocation at the NAT might as well be completely dynamic 🙈
the final possibility is where the NAT doesn't do per-session logging, even though the remote end might be failing to capture source ports. in that case, the port allocation policy proposed here can be used 🧩 the impact on traceability is that analyzing the logs only yields the list of all internal addresses mapped to a given public address during the time period in question 👀 that has an impact on privacy as well as traceability, depending on what follow-up actions get taken ⚠️
Part 1 of 2 — more to come, bestie 🌈
4. Other Considerations 🎀
okay so [RFC6269] points out a bunch of issues that come with dynamic port assignment, as opposed to static. Section 13.2 of that doc talks about how it messes with authentication procedures, bestie 😭. These issues must be resolved, but they aren't specific to the port allocation policy in this document, so it's not our mess to clean up here 💅
5. Security Considerations 🔒
this section is giving serious, so let's go 👑 The stuff we're about to talk about is especially relevant to the proposal in this document. The security considerations for NAT operation across different protocols, as documented in things like [RFC4787] and [RFC5382], also apply to this proposal. Not optional, sorry 🙈
[RFC6056] sums up the TCP port-guessing attack. This is where an attacker can hijack one end of a TCP connection 💀 One way to slow that down is to make the source port number used for a TCP connection harder to predict. [RFC6056] gives you a bunch of algorithms to do exactly that 🧚
As Section 3.1 of that RFC puts it: "...provided adequate algorithms are in use, the larger the range from which ephemeral ports are selected, the smaller the chances of an attacker are to guess the selected port number." So the reduced range sizes proposed by this document make it easier for an attacker to guess correctly 😬 We can't totally avoid that. BUT, mitigating measures can help a lot, both at port-block assignment time and when picking individual ports from the blocks a given user has been given ✨
At assignment time, one option is to assign ports as non-contiguous sets of values, as proposed in [RFC6431]. Lowkey it sounds cute, but it creates a lot of complexity for operations 😩 Also, the pseudorandomization can create uncertainty when the accuracy of logs is important to protect someone's life or liberty. That's a big deal, so please don't skip it 🫶
Another option is for the NAT to assign blocks of contiguous ports. Then, at assignment time, the NAT could try to randomize its choice of which of the available idle blocks it hands to a given user. This has to be traded off against the desire to minimize the chance of conflict between what [RFC6056] calls "transport protocol instances." That's why assigning the most-idle block, as suggested in Section 2, is the move. A compromise policy might be to assign blocks only if they've been idle for a certain amount of time, and then select pseudorandomly between the blocks available under that criterion 🦋 In this case, it is suggested that the time value used be greater than the guard timing mentioned in Section 2, and that no block should ever be reassigned until it has been idle at least for the duration given by the guard timer. Main character energy for the guard timer, no notes 👑
While the block assignment strategy gives some mitigation against the port-guessing attack, the biggest contribution will come from pseudorandomization at port-selection time 🔥 [RFC6056] offers a number of algorithms for doing that. When the available ports are contained in blocks, which aren't generally consecutive, those algorithms clearly need some adaptation. It gets messier because the number of blocks allocated to a user may change over time 😭 Adaptation is left as an exercise for the implementor. Good luck, future implementor, we believe in you 💖
6. Informative References 📚
[ALLOC-METHODS]
Chen, G., Li, W., Tsou, T., Huang, J., Taylor, T., and J.
Tremblay, "Analysis of NAT64 Port Allocation Methods for
Shared IPv4 Addresses", Work in Progress, draft-ietf-
sunset4-nat64-port-allocation-02, January 2016.
[APACHE_LOG_CONFIG]
The Apache Software Foundation, "Apache Module
mod_log_config", <http://httpd.apache.org/docs/2.4/mod/
mod_log_config.html>.
[POSTFIX_LOG_CONFIG]
"Postfix Configuration Parameters",
<http://www.postfix.org/postconf.5.html>.
[RFC4787] Audet, F., Ed. and C. Jennings, "Network Address
Translation (NAT) Behavioral Requirements for Unicast
UDP", BCP 127, RFC 4787, DOI 10.17487/RFC4787, January
2007, <http://www.rfc-editor.org/info/rfc4787>.
[RFC5382] Guha, S., Ed., Biswas, K., Ford, B., Sivakumar, S., and P.
Srisuresh, "NAT Behavioral Requirements for TCP", BCP 142,
RFC 5382, DOI 10.17487/RFC5382, October 2008,
<http://www.rfc-editor.org/info/rfc5382>.
[RFC6056] Larsen, M. and F. Gont, "Recommendations for Transport-
Protocol Port Randomization", BCP 156, RFC 6056,
DOI 10.17487/RFC6056, January 2011,
<http://www.rfc-editor.org/info/rfc6056>.
[RFC6269] Ford, M., Ed., Boucadair, M., Durand, A., Levis, P., and
P. Roberts, "Issues with IP Address Sharing", RFC 6269,
DOI 10.17487/RFC6269, June 2011,
<http://www.rfc-editor.org/info/rfc6269>.
[RFC6431] Boucadair, M., Levis, P., Bajko, G., Savolainen, T., and
T. Tsou, "Huawei Port Range Configuration Options for PPP
IP Control Protocol (IPCP)", RFC 6431,
DOI 10.17487/RFC6431, November 2011,
<http://www.rfc-editor.org/info/rfc6431>.
[RFC6888] Perreault, S., Ed., Yamagata, I., Miyakawa, S., Nakagawa,
A., and H. Ashida, "Common Requirements for Carrier-Grade
NATs (CGNs)", BCP 127, RFC 6888, DOI 10.17487/RFC6888,
April 2013, <http://www.rfc-editor.org/info/rfc6888>.
[SENDMAIL_LOG_CONFIG]
O'Reilly, "Sendmail, 3rd Edition, Page 798", December
2002.
[SSHD_LOG_CONFIG]
"sshd_config OpenSSH SSH daemon configuration file",
<http://www.openbsd.org/cgi-bin/
man.cgi?query=sshd_config&sektion=5>.Appendix A. Configure Server Software to Log Source Port 🌸
okay this appendix is basically a how-to for getting your servers to log the source port, and I'm obsessed 🥹
A.1. Apache
the user can use the LogFormat command to define a customized log format and use the CustomLog command to apply that log format. "%a" and "%{remote}p" can be used in the format string to require logging the client's IP address and source port, respectively. This feature has been available since Apache version 2.1 ✨
A detailed configuration guide can be found at [APACHE_LOG_CONFIG] 🧚
A.2. Postfix
in order to log the client source port, macro smtpd_client_port_logging should be set to "yes" in the configuration file [POSTFIX_LOG_CONFIG] 💅
This feature has been available since Postfix version 2.5 🍓
A.3. Sendmail
Sendmail has a macro ${client_port} storing the client port. To log the source port, the user can define some check rules. Here is an example that should be in the .mc configuration macro [SENDMAIL_LOG_CONFIG]:
LOCAL_CONFIG
Klog syslog
LOCAL_RULESETS
SLocal_check_mail
R $* $@ $(log Port_Stat $&{client_addr} $&{client_port} $)This feature has been available since version 8.10 ✨
A.4. sshd
sshd gets its own little man-page moment here 📖
SSHD_CONFIG(5) OpenBSD Programmer's Manual SSHD_CONFIG(5) NAME
sshd_config - OpenSSH SSH daemon configuration file LogLevel Gives
the verbosity level that is used when logging messages from sshd(8).
The possible values are: QUIET, FATAL, ERROR, INFO, VERBOSE, DEBUG,
DEBUG1, DEBUG2, and DEBUG3. The default is INFO. DEBUG and DEBUG1
are equivalent. DEBUG2 and DEBUG3 each specify higher levels of
debugging output. Logging with a DEBUG level violates the privacy of
users and is not recommended. SyslogFacility Gives the facility code
that is used when logging messages from sshd(8). The possible values
are: DAEMON, USER, AUTH, LOCAL0, LOCAL1, LOCAL2, LOCAL3, LOCAL4,
LOCAL5, LOCAL6, and LOCAL7. The default is AUTH.sshd supports logging the client IP address and client port when a client starts a connection since version 1.2.2. Here is the source code in sshd.c:
...
verbose("Connection from %.500s port %d", remote_ip, remote_port);
...sshd supports logging the client IP address when a client disconnects in version 1.2.2 to version 5.0. Since version 5.1, sshd supports logging the client IP address and source port. Here is the source code in sshd.c:
...
/* from version 1.2.2 to 5.0*/
verbose("Closing connection to %.100s", remote_ip);
...
/* since version 5.1*/
verbose("Closing connection to %.500s port %d",
remote_ip, remote_port);In order to log the source port, the LogLevel should be set to VERBOSE [SSHD_LOG_CONFIG] in the configuration file:
LogLevel VERBOSEA.5. Cyrus IMAP and UW IMAP
Cyrus IMAP and UW IMAP do not support logging the source port for the time being. Both software use syslog to create logs; it should not be too difficult to get it supported by adding some new code. Hot tip for the devs in the chat 🫶
Acknowledgements 💜
Mohamed Boucadair reviewed the initial document and provided useful comments to improve it. Reinaldo Penno, Joel Jaeggli, and Dan Wing provided comments on the subsequent draft version that resulted in major revisions. Serafim Petsis provided encouragement to publish the document after a hiatus of two years. Literally everyone who kept this going deserves a gold star ⭐
The authors are grateful to Dan Wing for his help in moving this document forward, and in particular for his helpful comments on its content. Dan Wing, you're iconic 👑
Authors' Addresses 🦋
Tina Tsou
Philips Lighting
3 Burlington Woods Dr #4t
Burlington, MA 01803
United States
Email: tina.tsou@philips.com
Weibo Li
China Telecom
109, Zhongshan Ave. West, Tianhe District
Guangzhou 510630
P.R. China
Email: mweiboli@gmail.com
Tom Taylor
Huawei Technologies
Ottawa
Canada
Email: tom.taylor.stds@gmail.com
James Huang
Huawei Technologies
Bantian, Longgang District
Shenzhen 518129
P.R. China
Email: James.huang@huawei.com📦 where this page comes from
- ietf: rfc7768 2016-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms