RFC 1027

Using ARP to implement transparent subnet gateways

📅 published October 1987, from ietf

Part 1 of 2 🎀

RFC 1027: Using ARP to Implement Transparent Subnet Gateways 💖

This one's from the Network Working Group, dated October 1987, and it's written by Smoot Carl-Mitchell and John S. Quarterman, both at Texas Internet Consulting ✨. The header doesn't name a category, and it doesn't list anything it obsoletes or updates 🧚. Its distribution info is in the Status of this Memo section right below.

Status of this Memo ✨

Okay so this RFC is all about how subnet gateways use the Ethernet Address Resolution Protocol (ARP) 💅 so that hosts on the connected subnets can talk to each other without even knowing subnets exist. It uses the "Proxy ARP" [6] technique 🦋. It's based on RFC-950 [1], RFC-922 [2], and RFC-826 [3], and it's a restricted subset of the mechanism from RFC-925 [4] 👑. Distribution of this memo is unlimited 🌈.

Acknowledgment 🫶

The authors did this work while they were employed by the Computer Sciences Department of the University of Texas at Austin.

Introduction 🌸

The whole point here is to walk through, in detail, how transparent subnet ARP gateways get implemented using Proxy ARP 🧩. The goal is to write down a technique that's already widely used ✍️.

1. Motivation 🏫

So the Ethernet at the University of Texas at Austin is HUGE 😭. It's a large installation connecting over ten buildings, and it has more than one hundred hosts hooked up to it [5]. The size of the Ethernet and how much traffic it handles means you can't just tie it all together with repeaters 🙅. So subnets came in as an attractive alternative for splitting the network into smaller distinct units 🧚.

This is literally the exact situation Internet subnets from RFC-950 were made for 💖. The tea is that lots of vendors hadn't implemented subnets yet 😤, and it wasn't practical to change the more than half a dozen different operating systems running on the hosts on the local networks 💀.

So a way to hide the existence of subnets from hosts was highly desirable 🙈. Since all the local area networks supported ARP, they went with an ARP-based method (commonly called "Proxy ARP" or the "ARP hack") 🎀. Fyi, in this memo, whenever "subnet" shows up, we're assuming the "RFC-950 subnet method" 💅.

2. Design 💅

2.1. Basic method 🦋

Picture a network that supports ARP 🌈. Host A (the source) broadcasts an ARP request for the network address that goes with host B's (the target) IP address 📣. Host B spots its own IP address, so it sends back a point-to-point ARP reply 💌. Host A saves that IP-to-network-address mapping in a local cache and uses it to keep talking to host B later 🫶.

If hosts A and B are on different physical networks, host B won't receive the ARP broadcast request from host A, and it can't respond to it 😵. But if the physical network of host A is connected by a gateway to the physical network of host B, the gateway will see the ARP request from host A 👀. Assuming subnet numbers correspond to physical networks, the gateway can also tell that the request is for a host on a different physical network from the requesting host 🧠. The gateway can then respond for host B, saying that the network address for host B is the gateway's own 🎭. Host A will see this reply, cache it, and send future IP packets for host B to the gateway 📦. The gateway will forward those packets to host B using the usual IP routing mechanisms 🛤️. The gateway is acting as an agent for host B, which is why this technique is called "Proxy ARP". We'll refer to this as a transparent subnet gateway or ARP subnet gateway 🕵️‍♀️.

When host B replies to traffic from host A, the same algorithm happens in reverse 🔁. The gateway connected to host B's network answers the request for host A's network address, and then host B sends IP packets for host A to the gateway 💫. The physical networks of host A and host B don't need to be connected to the same gateway. All that's necessary is that the networks be reachable from the gateway ✅.

With this approach, all ARP subnet handling happens in the ARP subnet gateways 🙌. No changes to the normal ARP protocol or routing need to be made to the source and target hosts 🚫. From the host's point of view, there are no subnets, and their physical networks are simply one big IP network 🌸. If a host has an implementation of subnets, its network masks MUST be set to cover only the IP network number, excluding the subnet bits, for the system to work properly ⚠️.

2.2. Routing 🛣️

As part of implementing subnets, it's expected that the elements of routing tables will include network numbers with both the IP network number and the subnet bits, as specified by the subnet mask, where appropriate 📚. When an ARP request is seen, the ARP subnet gateway can check whether it knows a route to the target host by looking in the ordinary routing table 🔍. If attempts to reach foreign IP networks are eliminated early (see Sanity Checks below), only a request for an address on the local IP network will reach this point 🚧. We'll assume the same network mask applies to every subnet of the same IP network 🎀. The network mask of the network interface where the ARP request arrived can then be applied to the target IP address, giving the network part to look up in the routing table 🧮.

In 4.3BSD (and probably in other operating systems), a default route is possible 🌐. This default route specifies an address to forward a packet to when no other route is found 📬. The default route MUST NOT be used when checking for a route to the target host of an ARP request 🛑. If the default route were used, the check would always succeed 🙃. But the host specified by the default route is unlikely to know about subnet routing (since it's usually an Internet gateway), so packets sent to it will probably be lost 😩. This special case in the routing lookup method is the only implementation change needed to the routing mechanism 💅.

If the network interfaces where the request was received and where the route to the target passes are the same, the gateway MUST NOT reply 🙅‍♀️. In this case, either the target host is on the same physical network as the gateway (and so the host SHOULD reply for itself), or this gateway isn't on the most direct path to the desired network. That means there's another gateway on the same physical network that's on a more direct path, and the other gateway SHOULD respond 🤝.

RFC-925 [4] describes a general mechanism for dynamic subnet routing using Proxy ARP and routing caches in the gateways 🌙. Our technique is a restricted subset of RFC-925, in which we use static subnet routes that are determined administratively 🗂️. As a result, our transparent subnet gateways require no new network routing table entries and no new ARP cache entries. The only tables affected are the ARP caches in the host 💜.

In our implementation, routing loops are prevented by proper administration of the subnet routing tables in the gateways 🔒.

2.3. Multiple gateways 🌈

The simplest subnet organization to administer is a tree structure, which can't have loops 🌳. However, it may be desirable for reliability or traffic accommodation to have more than one gateway (or path) between two physical networks ⚖️. ARP subnet gateways MAY be used in such a situation. A requesting host will use the first ARP response it receives, even if more than one gateway supplies one 🏃‍♀️. This may even provide a rudimentary load balancing service, since if two gateways are otherwise similar, the one most lightly loaded is more likely to reply first 🎈.

More complex mechanisms could be built as gateway-to-gateway protocols, and they'll no doubt become necessary in networks with large numbers of subnets and gateways 🚀. That's the same way gateway-to-gateway protocols are generally necessary among IP gateways 🧡.

2.4. Sanity checks 🧼

Care MUST be taken by the network and gateway administrators to keep the network masks the same on all the subnet gateway machines 🧐. The most common error is setting the network mask on a host without a subnet implementation to include the subnet number 🙈. This causes the host to fail to attempt sending packets to hosts that aren't on its local subnet 😱. Adjusting its routing tables won't help, since it won't know how to route to subnets 🤷‍♀️.

If the IP networks of the source and target hosts of an ARP request are different, an ARP subnet gateway implementation SHOULD NOT reply 🚫. This keeps the ARP subnet gateway from being used to reach foreign IP networks, which could otherwise bypass security checks provided by IP gateways 🔐.

An ARP subnet gateway implementation MUST NOT reply if the physical networks of the source and target of an ARP request are the same 🙅. In that case, either the target host presumably sits on the same physical network as the source host and can answer for itself, or the target host lies in the same direction from the gateway as the source host does. In the second case, an ARP reply from the gateway would cause a loop 🔄.

An ARP request for a broadcast address MUST elicit no reply, regardless of the source address or the physical networks involved 📢. If the gateway responded with an ARP reply in this situation, it would be inviting the original source to send actual traffic to a broadcast address 🙀. This could result in the "Chernobyl effect," where every host on the network replies to such traffic and causes network "meltdown" 💥🌈

2.5. Multiple logical subnets per physical network 👯

The most straightforward way to assign subnet numbers is one to one with physical networks 📏. There are, however, circumstances where multiple logical subnets per physical network are quite useful 💡. One of the more common is when a group of workstations is planned to go onto their own physical network, but the gateway to that new physical network needs testing first. (A repeater might be used when the gateway was not usable). If a rule of one subnet per physical network is enforced, the addresses of the workstations MUST be changed every time the gateway is tested 😩. If they MAY be assigned addresses using a new subnet number while they're still on the old physical network, no further address changes are needed 🎉.

To permit multiple subnets per physical network, an ARP subnet gateway MUST use the physical network interface, not the subnet number, to decide when to reply to an ARP request 🔌. That is, it SHOULD send a proxy ARP reply only when the source network interface differs from the target network interface 🪞. In addition, appropriate routing table entries for these "phantom" subnets MUST be added to the subnet gateway routing tables 👻✨

2.6. Broadcast addresses 💖

okay so there are two kinds of IP broadcast addresses 🎀: the main IP directed network broadcast and the subnet broadcast. A network broadcast address is the network number plus a well-known value in the rest of the address (the local part). A subnet broadcast is basically the same vibe, except it includes both the network number AND the subnet number bits 💅. RFC-922 standardized using all ones in the local part, but before that there were two conventions: all ones and all zeros 🙈. 4.2BSD used all zeros, and 4.3BSD uses all ones. So ngl, there are four kinds of IP directed broadcast addresses still in use on a bunch of networks 😭.

With transparent subnetting, a subnet gateway must not issue an IP broadcast using the subnet broadcast address, like 128.83.138.255. Hosts on the physical network that receive that broadcast won't understand it as a broadcast address, since they won't have subnets enabled (or won't have subnet implementations at all) 💀.

In fact, 4.2BSD hosts (with or without subnet implementations) will treat an address with all ones in the local part as a specific host address and try to forward the packet 🙃. There's no such target host, so there's no entry in the forwarding host's ARP tables, and it sends out an ARP request for the target host. This is the scenario that was actually observed 👀: a 4.3BSD gateway running the rwho program broadcasts a packet once a minute, which made every 4.2BSD host on the local physical network generate an ARP request at the same time. The same problem happens with any subnet broadcast address, whether the local part is all zeros or all ones. It's giving chaos 🌪️.

Thus, a subnet gateway in a network with hosts that don't understand subnets must take care not to use subnet broadcast addresses. Instead, it must use the IP network directed broadcast address 🫶.

Finally, since a lot of hosts running out-of-date software will still be using (and expecting) the old-style all-zeros IP network broadcast addresses, the gateway must send its broadcast addresses out in that form, like 128.83.0.0 🍓. It might be safe to also send a duplicate packet with all ones in the local part, like 128.83.255.255. It's not clear whether the local network broadcast address of all ones, 255.255.255.255, will cause ill effects, but it's very likely that many hosts running older software won't recognize it ✨.

3. Implementation in 4.3BSD 🎀

Subnet gateways using ARP have been implemented by a number of different people 🧚. The particular method described in this memo was first implemented in 4.2BSD on top of retrofitted beta-test 4.3BSD subnet code, and it has since been reimplemented as an add-on to the distributed 4.3BSD sources. The latter implementation is the one described here 💜.

Most of the new kernel code for the subnet ARP gatewaying function lives in the generic Ethernet interface module, netinet/if_ether.c. It's eight lines in in_arpinput that do a couple of quick checks: making sure the facility is enabled on the source interface, and that the source and target addresses are on different subnets. Then it calls a new routine, if_subarp, for further checks, and builds the ARP response if all the checks succeed 🔥. This code is only reached when an ARP request is received, and it does nothing if the facility isn't enabled on the source interface. So the performance of the gateway should be very little degraded by this addition 🌸. (The performance of the requesting host should also be similar to the latter case, since the only difference there is the efficiency of the ARP cache versus the routing tables.)

The routine if_subarp (about sixty lines) makes sure that the source and target addresses are on the same IP network, and that the target address is none of the four kinds of directed broadcast address 👑. It then tries to find a path to the target, either by finding a network interface with the desired subnet or by looking in the routing tables. Even if a network interface is found that leads to the target, for a reply to be sent, the ARP gateway must be enabled on that interface, and the target and source interfaces must be different 🙈.

The file netinet/route.c gets a static routing entry structure definition added, and about eight lines of modifications are made to the main routing table lookup routine, rtalloc. It now recognizes a pointer to that structure (when it's passed by if_subarp) as a direction to not use the default route in this routing check 🦋. The processor priority level (critical section protection) around the inner routing lookup check is changed to a higher value, because the routine may now be called from network interface interrupts as well as from the internal software interrupts that drive processing of IP and other high-level protocols. This raised processor priority could conceivably slow the whole kernel down a bit if there are lots of routing checks, but since the critical section is fast, the effect should be small 🫧.

A key kernel modification is about fifteen lines added to the routine ip_output in netinet/ip_output.c 🔑. It changes subnet broadcast addresses in packets originating at the gateway into IP network broadcast addresses, so that hosts without subnet code (or with their network masks set to ignore subnets) will recognize them as broadcast addresses 🥰. This section of code is only used if the ARP gateway is turned on for the outgoing interface, and it only affects subnet broadcast addresses.

A new routine, in_mainnetof, of about fifteen lines, is added to netinet/in.c to return the IP network number (without the subnet number) from an IP address. It gets called from if_subarp and ip_output 🧠.

Two kernel parameter files have one line added to each. net/if.h gets a definition of a bit in the network interface structure to indicate whether subnet ARP gateways are enabled, and netinet/in.h refers to in_mainnetof 🎀.

In addition to these approximately 110 lines of kernel source additions, there's one user-level modification. The source to the command ifconfig, which is used to set addresses and network masks of network interfaces, has four lines added so it can turn the subnet ARP gateway facility on or off for each interface 🌈. This is documented in eleven new lines in the manual entry for that command 📖.

4. Availability 🦋

The 4.3BSD implementation is currently available by anonymous FTP (login anonymous, password guest) from sally.utexas.edu as pub/subarp 💌. It's a 4.3BSD "diff -c" listing from the 4.3BSD sources that were distributed in September 1986.

This implementation was not included in the 4.3BSD distribution proper, because U.C. Berkeley CSRG thought that would reduce the incentive for vendors to implement subnets per RFC-950 🙅‍♀️. The authors concur. Nonetheless, there are circumstances in which the use of transparent subnet ARP gateways is indispensable 🫶✨.

References 🌸

  1. Mogul, J., and J. Postel, "Internet Standard Subnetting Procedure", RFC-950, Stanford University and USC/Information Sciences Institute, August 1985.
  2. Mogul, J., "Broadcasting Internet Datagrams in the Presence of Subnets", RFC-922, Computer Science Department, Stanford University, October 1984.
  3. Plummer, D., "An Ethernet Address Resolution Protocol or Converting Network Protocol Addresses to 48-bit Ethernet Addresses for Transmission on Ethernet Hardware", RFC-826, Symbolics, November 1982.
  4. Postel, J., "Multi-LAN Address Resolution", RFC-925, USC/Information Sciences Institute, October 1984.
  5. Carl-Mitchell, S., and J. S. Quarterman, "Nameservers in a Campus Domain", SIGCUE Outlook, Vol.19, No.1/2, pp.78-88, ACM SIG Computer Uses in Education, P.O. Box 64145, Baltimore, MD 21264, Spring/Summer 1986.
  6. Braden, R., and J. Postel, "Requirements for Internet Gateways", RFC-1009, USC/Information Sciences Institute, June 1987.
📦 where this page comes from