Our engineering team focuses on getting the maximum amount of information from the network while sending as little traffic as possible. This lean approach to network discovery is driven by our goal of being fast and safe for all networks. The more we can learn about a system from a single measurement, the less traffic we create and the quicker things run. In this post, I want to share how runZero uses one of the most common network protocols to obtain a wealth of information about network-attached devices.
Playing with ping #
The standard "ping" utility is one of the most commonly used network troubleshooting tools. This utility sends an ICMP Echo Request to a specific address and reports any replies it receives. This protocol is simple: the sender creates an IP header, appends an ICMP header, sets the Type and Code fields, and then adds the Echo Request data, consisting of an identifier, sequence number, and some data to echo. Finally, this protocol is written to the network, often with an Ethernet header.
The recipient receives the request, verifies the checksum, and responds with an almost identical reply where the Type is set to 0 instead of 8. The sender receives the response and verifies the data, calculating the time it took to get a reply, and shows this to the user. The sender increments the sequence number by one and then starts the cycle all over again. You can try this on your machine with the following command:
$ ping 13.248.161.247
Pinging 13.248.161.247 with 32 bytes of data:
Reply from 13.248.161.247: bytes=32 time=7ms TTL=120
This response tells us three things:
-
The system at address 13.248.161.247 exists and responds to ICMP Echo Requests.
-
The round-trip time was 7ms, so the system is likely between 300 and 700 miles away. Light and electricity have a top speed of 186 miles per millisecond. Routers take a non-zero amount of time to forward packets, and the recipient system requires some processing time to create a response. The network path between two points is also rarely a straight line. A good estimate of distance based on the round-trip time is around 25-50% of the speed of light in a vacuum.
-
There are likely 8 network hops between the system and our machine. The received time-to-live field (TTL) was decremented by one for each hop the reply traversed. The packet almost certainly started off with a TTL of 128 (a common value, like 64 and 255), which means there are likely 8 hops in the path from that system back to us. This is the inverse of a traceroute, which increments a TTL until it receives a response.
Digging into the raw packet, we can also see the MAC address on the Ethernet header, which belongs to the router that sent the packet back to us. Additionally, we can see other IP header fields like the ToS (0) and ID (0), which can provide some clues about the remote operating system and version.
This functionality is enough for us to start creating interesting measurements.
Local discovery #
Let's take a closer look at our local network. If our machine has an address in the subnet 192.168.0.0/24, what happens if we ping a machine in a different subnet?
$ ping 192.168.50.61
Pinging 192.168.50.61 with 32 bytes of data:
Reply from 192.168.50.61: bytes=32 time<1ms TTL=63
We received a reply in less than one millisecond, indicating that the destination is very close. The TTL field of 63 indicates that the remote host's starting TTL was likely 64 and that the response traversed one hop to get back to us. The estimated TTL also hints at the remote operating system: 64 is common for Linux, 128 for Windows, and 255 for BSD and macOS. We can guess that with a received TTL of 63, the remote system is likely running Linux.
Let's try pinging an IP address that is not in use:
$ ping 192.168.50.111
Pinging 192.168.50.111 with 32 bytes of data:
Reply from 192.168.0.1: Destination host unreachable.
We received a reply, but not from the address we pinged. Instead, we got a reply from our local router telling us that the host does not exist. This tells us two useful things:
-
The router at 192.168.0.1 also has an IP in the 192.168.50.0/24 subnet
-
An ARP request by the router on the subnet did not get a reply for 192.168.50.111
The last router in the path needs to resolve the layer 2 (usually Ethernet) address of the destination. Many routers will send up to three ARP requests for a local address before giving up. This means that our single ping request resulted in at least six packets on the wire:
-
Our local machine used ARP to get the Ethernet address of the 192.168.0.1 router.
-
Our local machine sent the ICMP packet to 192.168.0.1 to forward to 192.168.50.111.
-
The router at 192.168.0.1 sent three ARP requests on the 192.168.50.0/24 interface.
-
After failing to get a reply, the router sent us an ICMP Host Unreachable response
We don't control how the router does the final discovery and delivery, but we need to account for traffic multiplication when thinking about the network impact of our ping.
It's worth noting that these are all best-case assumptions. Any node can reply with any kind of ICMP response, and many firewalls are configured to send unreachable errors for systems where an ACL prevents access even if the host is online. ICMP responses provide a ton of information, but they can also be complete fabrications, and a paranoid approach to handling replies is important.
Broadcast responses #
Instead of pinging a specific machine, let's try pinging the network broadcast address:
$ ping 192.168.0.255
Pinging 192.168.0.255 with 32 bytes of data:
Reply from 192.168.0.5: bytes=32 time<1ms TTL=64
The ping utility shows that we received one reply from 192.168.0.5. The TTL and the time measurement both indicate this is a machine on the local network. Let's take a look at Wireshark:
Wow. OK, so we received seven different replies, and 192.168.0.5 was just the first one. Not shown here are the ARP requests and responses from each of these seven machines for the MAC address of our machine. These responses included the Ethernet, IP, and ICMP headers. Looking at the latency measurements, we can see two clear groups:
The first group, consisting of 192.168.0.5 and 192.168.0.153, have very low latency (near zero from when the request was sent). These IPs are associated with two Apple Mac Minis (Intel and ARM).
The second group, consisting of 192.168.0.21, 192.168.0.31, 192.168.0.33, and 192.168.0.34 are all various Netgear switches, with a distinctly higher response time.
These seven systems replied to a broadcast ping, but these are far from the only things on the network. These systems also had identical starting TTLs of 64. This tells us a few things right away:
-
Seven machines are configured to reply to local broadcast pings. These machines consist of Apple and Netgear products. Cisco products also commonly reply to broadcast traffic.
-
The latency measurements clearly separate PC-class processors (Intel, M1) from the embedded processors in the network switches. A ping alone can tell you a lot about the underlying hardware.
-
The MAC addresses in the reply are also useful for device fingerprinting and device age estimation.
Multihomed devices #
While most operating systems do not respond to pings of the network broadcast address, they do often reply from an IP address other than the one where the packet was received. Instead of sending ABCs in the data of the Echo body, what would happen if we encoded the destination IP address itself into the body and sent this to every IP on the network? Oddly enough, sometimes we get a reply from an IP we didn't send to (just like the broadcast address), but the echoed response data contains the original destination.
$ ping 192.168.30.10
Reply from 192.168.30.55: bytes=13 time=1ms TTL=63
Data: "192.168.30.10"
We now know that the system at IP address 192.168.30.10 has a second IP address at 192.168.30.55, and the "55" address is the default route back to our machine. This works great in practice, allowing us to correlate IP addresses on wildly different networks. On large internet scans, we often see ICMP replies from different /8 prefixes for the same physical system and can automatically correlate these in the results. This technique also works for many UDP-based services. The trick is tagging the request in a way that the response indicates the original destination.
Egress discovery #
Going a step further, what happens if we send an ICMP Echo Request to every IP in the local network, but spoof the source IP address so that it is an external internet address? Depending on the firewalls involved, these machines will reply to the external address and, more importantly, use their default internet route to send this reply.
If we listen for these replies on an internet-facing system, we can determine what default route was used for every internal machine. This technique can uncover unauthorized egress points, such as LTE modems and software VPNs.
In summary #
ICMP as a protocol is simple, but the amount of data that can be gleaned through a few creative packets is extensive. Systems that respond to ICMP Echo Requests are effectively providing a remote API: you give them a request, and depending on the configuration, they take various actions, which provides useful data. This "API" has limitations, including default rate limits, but it is available on nearly every networked device on the planet.
runZero uses ICMP responses for latency measurement, subnet identification, multihomed asset discovery, operating system fingerprinting, topology mapping, and more.
