Linux Firewalls - Michael Rash [40]
RST vs. RST/ACK
Many firewalls and intrusion detection systems can send TCP RST packets to knock down malicious connections, but the implementation details for sending such packets vary greatly. One detail often overlooked is whether a firewall or IDS sends a plain RST packet or a RST/ACK packet.
According to RFC 793, there are only three circumstances in which a TCP stack should generate a RST/ACK; the rest of the time, a RST packet is sent without the ACK bit set. Further, there is an inverse relationship between the ACK flag in the last packet seen in the TCP session and a RST packet used to tear down the connection. That is, if the last packet contained the ACK flag, a RST packet should not contain the flag. Conversely, if the last packet did not contain the ACK flag, a RST should.
For example, if a TCP SYN packet is sent to a port where no server is listening (i.e., the port is in the CLOSED state), a RST/ACK is sent back to the client. But if a SYN/ACK packet is sent to a CLOSED port, then a RST packet with no ACK bit is sent back to the client. These two scenarios are illustrated by the following example:
❶ [iptablesfw]# iptables -I INPUT 1 -p tcp --dport 5001 -j ACCEPT
❷ [ext_scanner]# nmap -P0 -sS -p 5001 71.157.X.X
[iptablesfw]# tcpdump -i eth0 -l -nn port 5001
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
17:10:24.693292 IP 144.202.X.X.33736 > 71.157.X.X.5001: S
522224616:522224616(0) win 2048 17:10:24.693413 IP 71.157.X.X.5001 > 144.202.X.X.33736: ❸R 0:0(0) ack 522224617 win 0 ❹ [ext_scanner]# nmap -P0 -sA -p 5001 71.157.X.X [iptablesfw]# tcpdump -i eth0 -l -nn port 5001 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode 17:11:03.985446 IP 144.202.X.X.62669 > 71.157.X.X.5001: . ack 1406759780 win 1024 17:11:03.985477 IP 71.157.X.X.5001 > 144.202.X.X.62669: ❺R 1406759780:1406759780(0) win 0 At ❶ above, iptables is taken out of the picture for TCP port 5001, and any client is allowed to talk directly with the Linux TCP stack on the iptablesfw system. This eliminates iptables as a potential factor that might otherwise skew our results. At ❷, a standard Nmap SYN scan is sent against port 5001 on the iptablesfw system, and the next line shows a tcpdump command to watch what happens. At ❸, the local TCP stack sends a RST back to the client, and this RST has a non-zero acknowledgment value; the ACK bit is set because the SYN packet from Nmap (displayed on the previous line in the tcpdump output) did not contain the ACK bit. At ❹, another Nmap scan is sent against port 5001: an ACK scan. The RST from the local TCP stack is seen at ❺, with no acknowledgment number and the ACK bit unset. This is because the packet from Nmap contained an acknowledgment number and had the ACK bit set. The iptables REJECT target implements the inverse relationship between the ACK flag on a matched TCP packet and the RST that it generates. This is enforced by the following code snippet from the linux/net/ipv4/netfilter/ipt_REJECT.c file in the kernel sources (see the send_reset() function—some of the code has been abbreviated for readability): static void send_reset(struct sk_buff *oldskb, int hook) { struct tcphdr *tcph; ❶ int needs_ack; ❷ if (tcph->ack) { ❸ needs_ack = 0; tcph->seq = oth->ack_seq; tcph->ack_seq = 0; } else { ❹ needs_ack = 1; tcph->ack_seq = htonl(ntohl(oth->seq) + oth->syn + oth->fin + oldskb->len - oldskb->nh.iph->ihl*4 - (oth->doff<<2)); tcph->seq = 0; } ❺ tcph->ack = needs_ack; At ❶, a flag needs_ack is declared that is used to determine whether the generated TCP RST packet contains the ACK control bit (and the corresponding nonzero acknowledgment value). If the original TCP packet