TCP MD5

TCP_MD5 (RFC 2385) is something that doesn’t come up often. There’s a couple of reasons for that, good and bad.

I used it with tlssh, but back then (2010) it was not practical due to the limitations in the API on Linux and OpenBSD.

This is an updated post, written after I discovered TCP_MD5SIG_EXT.

What it is

In short it’s a TCP option that adds an MD5-based signature to every TCP packet. It signs the source and destination IP addresses, ports, and the payload. That way the data is both authenticated and integrity protected.

When an endpoint enables TCP MD5, all unsigned packets (including SYN packets) are silently dropped. For a signed connection it’s not even possible for an eavesdropper to reset the connection, since the RST would need to be signed.

Because it’s on a TCP level instead of part of the protocol on top of TCP, it’s the only thing that can protect a TCP connection against RST attacks.

It’s used by the BGP protocol to set a password on the connection, instead of sending the password in the handshake. If the password doesn’t match the TCP connection doesn’t even establish.

But outside of BGP it’s essentially not used, which is a shame. If we could enable it for any TCP service it’d add a preshared key and if nothing else completely replace the silly port knocking. It probably couldn’t replace user passwords, but it could add a layer and greatly reduce attack surface much more than, say, a TLS certificate.

It’s MD5. Sure, MD5 still doesn’t have any preimage attack. Well, none that’s feasible anyway.

So that should be fine. And if not then there’s already TCP AO which is about the same but with other algorithms.

How to use it

For the server (on Linux)

constchar*password="hello";structtcp_md5sigsig;memset(&sig,0,sizeof(sig));memcpy(&sig.tcpm_addr,&peer_sockaddr,peer_sockaddr_len);sig.tcpm_flags=TCP_MD5SIG_FLAG_PREFIX;sig.tcpm_prefixlen=0;// Match any address.
sig.tcpm_keylen=std::min(TCP_MD5SIG_MAXKEYLEN,strlen(password));memcpy(sig.tcpm_key,password,sig.tcpm_keylen);if(setsockopt(s,IPPROTO_TCP,TCP_MD5SIG_EXT,&sig,sizeof(sig))==-1){fprintf(stderr,"Failed to setsockopt(): %s\n",strerror(errno));exit(1);}

For the client (on Linux)

constchar*password="hello";structtcp_md5sigsig;memset(&sig,0,sizeof(sig));memcpy(&sig.tcpm_addr,&peer_sockaddr,peer_sockaddr_len);sig.tcpm_keylen=std::min(TCP_MD5SIG_MAXKEYLEN,strlen(password));memcpy(sig.tcpm_key,password,sig.tcpm_keylen);if(setsockopt(s,IPPROTO_TCP,TCP_MD5SIG_EXT,&sig,sizeof(sig))==-1){fprintf(stderr,"Failed to setsockopt(): %s\n",strerror(errno));exit(1);}

On the client you can use TCP_MD5SIG instead of TCP_MD5SIG_EXT, since it won’t need to set the prefixlen.

The sad reason it’s not used: It doesn’t work through NAT

Because it signs the source and destination address and port. This is getting to be less and less of an issue as the world goes IPv6. So maybe we can see more of this in the future.

This was never really a problem for BGP, since production BGP doesn’t run through NAT.

It used to be impossible to use on Linux for most applications

The TCP_MD5SIG socket option on Linux requires specifying exactly what the remote address is. This doesn’t make sense for listening sockets, like an OpenSSH server, which won’t know ahead of time what the remote address is.

BGP doesn’t really have clients and servers, just mutually configured peers, so it works fine there.

It used to be possible (back in 2010) to enable MD5 after a connection is established. And indeed this is what I did with tlssh back in 2010. But trying that now it results in EINVAL. Which is odd, because TCP MD5 was made for routers, and routers most certainly allows enablin TCP MD5 on an existing connection.

You can actually make this work, and it’s what I did with tlssh: You enable TCP MD5 immediately after the connection is established. If the two ends don’t have the same password set, then no packets will go through in either direction, and the connections will just time out. Not the best experience.

The old TCP_MD5SIG api, generally do not use, but instead use TCP_MD5SIG_EXT, seen below:

constchar*password="hello";structtcp_md5sigsig;memset(&sig,0,sizeof(sig));memcpy(&sig.tcpm_addr,peer_sockaddr,sizeof(structsockaddr_storage));sig.tcpm_keylen=std::min(TCP_MD5SIG_MAXKEYLEN,strlen(password));memcpy(sig.tcpm_key,password,sig.tcpm_keylen);if(setsockopt(s,IPPROTO_TCP,TCP_MD5SIG,&sig,sizeof(sig))==-1){fprintf(stderr,"Failed to setsockopt(): %s\n",strerror(errno));exit(1);}

On OpenBSD it’s a system-level setting

On OpenBSD you set up the TCP MD5 as a security association between the two hosts, and then the program just enables MD5 on the socket. And it “just works” on both the client and the server. Well, a bit annoying that the key is set in hex.

# cat > /etc/tcpmd5.conf
tcpmd5 from 2001:db8::1111 to 2001:db8::2222 spi 0x100 authkey 0x68656c6c6f
tcpmd5 from 2001:db8::2222 to 2001:db8::1111 spi 0x101 authkey 0x68656c6c6f
^D
# ipsecctl -f /etc/tcpmd5.conf

constinton=1;if(setsockopt(s,IPPROTO_TCP,TCP_MD5SIG,&on,sizeof(on))==-1){fprintf(stderr,"Failed to setsockopt(): %s\n",strerror(errno));exit(1);}

Netcat with the -S option does the latter, but the password still needs to be set in the config.

So on OpenBSD it doesn’t really mesh well with adding TCP MD5 to any and all programs. Even if you set up the Security Association inside your program instead of in your config it seems more a state of the system than a state of your connection. Which makes some sense with BGP, but almost nothing else.

Patches

I’ve patched OpenSSH to enable TCP MD5. Works great. Right now it just uses a static password, but that still protects you against internet-wide spread port scans that don’t know what you’re running on port 2222.

$ sudo tcpdump -M openssh -nlpi lo port 2222
[…]
16:04:12.212566 IP6 ::1.2222 > ::1.43934: Flags [P.], seq 1828532:1828704, ack 1117, win 179, options [nop,nop,md5 valid], length 172
[…]

This sure is better than portknocking. The only problem is that it doesn’t work through IPv4 NAT. So just use IPv6.