TCP MD5

TCP_MD5 (RFC 2385) is something that doesn’t come up often. There’s a couple fo reasons for that, good and bad.

I used it with tlssh, but this should explain why I didn’t enable it by default.

What it is

In short it’s a TCP option that adds an MD5-based signature to every TCP packet. It signs the source and destination IP address, and the ports, and the payload. That way the data is both authenticated and integrity protected.

When an endpoint enables TCP MD5, all unsigned packets (including SYN packets) are silently dropped. For a signed connection it’s not even possible for an eavesdropper to reset the connection, since the RST would need to be signed.

It’s used by the BGP protocol to set a password on the connection, instead of sending the password in the handshake. If the password doesn’t match the TCP connection doesn’t even establish.

But outside of BGP it’s essentially not used, which is a shame. If we could enable it for any TCP service it’d add a preshared key and completely replace the silly port knocking. It probably couldn’t replace user passwords, but it could add a layer and greatly reduce attack surface much more than, say, a TLS certificate.

The bad reason it’s not used

It’s MD5. Sure, MD5 still doesn’t have any preimage attack. Well, none that’s feasible anyway.

So that should be fine. And if not then there’s already TCP AO which is about the same but with other algorithms.

The more real reasons

1. It’s impossible to use on Linux for most applications

I think this is the biggest reason it’s not used for playing around. This is the Linux API:

const char* password = "hello";
struct tcp_md5sig sig;
memset(&sig, 0, sizeof(sig));
memcpy(&sig.tcpm_addr, peer_sockaddr, sizeof(struct sockaddr_storage));
sig.tcpm_keylen = std::min(TCP_MD5SIG_MAXKEYLEN, strlen(password));
memcpy(sig.tcpm_key, Sflag_password, sig.tcpm_keylen);
if (setsockopt(s, IPPROTO_TCP, TCP_MD5SIG, &sig, sizeof(sig)) == -1) {
  fprintf(stderr, "Failed to setsockopt(): %s\n", strerror(errno));
  exit(1);
}

See the problem? Yeah. You need to know the address and port of the remote end. So it’s fine for clients, but not so much for servers. BGP doesn’t really have clients and servers, so it works fine there.

You can actually make this work, and it’s what I did with tlssh: You enable TCP MD5 immediately after the connection is established. If the two ends don’t have the same password set, then no packets will go through in either direction, and the connections will just time out. Not the best experience.

On OpenBSD it’s much easier. You set up the TCP MD5 as a security association between the two hosts, and then the program just enables MD5 on the socket. And it “just works” on both the client and the server. Well, a bit annoying that the key is set in hex.

# cat > /etc/tcpmd5.conf
tcpmd5 from 2001:db8::1111 to 2001:db8::2222 spi 0x100 authkey 0x68656c6c6f
tcpmd5 from 2001:db8::2222 to 2001:db8::1111 spi 0x101 authkey 0x68656c6c6f
^D
# ipsecctl -f /etc/tcpmd5.conf

const int on = 1;
if (setsockopt(s, IPPROTO_TCP, TCP_MD5SIG, &on, sizeof(on)) == -1) {
  fprintf(stderr, "Failed to setsockopt(): %s\n", strerror(errno));
  exit(1);
}

Or with netcat just use the -S option (but the password still needs to be set in the config).

But yes even on OpenBSD it doesn’t really mesh well with adding TCP MD5 to any and all programs. Even if you set up the Security Association inside your program instead of in your config it seems more a state of the system than a state of your connection. Which makes some sense with BGP, but almost nothing else.

2. It doesn’t work through NAT

Because it signs the source and destination address. This is getting to be less and less of an issue as the world goes IPv6.

Again, was never really a problem for BGP.

My wish

If Linux could have an API for enabling TCP MD5 on the server side for “any” peer address, that’d be great.