The wonder of UNIX is that you can delete running binaries and loaded shared libraries. The drawback is that you get no warning that you're still actually running old versions. E.g. old heartbleed-vulnerable OpenSSL.

Server binaries are often not forgotten by upgrade scripts, but client binaries almost certainly are. Did you restart your irssi? PostgreSQL client? OpenVPN client?

Find processes running with deleted OpenSSL libraries:

$ sudo lsof | grep DEL.*libssl
apache   17179      root  DEL       REG        8,1               24756 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0

Or if you're extra paranoid, and want to make sure everything is using the right OpenSSL version:

!/bin/sh
set -e
LIB="/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0"
if [ ! "$1" = "" ]; then
   LIB="$1"
fi
INODE="$(ls -i "$LIB" | awk '{print $1}')"
lsof | grep libssl.so | grep -v "$INODE"

• Run this as root in case lsof otherwise wouldn't be able to get at the data (e.g. if you run grsec)
• This assumes all libssl is on one filesystem, since it only checks inode number
• The easiest solution is of course to restart the whole machine, but there's really no reason to if you don't want to

Let's say you don't have a TPM chip, or you hate them, or for some other reason don't want to use it to protect your SSH keys. There's still hope! Here's a way to make it possible to use a key without having access to it. Meaning if you get hacked the key can't be stolen.

No TPM, but key can't be stolen anyway? Surely this is an elaborate ruse? Well yes, it is. My idea is that you essentially bounce off of a Raspberry Pi.

But doing that straightforward is too easy. I've instead made an SSH proxy, and will show you how to automatically bounce off of it. You could do the same by setting up a second SSH server (or the same one), and hack around with PAM and a restricted shell. But this solution can be run as any user, with just the binary and the set of keyfiles. Very simple.

The goal here is to log in to shell.foo.com from your workstation via a Raspberry Pi. The workstation SSH client presents its SSH client key to the SSH Proxy on the Raspberry Pi, and if allowed will connect on and present the SSH Proxy client key to shell.foo.com.

It doesn't have to be a Raspberry Pi, of course, but the idea here is to have it be a dedicated machine that you never log in to otherwise, and one nobody else has access to. A virtual machine will not do, since the host system has access to virtual machines.

1. Get a Raspberry Pi

   Get it up and running with SSH, and lock down every other port except, say 2022. Details on this are out of scope for this blog post.

2. Install Go

   Since it's ARM you're going to have to download and compile it.

   raspberrypi# cd /usr/local
   raspberrypi# wget http://golang.org/dl/go1.2.2.src.tar.gz  # Or whatever version is new.
   raspberrypi# tar fxz go1.2.2.src.tar.gz
   raspberrypi# cd go/src
   raspberrypi# ./make.bash
   [...]
   

3. Get SSHProxy

   raspberrypi$ cd $HOME
   raspberrypi$ mkdir go
   raspberrypi$ cd go
   raspberrypi$ GOPATH=$(pwd) /usr/local/go/bin/go get github.com/ThomasHabets/sshproxy
   raspberrypi$ ./bin/sshproxy -help
   Usage of ./sshproxy:
     -auth="": Auth mode (key, kbi).
     -authorized_keys="": auth=key: Authorized keys for clients.
     -client_keyfile="": auth=key: SSH client key file.
     -conn_fd="": File descriptor to work with.
     -forwarded="": Forwarded for. Used by sslserver.
     -keyfile="": SSH server key file.
     -log_downstream=false: Log data from downstream (client).
     -log_upstream=false: Log data from upstream (server).
     -logdir="": Directory in which to create logs.
     -target="": SSH server to connect to.
   

4. Generate SSHProxy server key

   $ ssh-keygen -N "" -f sshproxy -b 4096 -t rsa

5. Generate key SSHProxy will log in with

   $ ssh-keygen -N "" -f sshproxy-client -b 4096 -t rsa
   

6. Generate SSL key for SSHProxy (e.g. self signed)

   If someone breaks this key, or because it's self signed you are MITMed, then you will only reveal which hostname/port you tried to connect to.

   $ openssl req -new -x509 -nodes -sha256 -days 4096 -verbose -subj /CN=sshproxy.foo.com/ -newkey rsa:4096 -keyout sshproxy-ssl.key -out sshproxy-ssl.crt
   

7. Add your normal workstation SSH key to a new file, "authorized_proxy_clients"

   raspberrypi$ cat > authorized_proxy_clients
   ssh-rsa AAAA[...]uoeu== foo@bar.com
   ^D
   $
   

8. Start SSHProxy on the Raspberry Pi

   raspberrypi$ mkdir sshlogdir
   raspberrypi$ ./sslserver -key sshproxy-ssl.key -cert sshproxy-ssl.crt -listen :2023 \
           ./sshproxy \
           -auth key \
           -keyfile sshproxy \
           -authorized_keys authorized_proxy_clients \
           -client_keyfile sshproxy-client \
           -logdir sshlogdir
   

9. Install SSHProxy on your workstation too

   You'll only be using its sslclient. See above for instructions.

10. Add Proxy config to your workstation ~/.ssh/config

    Host *.foo.com
        ProxyCommand /path/to/sslclient -proxy 127.0.0.1:2023 -target %h:%p
    

11. Make sure *only* the SSHProxy client key is in authorized_keys on shell.foo.com

12. SSH from workstation to shell.foo.com

    workstation$ ssh shell.foo.com
    shell$
    

Links

• github.com/ThomasHabets/sshproxy
• TPM chip protecting SSH keys - properly

#!/bin/sh
NAME=$1
COLOR=$2
DESC="Some random machine"
QUALITY=h   # or l for low, m for medium
set -e

dispcal -m -H -q $QUALITY -y l -F -t $COLOR -g 2.2 $NAME
targen -v -d 3 -G -e 4 -s 5 -g 17 -f 64 $NAME
dispread -v -H -N -y l -F -k $NAME.cal $NAME
colprof -v -D $DESC -q m -a G -Z p -n c $NAME
dispwin -I $NAME.icc

I've previously blogged about a secure connection between browser and proxy. Unfortunately that doesn't work on Android yet, since except if you use Google for Work (an enterprise offering) you can't set Proxy Auto-Config.

This post shows you how to get that working for Android. Also it skips the stunnel hop since it doesn't add value and only makes Squid not know your real address. I'm here also using username and password to authenticate to the proxy instead of client certificates, to make it easier to set up.

First, why you would want to do this

• You have machines behind NAT, and a proxy that can see the inside while still accessible form the outside

  This way you can port forward one port from the NAT box to the proxy, and not have to use different ports everywhere. I'll call this proxy corp-proxy.example.com.

• You have servers that don't implement their own authentication, and you want the proxy to do it for you

  If you set up so that the only way to connect to the servers is via the proxy, then all access will be encrypted and password-protected. You won't get a green lock in your browser address bar since it's only protected between browser and proxy, not all the way to the web server. This also applies to corp-proxy.example.com.

• To encrypt traffic crossing country borders

  For all traffic going to Sweden, you want to securely connect to a proxy in Sweden, so that even traffic to unencrypted websites is encrypted when it's going across borders. I'll call this sweden-proxy.example.com.

How to do it

1. Get a real SSL certificate for your proxy

   StartSSL.com has free certificates. Put the .crt and .key files in /etc/squid3/. This blog post does not discuss the big topic that is the security of CAs and SSL.

2. Build and install Squid from source

   For licensing reasons, the Squid package on Debian is build without SSL. So let's make a .deb and install it.

   $ apt-get source squid3
    ... 
   $ sudo apt-get build-dep squid3
    ...
   $ sudo apt-get install devscripts build-essential fakeroot libssl-dev squid-langpack
    ...
   $ cd squid3-3*
   $ vim debian/rules  # Add --enable-ssl among the flags.
   $ ./configure
    ...
   $ debuild -us -uc -b
    ...
   $ cd ..
   $ sudo dpkg -i squid3_3*.deb squid3-common_3*.deb
    ...
   

3. Configure Squid

   $ sudo tee /etc/squid3/squid.conf > /dev/null
   acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
   auth_param basic program /usr/lib/squid3/ncsa_auth /etc/squid3/passwd
   auth_param basic realm proxy
   acl authenticated proxy_auth REQUIRED
   
   http_access deny to_localhost
   http_access allow authenticated
   
   # Effective turn off unencrypted connections.
   http_port 127.0.0.1:3127
   https_port 3128 cert=/etc/squid3/proxy.example.com.crt key=/etc/squid3/proxy.example.com.key
   ^D
   

4. Create allowed users

   $ sudo apt-get install apache2-utils
    ...
   $ sudo touch /etc/squid3/passwd
   $ sudo htpasswd /etc/squid3/passwd alice
    ...
   $ sudo htpasswd /etc/squid3/passwd bob
    ...
   

5. Restart squid

   $ sudo /etc/init.d/squid3 restart

6. Create a wpad.pac file somewhere that's reachable on the web

   $ cat > /var/www/wpad.dat
   function FindProxyForURL(url, host) {
       if (dnsDomainIs(host, ".corp.example.com")) {
           return "HTTPS corp-proxy.example.com:3128";
       }
       if (dnsDomainIs(host, ".se")) {
           return "HTTPS sweden-proxy.example.com:3128";
       }
       return "DIRECT";
   }
   ^D
   

   OK, that's it for the proxy. Now to set up the clients.

7. For Android devices: Install Drony

8. For Android devices: Set up Drony

   In Drony/Settings/"Wifi"/Name of network you're on ("Network not listed" is 3G), set:
   • Under "Script", set the URL, press the download button, and save.
   • Back under "network details", set "Proxy type" to "Script"
   Then in Drony, swipe right and press the "On" button.

9. For Desktop/Laptop: Set up proxy in Chrome (or other favourite browser with HTTPS proxy support)

   Menu->Preferences->Under the Bonnet->Change proxy settings->Automatic Proxy Configuration
   Set the URL to where the file is, including "https://".

10. Try to browse somewhere. When asked for username/password, enter them. Drony/Chrome will cache them

    Check /var/log/squid3/access.log for log lines.

Other notes

• You can set up authentications other than password, such as client certificates. But I don't know if Drony supports them.
• Please star the Chrome bug about .pac files.
• If your .pac file returns "PROXY " then it will connect unencrypted. HTTPS over it is still safe since that encrypts end to end.
• Yes, that means that if you run HTTPS to the proxy and connect to an HTTPS site, you are tunnelling HTTPS inside HTTPS.
• Proxy Auto-Config on Wikipedia.
• List of functions available in .pac files

Why

Let’s say you have some TCP connections to your local system that you want to kill. You could kill the process that handles the connection, but that may also kill other connections, so that’s not great. You could also put in a firewall rule that will cause the connection to be reset. But that won’t work on a connection that’s idle (also if one side is initiator then using this method the other side would not tear down its side of the connection). There’s tcpkill, but it needs to sniff the network to find the TCP sequence numbers, and again that won’t work for an idle connection.

Ideally for these long-running connections TCP keepalive would be enabled. But sometimes it’s not. (e.g. it’s not on by default for gRPC TCP connections, and they certainly can be long-running and idle).

You could also do this by attaching a debugger and calling shutdown(2) on the sockets, but having the daemon calling unexpected syscalls thus getting into an unexpected state doesn’t really make for a stable system. Also attaching a debugger hangs the daemon while you’re attached to it.

This post documents how to do this on a Debian system.

No, really. Why?

If a client connects to a dual-stack hostname it’ll (usually, see RFC3484) first try IPv6, and then IPv4 if that fails.

If a server comes up after the client tries IPv6 then it’ll fall back to IPv4, even though IPv6 would have worked at that time too.

I want to kick the IPv4 clients over to IPv6, since restarting the server (or even rebooting the server) doesn’t change anything about the race, and I don’t want to restart the clients because they’re doing long-running compute work that I don’t want to lose state on.

With IPv6 I can differentiate hosts behind NAT, for example.

How

1. Download debug kernel package for the kernel you’re running

Take the date from uname -a and add a week or so, and open the Debian archive for that day. E.g. http://snapshot.debian.org/archive/debian/2017031500T000000Z/pool/main/l/linux/.

Download the -dbg version of the kernel you’re running. E.g.:

linux-image-3.16.0-4-amd64-dbg_3.16.39-1+deb8u2_amd64.deb  351181890       2017-03-10 03:37:13

2. Unpack the .deb

mkdir tmpkernel
cd tmpkernel
dpkg -x ../linux-image….deb .
cp ./usr/lib/debug/lib/modules/*/vmlinux .

3. Find the address of the skbuf

$ ss -e -t dst 10.0.64.123
State      Recv-Q Send-Q         Local Address:Port    Peer Address:Port
ESTAB      0      0             ::ffff:192.0.2.1:22    ::ffff:10.0.64.123:30201    uid:1003 ino:68386802 sk:ffff88000caa2800 <->

3. Start kernel debugger

sudo apt-get install crash
sudo crash -e emacs ./vmlinux

4. Print the sequence numbers

crash> struct tcp_sock.rcv_nxt,snd_una ffff88000caa2800
  rcv_nxt = 2691239595
  snd_una = 3825672049

5. Kill both sides of the connection

hping3 -s 22    -c 1 -M 3825672049 -L 2691239595 -F -A -p 30201    10.0.64.123
hping3 -s 30201 -c 1 -L 2691239595 -M 3825672049 -F -A -p 22    -a 10.0.64.123 192.0.2.1

6. Verify that connection is closed

netstat -napW | grep 10.0.64.123

If possible you may want to check the remote end too. But if it’s the client that will eventually send traffic then it’ll be cleanly disconnected at that point.

Something I’ve been playing with lately is software defined radio with GNURadio. I’m not good at it yet, but I’ve managed to decode the signals from a couple of things.

This is my step-by-step for how I decoded data from a boiler thermostat. I’m not saying it’s the best way, or even a good way. But it’s what got me there.

0. Find the frequency

Often this is written on the device itself. Other times it’s in the manual. If not, then more research is needed, such as by trying to find the manufacturer on fcc.gov or similar.

In this case it was easy. The manual said “868 MHz”, which is in the SRD860 band.

1. Capture some data

When I poked at the controls of the thermostat, saying “please make the room 25 degrees”, the thermostat must send this data to the boiler. I could hear the boiler start up and shut down, so there must be something sent between me pressing the buttons and I heard the results.

I started by centering around 868.5 Mhz with 1Msps. The minimum for the RTL-SDR is 900ksps, so even if you wanted to see less than 1MHz you need to capture more first, and then downsample later.

A waterfall showed action around 868.288 MHz, that when slowed down became obvious 2FSK.

I started a new capture at 868.288Mhz at 1Msps of both “boiler goes on” and “boiler goes off”, and saved to file.

01-capture.grc

2. Cut away everything before and after the burst

Tweak “skip” and “save” variables (given in seconds) to get a smaller file that can be played with without needing to wait. Keep a bit of buffer on both sides though.

02-cut.grc

3. Filter the signal

An easy way to clean up the signal is to squelsh away the noise, and then band pass only the regions where there should be signal.

This is very likely not the best way to get the best range, but it does make it easy to work with the signal visually. And decoding a strong signal can be a first step to later tweaking to be able to receive weaker signals.

All inputs to a Frequency Sink must have the same sample rate, so compensate for the decimation by repeating the signal.

Because the signal coming out of the band pass filter is not still the original sample rate, it’ll create lots of aliasing in the graph. That’s fine. We’ll be working with the decimated sample rate anyway.

03-filter.grc

4. Quadrature demod

Quadrature demod is a magic box that turns water into wine, and 2FSK into floats. The output is essentially “>0 means the higher frequency was active, <0 means the lower frequency was active”, and the value shows how much more active that frequency was.

04-quad.grc

5. Moving average

What came out of quadrature demod was a square wave. The input to clock recovery needs to have peaks, or it won’t be able to center a bit around that peak. If the peak is flat then it won’t be able to adjust closer to the center of that peak.

A moving average will do this nicely. The number of samples to average is the width, in samples, of a peak.

The output is now a very nice and obvious bit pattern.

05-moving.grc

6. Clock recovery

Clock recovery takes a stream of data and picks out the “bits”, giving one output sample per bit. This is much better explained in the Guided tutorial PSK demodulation.

These output bits don’t actually look that good. I found that I get reliable output in the end, but clock recovery is clearly the weak part of my project.

06-clock-recovery.grc

7. Packets, assemble!

We now have a stream of bits, surrounded on both sides by zeroes. Turns out there’s no good standard block to turn this into packets. There’s “Tagged Stream to PDU”, but it wants the packet length.

So I had to write my own block. Because this block will be running at the bit rate of 2400sps and not the original sample rate of 1Msps or even the post-filter sample rate of 200ksps, I thought it would be fine to write it in Python. I have another block decoder for OOK designed to run at higher sample rates, and there the performance benefits of using C++ really mattered.

The block treats >0 as “one”, <0 as “zero”, and 0 as “end of packet”. At end of packets it tries to find the packet preamble, and if found will emit a PDU.

As expected, the packet for “on” and the packet for “off” are static, and they differ by one bit.

07-sink.grc

packet decoder

8. Ship it!

This is the same as step 7, except capturing data live. The flow graph is simple and fast enough to run on a Raspberry Pi 3.

08-live.grc

9. Data over time

After setting this up to log indefinitely I saw that the thermostat sends a command to the boiler once every 10 minutes in addition to when I triggered it with buttons. Presumably if the thermostat dies the boiler will automatically shut down after missed commands, so these transmissions every 10 minutes serve as keep-alive.

Two interesting things I saw when listening to the output from my microwave:

1. The frequency is not stable. I assume it’s not thermally controlled.
2. Probably not a good idea to use WiFi channel 6, being 2.437GHz+-11MHz and all.

Near the microwave

Some rooms away from the microwave

Separate measurement some rooms away.

How to generate the graphs

# Edit usrp_spectrum_sense.py so that it prints time.time() instead of datetime.now()
./usrp_spectrum_sense.py --dwell-delay=0.1 -A RX2 -s 8000000 --real-time 2.4e9 2.5e9 > near-microwave.txt
# Edit near-microwave to remove the stupid verbose messages from the top.
cat > microwave.plot << EOF
set view map
set size ratio .9

set object 1 rect from graph 0, graph 0 to graph 1, graph 1 back
set object 1 rect fc rgb "black" fillstyle solid 1.0

set xlabel 'Time in seconds'
set ylabel 'Frequency'
set zlabel 'dB'
set terminal epslatex color
set output "near-microwave.eps"
splot 'near-microwave.txt' using ($1-1496693552.11):((($5/1000-2400000)/1000)):4 with points pointtype 5 pointsize 3 palette linewidth 30 title 'Signal'
EOF
gnuplot microwave.plot
convert near-microwave.{eps,png}

I can’t have GNUPlot output PNG directly because it hangs.

The magic value 1496693552.11 is the first timestamp in the file.

I really don’t like the trend of filesystem authors to only care about filesystem integrity by default. How about having seat belt for your data integrity by default and let people turn it off if they want to compromise correctness for performance?

What I didn’t know is that ReFS integrity is not on by default. Only metadata integrity.

It’s also not visible or changeable in the UI (which is why I assumed they’d done the right thing), which is strange to me, this being Windows. No, you have to drop down into Microsoft’s crappy CLI.

How to check if it’s turned on.

Check files in one directory

PS E:\> Get-Item '*' | Get-FileIntegrity

FileName                       Enabled Enforced
--------                       ------- --------
E:\SomeDirectory               False   True
E:\SomeOtherDirectory          False   True
E:\SomeFile.txt                False   True
[...]

Fuck you, Microsoft.

Check recursively

Of course Get-Item doesn’t do recursion. Why would it? That would make sense.

PS E:\> Get-ChildItem -Recurse 'E:\SomeDirectory' | Get-FileIntegrity

FileName                       Enabled Enforced
--------                       ------- --------
E:\SomeDirectory\foo.txt       False   True
[...]

How to enable it

Both commands are needed. The first command sets the new default for the root directory, and the second adds checksums to all existing files and directories.

PS E:\> Get-Item 'E:\' | Set-FileIntegrity -Enable $True
PS E:\> Get-ChildItem -Recurse 'E:\' | Set-FileIntegrity -Enable $True

This will show a lame ASCII progress bar while it’s doing it. I say lame because this is 2017 and Microsoft managed to create PowerShell without 1970’s technology like SIGWINCH or equivalent for actually detecting a window resize. Not just “after the command was started”, mind you, but also if the window changed size before starting the command.

Oh, and run this as Administrator, because Microsoft will not only need that for some files, it’ll actually spit out error messages that do not contain the filename in question.

This is Microsoft’s “new and awesome” CLI, and it doesn’t do what CLI’s have done since the 70’s.

Also this can’t set integrity checking on files marked read-only. Why? Because Microsoft hates you, your data, and your cat.

So how do I trigger a scrub, a check of all checksums?

Ha ha ha, you can’t. Because Microsoft is retarded. They’re going with the bullshit “Oh you don’t need to!”, completely ignoring that what I want to find out is if my physical disks are failing, or have corrupted data.

I guess I could tar up the whole filesystem and send the output to the bitbucket. But oh wait… tar is not included in Windows so I need third party tools.

Because of the Infineon Disaster of 2017 lots of TPM and Yubikey keys have to be regenerated.

I have previously blogged about how to create these keys inside the yubikey, so here’s just the short version of how to redo it by generating the key in software and importing it into the yubikey.

When it appears to stall, that’s when it’s waiting for a touch.

openssl genrsa -out key.pem 2048
openssl rsa -in key.pem -outform PEM -pubout -out public.pem
yubico-piv-tool -s 9a -a import-key  --touch-policy=always -i key.pem
yubico-piv-tool -a verify-pin -a selfsign-certificate -s 9a -S '/CN=my SSH key/' -i public.pem -o cert.pem
yubico-piv-tool -a import-certificate -s 9a -i cert.pem
rm key.pem public.pem cert.pem
ssh-keygen -D /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so -e

Delete all mentions of previous key. It’s good to have a disaster plan ahead of time if keys need to be replaced, but if you don’t have one:

1. Inventory all bad keys. Make sure you have their fingerprints.
2. Inventory all places this key could be installed.
3. Generate new keys.
4. Distribute new keys. (in this case, add to all relevant ~/.ssh/authorized_keys)
5. Remove all old keys.
6. Grep for the keys found in step 1.
7. Try to log in with old key.

You could do 4 and 5 in one go, replacing key XXXXX with YYYYY (pick something large enough from the key to be unique) with something like:

OLD=XXXXXX
NEW=YYYYYY
NEW_URL=https://www.example.com/ssh-key.pub
for host in $(cat hosts); do
  echo -------------
  echo $host
  ssh $host "
set -e
cd .ssh
(grep -q $OLD authorized_keys && echo FIXING: old key there || echo OK: old key not there)
sed -i '/$OLD/d' authorized_keys
(grep -q $NEW authorized_keys && echo OK: new key already there || curl -s $NEW_URL >> authorized_keys)
" || echo FAILED: could not log in
done

Be prepared to touch the yubikey a lot.

PS

I made a tool to check if your TPM chip is bad. Well, it extracts the SRK public key and checks if it’s good or bad. If the SRK is bad then you can bet all other keys are bad too.

This will also detect the case where the firmware has been fixed, but you have not yet regenerated the key hierarchy on the TPM.

If the SRK is weak then not only are very likely all others keys you generated in the TPM weak, but also anything generated outside the TPM and imported is crackable, since your blobs are encrypted using this crackable SRK key.

In other words: After upgrading firmware you need to re-take ownership of the TPM, which will regenerate the SRK.

Example use:

$ g++ -o check-srk -std=gnu++11 check-srk.cc -ltspi -lssl -lcrypto 2>&1 && ./check-srk
Size: 2048
Outputting modulus…
8490234823904890234823904823904890238490238490238490238490[…]893428490823904231
--------------
THE KEY IS WEAK!

(use -s if you have an SRK PIN)

Thanks to marcan for a much better checking script than the one provided by original authors.

For more info about the Infineon disaster see this relevant paper.

In this post I’ll describe some experiences I had with the Internet in China, and what it means for people making websites in the west in order to reach expats, visitors, and anyone else in China. So this should be useful information even if you don’t care about China as a market at all.

This blog post may be updated, as I have more thoughts on Internet in China.

My subjective experience is that “Internet in China” is an oxymoron. How exactly is there “Internet” without Google, Facebook, and Twitter? When attaching an Android phone to a WiFi in China it even says “Wi-Fi has no Internet access”.

OK, that’s not entirely serious. Especially since I’m obviously not aware of what the Chinese language Internet looks like, not speaking or reading Chinese. Baidu looks like it largely provides the services Google does (search, maps, …), but they’re pretty much not translated. The Baidu Map app seems fine, but is almost useless if you don’t speak Chinese. The one thing it’s good for is that unlike Google Maps (if you can even get to it. see below) it actually shows you a correct location within China.

But more importantly it’s not just Google, YouTube, Facebook, Twitter, Bloomberg, New York Times, and all the other expected sites that are blocked. There’s also ALL the websites that assume that Google works from where you are. And that’s a lot.

Two out of the tree Swedish newspapers I tried did not work. DN and SvD are not blocked, but because they use Google resources they don’t actually work. They start to load, but then you only see a white blank page. Aftonbladet did work.

Picsearch (a previous employer of mine) also doesn’t load. Because I went to China I didn’t bring any access tokens or laptops, so I couldn’t dig too deep to verify for sure, but I’m pretty sure this is to blame:

<script type="text/javascript" src="//ajax.googleapis.com/ajax/libs/jquery/1.4/jquery.min.js"></script>

The fix here should be quite simple: Just host jQuery yourself. It’s 78kB (before gzip). Also the same for external fonts, css, and images. Yes, I realise I’m saying “don’t use CDNs”, but on the other hand it seemed that Cloudflare wasn’t blocked at all, so if you’re worried about resource use and latency then host your CDNable material on a Cloudflare-fronted subdomain.

Really I should also point out that there’s no reason to leave out the schema in this case. Just hardcode https:// there.

Recommendation 1: Don’t use CDNs

The more services you rely on, the higher the likelyhood that it’s blocked. Maybe one day your friendly CDN will draw the ire of the Chinese government, or they choose to redirect to a Google-hosted copy instead, which will fail in China because Google is blocked.

If all your resources are under one FQDN, then there’s less likelihood you you being blocked. With two domains you double your risk.

Recommendation 2: Actually test your site with Google/Facebook blocked

The list of networks here is probably not complete. I gathered it simply by resolving some domains, then doing whois on the address to get the whole range.

ipset create chinav4 nethash
for addr in \
  64.18.0.0/20 \
  64.233.160.0/19 \
  66.102.0.0/20 \
  66.249.64.0/19 \
  72.14.192.0/18 \
  74.125.0.0/16 \
  104.132.0.0/14 \
  108.177.0.0/17 \
  172.217.0.0/16 \
  207.126.144.0/20 \
  209.85.128.0/17 \
  216.58.192.0/19 \
  216.239.32.0/19 \
;do
  ipset add chinav4 "$addr"
done

ipset create chinav6 nethash family inet6
for addr in \
  2a00:1450::/29 \
  2a03:2880::/29 \
  2607:F8B0::/32 \
  2404:6800::/32 \
;do
  ipset add chinav6 "$addr"
done

iptables  -I INPUT -m set --match-set chinav4 src -j DROP
ip6tables -I INPUT -m set --match-set chinav6 src -j DROP

Does your site still load? If it doesn’t load fully, is it still at least usable?

(I’m pretty sure the list above is incomplete, because dn.se still works after doing this, but picsearch.com does not)

Recommendation 3: When visiting China, use roaming data with your home Telco

In other words: Get a data plan that’s not ridiculously expensive. My data plan vith Vodaphone UK charges me at £3 per megabyte. Yes, per megabyte. That would fall under “ridiculously expensive”.

The easiest way to get non-censored Internet, with working Google services (including Google maps and GMail) is to use roaming data from a western country.

The reason it’s not blocked is that roaming mobile data actually tunnels back to your home country, and it’s there that you connect to The Internet. China could do deep packet inspection on this tunnel, but they don’t.

You’ll have a very high latency to everywhere, since all packets have to go across the world first, but it’ll work. You could try getting a roaming mobile data plan in Hong Kong or a nearby country, which should help.

The recent bug in WPA2 has a worst case outcome that is the same as using a wifi without a password: People can sniff, maybe inject… it’s not great but you connect to open wifi at Starbucks anyway, and you’re fine with that because you visit sites with HTTPS and SSH. Eventually your client will get a fix too, so the whole thing is pretty “meh”.

But there’s a reason I call it “WPA2 bug” and I call the recent issue with Infineon key generation “the Infineon disaster”. It’s much bigger. It seems like the whole of Estonia needs to re-issue ID cards, and several years worth of PC-, smartcard-, Yubikey, and other production have been generating bad keys. And these keys will stick around.

From now until forever when you generate, use, or accept RSA keys you have to check for these weak keys. I assume OpenSSH will if it hasn’t already.

But then what? It’s not like servers can just reject these keys, or it’ll lock people out. And it’s not clear that an adversary even has your public key for SSH. And you can’t crack the key if you don’t have the public half. Maybe a warning, and then in a year start rejecting the keys?

And then you have to trust that every other implementation does the same.

But then you have all the clients and servers that just never get updated or audited…

So this is a disaster. It’s worse than the Debian randomness bug.

Previous blog posts on this issue

• Is my TPM affected by the Infineon disaster
• Yubikey for SSH after the Infineon disaster

For every day that passes I like bitcoin less. It’s bad for the world.

I have ranted about this many times, and it’s time I consolidate these rants into a blog post.

We’ll see with time if this rant ages poorly or not.

Section 1: Practicalities

What bitcoin is trying to achieve in payments

These would be good things:

• Anonymity
• Peer to peer and remote payments

What bitcoin is actually good for

Bitcoin today is pretty much only good for two things:

• Committing crimes
• Speculating on currency

If you’re not doing one of those, then don’t use Bitcoin.

If you are an online store then sure, accept Bitcoin. There’s moral problems with supporting Bitcoin (see rest of post), but as long as you immediately convert to fiat currency when you receive payment it’s fine for you. There are even companies out there that’ll guarantee an exchange rate so that you never have to get into the business of currency.

You may say that Bitcoin can be used to get money out of China, or into Brazil, or to enable shopping of “things that should not be called ‘drugs’ anyway”, but you have to admit that no matter what you think of laws, that’s just other words for “committing crimes”.

It does peer-to-peer and online payments, you left that out!

Is that really a hard problem?

• Paypal
• GMail Payments
• WeChat P2P
• Facebook is introducing it
• Wywallet
• Venmo (Owned by Paypal)
• Alipay
• Paytm
• Paym
• TWiNT
• Revolut

But wait, it’s still anonymous!

Is it though? In order to pay anonymously with bitcoin you need to jump through hoops and be improbably competent. You need to tumble your money very carefully, through multiple parties (how do you know they’re not fronts for the same party?), who you trust are not compromised.

… who you trust are not compromised.

If you’re the NSA/GCHQ/etc… and you haven’t hacked most bitcoin exchanges and tumblers then you’re incompetent, and essentially asleep at the wheel. They hack SIM card manufacturers and SWIFT, so of course they would hack these amateur tumbler and exchange websites.

… and be improbably competent

Attacks only get better. They never get worse. There are people today who claim to be able to de-anonymize bitcoin.

Can they? Maybe. But the transaction log is public. Do you think in 30 years it’ll be impossible? Do you think that your bitcoin tumbling strategy today will be impossible to decode after 30 years of research and statistics?

For those familiar with cryptography: Your use of Bitcoin anonymously is like you making your own security protocol, and thinking it’s secure because you base it on AES. But solid primitives don’t automatically make for a solid protocol.

There are vast opportunities to screw up a security protocol even with good primitives, and new ways are found every day. E.g. encrypt-then-sign, or sign-then-encrypt? If you’re not actually a cryptographer you shouldn’t put yourself into a position to answer that.

In 10-30 years when there’s a new paper showing how to de-anonymize your tumbling-strategy of today, do you want your weed purchases to be visible for the whole world to see? The government is unlikely to care (they likely already had the info anyway, and it’s beyond the statute of limitations), but now anyone with a grudge can make a “thing” out of it, socially or legally.

And do you truly believe that the people who tumble your cash aren’t secretly recording the in’s and out’s? If someone’s already immoral enough to launder money for anyone, what makes you think they’ll be loyal to you? That in/out mapping over a few years will be worth a lot.

Are you that competent? Is anyone? Seems improbable to me.

Section 2: Let’s talk about an anonymous global currency

If the end goal of Bitcoin is a currency as common as any other, legal and practical for goods and services on a large scale, then we need to answer some questions about what society that is.

Let’s assume that all technical hurdles of anonymity and scale are actually solved, and we have a perfectly anonymous currency, and compare it to digital non-anonymous currency like today’s “banks” and “credit cards”.

How would tax work?

Are you against taxes existing at all? All of it? Income tax, capital gains, VAT, duties, etc…

Sure, you can be an ultra-liberal to the point of being almost indistinguishable from an anarchist, but you and I both know that most people are not, and they won’t be.

Most people want and will want a government, and in the west even democracy.

If there’s no way for the government to check, then there’s no way to get caught. Why not pay everyone under the table? Today stores and taxis (bloody black cabs) dealing with cash have the opportunity to pocket money tax free. But there’s at least some work to keep that quiet. And it’s not always easy to spend the money either. It’s hard to pay rent, buy a car, a vacation, and all other things using untaxed money.

And if you get audited today it’ll look suspicious if 100% of your legal money goes towards rent, since the tax man knows that you need to eat too.

When paid in bitcoin both employer and employee have perfect deniability about how much is being paid. Bonuses can be simply transferred in bitcoin.

So in the end rich people will pay zero tax, and poor people much less than today. All untraceable. If there’s a social safety net for people who can’t afford food then everyone will be entitled to it.

Death

If I don’t tell you my password before I die then that money disappears.

If I do tell you my password before I die then it could disappear before I die.

What’s Bitcoin’s plan for this? Just suck it up as inflation? What if someone or something worth 10% of the currency dies? Which brings me to…

Money consolidation

Do you want a society where one family can collect 50% of the world’s money? To want a digital anonymous world you not only say “yes”, but you also want to say “yes, and future generations should not be allowed to know if this happens”.

Today we can at least investigate how much money rich people have. They hide it in complexity, but not with unbreakable math.

Individuals are not competent in every field humanity does

When was the last time you tried to get cash and the ATM said “sorry, we forgot the password to your account, so your money is gone”?

With Bitcoin people will have (and have had) hard drive crashes that have destroyed fortunes.

A society like this makes no sense.

Bitcoin exchanges are not regulated

… or at least not as regulated as banks.

Every idiot with a Cloud account can just become the bank and steal all the money.

Or claim that someone else broke in and stole all the money.

Or turn your bank into a Ponzi scheme after someone steals all the money.

• Mt. Gox. In one breach 6% of all Bitcoin in circulation was stolen (4% of all Bitcoin that will ever exist). Imagine if 6% of all cash were stolen. And realize that some of that bitcoin that you use is stolen property.
• Phaser Inc
• Nicehash
• Many many more

With game money you and the robber can just shrug and say “this is part of the game”, but when it’s about your non-disposable real money it’s not so funny anymore.

Fraud

Fraud (e.g. credit card fraud) is often solved by reversing the charge. This not only returns the money to the victim but also puts the incentives in the right place. Consumers don’t have the power to force chip-and-pin, so there would be no point in placing the incentives with them.

Thought it was annoying to get a virus before? Now it’ll take your bank account with no possibilty to fix it. You can’t really insure against it either, because another crime that’d be trivial and untraceable would be insurance fraud.

Fraud handling works much better “for the little guy” today than with Bitcoin. With Bitcoin fraud is not fixable.

Bank hacking

Banks get hacked, or sometimes just make mistakes in big transfers. How do you fix it? Yes, you reverse the transaction. What if the recipient has withdrawn all of it as cash? You send the police after them. And people know this, so they almost invariably don’t.

“If you are bank then don’t make a mistake” is obviously not the answer.

Money laundering

Are you simply OK with money laundering as a thing everyone should do? I’m not.

“Follow the money” for crime investigations

Banks

Bob is murdered. Alice transferred $100k to known criminal Dave 7 days prior. It’s not conclusive proof in itself, but you know where to start this investigation, don’t you?

Cash

Alice withdrew $100k 7 days prior, has been seen handing a bag to Dave, and Dave now needs to carefully and competently launder that without making a mistake.

Bitcoin

Bob is murdered. The End.

Without bitcoin ransomware would not exist

To participate in bitcoin is to enable ransomware. Yes, you can shrug and say “guns don’t kill people, people kill people”, but you have to admit that this is starting to add up. You have to shrug at a lot of suffering in the world in order to support bitcoin. People literally die who would not die if cryptocurrency didn’t exist.

How much shrugging is too much for your abstract notions of decentralized currency? (and it looks like it’s not even all that decentralized anyway, but this section assumes a perfect cryptocurrency)

Section 3: But it’s just like cash and that’s fine!

Bitcoin is really really not like cash. For small transactions, yes. But more of the same is not the same.

One example: Kamikaze-robbery

What happens if a huge pile of cash is left unguarded for 2 seconds? It might get stolen. Someone could grab the bag and run. They may even be armed. But the robbers can only grab as much as they can carry, and they have to keep it safe. The police will run after them, and often the money is recovered.

What happens when someone goes into a bitcoin exchange armed to the teeth, demanding all bitcoins in the exchange or the hostage will die?

What happens when a rogue employee of a company holding bitcoin (e.g. an exchange) realizes that due to a security mistake they have access to $1B worth of bitcoin, and they take it?

Well… they’ll probably get caught. Let’s assume the government exists, or at least the police does.

Now what? The money is gone. The robber didn’t even need an exit strategy, and can just surrender. Their family is set for life, and only one person went to jail, and maybe only for a few years. When they get out they’re rich.

ISIS could do this today. They could even go in, steal a billion dollars, and then blow themselves up.

You could literally kill a bank and take all its money.

Somewhat relevant.

HSBC got a slap on the wrist for laundering terrorist money, but at least we know about it

Yeah that happened.

If the world were Bitcoin then HSBC would not be needed. Bitcoin is dishonest money laundering because “money laundering” is just the flip side of honest “anonymity”.

Section 3: The electricity

Some of the electricity aspects are in theory possible to fix, I’ll admit. But the current state is absolutely horrible.

The numbers

As of 2017-11-10 Bitcoin consumes about 1.7GW of power. That’s about 1.5-4 nuclear reactor’s worth of output (depending on reactor size) at a cost of about 1.3 BILLION dollars annually.

All this to support 7 transactions per second.

Bitcoin is built on burglary and theft

As of approximately now-ish (depends on BTC value of the day) it’s not economically feasible to pay for electricity to run Bitcoin. So why does Bitcoin run?

Because it’s run on stolen power. Hacked machines, fraud, abuse, and malware runs bitcoin.

You don’t need to go to Silk Road to be part of the criminal parts of Bitcoin; Bitcoin itself runs on crime.

Electricity cost floor

This is another major reason I strongly dislike Bitcoin. As long as participating in mining or transaction validation is compensated this sets a floor on the cost of electricity.

Imagine I invented a device that lowered the cost of electricity production to 10% of what it is today. How would that affect the price of energy for consumers, to save money for consumers, resources for the world, and enable new uses of energy previously not economical?

In a Bitcoin world: Not at all. The price was already at the level where it was rational for someone to pay that in order to make Bitcoin. If the price drops, then that pushes up demand until supply runs low causing the price to be back to exactly where it started.

Placing a floor on the price of energy is just about the most evil thing that can be done to civilization, the world economy, and everything affected by it.

I’ve blogged before about how the new integrity-checking filesystem in Windows, ReFS, doesn’t actually have integrity checking turned on by default. It’s pretty silly that for a modern filesystem meant to compete with ZFS and BtrFS, to have the main 21st century feature turned off by default. But it’s not quite ridiculous. Not yet.

Now it turns out that scrubbing is only supported on Windows Server 2016. Microsoft honestly shipped an integrity-checking filesystem in Windows 10 with no way to repair or scrub it.

I used to say that Windows 10 is the best Windows ever, and that Microsoft kinda won my trust back. But what the hell?

I contacted Microsoft support over chat, who first suggested I do a system restore (sigh). But after I insisted that they please confirm that it’s supposed to work confirmed that no that only ships with Windows Server.

It’s not even clear from their pricing if I need the $882 Standard Edition or the $6,155 Datacenter Edition. Either one is way too much for such a standard feature.

What the hell, Microsoft? All I want is a checksumming file system. Either provide it, or don’t. Don’t give me a checksumming filesystem that can’t be repaired or verified.

It may even be worse. Rumors seem to be that Microsoft turned off this feature in the 1709 release. How’s that for a bait-and-switch?

It’s cheaper to buy a whole new server just to run ZFS or BtrFS on it, and use it from Windows, that it is to actually store files reliably on the Windows machine itself. Absolutely ridiculous.

For the record, the way you would run a scrub, if you could, is:

Start-ScheduledTask -TaskPath "\Microsoft\Windows\Data Integrity Scan\" -TaskName "Data Integrity Scan"

Repair one file with:

Repair-FileIntegrity -FileName 'E:\somefile.png'

Or repair a whole tree with:

Get-ChildItem -Path 'E:\' -Recurse | Repair-FileIntegrity

Short post this time.

If you on Linux set up your Yubikey in smartcard mode then you can use that Yubikey without any setup at all on Windows.

Just open PuttyWincrypt, put in the host to log in to, and under Connection > SSH > Auth set Private key file for authentication to cert://*, then click Open.

It’ll ask for the PIN, you’ll have to touch the Yubikey when it’s blinking, and you’re in.

My interest in SDR got me into Amateur Radio. One reason was that so that I could transmit on non-ISM bands and with more power. Turns out the 2.3GHz band available to Amateur Radio operators is much quieter than the 2.4GHz band where WiFi, bluetooth, microwave ovens, drones, cordless phones and everything else lives. Shocker, I know.

Amateur radio doesn’t just have voice and morse code, there’s also digital modes.

A popular mode is FT8. It’s only used to exchange signal reports, so there’s no chatting. It’s in fact often practically unattended.

It’s a good way to check the quality of your radio setup, and the radio propagation properties that depend on how grumpy the ionosphere is at the moment.

If you transmit, even if you nobody replies, you’ll be able to see on PSKReporter who heard you, which is useful.

Because propagation should be pretty much symmetric, receiving a strong signal should mean that two-way communication is possible with the station. Though FT8 is a slow mode that will get through where others won’t, so a weak FT8 signal means that any voice communication is very unlikely to get through.

Unfortunately unlike WSPR the standard FT8 protocol doesn’t include transitter power. So you don’t know if the reason you can hear a remote station is that they’ve cranked up way beyond QRP. The only way to correlate transmitter power with signal report is if you transmit (and thus know your transmitted power) and someone replies with a signal report.

I set up a listening station with a Raspberry Pi 3, a SignaLink USB, and an Elecraft KX2. I tried using a Raspberry Pi Zero W, but connecting two USB devices plus a USB hub caused spontaneous reboots due to power drain. I could use a USB hub with its own power source, but the Zero CPU is so slow that you have to set it to do a Fast Decode, meaning it’ll be less sensitive to weak signals. Also I already had a Raspberry Pi 3 doing other things.

The antenna is an AlexLoopmagnetic loop antenna, so it requires tuning to the frequency in use. That’s no problem since I’m not tuning around.

The antenna is just leaning against a window inside a flat, so it’s amazing how well it works. It even heard a few stations in South America!

Steps

1. Dial an FT8 frequency. In my case 14.074MHz.
2. Tune the antenna either by ear or with the help of this tool
3. Start wsjtx, and set it to report to 127.0.0.1:2237
4. Build and run wxlog, logging to disk.
5. Create fancy output like this map using the mapper tool

Random Notes

• “[ISM] applications (of radio frequency energy): Operation of equipment or appliances designed to generate and use locally radio frequency energy for industrial, scientific, medical, domestic or similar purposes” — ITU Radio Regulations Article 1.15. (my emphasis)
• VOACAP propagation prediction

The Go standard library is generally great, but some parts have replacements that are just plain better and remove frustrations that you may have not even realised were frustrations. Here are my recommendations for every Go program.

I wouldn’t recommend that anyone use the standard library version of these for any purpose, since better alternatives exist.

This list may expand in the future.

gorilla/mux

The standard router is fine, but very low level. Here’s some of the features that makes it vital.

Filter on HTTP method

With the standard router you have to manually check that the method is what you expect it to be, and if the same endpoint has both GET and POST then you have to route that yourself. With gorilla/mux it’s as simple as:

r := mux.NewRouter()
get := r.Method("GET").Subrouter()
post := r.Method("POST").Subrouter()
get.HandleFunc("/", handleRoot)
get.HandleFunc("/items", handleListItems)
post.HandleFunc("/items", handleUploadItem)

You can also assert that headers are in place, for example to check X-Requested-With because some API endpoints should not be allowed in cross-domain XHR requests. Adding it to the router instead of manual checks simplifies code and reduces risk of forgetting to add the check.

Pattern URLs

With the standard router you have to set up a prefix handler and parse the URL yourself… and have your “/” handler handle all 404s that it’ll get.

get.HandleFunc("/items/{item_id:[0-9-]+}", handleGetItem)
[...]
func handleGetItem(w http.ResponseWriter, r *http.Request) {
  itemID := mux.Vars(r)["item_id"]
}

sirupsen/logrus

Unstructured logging is frustrating. Merely searching for all log entries in a given time range is a whole project because timestamps need to be parsed, daylight savings taken into account, and then the rest of the line needs to be parsed to extract severity and message.

And that doesn’t even enable multi-line logs, labels and other structured data.

func handleGetItem(w http.ResponseWriter, r *http.Request) {
  log := logrus.WithFields(logrus.Fields{
        "client_address": r.RemoteAddr,
      },
  )
  itemID := mux.Vars(r)["item_id"]

  if secretItem(itemID) {
    log.Warningf("Attempt to access secret item %q", itemID)
    code := http.StatusForbidden
    http.Error(w, http.StatusText(code), code)
    return
  }
}

If outputting in JSON format that becomes:

time="2018-09-15T13:34:18+01:00" level=warning msg="Attempt to access secret item \"42\"" client_address=192.0.2.123

You can even add the item ID as another field, incrementally, with:

  log = log.WithField("item_id", itemID)

To have these fields traverse code stacks I’d probably have my extra label data attached to the context, so that I don’t have to pass down both context and log objects.

errors

This is not so much a replacement for a package as filling a need that plain errors have.

The biggest problem standard Go errors have is that they are pretty much untyped. Yes, they have a type, but then they get annotated as they traverse up the stack so that in the end they’re more like freeform strings. Once your os.PathError is wrapped via standard means (return fmt.Errorf("failed to read config: %v", err)) the type is obliterated.

func readSomething() error {
  _, err := ioutil.ReadAll(f)
  if err != nil {
    return errors.Wrap(err, "reading the thing failed")
  }
  return nil
}

type stackTracer interface {
    StackTrace() errors.StackTrace
}

func main() {
  if err := readSomething(); err != nil {
    log := logrus.WithFields(log.Fields{
      "error": err.Error(),
      "cause": errors.Cause(err),
    })
    if e, ok := err.(stackTracer); ok {
      log = log.WithField("stack", e.StackTrace())
    }
    if e, ok := errors.Cause(err).(*os.PathError); ok {
      log = log.WithFields(log.Fields{
        "os.PathError.Op": e.Op,
        "os.PathError.Path": e.Path,
        "os.PathError.Err": e.Err,
      })
    }
    log.Fatalf("Failed to read something")
  }
}

Now the original error (or as close to it as possible) can be found with errors.Cause(err). err.String() is still the most human-readable full error, but by wrapping errors you don’t have to choose between annotating, and keeping the error type and exact value. And also not resort to parsing error strings.

JT65 is a slow protocol for propagation reports. In short it takes 60 seconds to send 13 characters. Then you wait 60 seconds for a reply, and repeat.

The 60 seconds are actually 1 second silence, 46.872 second of signal, then another 12.128 seconds of silence, allowing for clock drifts and for a human to choose the reply.

The mode is this slow in order to add a lot of redundancy and to make it easier for the receiver to dig out a signal way below the noise floor. It was originally meant for making contacts by bouncing signals off the moon, which has a path loss of ~250dB. Someone even managed a JT65 moonbounce on 10 Watts using JT65 with gear you and I could buy/build!. That’s the power of a low energy light bulb!

I wanted to do propagation experiments with SDR, with low power in various frequency bands, but couldn’t find a GNURadio module for JT65. So I made one.

The JT65 specification is very well written, except for the parts it says “the code is the specification”. Which would normally be fine, but the code is in Fortran and Fortran is terrible.

JT65A is under 200Hz wide, which means that with an SDR you can put many many channels in parallel. You could run Codec2 at the highest quality in 500kHz, if you don’t mind the 60 second delay. Sure, half a MHz is a lot, but the signal can be picked up way below the noise floor with a good receiver. (my receiver is not yet good)

And with everything in GNURadio you can speed up transmissions arbitrarily, or more accurately as far as SNR will allow. This is also why simply taking the output of WSJT-X as audio was sufficient. I want to be able to play with all the parameters fully.

Successful decode of JT65C sent from BladeRF x40 to USRP B200 on 2.3GHz:

Links

• JT65 on Signal Identification Guide
• JT65 specification, retrieved from ARRL on 2018-07-01.
• WSJT on Wikipedia

What is the point of amateur radio? To learn about radio, propagation, and the electromagnetic spectrum in general. To understand how it works, and maybe even build or modify your own equipment. The license, after all, is the only legal way to use the electromagnetic spectrum at interesting power levels.

In order to learn we must be able to inspect; To tinker, or at the very least have access to a specification we can build from.

Some amateur radio operators seem to complain that people don’t build their own radios anymore. That they just buy a box and antenna and are now consumers. This is not what I’m talking about here. First, you know in principle how your radio works. And you could build one that could replace it. Would it be as good as a modern fancy rig? Of course not. It wouldn’t be as good, but you could build one, and you could use it just as well as the bought one.

And if you learn enough, and tweak enough with the rig and antenna system, you could build something better for your particular environment.

When I first learned that D-Star used a proprietary voice codec I couldn’t understand the point of D-Star at all. Some black box you just buy and are a passive consumer? What’s the point? Why even bother with a hobby of sharing and learning, if there’s a big wall saying “this far, no further!”, that you are not permitted to create your own radio for. Or permitted to understand.

If it’s just “buy this consumer device and use it, but don’t learn” then why am I not just using skype, or whatsapp, if it’s just about making free phone calls?

I would like to learn and create in all aspects of radio. One of these is Software Defined Radio (SDR), which this proprietary technology prevents me from using to explore D-Star in a meaningful way.

I’m hopeful that this will change, that proprietary technologies like the D-Star voice codec go away, now that SDR is starting to be introduced to amateur radio tests. I hope that’ll bring the digital knowledge and culture of open source together with the radio knowledge of amateur radio, to create the best of both worlds. As-is D-Star voice codec is a dead end. It can’t lead to new things like JT65 led to FT8, and FT8 to JS8Call, or other inventions.

It’s just there, static, opaque, and for the main purpose of amateur radio: useless.

If SDR on amateur radio tests brings open source people to amateur radio, then maybe it can bring openness of technology to the amateur radio community. (yes, it’s weird to me that openness of technology needs to be brought to, of all things, the xamateur radio community)

Some call codec openness “ideological purity of dubious usefulness”. In my opinion the closed codec is what makes D-Star of dubious usefulness. It’s a dead end.

Then there’s the registration system. If you ever had to start using D-Star and didn’t live near a D-Star enabled repeater then you know how annying this could be. In theory you should just be able to register, but in practice registrations not at “home” seem to be ignored or denied.

It seems like the AMBE patent has expired now, but until there’s an implementation of codec (not just decoder) that’s not all that helpful.

I have hopes for Codec2 and FreeDV, but there’s a whole legacy proprietary repeater and reflector system that needs to be dismantled, not to mention all the handsets that have to be exchanged since none of them currently support FreeDV, and likely never will get a firmware update to support them.

What amateur radio really needs is open source digital radios. And it shouldn’t be that unrealistic. Surely it’s not that expensive or hard if only opensource people can combine their skills and let affordable SDRs be the unifying banner under which a new better amateur radio world is built.

I’m doing my part, learning enough of the hardware parts to combine a Raspberry Pi, a USB SDR, a RF Power amplifier, and some other parts to create an open and more importantly debuggable and software-extendable radio.

TCP_MD5 (RFC 2385) is something that doesn’t come up often. There’s a couple fo reasons for that, good and bad.

I used it with tlssh, but this should explain why I didn’t enable it by default.

What it is

In short it’s a TCP option that adds an MD5-based signature to every TCP packet. It signs the source and destination IP address, and the ports, and the payload. That way the data is both authenticated and integrity protected.

When an endpoint enables TCP MD5, all unsigned packets (including SYN packets) are silently dropped. For a signed connection it’s not even possible for an eavesdropper to reset the connection, since the RST would need to be signed.

It’s used by the BGP protocol to set a password on the connection, instead of sending the password in the handshake. If the password doesn’t match the TCP connection doesn’t even establish.

But outside of BGP it’s essentially not used, which is a shame. If we could enable it for any TCP service it’d add a preshared key and completely replace the silly port knocking. It probably couldn’t replace user passwords, but it could add a layer and greatly reduce attack surface much more than, say, a TLS certificate.

The bad reason it’s not used

It’s MD5. Sure, MD5 still doesn’t have any preimage attack. Well, none that’s feasible anyway.

So that should be fine. And if not then there’s already TCP AO which is about the same but with other algorithms.

The more real reasons

1. It’s impossible to use on Linux for most applications

I think this is the biggest reason it’s not used for playing around. This is the Linux API:

const char* password = "hello";
struct tcp_md5sig sig;
memset(&sig, 0, sizeof(sig));
memcpy(&sig.tcpm_addr, peer_sockaddr, sizeof(struct sockaddr_storage));
sig.tcpm_keylen = std::min(TCP_MD5SIG_MAXKEYLEN, strlen(password));
memcpy(sig.tcpm_key, Sflag_password, sig.tcpm_keylen);
if (setsockopt(s, IPPROTO_TCP, TCP_MD5SIG, &sig, sizeof(sig)) == -1) {
  fprintf(stderr, "Failed to setsockopt(): %s\n", strerror(errno));
  exit(1);
}

See the problem? Yeah. You need to know the address and port of the remote end. So it’s fine for clients, but not so much for servers. BGP doesn’t really have clients and servers, so it works fine there.

You can actually make this work, and it’s what I did with tlssh: You enable TCP MD5 immediately after the connection is established. If the two ends don’t have the same password set, then no packets will go through in either direction, and the connections will just time out. Not the best experience.

On OpenBSD it’s much easier. You set up the TCP MD5 as a security association between the two hosts, and then the program just enables MD5 on the socket. And it “just works” on both the client and the server. Well, a bit annoying that the key is set in hex.

# cat > /etc/tcpmd5.conf
tcpmd5 from 2001:db8::1111 to 2001:db8::2222 spi 0x100 authkey 0x68656c6c6f
tcpmd5 from 2001:db8::2222 to 2001:db8::1111 spi 0x101 authkey 0x68656c6c6f
^D
# ipsecctl -f /etc/tcpmd5.conf

const int on = 1;
if (setsockopt(s, IPPROTO_TCP, TCP_MD5SIG, &on, sizeof(on)) == -1) {
  fprintf(stderr, "Failed to setsockopt(): %s\n", strerror(errno));
  exit(1);
}

Or with netcat just use the -S option (but the password still needs to be set in the config).

But yes even on OpenBSD it doesn’t really mesh well with adding TCP MD5 to any and all programs. Even if you set up the Security Association inside your program instead of in your config it seems more a state of the system than a state of your connection. Which makes some sense with BGP, but almost nothing else.

2. It doesn’t work through NAT

Because it signs the source and destination address. This is getting to be less and less of an issue as the world goes IPv6.

Again, was never really a problem for BGP.

My wish

If Linux could have an API for enabling TCP MD5 on the server side for “any” peer address, that’d be great.