Mitigating Poisoned PGP Certificates (CVE-2019-13050)

This article will describe PGP Certificate Flooding attacks as well as inform the reader

1. How to detect if you have a poisoned certificate in your keyring,
2. How to identify & clean the poisoned cert, and
3. How to update the configuration to prevent it from importing poisoned certs in the future

Last month, an attacker spammed several high-profile PGP certificates with tens of thousands (or hundreds of thousands) of signatures (CVE-2019-13050) and uploaded these signatures to the SKS keyservers.

Without looking very deep, I quickly stumbled on 4 keys that were attacked last month:

The Problem

If your GnuPG client or thunderbird's enigmail is configured to automatically `--refresh-keys`, then you could also automatically break your GnuPG install if it downloads one of these poisoned certificates into your keyring.

Note that this problem is further exacerbated by the fact that keyservers by design do not delete keys. Though this introduces obvious vulnerabilities, it was a decision made to make the network resilient to government tampering & censorship.

Personally, I noticed this issue when my email stopped working. Debugging thunderbird & enigmail pointed me to issues with PeP, but--as I was debugging--I noticed that `gpg` was hanging on `removing stale lockfile`. This was because thunderbird was already running, and it was hanging on the poisoned certificate.

user@host:~$ gpg -vvvv --list-keys
gpg: using character set 'utf-8'
gpg: using pgp trust model
...
gpg: checking the trustdb
gpg: removing stale lockfile (created by 7055)

The Solution

This section will describe how to fix your GnuPG keyring and configuration to clean out poisoned public keys and prevent it from automatically importing them in the future.

1. Identifying the poisoned key

First, we can list the size the public keys in our keyring (in bytes) using the following command (as reported on the GnuPG issue tracker):

user@disp1754:~$ gpg --export | gpg --list-packets | awk -F= -v oldoff=-1 -v keyid=unset '
/^# off=/{ off = $2 + 0 }
/^:public key/{ if (oldoff>-1) { print (off - oldoff) " " keyid }; oldoff = off; keyid = "unset"; }
/keyid:/ {if (keyid == "unset") { keyid = $1; } }
END { print (off - oldoff) " " keyid ; };' | sort -n
7284 	keyid: 1DCBDC01B44427C7
119748 	keyid: 4E2C6E8793298290
124557 	keyid: 403C2657CD994F73
16934647 	keyid: F20691179038E5C6
user@disp1754:~$ 

If the above command takes longer than a few seconds to run, then you have a problem. Give it 20 minutes or so, and you'll see problematic keys at the bottom. Anything with 8 digits (>10 MB) is a red flag.

2. Exporting the poisoned key

Now that we've identified the poisoned key, let's export it for safe keeping before we delete it.

user@disp1754:~$ time gpg -a --export 'F20691179038E5C6' > pubkey.asc
  
real	3m30.950s
user	3m24.430s
sys	0m0.322s
user@disp1754:~$ du -sh pubkey.asc 
22M	pubkey.asc
user@disp1754:~$ 

After a few minutes, the command will finish, and you should have a file named `pubkey.asc` with the contents of the poisoned public key in it. Note that this ASCII armored file containing exactly one public key is 22M!

3. Deleting the poisoned key

Now that we have a safe backup of the poisoned key on disk, let's delete it from our keyring.

user@disp1754:~$ time gpg --delete-key 'F20691179038E5C6'
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

pub  ed25519/F20691179038E5C6 2019-01-19 Daniel Kahn Gillmor <dkg@fifthhorseman.net>

Delete this key from the keyring? (y/N) y

real	12m15.265s
user	11m54.242s
sys	0m0.715s
user@disp1754:~$ 

4. Re-importing the cleaned key

To re-import a clean copy of the public key, we'll use the `gpg` argument `--import-filters` to drop all signatures (`drop-sig`) made on the date when the certificate was flooded with signatures.

Given a public key file (like the backup we just exported above), we can list a count of the number of signatures that certificate received for each day with the following command:

user@disp1754:~$ time gpg --list-packets pubkey.asc | grep -i 'sig created ' | sort -n | uniq -c
     11 	hashed subpkt 2 len 4 (sig created 2019-01-19)
      2 	hashed subpkt 2 len 4 (sig created 2019-01-20)
      4 	hashed subpkt 2 len 4 (sig created 2019-01-21)
      2 	hashed subpkt 2 len 4 (sig created 2019-01-28)
  14400 	hashed subpkt 2 len 4 (sig created 2019-06-17)
  40200 	hashed subpkt 2 len 4 (sig created 2019-06-18)

real	0m23.061s
user	0m17.803s
sys	0m0.150s
user@disp1754:~$ 

The above output shows that

1. 14,400 signatures were made to the key on 2019-06-17 and
2. 40,200 signatures were made to the key on 2019-06-18

We can import the key while omitting these spammed signatures on these two days as follows (be sure to replace the dates with the corresponding days printed from the above command on your machine):

user@disp1754:~$ time gpg --import-filter drop-sig="sig_created_d=2019-06-17 || sig_created_d=2019-06-18" --import pubkey.asc
gpg: key F20691179038E5C6: 54614 signatures not checked due to missing keys
gpg: key F20691179038E5C6: public key "Daniel Kahn Gillmor <dkg@fifthhorseman.net>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: no ultimately trusted keys found

real	3m12.091s
user	3m6.991s
sys	0m0.284s
user@disp1754:~$ 

And now things should be much more sane:

user@disp1754:~$ time gpg --export | gpg --list-packets | awk -F= -v oldoff=-1 -v keyid=unset '
> /^# off=/{ off = $2 + 0 }
> /^:public key/{ if (oldoff>-1) { print (off - oldoff) " " keyid }; oldoff = off; keyid = "unset"; }
> /keyid:/ {if (keyid == "unset") { keyid = $1; } }
> END { print (off - oldoff) " " keyid ; };' | sort -n
7284 	keyid: 1DCBDC01B44427C7
8930 	keyid: F20691179038E5C6
119748 	keyid: 4E2C6E8793298290
124557 	keyid: 403C2657CD994F73

real	0m0.063s
user	0m0.059s
sys	0m0.016s
user@disp1754:~$ gpg -a --export '403C2657CD994F73' > pubkey2.asc
user@disp1754:~$ du -sh pubkey2.asc 
168K	pubkey2.asc
user@disp1754:~$ 

5. Updating your GnuPG config

As Robert J. Hansen (whoose pgp key was spammed with 149,100 signatures on 2019-06-19) pointed out in their excellent comprehensive gist about this issue, you can prevent your gpg client from breaking itself by:

1. Removing any lines that start with `keyserver` in your `gpg.conf` file and
2. Updating `dirmngr.conf` so that it has only one keyserver: `keyserver hkps://keys.openpgp.org`

That `keys.openpgp.org` keyserver is a new experimental server (interestingly, it went live just weeks before these poisoned certificates were uploaded) that is more resistant to these attacks. Note that the certificates it serves entirely lack third party signatures, and it also strips the UID packets from the key unless a user explicitly opts-in.

6. Updating your MUA config

Update 2019-07-16: Enigmail changed their default keyserver to `keys.openpgp.org` in v2.0.12, so updating engimail >= this version is preferred to disabling `keyRefershOn` as described below, if possible. (ty Wiktor)

Firefox Advanced Config Editor

You may also need to update your MUA. For example, enigmail in thunderbird may also be configured to update the keys in your keyring.

Disable Enigmail's "keyRefreshOn" setting

To prevent enigmail from refreshing your keys from a keyserver, go to your thunderbird preferences -> Advanced -> Config Editor... -> I accept the risk!

And set `extensions.enigmail.keyRefreshOn` to `false`

Addendum

Note that the keybox keyring format will refuse to import posioned keys as it has a max key size of 5 MiB, and that users with old installs should consider migrating their keyring to keybox format.

This can be done easily in debian-based systems using the migrate-pubring-from-classic-gpg command

Further Reading

1. https://dkg.fifthhorseman.net/blog/openpgp-certificate-flooding.html
2. https://access.redhat.com/articles/4264021
3. https://www.vice.com/en_us/article/8xzj45/someone-is-spamming-and-breaking-a-core-component-of-pgps-ecosystem
4. https://www.zdnet.com/article/openpgp-flooded-with-spam-by-unknown-hackers/
5. https://daniel-lange.com/archives/159-Cleaning-a-broken-GNUpg-gpg-key.html

Michael Altfield

Hi, I’m Michael Altfield. I write articles about opsec, privacy, and devops ➡

About Michael