Decrypting a GPG-encrypted file in Python with raw key

Question

I'm trying to use Python to decrypt a GPG-encrypted file using the raw key. Not the passphrase, not a nicely formatted file from a keyring, just the literal raw bytes of the key that the file was encrypted with.

I first created a test file:

~$ echo "It would be really cool if this worked" >> PGPDecryptorTest1.txt

I then encrypted the file using AES256 with the passphrase "a" and SHA256 key derivation:

~$ gpg --symmetric --s2k-mode 0 --s2k-digest-algo SHA256 --cipher-algo AES256 PGPDecryptorTest1.txt

I wrote the following short script to decode the file:

import sys 
from Crypto.Cipher import AES 

# With s2k-mode 0 specified, key is just SHA256 hash of passphrase
hash_a = b"\xca\x97\x81\x12\xca\x1b\xbd\xca\xfa\xc2\x31\xb3\x9a\x23\xdc\x4d\xa7\x86\xef\xf8\x14\x7c\x4e\x72\xb9\x80\x77\x85\xaf\xee\x48\xbb"
key = hash_a

def main(filename):
    
    with open(filename, "rb") as f:

        # First 9 bytes are header, ignore them and read the rest
        contents = f.read()[9:]

        # IV is size (block size + 2)
        # AES uses 16-byte (128-bit) blocks
        # Last two bytes are for checksum
        iv = contents[0:16 + 2]

        # Rest of contents should be ciphertext
        ciphertext = contents[16 + 2:]

        # Use openPGP special cipher mode
        cipher = AES.new(key, AES.MODE_OPENPGP, iv=iv)
        plaintext = cipher.decrypt(ciphertext)

        print("Output: " + str(plaintext))

if __name__ == "__main__":
    if (len(sys.argv) > 1): 
        main(sys.argv[1])
    else:
        main(input("Please specify an input file: "))

However, the output for this program is unintelligible garbage.

~$ python3 PGPDecryptor.py PGPDecryptorTest1.txt.gpg
Output: b'\x11\xd6\xf4\x8d\xf7/o.\x13k#D\xd1!\xce\xf5\xf9\xd9\x0b,\xdb\xe4\xd6,\xb8\x80\xcb2N\xd1^\x96\x8chP\xfb\xb0?Z\xb2\xed?\xce==\xfb9\xcf5o{\xb6\x12\xf3\xf7\xc9QC\xc3\xb5\xe4\x95ab?\x17\x9d\xd3\xd3\xc6\xa8j#K\x8cMf\xc6\x00V\x89Y\xe2\xe7~\xc4B\xd5\x1b\x8f\xe9&t'

I have verified the key by other methods, so I'm confident that it's correct. I must be very close to a proper solution, because changing either the IV or the key even slightly causes the following error to appear:

ValueError: Failed integrity check for OPENPGP IV

This suggests that I'm getting the key and IV correct. I've tried a nested for loop to try every valid combination of start and end indices for the ciphertext, just in case there was some additional garbage/header data somewhere, but with equally useless output for every combination.

If anyone can tell me what I'm doing wrong/how to correct it, I'd be very grateful. I suspect the error is very simple, but the nature of the problem makes it difficult to troubleshoot.

I currently have a janky alternative solution that involves modifying the pgpy library, but my problem with this is that importing large files to process (~500MB) takes a long time (~20-30 minutes). I looked at gnupg as well, but it's just a wrapper--it can decrypt with passphrases, but not with raw keys.

Answer 1

Using AES.MODE_OPENPGP would probably work for a Symmetrically Encrypted Data packet (tag 9), as it simply contains the encrypted data (reference).

However, that's not what you've produced with your gpg invocation. To get some insight into what we're actually dealing with, you can use the --list-packets command:

$ gpg --list-packets --verbose PGPDecryptorTest1.txt.gpg
gpg: AES256.CFB encrypted data
gpg: pinentry launched (90945 curses 1.1.0 /dev/pts/6 screen -)
gpg: encrypted with 1 passphrase
# off=0 ctb=8c tag=3 hlen=2 plen=4
:symkey enc packet: version 4, cipher 9, aead 0,s2k 0, hash 8
# off=6 ctb=d2 tag=18 hlen=2 plen=112 new-ctb
:encrypted data packet:
        length: 112
        mdc_method: 2
# off=27 ctb=a3 tag=8 hlen=1 plen=0 indeterminate
:compressed packet: algo=1
# off=29 ctb=ac tag=11 hlen=2 plen=66
:literal data packet:
        mode b (62), created 1652044966, name="PGPDecryptorTest1.txt",
        raw data: 39 bytes

Two things of note:

• The encrypted data packet is tag 18, which is a Symmetrically Encrypted Integrity Protected Data packet. We're no longer dealing with only the output of the cipher, but data preceded with a version # and suffixed with with a Modification Detection Code packet (reference).
• The encrypted data packets contents are compressed.

---

WARNING: The code below is just a rough demonstration of poking around OpenPGP message format. It is brittle and shouldn't be reused. The main takeaway is that reliably parsing OpenPGP messages isn't trivial and you should use a well tested library.

The main references I used are:

• RFC 4880 - OpenPGP Message Format
• GPGLib2 (Another pure Python GPG implementation that might be worth testing)

---

To more easily demonstrate digging into this, I've produced an encrypted message with compression turned off:

$ cat PGPDecryptorTest1.txt
It would be really cool if this worked

$ gpg --symmetric -o PGPDecryptorTest1.txt.uncompressed.gpg --compress-level 0 --s2k-mode 0 --s2k-digest-algo SHA256 --cipher-algo AES256 PGPDecryptorTest1.txt
gpg: Note: simple S2K mode (0) is strongly discouraged

$ python3 solution.py PGPDecryptorTest1.txt.uncompressed.gpg
contents(len: 117): b'8c0404090008d26d017795712a4686d1a176a0f150a33b9c972d876948df739b1058a513f916ef8094c80ae65ed022c30e1108d20dbeaeee70285e8736e8184520ceb0c435feafdd856051eb166e96e32e82ba51a3af4d230174e97a8f3a3529606b6558fce716bf3b0e9b856d442f5104f3647af0'
decrypted_iv(len: 16): b'49355c68e8e3eba7cc5ccb529d158a2c'
first_block(len: 16): b'8a2cac42621550475044656372797074'
decrypted_data (first block)(len: 14): b'ac42621550475044656372797074'
decrypted_data(len: 68): b'ac426215504750446563727970746f7254657374312e74787462783732497420776f756c64206265207265616c6c7920636f6f6c206966207468697320776f726b65640a'
plaintext(len: 39): b'497420776f756c64206265207265616c6c7920636f6f6c206966207468697320776f726b65640a'
It would be really cool if this worked

Here's the solution implementation:

import binascii
import hashlib
import sys
from Cryptodome.Cipher import AES

def print_bytes(name, data):
    print("%s(len: %d): %s" % (name, len(data), str(binascii.hexlify(data))))

def main(filename):
    # Generate key material from the passphrase.
    passphrase = b"a"
    m = hashlib.sha256()
    m.update(passphrase)
    key = m.digest()

    # Get file contents.
    with open(filename, "rb") as f:
        contents = f.read()
    print_bytes("contents", contents)

    # Constants
    header_len = 9  # including the 1-octet type-19 version identifier
    block_size = 16 # alogorithm details should normally be extracted from the header
    segment_size = block_size * 8
    iv_len = block_size
    iv_tag_len = 2
    mdc_len = 22

    # "Manually" decrypting to adhere to
    # https://datatracker.ietf.org/doc/html/rfc4880#section-5.13
    # Doing it this way helps with the integrity check, which I ended
    # skipping.
    cipher = AES.new(key, AES.MODE_CFB, iv=b"\x00" * block_size, segment_size=segment_size)

    offset = header_len

    decrypted_iv = cipher.decrypt(contents[offset:offset+block_size])
    print_bytes("decrypted_iv", decrypted_iv)

    decrypted_data = bytearray()

    offset += block_size

    first_block = cipher.decrypt(contents[offset:offset+block_size])
    print_bytes("first_block", first_block)

    offset += block_size

    if first_block[:2] != decrypted_iv[-2:]:
        print("IV check failed")
        sys.exit(1)

    decrypted_data.extend(first_block[2:])
    print_bytes("decrypted_data (first block)", decrypted_data)

    padding = block_size - (len(contents)-offset) % block_size
    contents += b"\x00" * padding

    decrypted_data.extend(cipher.decrypt(contents[offset:]))

    # Here is where you should parse the MDC packet and use it for integrity
    # checking. Instead, skipping the check and discarding the packet for 
    # brevity.
    decrypted_data = decrypted_data[:-(mdc_len+padding)]
    print_bytes("decrypted_data", decrypted_data)

    # Extract filename length so we can find the plaintext offset.
    # see https://datatracker.ietf.org/doc/html/rfc4880#section-5.9
    filename_len = decrypted_data[3]
    plaintext_offset = (
        2  # header
        + 1  # file type
        + 1  # filename length
        + filename_len  # filename contents
        + 4  # timestamp
    )
    plaintext = decrypted_data[plaintext_offset:]
    print_bytes("plaintext", plaintext)
    print(plaintext.decode())


if __name__ == "__main__":
    if (len(sys.argv) > 1):
        main(sys.argv[1])
    else:
        print("first argument must be file to decrypt")
        sys.exit(1)