Why does creating ECDSA keypairs via python differ from ssh-keygen -t ecdsa and how can I avoid that?

Question

For openstack apparently I need the key-output like ssh-keygen -t ecdsa generates it, but I am not getting similar output when using the ecdsa modul in python.

def createECDSAKeyPairLocally(projectName="current"):
    sk = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1)
    vk = sk.verifying_key
    with open(f"privateKey_bibi2_{projectName}.pem", "wb") as f:
        f.write(sk.to_pem())
    with open(f"publicKey_bibi2_{projectName}.pem", "wb") as f:
        f.write(vk.to_pem())
    return vk

vk will be something like:

b'-----BEGIN PUBLIC KEY-----[...]

while the ssh-keygen will be like:

ecdsa-sha2-nistp256[...]

While openstack accepts ssh-keygen's output it does not accept python exdsa's. I think I am misunderstanding something regarding the .pem files created. How can I generate similar output or what am I understanding wrong?

EDIT1

I tried

def createECDSAKeyPairLocally(projectName="current", comment = "no comment"):
    sk = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1)
    vk = sk.verifying_key
    with open(f"privateKey_bibi2_{projectName}.pem", "wb") as f:
        f.write(sk.to_pem())
    first = "ecdsa-sha2-nistp256"
    second = base64.b64encode(vk.to_string()).decode("utf-8")
    third = comment
    sshKeygenFormat = " ".join([first, second, third])
    with open(f"publicKey_bibi2_{projectName}.pub", "w") as f:
        f.write(sshKeygenFormat)
    return sshKeygenFormat

But openstack says that it is invalid. In the output I noticed that the python ecdsa version includes "" which I haven't seen yet in ssh.keygen's keys. Do they maybe use a different alphabet?

EDIT2

I attempted using

def createECDSAKeyPairLocally(projectName="current", comment = "no comment"):
    sk = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1)
    vk = sk.verifying_key
    with open(f"privateKey_bibi2_{projectName}.pem", "wb") as f:
        f.write(sk.to_pem())
    with open(f"publicKey_bibi2_{projectName}.pub", "w") as f:
        first = "ecdsa-sha2-nistp256"
        prefix = b"\x00\x00\x00\x13ecdsa-sha2-nistp256\x00\x00\x00\x08nistp256\x00\x00\x00A"
        second = base64.b64encode(prefix+vk.to_string()).decode("utf-8")
        third = comment
        sshkeygen = " ".join([first, second, third])
        f.write(sshkeygen)
    return sshkeygen

But the response is also that it's invalid.

EDIT3 I now tried:

def createECDSAKeyPairLocally(projectName="current", comment = "no comment"):
    sk = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1)
    vk = sk.verifying_key
    with open(f"privateKey_bibi2_{projectName}", "wb") as f:
        f.write(sk.to_pem())
    with open(f"publicKey_bibi2_{projectName}.pub", "w") as f:
        first = "ecdsa-sha2-nistp256"
        prefix = b"\x00\x00\x00\x13ecdsa-sha2-nistp256\x00\x00\x00\x08nistp256\x00\x00\x00A"
        second = base64.b64encode(
            prefix+vk.to_string(encoding="uncompressed")
         ).decode("utf-8")
        third = comment
        keygen = " ".join([first, second, third])
        f.write(keygen)
    return keygen

But the generated key is invalid (by openstack standards) as well. Someone outside of StackOverflow suggested that the ecdsa modul might use an older standard, but I don't know how to verify that.

EDIT 4.2

def createECDSAKeyPairLocally(projectName="current", comment = "no"):
    sk = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1)
    vk = sk.verifying_key
    with open(f"privateKey_bibi2_{projectName}", "wb") as f:
        f.write(sk.to_pem())
    with open(f"publicKey_bibi2_{projectName}.pub", "w") as f:
        first = "ecdsa-sha2-nistp256"
        prefix = b"\x00\x00\x00\x13ecdsa-sha2-nistp256\x00\x00\x00\x08nistp256\x00\x00\x00A"
        all_bytes = vk.to_string(encoding="uncompressed")
        prepending_byte = all_bytes[:1] # necessary to get bytes instead of int
        first_key_part = all_bytes[1:33]
        second_key_part = all_bytes[33:]
        second = base64.b64encode(
            b''.join([prefix,prepending_byte,second_key_part,first_key_part])
         ).decode("utf-8")
        third = comment
        keygen = " ".join([first, second, third])
        f.write(keygen)
    return keygen

Fixed the byte slicing by using b''.join. Still not accepted by openstack.

Answer 1

From the OpenSSH manual page:

ssh-keygen will by default write keys in an OpenSSH-specific format.

and

It is still possible for ssh-keygen to write the previously-used PEM format private keys using the -m flag.

The OpenSSH public key file is not difficult. It is basically a line of text consisting of three parts, separated by spaces.

• type name of the key used
• public key, base64 encoded
• comment (often user@host or an e-mail address)

EDIT:

The format of the second part is slightly more complicated than just base64 encoded binary. Let's generate a test key;

> ssh-keygen -t ecdsa -f testkey
Generating public/private ecdsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in testkey
Your public key has been saved in testkey.pub
The key fingerprint is:
SHA256:lsHGr4B4kIDxHsKgdhDengrFMzKu4kXASvTrKG/QLig
The key's randomart image is:
+---[ECDSA 256]---+
|==.              |
|O++.   o         |
|=BX+    =        |
|=*+*o. . +       |
|ooo=o . S .      |
|+ *.   o .       |
|== o    .        |
|E.+              |
|.+.              |
+----[SHA256]-----+

Decode the contents of the public key;

> python
Python 3.9.9 (main, Dec 11 2021, 14:34:11) 
Type "help", "copyright", "credits" or "license" for more information.
>>> import base64
>>> base64.b64decode("AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCFeupSBqgm0Glpskrctk/iyRcmdzRLnyiept0cIXtP4XygWiiptxcrvJ3iFhyYxxV6a26gkvn8Ub2QGn5k1gsE=")
b"\x00\x00\x00\x13ecdsa-sha2-nistp256\x00\x00\x00\x08nistp256\x00\x00\x00A\x04!^\xba\x94\x81\xaa\t\xb4\x1aZl\x92\xb7-\x93\xf8\xb2E\xc9\x9d\xcd\x12\xe7\xca'\xa9\xb7G\x08^\xd3\xf8_(\x16\x8a*m\xc5\xca\xef'x\x85\x87&1\xc5^\x9a\xdb\xa8$\xbe\x7f\x14od\x06\x9f\x995\x82\xc1"

Look carefully at the beginning of the decoded string:

b"\x00\x00\x00\x13ecdsa-sha2-nistp256\x00\x00\x00\x08nistp256\x00\x00\x00A"

What do we see?

• Three zero bytes, followed by a non-zero byte (1), value 0x13
• A string cdsa-sha2-nistp256
• Three zero bytes, followed by a non-zero byte (2), value 0x08
• A string nistp256
• Three zero bytes, followed by a non-zero byte (3), value A

My hunch is that the four byte entities are lengths, in big-endian integers. Let's test that.

>>> import struct
>>> struct.unpack(">i", b'\x00\x00\x00\x13')
(19,)
>>> len("ecdsa-sha2-nistp256")
19
>>> struct.unpack(">i", b'\x00\x00\x00\x08')
(8,)
>>> len("nistp256")
8
>>> struct.unpack(">i", b'\x00\x00\x00A')
(65,)
>>> len(b"\x04!^\xba\x94\x81\xaa\t\xb4\x1aZl\x92\xb7-"
... b"\x93\xf8\xb2E\xc9\x9d\xcd\x12\xe7\xca'\xa9\xb7G\x08^"
... b"\xd3\xf8_(\x16\x8a*m\xc5\xca\xef'x\x85\x87&1\xc5^\x9a"
... b"\xdb\xa8$\xbe\x7f\x14od\x06\x9f\x995\x82\xc1")
65

The 4-byte numbers are indeed lengths. The length of the key seems to be correct, see this answer.

EDIT2:

Looking here, which leads to to_bytes, it seems you need to use the "uncompressed" encoding (instead of the default "raw"). Because that starts with 0x04. Unfortunately the docstrings aren't very clear about what these encodings actually mean.

import base64

def createECDSAKeyPairLocally(projectName="current", comment = "no comment"):
    sk = ecdsa.SigningKey.generate(curve=ecdsa.SECP256k1)
    vk = sk.verifying_key
    with open(f"privateKey_bibi2_{projectName}.pem", "wb") as f:
        f.write(sk.to_pem())
    with open(f"publicKey_bibi2_{projectName}.pub", "wb") as f:
        first = "ecdsa-sha2-nistp256"
        prefix = b"\x00\x00\x00\x13ecdsa-sha2-nistp256\x00\x00\x00\x08nistp256\x00\x00\x00A"
        second = base64.b64encode(
            prefix+vk.to_string(encoding="uncompressed")
         ).decode("utf-8")
        third = comment
        f.write(" ".join([first, second, third]))
    return vk

Edit3:

Looking at the header in keys.py:

    raw encoding
        Conversion of public, private keys and signatures (which in
        mathematical sense are integers or pairs of integers) to strings of
        bytes that does not use any special tags or encoding rules.
        For any given curve, all keys of the same type or signatures will be
        encoded to byte strings of the same length. In more formal sense,
        the integers are encoded as big-endian, constant length byte strings,
        where the string length is determined by the curve order (e.g.
        for NIST256p the order is 256 bits long, so the private key will be 32
        bytes long while public key will be 64 bytes long). The encoding of a
        single integer is zero-padded on the left if the numerical value is
        low. In case of public keys and signatures, which are comprised of two
        integers, the integers are simply concatenated.
    uncompressed
        The most common formatting specified in PKIX standards. Specified in
        X9.62 and SEC1 standards. The only difference between it and
        :term:`raw encoding` is the prepending of a 0x04 byte. Thus an
        uncompressed NIST256p public key encoding will be 65 bytes long.

So the uncompressed format should be OK. I have two more ideas.

First, your comment has a space in it; comment = "no comment". Since a space is the separator in the OpenSSH key format, it could be that a space is not allowed. Try changing it to comment = "no_comment".

Second, since the key consists of two coordinates, say X and Y, you would expect them to be in the sequence 0x04XY. But maybe they are in the sequence 0x04YX in OpenSSH? So try putting the parts of the key in the other order.

EDIT4:

Slicing example (with characters for easy visualisation).

In [1]: a = '4'
Out[1]: '4'

In [2]: b = 32*'x'
Out[2]: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

In [3]: c = 32*'y'
Out[3]: 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'

In [4]: key = a+b+c
Out[4]: '4xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'

In [5]: key[0]
Out[5]: '4'

In [6]: key[1:33]
Out[6]: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

In [7]: key[33:]
Out[7]: 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'