Pysftp fails with "Authentication failed" and "Server did not send a server-sig-algs list; defaulting to our first preferred algo ('rsa-sha2-512')"

Question

I have ec2 instance with ubuntu v20.04 and it has python v3.8.10 and pysftp 0.2.9.

I have generate .pem file from .ppk file using below command

puttygen sftp_server.ppk -O private-openssh -o sftp_server.pem

I am able to connect successfully to sftp server using command line-

sftp -o "IdentityFile=sftp_server.ppk" sftp_user@ip_address
sftp -o "IdentityFile=sftp_server.pem" sftp_user@ip_address

#Output of above command
Connected to ip_address
sftp> pwd
Remote working directory: /

I have pasted the output of above sftp command for .pem file -

OpenSSH_8.2p1 Ubuntu-4ubuntu0.3, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /home/user/.ssh/config
debug1: /home/user/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname IP_ADDRESS is address
debug2: ssh_connect_direct
debug1: Connecting to IP_ADDRESS [IP_ADDRESS] port 22.
debug1: Connection established.
debug1: identity file sftp_server.pem type -1
debug1: identity file sftp_server.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.3
debug1: Remote protocol version 2.0, remote software version 5.17 FlowSsh: Bitvise SSH Server (WinSSHD) 5.58
debug1: no match: 5.17 FlowSsh: Bitvise SSH Server (WinSSHD) 5.58
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to IP_ADDRESS:22 as 'sftp_user'
debug3: hostkeys_foreach: reading file "/home/user/.ssh/known_hosts"
debug3: record_hostkey: found key type DSA in file /home/user/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from IP_ADDRESS
debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-dss
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ssh-dss,[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==
debug2: host key algorithms: ssh-dss
debug2: ciphers ctos: aes256-ctr,aes256-cbc,aes192-ctr,aes192-cbc,aes128-ctr,aes128-cbc,3des-ctr,3des-cbc
debug2: ciphers stoc: aes256-ctr,aes256-cbc,aes192-ctr,aes192-cbc,aes128-ctr,aes128-cbc,3des-ctr,3des-cbc
debug2: MACs ctos: hmac-sha2-256,hmac-sha1,hmac-md5,hmac-sha2-256-96,hmac-sha1-96,hmac-md5-96
debug2: MACs stoc: hmac-sha2-256,hmac-sha1,hmac-md5,hmac-sha2-256-96,hmac-sha1-96,hmac-md5-96
debug2: compression ctos: zlib,none
debug2: compression stoc: zlib,none
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: diffie-hellman-group-exchange-sha256
debug1: kex: host key algorithm: ssh-dss
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug3: send packet: type 34
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(2048<8192<8192) sent
debug3: receive packet: type 31
debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
debug2: bits set: 1039/2048
debug3: send packet: type 32
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug3: receive packet: type 33
debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: ssh-dss SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
debug3: hostkeys_foreach: reading file "/home/user/.ssh/known_hosts"
debug3: record_hostkey: found key type DSA in file /home/user/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from IP_ADDRESS
debug1: Host 'IP_ADDRESS' is known and matches the DSA host key.
debug1: Found key in /home/user/.ssh/known_hosts:1
debug2: bits set: 1016/2048
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: sftp_server.pem  explicit
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: sftp_server.pem
debug3: sign_and_send_pubkey: RSA SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
debug3: sign_and_send_pubkey: signing using ssh-rsa SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 52
debug1: Authentication succeeded (publickey).
Authenticated to IP_ADDRESS ([IP_ADDRESS]:22).
debug2: fd 4 setting O_NONBLOCK
debug3: fd 5 is O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x08
debug2: client_session2_setup: id 0
debug1: Sending environment.
debug3: Ignored env SHELL
debug3: Ignored env PWD
debug3: Ignored env LOGNAME
debug3: Ignored env XDG_SESSION_TYPE
debug3: Ignored env MOTD_SHOWN
debug3: Ignored env HOME
debug1: Sending env LANG = C.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env LS_COLORS
debug3: Ignored env SSH_CONNECTION
debug3: Ignored env LESSCLOSE
debug3: Ignored env XDG_SESSION_CLASS
debug3: Ignored env TERM
debug3: Ignored env LESSOPEN
debug3: Ignored env USER
debug3: Ignored env SHLVL
debug3: Ignored env XDG_SESSION_ID
debug3: Ignored env XDG_RUNTIME_DIR
debug3: Ignored env SSH_CLIENT
debug3: Ignored env XDG_DATA_DIRS
debug3: Ignored env PATH
debug3: Ignored env DBUS_SESSION_BUS_ADDRESS
debug3: Ignored env SSH_TTY
debug3: Ignored env _
debug3: Ignored env OLDPWD
debug1: Sending subsystem: sftp
debug2: channel 0: request subsystem confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 262144 rmax 65536
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: subsystem request accepted on channel 0
debug2: Remote version: 3
Connected to IP_ADDRESS.
debug3: Sent message fd 3 T:16 I:1
debug3: SSH_FXP_REALPATH . -> / size 0
debug3: receive packet: type 2
debug3: Received SSH2_MSG_IGNORE
debug3: receive packet: type 2
debug3: Received SSH2_MSG_IGNORE
^Cdebug3: receive packet: type 2
debug3: Received SSH2_MSG_IGNORE

I have used following code in python.

with pysftp.Connection(host=hostname, username=username, private_key="sftp_server.ppk") as sftp:
        print ("SFTP Connection succesfully established ...")

I have tried with ppk/pem file also with pysftp v0.2.8 but i get following error

Traceback (most recent call last):
  File "sftp_conn_test.py", line 39, in <module>
    with pysftp.Connection(host=hostname, username=username, private_key="sftp_server.ppk") as sftp:
  File "/home/user/.local/lib/python3.8/site-packages/pysftp.py", line 187, in __init__
    self._transport.connect(username=username, pkey=prv_key)
  File "/home/user/.local/lib/python3.8/site-packages/paramiko/transport.py", line 1387, in connect
    self.auth_publickey(username, pkey)
  File "/home/user/.local/lib/python3.8/site-packages/paramiko/transport.py", line 1634, in auth_publickey
    return self.auth_handler.wait_for_response(my_event)
  File "/home/user/.local/lib/python3.8/site-packages/paramiko/auth_handler.py", line 258, in wait_for_response
    raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

I have pasted DEBUG mode log of paramiko/pysftp -

2022-01-23 09:59:32,048 -  DEBUG - transport - 1873 - starting thread (client mode): 0xfca21a30
2022-01-23 09:59:32,048 -  DEBUG - transport - 1873 - Local version/idstring: SSH-2.0-paramiko_2.9.2
2022-01-23 09:59:32,137 -  DEBUG - transport - 1873 - Remote version/idstring: SSH-2.0-5.17 FlowSsh: Bitvise SSH Server (WinSSHD) 5.58
2022-01-23 09:59:32,137 -  INFO - transport - 1873 - Connected (version 2.0, client 5.17)
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - === Key exchange possibilities ===
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - kex algos: diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1, diffie-hellman-group14-sha1, diffie-hellman-group1-sha1, gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==, gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - server key: ssh-dss
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - client encrypt: aes256-ctr, aes256-cbc, aes192-ctr, aes192-cbc, aes128-ctr, aes128-cbc, 3des-ctr, 3des-cbc
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - server encrypt: aes256-ctr, aes256-cbc, aes192-ctr, aes192-cbc, aes128-ctr, aes128-cbc, 3des-ctr, 3des-cbc
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - client mac: hmac-sha2-256, hmac-sha1, hmac-md5, hmac-sha2-256-96, hmac-sha1-96, hmac-md5-96
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - server mac: hmac-sha2-256, hmac-sha1, hmac-md5, hmac-sha2-256-96, hmac-sha1-96, hmac-md5-96
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - client compress: zlib, none
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - server compress: zlib, none
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - client lang: <none>
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - server lang: <none>
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - kex follows: False
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - === Key exchange agreements ===
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - Kex: diffie-hellman-group-exchange-sha256
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - HostKey: ssh-dss
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - Cipher: aes128-ctr
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - MAC: hmac-sha2-256
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - Compression: none
2022-01-23 09:59:32,138 -  DEBUG - transport - 1873 - === End of kex handshake ===
2022-01-23 09:59:32,359 -  DEBUG - transport - 1873 - Got server p (2048 bits)
2022-01-23 09:59:32,527 -  DEBUG - transport - 1873 - kex engine KexGexSHA256 specified hash_algo <built-in function openssl_sha256>
2022-01-23 09:59:32,528 -  DEBUG - transport - 1873 - Switch to new keys ...
2022-01-23 09:59:32,528 -  DEBUG - transport - 1873 - Attempting public-key auth...
2022-01-23 09:59:32,750 -  DEBUG - transport - 1873 - userauth is OK
2022-01-23 09:59:32,750 -  DEBUG - transport - 1873 - Finalizing pubkey algorithm for key of type 'ssh-rsa'
2022-01-23 09:59:32,750 -  DEBUG - transport - 1873 - Our pubkey algorithm list: ['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa']
2022-01-23 09:59:32,750 -  DEBUG - transport - 1873 - Server did not send a server-sig-algs list; defaulting to our first preferred algo ('rsa-sha2-512')
2022-01-23 09:59:32,750 -  DEBUG - transport - 1873 - NOTE: you may use the 'disabled_algorithms' SSHClient/Transport init kwarg to disable that or other algorithms if your server does not support them!
2022-01-23 09:59:32,875 -  INFO - transport - 1873 - Authentication (publickey) failed.

Could someone please help here. thanks

Answer 1

The error comes from underlying Paramiko and is discussed here:
Paramiko authentication fails with "Agreed upon 'rsa-sha2-512' pubkey algorithm" (and "unsupported public key algorithm: rsa-sha2-512" in sshd log)

Though pysftp does not expose the disabled_algorithms parameter.

You better switch to using Paramiko directly. The pysftp is abandoned project. See pysftp vs. Paramiko.