'HSM Error | Private key must be instance of RSAPrivate(Crt)Key or have PKCS#8

Error received while decrypting data when private key is retrieved from HSM.

I have added sunpkcs11 provider in java.security. Hence, NOT adding provider via code. Text gets encrypted successfully. However, while decrypting the encrypted text, I am getting below error at below line:

cipher.init(Cipher.DECRYPT_MODE, privateKey);

What is that i am missing here?

Error:

    Caused by: java.security.InvalidKeyException: Private key must be instance of RSAPrivate(Crt)Key or have PKCS#8 encoding
        at sun.security.pkcs11.P11RSAKeyFactory.implTranslatePrivateKey(P11RSAKeyFactory.java:101) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11KeyFactory.engineTranslateKey(P11KeyFactory.java:132) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11KeyFactory.convertKey(P11KeyFactory.java:65) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11RSACipher.implInit(P11RSACipher.java:199) [sunpkcs11.jar:1.7.0_85]
        at sun.security.pkcs11.P11RSACipher.engineInit(P11RSACipher.java:168) [sunpkcs11.jar:1.7.0_85]
        at javax.crypto.Cipher.init(Cipher.java:1068) [jce.jar:1.7.0_85]
        at javax.crypto.Cipher.init(Cipher.java:1012) [jce.jar:1.7.0_85]enter code here

Below is the code:

import java.io.ByteArrayOutputStream;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.cert.Certificate;

import javax.crypto.Cipher;
import javax.xml.bind.DatatypeConverter;

import sun.security.pkcs11.SunPKCS11;

public class App {

    public static void main(String[] args) throws Exception {

        try {
            String passphrase = "mysecretkey";
            SunPKCS11 provider = new SunPKCS11("/home/user/pkcs11.cfg");
            KeyStore keystore = KeyStore.getInstance("PKCS11", provider);
            keystore.load(null, passphrase.toCharArray());
            String textToEncrypt = "this is my text";
            Certificate cert = keystore.getCertificate("my-SHA1WITHRSA-2048-bits-key");
            PublicKey publicKey = cert.getPublicKey();
            Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding", provider);
            cipher.init(Cipher.ENCRYPT_MODE, publicKey);
            String encryptedData = DatatypeConverter.printBase64Binary(cipher.doFinal(textToEncrypt.getBytes()));

            PrivateKey privateKey = (PrivateKey) keystore.getKey("my-SHA1WITHRSA-2048-bits-key",
                    passphrase.toCharArray());
            cipher.init(Cipher.DECRYPT_MODE, privateKey);
            byte[] decodedEncryptedData = DatatypeConverter.parseBase64Binary(encryptedData);
            ByteArrayOutputStream stream = new ByteArrayOutputStream();
            int blocks = decodedEncryptedData.length / 256;
            int offset = 0;
            for (int blockIndex = 0; blockIndex < blocks; blockIndex++) {
                byte[] nextBlock = getNextBlock(decodedEncryptedData, offset);
                stream.write(cipher.doFinal(nextBlock));
                offset += 256;
            }
        } catch (Exception e) {
            e.printStackTrace();
        }

    }

    private static byte[] getNextBlock(byte[] cipherText, int offset) {
        byte[] block = new byte[256];
        System.arraycopy(cipherText, offset, block, 0, 256);
        return block;
    }

}
javaencryptionrsapublic-key-encryptionhsm


Solution 1:[1]

How I resolved:

Root cause of this issue was that sunpkcs11 provider was getting loaded both statically and dynamically.

i.e. in java.security, provider entry along with cfg path was already added.

Also, in code, provider was initialized again with the cfg file.

This was causing the issue.

After changing:

SunPKCS11 provider = new SunPKCS11("/home/user/pkcs11.cfg");

TO:

SunPKCS11 sunPKCS11Provider = (SunPKCS11) Security.getProvider("SunPKCS11");

issue got resolved.

Solution 2:[2]

I have used following code and issue has been resolved

    SunPKCS11 provider = new SunPKCS11("/home/user/pkcs11.cfg");
    Security.addProvider(provider);
    KeyStore keystore = KeyStore.getInstance("PKCS11");
    keystore.load(null, passphrase.toCharArray());

Solution 3:[3]

I was getting this same error when trying to get a ContentSigner with SHA1 using Linux and Jdk 1.8u202. I haven't found a report, but I suspect there's a bug in that JDK because it worked on newer JDKs (1.8u322), JDK11, JDK17 on Linux and on 1.8u202 in Windows.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Smart B0y
Solution 2 Dharmendrasinh Chudasama
Solution 3 Amber

Related Questions