Store .jks file in google secret manager without getting non UTF-8 file?

Question

Thank you for all the replies, I am still facing the issue, just to add more clarity I've given all the details of cloudbuild.yaml and build.gradle and key property details, please let me know if the configuration is correct and let me know how to fix the JKS issue.

I am working on integrating the CI\CD pipeline to a flutter project using GCP, which I am trying to store.JKS file in google secret manager and call it from inside the code, but it is giving an error saying secret env variable cannot be the non-UTF-8 format.

So I tried a couple of things,

1. I tried to convert the.JKS file to txt viewable - after that it gave me an error saying secret env variable cannot have null values.

2. I tried to store the JKS file in cloud storage - but the code is not able to get the content of the JKS even if I give the link and all the necessary permissions for cloud build.

Please suggest some fix or alternate storage area in GCP.

I am adding the code for more clarity. ------------------------------------------***-----------------------------------------

APPROACH 1: TRYING TO ACCESS JKS FILE FROM SECRET MANAGER

Secret Manager KEY VALUE structure

key                      value
KEYSTORE_PASSWORD       xxxxxxxxxxxx        
KEY_PASSWORD            xxxxxxxxxxxx
KEY_ALIAS               upload
JKS                     fe ed fe ed 00 00 00.....

---

build.sh

cd /workspace/$1
VERSION_NAME=$(git describe)
VERSION_CODE=$(git rev-list --count master)
flutter build apk --build-name=$VERSION_NAME --build-number=$VERSION_CODE

---

---

cloudbuild.yaml:

---

# Flutter CD configuration file with Cloud build

steps:

  # clone the latest source codes
  - name: 'gcr.io/cloud-builders/git'
    args: ['clone', 'https://XXXXX:[email protected]/XXXXXXXX/XX.git']
    dir: '/workspace'

  # using flutter builder Docker image we have built previously to compile the repo
  - name: 'gcr.io/$PROJECT_ID/flutter'
    entrypoint: 'bash'
    args: [ 'build.sh']
    secretEnv: ['KEYSTORE_PASSWORD','KEY_PASSWORD', 'KEY_ALIAS', 'JKS']

  # Push the APK Output to your GCS Bucket with Short Commit SHA.
  - name: 'gcr.io/cloud-builders/gsutil'
    args: [ 'cp', 'build/app/outputs/flutter-apk/app-release.apk', 'gs://BUCKET_NAME' ]

availableSecrets:
  secretManager:
  - versionName: projects/xxxxxx/secrets/KEYSTORE_PASSWORD/versions/1
    env: 'KEYSTORE_PASSWORD'
  - versionName: projects/xxxxxxx/secrets/KEY_PASSWORD/versions/1
    env: 'KEY_PASSWORD'
  - versionName: projects/xxxxxx/secrets/KEY_ALIAS/versions/1
    env: 'KEY_ALIAS'
  - versionName: projects/xxxxxxx/secrets/upload-keystore-jks/versions/1
    env: 'JKS'

---

build.gradle:

---

def localProperties = new Properties()
def localPropertiesFile = rootProject.file('local.properties')
if (localPropertiesFile.exists()) {
    localPropertiesFile.withReader('UTF-8') { reader ->
        localProperties.load(reader)
    }
}

def flutterRoot = localProperties.getProperty('flutter.sdk')
if (flutterRoot == null) {
    throw new GradleException("Flutter SDK not found. Define location with flutter.sdk in the local.properties file.")
}

def flutterVersionCode = localProperties.getProperty('flutter.versionCode')
if (flutterVersionCode == null) {
    flutterVersionCode = '1'
}

def flutterVersionName = localProperties.getProperty('flutter.versionName')
if (flutterVersionName == null) {
    flutterVersionName = '1.0'
}

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply from: "$flutterRoot/packages/flutter_tools/gradle/flutter.gradle"

android {
    compileSdkVersion flutter.compileSdkVersion

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }

    kotlinOptions {
        jvmTarget = '1.8'
    }

    sourceSets {
        main.java.srcDirs += 'src/main/kotlin'
    }

    defaultConfig {
        // TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
        applicationId "com.XXX.XXX"
        minSdkVersion flutter.minSdkVersion
        targetSdkVersion flutter.targetSdkVersion
        versionCode flutterVersionCode.toInteger()
        versionName flutterVersionName
    }

    signingConfigs {
        release {
            keyAlias System.getenv("KEY_ALIAS")
            keyPassword System.getenv("KEY_PASSWORD")
            storePassword System.getenv("KEYSTORE_PASSWORD")
            storeFile System.getenv("JKS")
        }
    }
    buildTypes {
        release {
            signingConfig signingConfigs.release
        }
    }

}

flutter {
    source '../..'
}

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
}   


ERROR
==============================================================================================================
build step 1 "gcr.io/buildtrial-1/flutter" failed: secret projects/xxxxxxx/secrets/upload-keystore-jks/versions/1 value is not valid UTF-8

==============================================================================================================

---

NOTE: 
1.build.gradle file is same for both the approches, as the JKS variable name is conssistent in both the approches
2. I've verified that, 'KEYSTORE_PASSWORD','KEY_PASSWORD', 'KEY_ALIAS' is working properly, only problem is with accessing JKS file in both the approches.

-------------------------------------------------------------------------**********-----------------------------------------------------------------------------------------------------------------------

APPROCH 2: TRYING TO ACCESS JKS FILE FROM CLOUD STORAGE.

cloudbuild.yaml:

---

# Flutter CD configuration file with Cloud build

steps:

  # clone the latest source codes
  - name: 'gcr.io/cloud-builders/git'
    args: ['clone', 'https://XXXXX:[email protected]/XXXXXXXX/XX.git']
    dir: '/workspace'

  #accessing the JKS file stored in cloud storage through environment variable
  - name: 'gcr.io/cloud-builders/gsutil'
    env:
      - 'JKS=gs://BUCKET_NAME/KEYSTORE.jks'

  # using flutter builder Docker image we have built previously to compile the repo
  - name: 'gcr.io/$PROJECT_ID/flutter'
    entrypoint: 'bash'
    args: [ 'build.sh']
    secretEnv: ['KEYSTORE_PASSWORD','KEY_PASSWORD', 'KEY_ALIAS']

  # Push the APK Output to your GCS Bucket with Short Commit SHA.
  - name: 'gcr.io/cloud-builders/gsutil'
    args: [ 'cp', 'build/app/outputs/flutter-apk/app-release.apk', 'gs://BUCKET_NAME' ]

availableSecrets:
  secretManager:
  - versionName: projects/xxxxxx/secrets/KEYSTORE_PASSWORD/versions/1
    env: 'KEYSTORE_PASSWORD'
  - versionName: projects/xxxxxxx/secrets/KEY_PASSWORD/versions/1
    env: 'KEY_PASSWORD'
  - versionName: projects/xxxxxx/secrets/KEY_ALIAS/versions/1
    env: 'KEY_ALIAS'

============================================================================= ERROR

Step #2: Execution failed for task ':app:validateSigningRelease'.
Step #2: > Keystore file not set for signing config release

=============================================================================

Answer 1

When you have binary files, you have to convert them in base64 and then to store them encoded.

In your application, read the base64 content of Secret Manager, decode it, and use it.