Deleting Azure Key Vault secret fails in Azure Pipeline

Question

I'm trying to use an Azure Pipeline to delete a secret in Azure Key Vault. The Key Vault has soft-delete enabled and purge protection disabled. I want to delete the secret and then purge the secret in a later task.

I use the following task:

- task: AzureCLI@2
      displayName: 'Delete <secret> from Azure Key Vault'
      inputs:
        azureSubscription: <Valid service connection>
        scriptType: 'pscore'
        scriptLocation: 'inlineScript'
        inlineScript: 'az keyvault secret delete --name "<secret name>" --vault-name "<Key Vault Name>"'
        failOnStandardError: true

When I execute this the secret is deleted but the task fails with the following:

{
  "attributes": {
    "created": "2022-03-03T08:16:55+00:00",
    "enabled": true,
    "expires": null,
    "notBefore": null,
    "recoveryLevel": "Recoverable+Purgeable",
    "updated": "2022-03-03T08:16:55+00:00"
  },
  "contentType": null,
  "deletedDate": "2022-03-03T09:44:49+00:00",
  "id": "https://<vault name>.vault.azure.net/secrets/<secret name>/<ID>",
  "kid": null,
  "managed": null,
  "name": "<secret name>",
  "recoveryId": "https://<vault name>.vault.azure.net/deletedsecrets/<secret name>",
  "scheduledPurgeDate": "2022-06-01T09:44:49+00:00",
  "tags": null,
  "value": null
}

##[error]WARNING: Warning! If you have soft-delete protection enabled on this key vault, this secret will be moved to the soft deleted state. You will not be able to create a secret with the same name within this key vault until the secret has been purged from the soft-deleted state. Please see the following documentation for additional guidance. https://docs.microsoft.com/azure/key-vault/general/soft-delete-overview

##[error]Script has output to stderr. Failing as failOnStdErr is set to true.

Why does the script fail when the error is really just a warning? Any ideas how to solve this?