envsubst: default values for unset variables

Question

I've got a json file input.json like the following one:

{
  "variable" : "${ENV_VAR}"
}

of course, I can invoke envsubst from bash like the following:

$ export ENV_VAR=myvalue
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "myvalue"
}

Now, I wish I could set default values for variables in the input.json for the case when ENV_VAR is not set, like in the following example which, as unfortunately can be seen in the example below, doesn't work:

$ cat input.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}
$ export ENV_VAR=newvalue
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}
$ unset ENV_VAR
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}

What's curious, if I execute the envsubst like in the following example (without involving an input file), it works

$ export ENV_VAR=myvalue
$ echo "value is ${ENV_VAR:=defaultvalue}" | envsubst
value is myvalue
$ unset ENV_VAR
$ echo "value is ${ENV_VAR:=defaultvalue}" | envsubst
value is defaultvalue

Where is the problem with the files?

Answer 1

According to man envsubst, envsubst will only ever replace references to environment variables in the form of ${VAR} or $VAR. Special shell features like ${VAR:-default} are not supported. The only thing you could do is to (re)define all variables in the environment of the envsubst invocation and assign local default values, if they are missing:

ENV_VAR="${ENV_VAR:-defaultvalue}" \
OTHER_VAR="${OTHER_VAR:-otherdefault}" \
envsubst < input.json > output.json

Note, that this is actually a single command line split into multiple lines each ending with a line continuation \. The first two lines are variable assignments, that are only effective in the environment of the executed command envsubst in the last line. What's happening is, that the shell will create an environment for the execution of the command (as it would always do). That environment is initially a copy of the current shell environment. Within that new environment ENV_VAR and OTHER_VAR are assigned the values of expanding the expression ${VAR:-default}, which essentially expands to default unless VAR is defined and has a none-empty value. The command envsubst is executed, receiving the file input.json as standard-input and having its standard-output redirected to output.json (both is done by the shell, transparent to the command). After the command execution, the shell deletes the command environment returning to its original environment, i.e. the local variable assignments are no longer effective.

There is no way to define default values from inside the JSON file, unless you implement a program to do so yourself, or use another tool that can to that.

You could do something like the following, but it is NOT RECOMMENDED:

eval echo "$(cat input.json)" > output.json

which will read input.json into a string, and than evaluate the command echo <string> as if it was type literally, which means that any embedded ${VAR:-default} stuff should be expanded by the shell before the string is passed to echo. BUT any other embedded shell feature will be evaluated as well, which poses a HUGE SECURITY RISK.

Answer 2

I'm using https://github.com/a8m/envsubst and it has enhancements over the original gettext envsubst that the expressions in the template file supports default values.

The example in the README just works.

echo 'welcome $HOME ${USER:=a8m}' | envsubst

Answer 3

Blockquote I'm using https://github.com/a8m/envsubst and it has enhancements over the original gettext envsubst that the expressions in the template file supports default values.

Similarly, a Rust variant called 'envsub' is available which also supports the default values. See https://github.com/stephenc/envsub .