Python from a different environment in a shell script

Question

I have a .yaml file which specifies a command to be executed, and where I want to call a bash script:

cmd: bash python_exec.sh 1 src/sample.py

Note the argument 1 in the above case: this argument 'determines' the python executable which corresponds either to a Jupyter lab or to my machine. The content of python_exec.sh is the following:

#!/bin/bash

if [[ $1 -eq 0 ]]
then
  python -c "import sys; print(sys.executable)" $2  
else
  python -c "import jupyter_client; print(jupyter_client.kernelspec.get_kernel_spec('evaluation').argv[0])" $2
fi

As you may see, I am trying to extract python executable, and run it with 'src/sample.py'. The above doesn't work for some reason. The error message I get is:

Running stage 'train':                                                                                                                                        
> bash python_exec.sh 1 src/sample.py                                                                                                                         
/home/jovyan/.local/share/virtualenvs/.evaluation-kernel-CVTAJ6_f/bin/python
ERROR: failed to reproduce 'dvc.yaml': output 'models' does not exist                                                                                         
ERROR: failed to reproduce 'dvc.yaml': output 'models' does not exist

Note that I'm using DVC experiment management, dvc exp run to fetch the .yaml file.

However, simply putting the following in a .yaml file works:

cmd: $(python -c "import jupyter_client; print(jupyter_client.kernelspec.get_kernel_spec('evaluation').argv[0])") src/sample.py

Any ideas on how I might achieve the desired?

Answer 1

The thing is, I forgot to wrap the python command with $(...). The following bash file works:

#!/bin/bash

if [[ $1 -eq 0 ]]
then
  $(python -c "import sys; print(sys.executable)") $2  
else
  $(python -c "import jupyter_client; print(jupyter_client.kernelspec.get_kernel_spec('evaluation').argv[0])") $2
fi