How to save an error message from subprocess command to file

Question

When running bash command using subprocess, I might run into situation where the command is not valid. In this case, bash would return an error messsage. How can we catch this message? I would like to save this message to a log file. The following is an example, where I try to list files in a non-existed directory.

try:
    subprocess.check_call(["ls", "/home/non"]) 
    df = subprocess.Popen(["ls", "/home/non"], stdout=subprocess.PIPE)        
    output, err = df.communicate()
    # process outputs
except Exception as error:        
    print error
    sys.exit(1)

Bash would prints "ls: cannot access /home/non: No such file or directory". How can I get this error message? The error caught by the except line is clearly different, it says "Command '['ls', '/home/non']' returned non-zero exit status 2".

Answer 1

"ls: cannot access /home/non: No such file or directory" is generated by ls command, not bash here.

If you want to handle non-existing files using exception handling then use subprocess.check_output():

#!/usr/bin/env python
from subprocess import check_output, STDOUT, CalledProcessError

try:
    output = check_output(['ls', 'nonexistent'], stderr=STDOUT)
except CalledProcessError as exc:
    print(exc.output)
else:
    assert 0

Output

ls: cannot access nonexistent: No such file or directory

Answer 2

Per the subprocess docs, the command run is now prefered.

Example:

import logging
import subprocess

logger = logging.getLogger(__name__)

try:
    subprocess.run(["ls", "-l"], shell=True, check=True, capture_output=True)
except subprocess.CalledProcessError as err:
    logger.error(f"{err} {err.stderr.decode('utf8')}")

As others have mentioned, if you want to save to a file, you cand use the stdout param to run; however, you may as well use logging to do that, and then just log the error in your method.