Flask API Not Writing Out Logs on Elastic Beanstalk

Question

I have a Flask API and I am attempting to implement logging. Everything is working fine in my local machine, but nothing seems to work when deployed on AWS Elastic Beanstalk.

I have a file logger.py that creates and configures the logger that looks like this:

import logging
import os

if not os.path.isdir(os.environ.get('LOG_PATH', 'log')):
    os.mkdir(os.environ.get('LOG_PATH', 'log'))

# Configure logger
logger = logging.getLogger(__name__)
formatter = logging.Formatter('[%(asctime)s] %(levelname)s in %(module)s: %(message)s')

debug_file_handler = logging.FileHandler(f'{os.environ.get("LOG_PATH", "log")}/debug.log')
info_file_handler = logging.FileHandler(f'{os.environ.get("LOG_PATH", "log")}/info.log')
error_file_handler = logging.FileHandler(f'{os.environ.get("LOG_PATH", "log")}/error.log')

debug_file_handler.setLevel(logging.DEBUG)
info_file_handler.setLevel(logging.INFO)
error_file_handler.setLevel(logging.ERROR)

debug_file_handler.setFormatter(formatter)
info_file_handler.setFormatter(formatter)
error_file_handler.setFormatter(formatter)

logger.addHandler(debug_file_handler)
logger.addHandler(info_file_handler)
logger.addHandler(error_file_handler)

I import this file int my app factory and add the handlers to my app.

# Configure logger
app.logger.removeHandler(default_handler)
app.logger.addHandler(debug_file_handler)
app.logger.addHandler(info_file_handler)
app.logger.addHandler(error_file_handler)

Then in the app factory still, I have a @app.before_request so that logging should occur on every request (for testing purposes).

@app.before_request
def before_request():
    elif request.method == 'GET':
        app.logger.debug(f'{request.method} {request.base_url}: parameters {dict(request.args)}')
    else:
        app.logger.debug(f'{request.method} {request.base_url}: parameters {request.json}')

This all works fine on my local machine without a problem. However when I try to deploy to AWS EB, nothing is written to the log files. The log files appear to be created and present when I pull them down via the cli or gui, but nothing is being written to it. I've followed a few tutorials and have added to my .ebextensions file but am still having no luck. Currently my .ebextensions includes a logging.config that looks like this:

files: 
  "/opt/elasticbeanstalk/tasks/bundlelogs.d/applogs.conf" :
    mode: "000777"
    owner: root
    group: root
    content: |
      /tmp/*.log
  "/opt/elasticbeanstalk/tasks/taillogs.d/applogs.conf" :
    mode: "000777"
    owner: root
    group: root
    content: |
      /tmp/*.log

The log files appear to be included when I request the logs (both tail and full), but the logs are completely empty even after making multiple request. I've also seen a few answers on here saying I need to include chown wsgi:wsgi in this file, but that has not worked either. I've also tried writing the logs to var/app/current/log and that doesn't work either. The log files are always created, but not written to.

For context, the Elastic Beanstalk platform is Python 3.8 running on Amazon Linux 2.

Any help would be greatly appreciated.

Answer 1

Turns out all I needed to do was set the level of the logger to logging.DEBUG so I could see debug logs.

in __init__.py:

app.logger.setLevel(logging.DEBUG)

And since my app name is src when I configure it, I can grab this logger in other modules with:

import logging
logger = logging.getLogger('src')