Question 95477 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

How to configure FastAPI to publish logs to CloudWatch? – Amazon web services

I have a FastAPI service that works as expected in every regard except the logging, only when it runs as a AWS Lambda function.

When running it locally the logs are displayed on the console as expected:

INFO:     127.0.0.1:62160 - "POST /api/v1/feature-requests/febbbc21-9650-44e6-8df5-80c8bb33b6ea/upvote HTTP/1.1" 200 OK
INFO:     127.0.0.1:62158 - "OPTIONS /api/v1/feature-requests HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests HTTP/1.1" 200 OK
INFO:     127.0.0.1:62158 - "OPTIONS /api/v1/feature-requests-meta HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests-meta HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests/febbbc21-9650-44e6-8df5-80c8bb33b6ea HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests-meta/febbbc21-9650-44e6-8df5-80c8bb33b6ea HTTP/1.1" 200 OK

However, when deployed as a Lambda function the logs are not there:

2022-09-07T10:44:57.426+02:00   START RequestId: fd44ae47-5bfb-42e3-aeb4-d9f29857bb39 Version: $LATEST
2022-09-07T10:44:57.604+02:00   END RequestId: fd44ae47-5bfb-42e3-aeb4-d9f29857bb39
2022-09-07T10:44:57.604+02:00   REPORT RequestId: fd44ae47-5bfb-42e3-aeb4-d9f29857bb39 Duration: 177.85 ms Billed Duration: 178 ms Memory Size: 2048 MB Max Memory Used: 152 MB Init Duration: 1733.88 ms
2022-09-07T10:45:00.299+02:00   START RequestId: 08a7a6da-c2c6-446c-baa3-1d08c9816f5b Version: $LATEST
2022-09-07T10:45:00.318+02:00   END RequestId: 08a7a6da-c2c6-446c-baa3-1d08c9816f5b

Even for the logs that are produced by our code (as opposed to the framework) are not visible when running as a Lambda function.

Configuration:

In app.py

LOG = logging.getLogger()
log_format = "%(asctime)s %(levelname)s %(message)s"
log_date_fmt = "%Y-%m-%d %H:%M:%S"
logging.basicConfig(
    format=log_format,
    level=logging.INFO,
    datefmt=log_date_fmt,
)

In every other Python file:

LOG = logging.getLogger(__name__)

logging.conf

[loggers]
keys=root,api,config

[handlers]
keys=console_handler

[formatters]
keys=normal_formatter

[logger_root]
level=INFO
handlers=console_handler

[logger_api]
level=INFO
handlers=console_handler
qualname=api
propagate=0

[logger_config]
level=INFO
handlers=console_handler
qualname=config
propagate=0

[handler_console_handler]
class=StreamHandler
level=INFO
formatter=normal_formatter
args=(sys.stdout,)

[formatter_normal_formatter]
format=%(asctime)s %(levelname)s %(name)s %(message)s
datefmt=%Y-%m-%d %H:%M:%S

I am not sure what else needs to happen to get the logs in CloudWatch.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    As it turns out the following setup is needed:

    On the top of the logging.conf above uvicorn has to be imported that creates an extra property on logging and than fileConfig has to be used like this:

    import uvicorn
logging.config.fileConfig("logging.conf", disable_existing_loggers=False)
LOG = logging.getLogger(__name__)

  2. 0

    try cloudwatch library

    Run pip install cloudwatch in the console

    then in your code:

    import logging
from cloudwatch import cloudwatch

logger = logging.getLogger('cloudwatch_logger')
formatter = logging.Formatter('%(asctime)s : %(levelname)s - %(message)s')
handler = cloudwatch.CloudwatchHandler(log_group = 'cloudwatch_log_group')
handler.setFormatter(formatter)
logger.setLevel(logging.DEBUG)
logger.addHandler(handler)

    and then either use this logger to log to CloudWatch
    logger.warning('I am here') or add the handler to the root logger via logging configuration

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search