The Solution: Applying Log Filtering
- Step 1: Define a Filter Class
- Step 2: Add the Filter to the Logger
Conclusion

Managing logs effectively is crucial when running a FastAPI application with Uvicorn. By default, Uvicorn generates access logs for every incoming request, which can quickly accumulate and potentially impact log readability.

A question was raised on Stack Overflow about a logging issue faced by a user working with a FastAPI service deployed on AWS ECS, specifically logging to Cloudwatch. The user expressed concern about the logs being filled with "200 OK" entries whenever the ALB (Application Load Balancer) hit the /health endpoint. Their goal was to maintain a clean log stream and prioritize essential log entries, which prompted them to seek a solution. The original question can be found here.

In this blog post, we will address this problem and explore how to apply log filtering in a FastAPI application to avoid unnecessary log entries. We will provide a solution that allows you to filter out specific log entries, such as those generated by health endpoint checks, while retaining important log information. By implementing this solution, you'll be able to improve the log readability of your FastAPI applications and streamline the troubleshooting process.

The Solution: Applying Log Filtering

To filter out specific log entries in a FastAPI application running with Uvicorn, we can leverage the logging capabilities in Python. By applying log filtering, we can selectively exclude certain log entries based on predefined criteria. Let's walk through the steps to implement log filtering in your FastAPI application.

Step 1: Define a Filter Class

The first step is to define a filter class that will be responsible for filtering out unwanted log entries. Create a Python file (e.g. log_filters.py) and add the following code:

import logging

class EndpointFilter(logging.Filter):
    """Filter class to exclude specific endpoints from log entries."""

    def __init__(self, excluded_endpoints: list[str]) -> None:
        """
        Initialize the EndpointFilter class.

        Args:
            excluded_endpoints: A list of endpoints to be excluded from log entries.
        """
        self.excluded_endpoints = excluded_endpoints

    def filter(self, record: logging.LogRecord) -> bool:
        """
        Filter out log entries for excluded endpoints.

        Args:
            record: The log record to be filtered.

        Returns:
            bool: True if the log entry should be included, False otherwise.
        """
        return record.args and len(record.args) >= 3 and record.args[2] not in self.excluded_endpoints

The EndpointFilter class above takes a list of excluded endpoints as a parameter. It filters out log entries for any endpoint specified in the excluded_endpoints list.

Step 2: Add the Filter to the Logger

In your main application file (e.g. main.py), import the filter class from the previously created file and add the filter to the desired logger. Here's an example of how to apply the filter to the uvicorn.access logger:

import logging
from fastapi import FastAPI
from log_filters import EndpointFilter

app = FastAPI()

# Define excluded endpoints
excluded_endpoints = ["/health"]

# Add filter to the logger
logging.getLogger("uvicorn.access").addFilter(EndpointFilter(excluded_endpoints))

# Define the API endpoints
@app.get('/health')
def health():
    """The health endpoint of the application."""
    print('health endpoint')

@app.get('/test')
def test():
    """A test endpoint of the application."""
    print('test endpoint')

In the example above, we've defined /health as the excluded endpoint. As a result, log entries for the /health endpoint will be filtered out by the EndpointFilter. This allows you to focus on other relevant log entries and avoid unnecessary noise in your log stream.

When we run the example application and send a request to the /test endpoint, we observe that an access log entry is generated along with the function's output. However, for the /health endpoint, we notice that there is no access log entry recorded; only the function's output is displayed.

INFO:     Started server process [11272]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
test endpoint
INFO:     127.0.0.1:54014 - "GET /test HTTP/1.1" 200 OK
health endpoint

Conclusion

In this blog post, we learned how to avoid excessive access logs for specific endpoints when running a FastAPI application with Uvicorn. By leveraging the logging capabilities and filters provided by the Python logging module, we can selectively filter out access logs for the /health endpoint.

Remember to consider your application's logging requirements and strike a balance between log verbosity and the information needed for effective monitoring and debugging.

By understanding and utilizing the logging capabilities in Python and FastAPI, you can fine-tune your log output and ensure it provides meaningful information without unnecessary noise.

If you have any questions or would like to discuss this topic further, feel free to connect with me on LinkedIn. I would love to hear your thoughts and continue the conversation!

Happy logging!