How to design logger system

LldHow-ToBeginner · 4 min read

How to Design a Logger System: Simple and Scalable Approach

To design a logger system, create a component that captures and stores log messages with different severity levels using loggers, handlers, and formatters. Ensure it supports asynchronous writing, log rotation, and configurable output destinations for scalability and reliability.

📐

Syntax

A logger system typically has these parts:

Logger: The main interface to send log messages.
Handler: Sends logs to destinations like files or consoles.
Formatter: Defines how log messages look.
Level: Severity of logs (e.g., DEBUG, INFO, ERROR).

These parts work together to capture, format, and store logs efficiently.

python

class Logger:
    def __init__(self, level):
        self.level = level
        self.handlers = []

    def add_handler(self, handler):
        self.handlers.append(handler)

    def log(self, level, message):
        if level >= self.level:
            for handler in self.handlers:
                handler.emit(level, message)

class Handler:
    def emit(self, level, message):
        pass

class ConsoleHandler(Handler):
    def emit(self, level, message):
        print(f"[{level}] {message}")

💻

Example

This example shows a simple logger that prints messages to the console with levels and formats.

python

class Logger:
    LEVELS = {"DEBUG": 10, "INFO": 20, "WARNING": 30, "ERROR": 40}

    def __init__(self, level="DEBUG"):
        self.level = self.LEVELS[level]
        self.handlers = []

    def add_handler(self, handler):
        self.handlers.append(handler)

    def log(self, level, message):
        if self.LEVELS[level] >= self.level:
            for handler in self.handlers:
                handler.emit(level, message)

class ConsoleHandler:
    def emit(self, level, message):
        print(f"[{level}] - {message}")

# Usage
logger = Logger(level="INFO")
console_handler = ConsoleHandler()
logger.add_handler(console_handler)

logger.log("DEBUG", "This is a debug message")
logger.log("INFO", "This is an info message")
logger.log("ERROR", "This is an error message")

Output

[INFO] - This is an info message [ERROR] - This is an error message

⚠️

Common Pitfalls

Common mistakes when designing a logger system include:

Logging synchronously in main threads causing slowdowns.
Not handling log file rotation leading to huge files.
Ignoring log levels and flooding logs with unnecessary data.
Hardcoding output destinations instead of making them configurable.

Always design for asynchronous logging and flexible configuration.

python

import threading

class SyncLogger:
    def log(self, message):
        # This blocks main thread
        with open('log.txt', 'a') as f:
            f.write(message + '\n')

class AsyncLogger:
    def __init__(self):
        self.lock = threading.Lock()

    def log(self, message):
        def write():
            with self.lock:
                with open('log.txt', 'a') as f:
                    f.write(message + '\n')
        threading.Thread(target=write).start()

📊

Quick Reference

Logger system design tips:

Use levels to filter important logs.
Separate loggers, handlers, and formatters for flexibility.
Implement asynchronous logging to avoid blocking.
Support log rotation to manage file sizes.
Make output destinations configurable (files, console, remote servers).

✅

Key Takeaways

Design logger systems with clear separation of logger, handler, and formatter components.

Use log levels to control the importance and volume of logged messages.

Implement asynchronous logging to prevent performance bottlenecks.

Support log rotation and configurable outputs for scalability and maintenance.

Avoid hardcoding and make the logger flexible for different environments.