Bird
Raised Fist0
HLDsystem_design~25 mins

Logging strategies in HLD - System Design Exercise

Choose your learning style10 modes available
LearnWhyDeepArchPracticeChallengeDesignRecallScale
Design: Logging System for Distributed Applications
Design focuses on log collection, storage, search, and alerting. Excludes detailed UI design and log analysis algorithms.
Functional Requirements
FR1: Collect logs from multiple services and servers
FR2: Support different log levels (info, warning, error, debug)
FR3: Allow searching and filtering logs by time, service, and level
FR4: Ensure logs are stored reliably and durably
FR5: Provide real-time monitoring and alerting on critical errors
FR6: Support high write throughput (up to 100,000 logs per second)
FR7: Allow log retention policies and archiving
Non-Functional Requirements
NFR1: System must handle 100K log entries per second
NFR2: Search queries should return results within 2 seconds
NFR3: System availability must be 99.9%
NFR4: Logs must be stored for at least 30 days before archiving
NFR5: Latency for log ingestion should be under 500ms
Think Before You Design
Questions to Ask
❓ Question 1
❓ Question 2
❓ Question 3
❓ Question 4
❓ Question 5
❓ Question 6
Key Components
Log collectors/agents on servers
Message queue or streaming platform for log transport
Centralized log storage (e.g., Elasticsearch, cloud storage)
Indexing and search engine
Alerting and monitoring system
Log archiving and retention manager
Design Patterns
Log aggregation pattern
Event streaming with backpressure handling
Tiered storage for hot and cold logs
Circuit breaker for log ingestion failures
Structured logging and log enrichment
Reference Architecture
  +-------------+       +----------------+       +-------------------+       +----------------+
  | Application | ----> | Log Collector  | ----> | Message Queue     | ----> | Log Storage    |
  | Services    |       | (Agent)        |       | (Kafka/RabbitMQ)  |       | (Elasticsearch) |
  +-------------+       +----------------+       +-------------------+       +----------------+
                                                                                  |
                                                                                  v
                                                                           +----------------+
                                                                           | Alerting &     |
                                                                           | Monitoring     |
                                                                           +----------------+
Components
Log Collector (Agent)
Fluentd, Logstash, or custom agent
Collect logs from applications and forward them reliably
Message Queue
Apache Kafka or RabbitMQ
Buffer and transport logs asynchronously to storage
Log Storage
Elasticsearch or similar search engine
Store, index, and allow fast search of logs
Alerting & Monitoring
Prometheus + Alertmanager or custom system
Monitor logs for critical errors and send alerts
Retention Manager
Custom scripts or lifecycle policies
Manage log retention and archiving to cheaper storage
Request Flow
1. 1. Application services generate logs with structured format and levels.
2. 2. Log collectors running on servers capture logs and batch them.
3. 3. Logs are sent asynchronously to a message queue to handle bursts.
4. 4. Consumers read logs from the queue and index them into Elasticsearch.
5. 5. Users query Elasticsearch to search and filter logs by criteria.
6. 6. Alerting system monitors logs for error patterns and triggers notifications.
7. 7. Retention manager deletes or archives logs older than retention period.
Database Schema
Entities: LogEntry(id, timestamp, service_name, log_level, message, metadata_json), Service(id, name, owner), Alert(id, condition, severity, status). Relationships: LogEntry linked to Service by service_name; Alerts configured on LogEntry patterns.
Scaling Discussion
Bottlenecks
Message queue throughput limits under high log volume
Storage capacity and indexing speed in Elasticsearch
Search query latency with large datasets
Alerting system overload with many error events
Solutions
Partition message queue topics and scale consumers horizontally
Use tiered storage: hot nodes for recent logs, cold storage for older logs
Implement query caching and limit query scope for faster responses
Use sampling and rate limiting in alerting to reduce noise
Interview Tips
Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.
Emphasize reliability and durability of log collection
Discuss asynchronous log transport to handle bursts
Explain indexing and search for fast log retrieval
Highlight alerting importance for operational monitoring
Address scaling challenges and mitigation strategies

Practice

(1/5)
1. What is the main purpose of using different log levels like INFO, WARN, and ERROR in logging strategies?
easy
A. To categorize logs by importance and help filter relevant information
B. To make logs look colorful and attractive
C. To increase the size of log files for better storage
D. To encrypt log messages for security

Solution

  1. Step 1: Understand log levels

    Log levels like INFO, WARN, and ERROR help mark how important or severe a log message is.

  2. Step 2: Purpose of categorizing logs

    This categorization helps users quickly find important messages and ignore less critical ones.

  3. Final Answer:

    To categorize logs by importance and help filter relevant information -> Option A

  4. Quick Check:

    Log levels = Categorize importance [OK]
Hint: Log levels help sort messages by importance fast [OK]
Common Mistakes:
  • Thinking log levels are for decoration
  • Assuming log levels increase file size intentionally
  • Confusing log levels with encryption
2. Which of the following is the correct syntax to configure a log message format in a typical logging configuration file?
easy
A. format: asctime, levelname, message
B. format = "%(asctime)s - %(levelname)s - %(message)s"
C. format -> asctime | levelname | message
D. format = asctime + levelname + message

Solution

  1. Step 1: Recognize standard log format syntax

    Typical logging uses placeholders like %(asctime)s for timestamp, %(levelname)s for log level, and %(message)s for the log text.

  2. Step 2: Match correct syntax

    format = "%(asctime)s - %(levelname)s - %(message)s" uses the correct placeholder format with percent signs and parentheses, matching common logging config files.

  3. Final Answer:

    format = "%(asctime)s - %(levelname)s - %(message)s" -> Option B

  4. Quick Check:

    Correct placeholders = format = "%(asctime)s - %(levelname)s - %(message)s" [OK]
Hint: Look for %()s placeholders in format strings [OK]
Common Mistakes:
  • Using commas or arrows instead of placeholders
  • Missing percent signs and parentheses
  • Concatenating strings without placeholders
3. Given this log configuration snippet:
log_level = "WARN"
log_destination = "file"
What will happen if the system logs an INFO message?
medium
A. The INFO message will be saved to the file
B. The system will crash due to invalid log level
C. The INFO message will be printed to the console only
D. The INFO message will be ignored and not saved

Solution

  1. Step 1: Understand log level filtering

    Setting log_level to WARN means only WARN and ERROR messages (higher importance) are saved; INFO is lower priority.

  2. Step 2: Effect on INFO messages

    INFO messages are below WARN level, so they will be ignored and not saved to the file.

  3. Final Answer:

    The INFO message will be ignored and not saved -> Option D

  4. Quick Check:

    Log level filter blocks lower priority logs [OK]
Hint: Logs below set level are ignored [OK]
Common Mistakes:
  • Assuming all logs are saved regardless of level
  • Thinking INFO logs print to console automatically
  • Believing invalid log level causes crash
4. You have a logging setup that writes logs to a file, but the file remains empty even after errors occur. Which is the most likely cause?
medium
A. The log level is set too high, filtering out error messages
B. The logging format string is missing placeholders
C. The log file path is incorrect or lacks write permission
D. The system does not support file logging

Solution

  1. Step 1: Check log level filtering

    Error messages are high priority, so a high log level would not filter them out; this is unlikely the cause.

  2. Step 2: Verify file path and permissions

    If the file path is wrong or the system user cannot write to the file, logs won't be saved, causing an empty file.

  3. Final Answer:

    The log file path is incorrect or lacks write permission -> Option C

  4. Quick Check:

    File permission issues cause empty logs [OK]
Hint: Check file path and permissions first [OK]
Common Mistakes:
  • Blaming log level for missing error logs
  • Ignoring file system permissions
  • Assuming format string affects file writing
5. You want to design a logging strategy for a web app that must:
- Capture all errors and warnings
- Save detailed info logs only during debugging
- Send critical errors to an external alert system

Which combination of strategies best meets these needs?
hard
A. Use log level WARN for normal runs, switch to INFO for debugging, and configure alerts for ERROR logs
B. Always use log level INFO and save all logs to a single file without alerts
C. Use log level ERROR only and ignore INFO and WARN logs
D. Send all logs to the alert system regardless of level

Solution

  1. Step 1: Match log levels to requirements

    WARN level captures warnings and errors; INFO level includes detailed info logs for debugging.

  2. Step 2: Configure alerting for critical errors

    Setting alerts on ERROR logs ensures critical issues trigger notifications externally.

  3. Step 3: Evaluate options

    Use log level WARN for normal runs, switch to INFO for debugging, and configure alerts for ERROR logs matches all requirements: dynamic log level and alerting. Others miss one or more needs.

  4. Final Answer:

    Use log level WARN for normal runs, switch to INFO for debugging, and configure alerts for ERROR logs -> Option A

  5. Quick Check:

    Dynamic levels + alerts = Use log level WARN for normal runs, switch to INFO for debugging, and configure alerts for ERROR logs [OK]
Hint: Adjust log level by mode and alert on errors [OK]
Common Mistakes:
  • Logging everything all the time wastes resources
  • Ignoring alerting for critical errors
  • Using too high log level and missing info logs
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard