Centralized logging setup in Docker - Mini Project: Build & Apply

Centralized logging setup

📖 Scenario: You are managing multiple Docker containers running different applications. To keep track of all logs in one place, you want to set up a centralized logging system using Docker's built-in logging drivers.This helps you easily monitor and troubleshoot your applications without checking each container separately.

🎯 Goal: Build a simple Docker Compose setup with two containers that send their logs to a centralized logging container using the json-file logging driver. You will configure the logging options and verify the logs are collected centrally.

📋 What You'll Learn

Create a Docker Compose file with two application containers named app1 and app2.

Add a centralized logging container named logcollector.

Configure app1 and app2 to use the json-file logging driver with a max file size of 10m.

Print the logging configuration of app1 to verify the setup.

💡 Why This Matters

🌍 Real World

Centralized logging helps developers and operators monitor multiple containers easily, improving troubleshooting and system reliability.

💼 Career

Understanding Docker logging drivers and centralized log collection is essential for DevOps roles managing containerized applications.

1

Create Docker Compose services

Create a Docker Compose file named docker-compose.yml with two services: app1 and app2. Both should use the alpine image and run the command sh -c "while true; do echo Hello from app1; sleep 5; done" for app1 and sh -c "while true; do echo Hello from app2; sleep 5; done" for app2.

Docker

# Create docker-compose.yml with app1 and app2 services
# Your code here

Hint

Define two services under services: with the exact names app1 and app2. Use the alpine image and the given commands.

2

Add centralized logging container

Add a third service named logcollector to the docker-compose.yml. Use the busybox image and run the command sh -c "tail -f /var/log/app.log".

Docker

version: '3.8'
services:
  app1:
    image: alpine
    command: sh -c "while true; do echo Hello from app1; sleep 5; done"
  app2:
    image: alpine
    command: sh -c "while true; do echo Hello from app2; sleep 5; done"
# Add logcollector service below
# Your code here

Hint

Add a new service named logcollector with the busybox image and the specified tail command.

3

Configure logging driver for app containers

Configure both app1 and app2 services in docker-compose.yml to use the json-file logging driver with the option max-size: 10m.

Docker

version: '3.8'
services:
  app1:
    image: alpine
    command: sh -c "while true; do echo Hello from app1; sleep 5; done"
    # Add logging config here
  app2:
    image: alpine
    command: sh -c "while true; do echo Hello from app2; sleep 5; done"
    # Add logging config here
  logcollector:
    image: busybox
    command: sh -c "tail -f /var/log/app.log"

Hint

Under each app service, add a logging section with driver: json-file and options: max-size: 10m.

4

Print logging configuration of app1

Run the command docker inspect --format='{{json .HostConfig.LogConfig}}' $(docker-compose ps -q app1) in your terminal to print the logging configuration of the app1 container.

Docker

# Run the command below in terminal to print app1 logging config
# Your code here

Hint

Use docker inspect with --format='{{json .HostConfig.LogConfig}}' and the container ID of app1 from docker-compose ps -q app1.

Practice

1. What is the main purpose of using a centralized logging setup in Docker?

easy

A. To reduce the size of container images

B. To increase container startup speed

C. To collect logs from multiple containers in one place for easier monitoring

D. To automatically restart containers on failure

Solution

1. Step 1: Understand centralized logging concept

   Centralized logging means gathering logs from many containers into a single location.

2. Step 2: Identify the benefit

   This helps in easier monitoring and troubleshooting by having all logs accessible in one place.

3. Final Answer:

   To collect logs from multiple containers in one place for easier monitoring -> Option C

4. Quick Check:

   Centralized logging = collect logs centrally [OK]

Hint: Centralized logging means one place for all container logs [OK]

Common Mistakes:

• Confusing logging with container performance
• Thinking it reduces image size
• Assuming it restarts containers automatically

2. Which Docker command option correctly sets the log driver to Fluentd for a container?

easy

A. docker run --log-driver fluentd myimage

B. docker run --log-driver=fluentd myimage

C. docker run --log-driver:fluentd myimage

D. docker run --logdriver=fluentd myimage

Solution

1. Step 1: Recall Docker log driver syntax

   The correct syntax uses double dashes and equals sign: --log-driver=drivername

2. Step 2: Match the option

   docker run --log-driver=fluentd myimage uses --log-driver=fluentd which is correct syntax for setting Fluentd as log driver.

3. Final Answer:

   docker run --log-driver=fluentd myimage -> Option B

4. Quick Check:

   Correct syntax uses '=' after --log-driver [OK]

Hint: Use --log-driver=drivername with equals sign [OK]

Common Mistakes:

• Using space instead of '=' after --log-driver
• Using colon ':' instead of '='
• Misspelling option name

3. Given the command:

docker run --log-driver=fluentd --log-opt fluentd-address=localhost:24224 myapp

What will happen to the container logs?

medium

A. Logs will be sent to a Fluentd server running on localhost port 24224

B. Logs will be stored only inside the container

C. Logs will be sent to Docker's default json-file driver

D. Container will fail to start due to invalid log options

Solution

1. Step 1: Analyze log driver and options

   The command sets log driver to fluentd and specifies fluentd-address as localhost:24224.

2. Step 2: Understand effect on logs

   Logs will be forwarded to the Fluentd server at the given address instead of default storage.

3. Final Answer:

   Logs will be sent to a Fluentd server running on localhost port 24224 -> Option A

4. Quick Check:

   Fluentd driver + address = logs sent to Fluentd [OK]

Hint: Fluentd log driver sends logs to specified Fluentd address [OK]

Common Mistakes:

• Assuming logs stay inside container
• Thinking default json-file driver is used
• Believing the command causes failure

4. You tried to run:

docker run --log-driver=fluentd --log-opt fluentd-address=localhost:24224 myapp

but logs are not appearing in Fluentd. What is a likely cause?

medium

A. The container image 'myapp' does not support logging

B. The --log-driver option must be set after the image name

C. Docker does not support Fluentd as a log driver

D. Fluentd server is not running or unreachable at localhost:24224

Solution

1. Step 1: Check Fluentd server availability

   If Fluentd server is not running or unreachable at the specified address, logs won't be received.

2. Step 2: Verify Docker and options correctness

   Docker supports Fluentd log driver and the command syntax is correct, so these are unlikely causes.

3. Final Answer:

   Fluentd server is not running or unreachable at localhost:24224 -> Option D

4. Quick Check:

   Fluentd server must be running to receive logs [OK]

Hint: Check Fluentd server is running and reachable [OK]

Common Mistakes:

• Blaming container image for logging issues
• Thinking Docker lacks Fluentd support
• Misplacing --log-driver option

5. You want to centralize logs from multiple Docker containers using syslog. Which is the correct way to run a container to send logs to a syslog server at 192.168.1.100 port 514?

hard

A. docker run --log-driver=syslog --log-opt syslog-address=udp://192.168.1.100:514 myapp

B. docker run --log-driver=json-file --log-opt syslog-address=tcp://192.168.1.100:514 myapp

C. docker run --log-driver=fluentd --log-opt syslog-address=udp://192.168.1.100:514 myapp

D. docker run --log-driver=syslog --log-opt syslog-address=tcp://192.168.1.100:514 myapp

Solution

1. Step 1: Identify correct log driver for syslog

   Use --log-driver=syslog to send logs to a syslog server.

2. Step 2: Choose correct syslog protocol and address

   Syslog commonly uses UDP protocol, so syslog-address=udp://192.168.1.100:514 is correct.

3. Step 3: Verify options match

   docker run --log-driver=syslog --log-opt syslog-address=udp://192.168.1.100:514 myapp correctly sets syslog driver and UDP address; others use wrong driver or protocol.

4. Final Answer:

   docker run --log-driver=syslog --log-opt syslog-address=udp://192.168.1.100:514 myapp -> Option A

5. Quick Check:

   Syslog driver + UDP address = correct syslog logging [OK]

Hint: Syslog driver with udp:// address sends logs correctly [OK]

Common Mistakes:

• Using TCP instead of UDP for syslog
• Mixing Fluentd driver with syslog options
• Using json-file driver for centralized logging