How to Use Fluentd in Kubernetes for Log Collection

To use Fluentd in Kubernetes, deploy it as a DaemonSet to run on each node and configure it to collect logs from container files or system logs. Fluentd then forwards these logs to your chosen storage or analysis system like Elasticsearch or a cloud logging service.

Syntax

Fluentd is typically deployed in Kubernetes using a DaemonSet so it runs on every node. The key parts include:

• apiVersion, kind: Define the resource type (DaemonSet).
• metadata: Names and labels for the DaemonSet.
• spec: Defines the pod template and selector.
• containers: The Fluentd container image and configuration.
• volumeMounts and volumes: Mount log directories and config files inside the container.

This setup lets Fluentd access logs on each node and forward them as configured.

yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: fluentd
  template:
    metadata:
      labels:
        name: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd:v1.15-debian-1
        env:
        - name: FLUENTD_ARGS
          value: "-c /fluentd/etc/fluent.conf"
        volumeMounts:
        - name: config-volume
          mountPath: /fluentd/etc
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: config-volume
        configMap:
          name: fluentd-config
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

Example

This example shows a Fluentd DaemonSet that collects logs from Docker containers and forwards them to Elasticsearch. It mounts the host's log directories and uses a ConfigMap for Fluentd configuration.

yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-config
  namespace: kube-system
data:
  fluent.conf: |
    <source>
      @type tail
      path /var/lib/docker/containers/*/*.log
      pos_file /var/log/fluentd-containers.log.pos
      tag kubernetes.*
      format json
      time_key time
      time_format %Y-%m-%dT%H:%M:%S.%N%z
    </source>

    <match kubernetes.**>
      @type elasticsearch
      host elasticsearch.logging.svc.cluster.local
      port 9200
      logstash_format true
      include_tag_key true
      tag_key @log_name
      flush_interval 5s
    </match>
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: fluentd
  template:
    metadata:
      labels:
        name: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd:v1.15-debian-1
        env:
        - name: FLUENTD_ARGS
          value: "-c /fluentd/etc/fluent.conf"
        volumeMounts:
        - name: config-volume
          mountPath: /fluentd/etc
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: config-volume
        configMap:
          name: fluentd-config
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

Output

DaemonSet "fluentd" created
ConfigMap "fluentd-config" created
Fluentd pods start on each node, collecting container logs and forwarding to Elasticsearch.

Common Pitfalls

Common mistakes when using Fluentd in Kubernetes include:

• Not mounting the correct host log directories, so Fluentd cannot read logs.
• Incorrect Fluentd configuration syntax causing Fluentd to fail starting.
• Forgetting to create or properly reference the ConfigMap with Fluentd config.
• Using the latest image tag, which can cause unpredictable behavior.
• Not setting proper permissions on log files or volumes.

Always check pod logs for Fluentd errors and validate your config with fluentd --dry-run locally.

yaml

Wrong example (missing volume mount):

containers:
- name: fluentd
  image: fluent/fluentd:v1.15-debian-1
  env:
  - name: FLUENTD_ARGS
    value: "-c /fluentd/etc/fluent.conf"
  volumeMounts:
  - name: config-volume
    mountPath: /fluentd/etc

Correct example (includes hostPath mounts):

containers:
- name: fluentd
  image: fluent/fluentd:v1.15-debian-1
  env:
  - name: FLUENTD_ARGS
    value: "-c /fluentd/etc/fluent.conf"
  volumeMounts:
  - name: config-volume
    mountPath: /fluentd/etc
  - name: varlog
    mountPath: /var/log
  - name: varlibdockercontainers
    mountPath: /var/lib/docker/containers
    readOnly: true

Quick Reference

Fluentd in Kubernetes Cheat Sheet:

• Deploy Fluentd as a DaemonSet for node-wide log collection.
• Mount host log directories like /var/log and /var/lib/docker/containers.
• Use a ConfigMap to manage Fluentd configuration files.
• Forward logs to storage or analysis backends (Elasticsearch, cloud logging).
• Check Fluentd pod logs for troubleshooting.

Key Takeaways

Deploy Fluentd as a DaemonSet to collect logs from all Kubernetes nodes.

Mount host log directories inside Fluentd pods to access container logs.

Use a ConfigMap to provide Fluentd configuration for log parsing and forwarding.

Avoid using the latest image tag; specify a stable Fluentd version.

Check Fluentd pod logs to troubleshoot configuration or permission issues.