Log management and rotation in Jenkins - Step-by-Step Execution

Process Flow - Log management and rotation

Jenkins generates logs

↓

Log file grows

↓

Check log size or age

↓

Rotate logs

↓

Archive old logs

↓

Delete logs older than retention

↓

New log file created

↓

Repeat cycle

Jenkins writes logs continuously. When logs get too big or old, Jenkins rotates them by archiving and starting fresh logs, then deletes old archives based on retention.

Execution Sample

Jenkins

pipeline {
  agent any
  options {
    buildDiscarder(logRotator(numToKeepStr: '3'))
  }
  stages {
    stage('Example') { steps { echo 'Hello' } }
  }
}

This Jenkins pipeline configures log rotation to keep only the last 3 builds' logs.

Process Table

Step	Action	Log Size or Age Check	Rotation Triggered	Result
1	Build #1 runs	Log size < limit	No	Log appended, no rotation
2	Build #2 runs	Log size < limit	No	Log appended, no rotation
3	Build #3 runs	Log size < limit	No	Log appended, no rotation
4	Build #4 runs	Log size > limit	Yes	Old logs archived, oldest deleted, new log file created
5	Build #5 runs	Log size < limit	No	Log appended, no rotation

💡 Rotation stops when logs are within size limits and retention count is maintained

Status Tracker

Variable	Start	After Build #1	After Build #2	After Build #3	After Build #4	After Build #5
Log Size	0KB	50KB	100KB	150KB	50KB (rotated)	100KB
Archived Logs Count	0	0	0	0	1	1
Current Log File	log-0.txt	log-0.txt	log-0.txt	log-0.txt	log-1.txt	log-1.txt

Key Moments - 3 Insights

Why does the log size drop after Build #4?

Because at Build #4, log rotation happens (see execution_table row 4), old logs are archived and a new log file starts fresh, reducing current log size.

What triggers the log rotation?

The log rotation triggers when the log size exceeds the configured limit or the number of builds exceeds retention (execution_table row 4).

Why are some old logs deleted after rotation?

To keep only a set number of archived logs as per retention policy, oldest logs are deleted to save space (see variable_tracker Archived Logs Count).

Visual Quiz - 3 Questions

Look at the execution table, what is the log size after Build #3?

A 100KB

B 150KB

C 50KB

D 0KB

💡 Hint

Check the 'Log Size' column in variable_tracker after Build #3

At which build does log rotation occur?

A Build #2

B Build #3

C Build #4

D Build #5

💡 Hint

See execution_table row where 'Rotation Triggered' is Yes

If retention is set to keep 5 builds instead of 3, what changes in the execution table?

A Rotation triggers later, after Build #6

B Rotation triggers earlier, after Build #2

C Rotation never triggers

D Rotation triggers at Build #4 as before

💡 Hint

Retention controls when rotation triggers based on number of builds kept

Concept Snapshot

Jenkins log rotation keeps logs manageable.
Configure with buildDiscarder(logRotator).
Set number of builds or days to keep.
When logs exceed limits, Jenkins archives old logs.
Oldest logs deleted to save space.
New log files start fresh after rotation.

Full Transcript

Jenkins creates logs for each build. As logs grow, Jenkins checks if they exceed size or age limits. If yes, Jenkins rotates logs by archiving old ones and deleting the oldest beyond retention. This keeps logs from using too much space. The pipeline example uses buildDiscarder with logRotator to keep only the last 3 builds' logs. The execution table shows builds 1 to 5, with rotation happening at build 4 when logs exceed size. Variables track log size, archived logs count, and current log file name. Key moments clarify why log size drops after rotation and what triggers rotation. The quiz tests understanding of log size, rotation timing, and retention effects.

Practice

1. What is the main purpose of log rotation in Jenkins?

easy

A. To move old logs and save disk space

B. To delete all Jenkins jobs automatically

C. To increase Jenkins build speed

D. To backup Jenkins configuration files

Solution

1. Step 1: Understand what logs do in Jenkins

   Logs record Jenkins activity and can grow large over time.

2. Step 2: Purpose of log rotation

   Log rotation moves or archives old logs to save disk space and keep the system tidy.

3. Final Answer:

   To move old logs and save disk space -> Option A

4. Quick Check:

   Log rotation = save space [OK]

Hint: Log rotation helps keep disk space free by moving old logs [OK]

Common Mistakes:

• Thinking log rotation deletes Jenkins jobs
• Confusing log rotation with backup
• Believing log rotation speeds up builds

2. Which of the following is the correct syntax to configure log rotation using copytruncate in a Linux logrotate config for Jenkins logs?

easy

A. /var/log/jenkins/jenkins.log {
truncatecopy
weekly
rotate 5
}

B. /var/log/jenkins/jenkins.log {
copytruncate
monthly
rotate seven
}

C. /var/log/jenkins/jenkins.log {
copytruncate
daily
rotate 7
}

D. /var/log/jenkins/jenkins.log {
copytruncate
daily
rotate seven
}

Solution

1. Step 1: Identify correct logrotate syntax

   The copytruncate option is valid and used to copy then truncate the log file without stopping Jenkins.

2. Step 2: Check rotation frequency and count

   Using daily and rotate 7 is correct syntax for daily rotation keeping 7 files.

3. Final Answer:

   Correct syntax with copytruncate, daily rotation, and rotate 7 -> Option C

4. Quick Check:

   copytruncate + daily + rotate 7 = correct [OK]

Hint: copytruncate keeps Jenkins running during log rotation [OK]

Common Mistakes:

• Using 'truncatecopy' instead of 'copytruncate'
• Writing 'rotate seven' instead of 'rotate 7'
• Using invalid rotation frequency like 'monthly' with wrong number

3. Given this logrotate config snippet for Jenkins logs:

/var/log/jenkins/jenkins.log {
    copytruncate
    weekly
    rotate 4
}

What will happen after 5 weeks of running this rotation?

medium

A. Jenkins service will stop during log rotation

B. All 5 weeks of logs are kept without deletion

C. Logs are deleted every week without keeping any backups

D. Only the latest 4 weekly logs are kept; older logs are deleted

Solution

1. Step 1: Understand rotate count meaning

   Rotate 4 means keep 4 old log files; older ones get deleted.

2. Step 2: Apply to 5 weeks

   After 5 weeks, the oldest log (week 1) is deleted, keeping only weeks 2-5.

3. Final Answer:

   Only the latest 4 weekly logs are kept; older logs are deleted -> Option D

4. Quick Check:

   rotate 4 means keep 4 logs [OK]

Hint: Rotate number = how many old logs to keep [OK]

Common Mistakes:

• Thinking all logs are kept forever
• Assuming logs delete immediately without backups
• Believing Jenkins stops during copytruncate rotation

4. You configured Jenkins log rotation with copytruncate, but logs keep growing without rotation. What is the most likely cause?

medium

A. The Jenkins service is stopped

B. The logrotate service is not running or not scheduled

C. The copytruncate option is misspelled as copytrunc

D. The log file path is incorrect in the config

Solution

1. Step 1: Check if logrotate runs

   If logrotate is not running or scheduled, logs won't rotate despite correct config.

2. Step 2: Consider other causes

   Misspelling or wrong path cause errors, but usually logs won't grow if Jenkins stopped.

3. Final Answer:

   The logrotate service is not running or not scheduled -> Option B

4. Quick Check:

   Logrotate must run to rotate logs [OK]

Hint: Ensure logrotate runs regularly to rotate logs [OK]

Common Mistakes:

• Assuming Jenkins must be stopped for rotation
• Ignoring logrotate service status
• Overlooking typos in config file

5. You want to rotate Jenkins logs daily without stopping Jenkins and keep logs for 10 days. Which logrotate config snippet achieves this safely?

hard

A. /var/log/jenkins/jenkins.log {
copytruncate
daily
rotate 10
missingok
notifempty
}

B. /var/log/jenkins/jenkins.log {
delaycompress
weekly
rotate 10
}

C. /var/log/jenkins/jenkins.log {
copytruncate
monthly
rotate 10
}

D. /var/log/jenkins/jenkins.log {
compress
daily
rotate 5
}

Solution

1. Step 1: Identify safe rotation without stopping Jenkins

   copytruncate copies and truncates logs safely without stopping Jenkins.

2. Step 2: Check rotation frequency and retention

   Daily rotation with rotate 10 keeps 10 days of logs. missingok and notifempty handle missing or empty logs safely.

3. Final Answer:

   Config with copytruncate, daily rotation, rotate 10, missingok, notifempty -> Option A

4. Quick Check:

   copytruncate + daily + rotate 10 = safe rotation [OK]

Hint: Use copytruncate for safe daily rotation without stopping Jenkins [OK]

Common Mistakes:

• Using monthly or weekly instead of daily rotation
• Not using copytruncate causing Jenkins stop
• Setting rotate less than 10 days