Docker compose in pipelines in Jenkins - Step-by-Step Execution

Process Flow - Docker compose in pipelines

Start Jenkins Pipeline

↓

Checkout Code

↓

Run 'docker-compose up'

↓

Services Start

↓

Run Tests or Build Steps

↓

Run 'docker-compose down'

↓

Cleanup and Finish Pipeline

This flow shows how a Jenkins pipeline uses docker-compose to start services, run tests, then clean up.

Execution Sample

Jenkins

pipeline {
  agent any
  stages {
    stage('Start Services') {
      steps { sh 'docker-compose up -d' }
    }
    stage('Test') {
      steps { sh './run-tests.sh' }
    }
    stage('Cleanup') {
      steps { sh 'docker-compose down' }
    }
  }
}

A Jenkins pipeline that starts services with docker-compose, runs tests, then stops services.

Process Table

Step	Action	Command Run	Result	Pipeline State
1	Checkout Code	git clone repo	Code ready	Ready to start services
2	Start Services	docker-compose up -d	Containers started	Services running
3	Run Tests	run-tests.sh	Tests passed	Testing
4	Cleanup	docker-compose down	Containers stopped	Services stopped
5	Pipeline End	N/A	Pipeline success	Finished

💡 Pipeline ends after cleanup stage stops all docker-compose services.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
Code	Not checked out	Checked out	Checked out	Checked out	Checked out
Services	Stopped	Running	Running	Stopped	Stopped
Tests	Not run	Not run	Passed	Passed	Passed
Pipeline State	Idle	Running services	Testing	Services stopped	Finished

Key Moments - 3 Insights

Why do we run 'docker-compose up' before tests?

Because the tests need the services running to connect to them. See execution_table step 2 and 3 where services start before tests run.

What happens if we skip 'docker-compose down'?

Containers keep running and may cause conflicts later. The cleanup step (step 4) stops containers to keep the environment clean.

Why use '-d' flag with 'docker-compose up'?

The '-d' runs containers in the background so the pipeline can continue to the next steps without waiting.

Visual Quiz - 3 Questions

Look at the execution_table, what is the pipeline state after step 3?

A Services running

B Cleaning up

C Testing

D Finished

💡 Hint

Check the 'Pipeline State' column in execution_table row for step 3.

At which step are the docker containers stopped?

A Step 4

B Step 3

C Step 2

D Step 5

💡 Hint

Look for 'Containers stopped' in the 'Result' column of execution_table.

If we remove '-d' from 'docker-compose up', what changes in the pipeline?

A Containers start in background as usual

B Pipeline waits until containers stop before next step

C Tests run before containers start

D Pipeline fails immediately

💡 Hint

Recall that '-d' runs containers detached; without it, the command blocks.

Concept Snapshot

Jenkins pipeline with docker-compose:
- Use 'docker-compose up -d' to start services in background
- Run tests after services are ready
- Use 'docker-compose down' to stop and clean containers
- Keeps pipeline clean and services isolated
- Order matters: start -> test -> cleanup

Full Transcript

This visual execution shows how a Jenkins pipeline uses docker-compose commands step-by-step. First, the pipeline checks out the code. Then it runs 'docker-compose up -d' to start services in the background. After services are running, tests execute. Finally, 'docker-compose down' stops the containers to clean up. Variables like services state and pipeline state update at each step. Key points include why '-d' is used to run containers detached and why cleanup is important to avoid leftover containers. The quiz questions check understanding of pipeline states and command effects.

Practice

1. What is the main purpose of using docker-compose in Jenkins pipelines?

easy

A. To manage multiple containers together easily

B. To write Jenkins pipeline scripts

C. To build Java applications

D. To deploy Jenkins itself

Solution

1. Step 1: Understand Docker Compose role

   Docker Compose is designed to manage multiple containers as a group, making it easier to start, stop, and configure them together.

2. Step 2: Relate to Jenkins pipelines

   In Jenkins pipelines, Docker Compose helps run multiple related containers needed for testing or deployment in one step.

3. Final Answer:

   To manage multiple containers together easily -> Option A

4. Quick Check:

   Docker Compose = Manage multiple containers [OK]

Hint: Docker Compose groups containers for easy control [OK]

Common Mistakes:

• Confusing Docker Compose with Jenkins scripting
• Thinking it builds applications
• Assuming it deploys Jenkins itself

2. Which command correctly starts services defined in a docker-compose.yml file in detached mode within a Jenkins pipeline?

easy

A. docker-compose start -d

B. docker-compose up -d

C. docker-compose run -d

D. docker-compose exec -d

Solution

1. Step 1: Identify the command to start services

   The correct command to start all services in detached mode is docker-compose up -d. The -d flag runs containers in the background.

2. Step 2: Check other options

   docker-compose start starts existing containers but does not create new ones. docker-compose run runs a one-off container, not the whole set. docker-compose exec runs commands inside running containers, not start them.

3. Final Answer:

   docker-compose up -d -> Option B

4. Quick Check:

   Start services detached = up -d [OK]

Hint: Use 'up -d' to start all services in background [OK]

Common Mistakes:

• Using 'start -d' which is invalid
• Confusing 'run' with 'up'
• Using 'exec' to start containers

3. Given this Jenkins pipeline snippet, what will be the output of the sh step?

stage('Test') {
  steps {
    sh 'docker-compose up -d'
    sh 'docker-compose exec web curl -s http://localhost'
    sh 'docker-compose down'
  }
}

medium

A. Starts containers, runs curl inside 'web' container, then stops containers

B. Only starts containers, no curl command runs

C. Runs curl on host machine, then stops containers

D. Fails because 'docker-compose exec' cannot run in pipeline

Solution

1. Step 1: Analyze each command in the pipeline

   docker-compose up -d starts all services in background. docker-compose exec web curl -s http://localhost runs curl inside the 'web' container. docker-compose down stops and removes containers.

2. Step 2: Understand the sequence and output

   The commands run in order: start containers, execute curl inside 'web', then stop containers. This is the expected behavior in a pipeline.

3. Final Answer:

   Starts containers, runs curl inside 'web' container, then stops containers -> Option A

4. Quick Check:

   Up, exec inside container, down = Starts containers, runs curl inside 'web' container, then stops containers [OK]

Hint: Exec runs commands inside containers after 'up' [OK]

Common Mistakes:

• Thinking exec runs on host, not inside container
• Assuming curl won't run
• Believing exec can't run in pipeline

4. You wrote this Jenkins pipeline step:

sh 'docker-compose exec web curl http://localhost'

But it fails with "No such container: web" error. What is the most likely cause?

medium

A. The docker-compose.yml file is missing

B. The curl command is incorrect

C. docker-compose exec cannot run curl

D. The 'web' container is not running yet

Solution

1. Step 1: Understand the error message

   "No such container: web" means the container named 'web' is not running or does not exist at the time of the command.

2. Step 2: Identify the cause in pipeline context

   If the container is not started before the exec command, this error occurs. The curl command or docker-compose.yml presence is unrelated to this specific error.

3. Final Answer:

   The 'web' container is not running yet -> Option D

4. Quick Check:

   Exec needs running container = The 'web' container is not running yet [OK]

Hint: Start containers before exec commands [OK]

Common Mistakes:

• Blaming curl command syntax
• Assuming exec can't run curl
• Ignoring container start step

5. You want to run integration tests inside a Jenkins pipeline using Docker Compose. The tests run inside a container named app. Which pipeline snippet correctly starts services, runs tests inside app, and then cleans up?

hard

A. sh 'docker-compose exec app ./run-tests.sh'
sh 'docker-compose up -d'
sh 'docker-compose down'

B. sh 'docker-compose run app ./run-tests.sh'
sh 'docker-compose up -d'
sh 'docker-compose down'

C. sh 'docker-compose up -d'
sh 'docker-compose exec app ./run-tests.sh'
sh 'docker-compose down'

D. sh 'docker-compose down'
sh 'docker-compose up -d'
sh 'docker-compose exec app ./run-tests.sh'

Solution

1. Step 1: Start services before running tests

   Services must be started with docker-compose up -d before running commands inside containers.

2. Step 2: Run tests inside the 'app' container

   Use docker-compose exec app ./run-tests.sh to execute tests inside the running 'app' container.

3. Step 3: Clean up containers after tests

   Finally, stop and remove containers with docker-compose down to free resources.

4. Final Answer:

   sh 'docker-compose up -d' sh 'docker-compose exec app ./run-tests.sh' sh 'docker-compose down' -> Option C

5. Quick Check:

   Up, exec test, down = sh 'docker-compose up -d' sh 'docker-compose exec app ./run-tests.sh' sh 'docker-compose down' [OK]

Hint: Always start containers before exec, then clean up [OK]

Common Mistakes:

• Running exec before starting containers
• Using run instead of exec for tests
• Stopping containers before tests run