GitLab CI with Docker - Deep Dive

Overview - GitLab CI with Docker

What is it?

GitLab CI with Docker is a way to automate software building, testing, and deployment using GitLab's continuous integration system combined with Docker containers. Docker packages applications and their environments into containers, making them easy to run anywhere. GitLab CI uses configuration files to define steps that run inside Docker containers, ensuring consistent and isolated environments for each task.

Why it matters

Without GitLab CI and Docker working together, developers face problems like inconsistent environments, manual testing, and slow deployment. This combination solves these by automating workflows and packaging everything needed to run software reliably. It saves time, reduces errors, and helps teams deliver software faster and more confidently.

Where it fits

Before learning this, you should understand basic GitLab usage and Docker container concepts. After mastering GitLab CI with Docker, you can explore advanced CI/CD pipelines, Kubernetes deployments, and multi-stage Docker builds for production-ready workflows.

Mental Model

Core Idea

GitLab CI runs automated tasks inside Docker containers to ensure consistent, repeatable, and isolated software workflows.

Think of it like...

It's like a kitchen where each recipe (CI job) is cooked inside its own clean, identical cooking pot (Docker container), so the taste never changes no matter who cooks it or where.

┌─────────────┐      ┌─────────────┐      ┌─────────────┐
│ GitLab Repo │─────▶│ GitLab CI   │─────▶│ Docker      │
│ (Source)    │      │ Runner      │      │ Container   │
└─────────────┘      └─────────────┘      └─────────────┘
       │                    │                   │
       │                    │                   │
       │                    │                   ▼
       │                    │           ┌─────────────┐
       │                    │           │ Build/Test  │
       │                    │           │ Environment │
       │                    │           └─────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding GitLab CI Basics

Concept: Learn what GitLab CI is and how it automates tasks using pipelines.

GitLab CI is a tool inside GitLab that runs jobs automatically when you push code. These jobs are defined in a file called .gitlab-ci.yml in your project. Each job can do things like build code, run tests, or deploy software. Pipelines are sequences of these jobs that run in order or in parallel.

Result

You know how to create a simple pipeline that runs a job when you push code.

Understanding GitLab CI basics is essential because it shows how automation replaces manual work, making software delivery faster and less error-prone.

2

Foundation

Introduction to Docker Containers

Concept: Learn what Docker containers are and why they matter for consistent environments.

Docker containers are like lightweight boxes that hold your application and everything it needs to run. This means your app works the same on your computer, a server, or anywhere else. Containers start quickly and isolate your app from other software, preventing conflicts.

Result

You understand how Docker packages apps and why containers help avoid 'it works on my machine' problems.

Knowing Docker containers helps you see why running CI jobs inside containers ensures consistency and reliability.

3

Intermediate

Configuring GitLab CI to Use Docker

🤔Before reading on: do you think GitLab CI runs jobs inside Docker containers by default or do you need to specify it? Commit to your answer.

Concept: Learn how to tell GitLab CI to run jobs inside Docker containers using the .gitlab-ci.yml file.

In your .gitlab-ci.yml file, you specify an 'image' keyword to tell GitLab CI which Docker image to use for the job. For example: image: python:3.10 This means the job runs inside a container based on the python:3.10 image. You can run commands inside this container as if you were inside a fresh machine with Python installed.

Result

Jobs run inside the specified Docker container, ensuring the environment matches the image.

Knowing how to specify Docker images in GitLab CI lets you control the environment precisely, avoiding surprises from different software versions.

4

Intermediate

Using Docker-in-Docker for Building Images

🤔Before reading on: do you think you can build Docker images inside a GitLab CI job without special setup? Commit to your answer.

Concept: Learn how to build Docker images inside GitLab CI jobs using Docker-in-Docker (DinD).

To build Docker images in GitLab CI, you need to run Docker inside the CI job's container. This is called Docker-in-Docker. You enable it by using a special service in your .gitlab-ci.yml: services: - docker:dind And set variables: variables: DOCKER_HOST: tcp://docker:2375/ DOCKER_TLS_CERTDIR: "" This setup lets your job run Docker commands like 'docker build' to create images.

Result

You can build and push Docker images as part of your CI pipeline.

Understanding Docker-in-Docker is key to automating image builds, which is essential for containerized deployments.

5

Intermediate

Caching Docker Layers to Speed Pipelines

🤔Before reading on: do you think Docker caches layers automatically in GitLab CI jobs? Commit to your answer.

Concept: Learn how to speed up Docker builds in GitLab CI by caching layers between jobs.

Docker caches layers to avoid rebuilding unchanged parts, but in CI jobs, containers are fresh each time, so cache is lost. To fix this, you can use GitLab's cache or registry to save and reuse layers. For example, pushing intermediate images to GitLab's container registry and pulling them in later jobs helps reuse layers and speeds up builds.

Result

Docker builds run faster by reusing cached layers across pipeline runs.

Knowing how to cache Docker layers prevents slow builds and wasted resources in CI pipelines.

6

Advanced

Multi-Stage Docker Builds in CI Pipelines

🤔Before reading on: do you think multi-stage builds reduce final image size or increase it? Commit to your answer.

Concept: Learn how to use multi-stage Docker builds in GitLab CI to create small, efficient images.

Multi-stage builds let you use multiple FROM statements in a Dockerfile. You build your app in one stage with all tools, then copy only the final artifacts to a smaller base image. This reduces image size and attack surface. In GitLab CI, you build this multi-stage Dockerfile as usual, producing optimized images ready for deployment.

Result

Your Docker images are smaller and more secure, improving deployment speed and reliability.

Understanding multi-stage builds helps you produce production-ready images that are efficient and secure.

7

Expert

Security and Performance Considerations in GitLab CI with Docker

🤔Before reading on: do you think running Docker-in-Docker with root privileges is safe by default? Commit to your answer.

Concept: Explore advanced security and performance trade-offs when using Docker in GitLab CI pipelines.

Running Docker-in-Docker requires privileged mode, which can expose security risks. Alternatives include using Docker socket binding or remote Docker daemons. Also, running many containers can strain CI runners, so resource limits and runner scaling are important. Experts use dedicated runners, minimal images, and secrets management to secure pipelines and optimize performance.

Result

You can design secure, efficient CI pipelines that scale and protect your infrastructure.

Knowing these trade-offs prevents common security pitfalls and performance bottlenecks in production CI/CD systems.

Under the Hood

GitLab CI uses runners that execute jobs defined in .gitlab-ci.yml. When Docker is specified, the runner pulls the Docker image and starts a container for the job. Commands run inside this container, isolated from the host and other jobs. For Docker-in-Docker, a special Docker daemon runs inside a service container, allowing nested Docker commands. The runner communicates with this daemon over a network socket, enabling image builds and pushes.

Why designed this way?

This design isolates jobs to prevent interference and ensures consistent environments. Using containers avoids dependency conflicts and makes pipelines portable. Docker-in-Docker was chosen to enable image building inside CI without needing direct host Docker access, balancing flexibility and security. Alternatives like mounting the host Docker socket exist but have higher security risks.

┌─────────────┐
│ GitLab CI   │
│ Runner      │
└─────┬───────┘
      │
      │ pulls Docker image
      ▼
┌─────────────┐
│ Docker      │
│ Container   │
│ (Job Env)   │
└─────┬───────┘
      │
      │ runs job commands
      ▼
┌─────────────┐
│ Docker-in-  │
│ Docker      │
│ Service     │
└─────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does specifying 'image' in .gitlab-ci.yml guarantee the job runs inside that Docker container? Commit yes or no.

Common Belief: Specifying 'image' means the job runs inside that Docker container automatically.

Reality: The job runs inside the Docker container only if the runner is configured to use the Docker executor. If the runner uses a shell executor, the 'image' setting is ignored.

Why it matters: Assuming 'image' always works can cause confusion when jobs run in unexpected environments, leading to build failures or inconsistent results.

Quick: Can you build Docker images inside GitLab CI jobs without enabling Docker-in-Docker or special services? Commit yes or no.

Common Belief: You can run 'docker build' commands in any GitLab CI job without extra setup.

Reality: Docker build commands require Docker daemon access, which is not available by default. You must enable Docker-in-Docker or bind the Docker socket for builds to work.

Why it matters: Without this setup, build commands fail, blocking image creation and deployment.

Quick: Does Docker caching work automatically across different GitLab CI pipeline runs? Commit yes or no.

Common Belief: Docker caches layers automatically between CI pipeline runs, speeding up builds.

Reality: Each CI job runs in a fresh container without cache. Docker caching does not persist between jobs unless explicitly managed with caching strategies or registries.

Why it matters: Ignoring this leads to slow builds and wasted resources, frustrating developers and delaying delivery.

Quick: Is running Docker-in-Docker in privileged mode safe by default? Commit yes or no.

Common Belief: Running Docker-in-Docker with privileged mode is safe and recommended for all pipelines.

Reality: Privileged mode grants elevated permissions that can expose security risks. It should be used carefully with trusted code and runners.

Why it matters: Misusing privileged mode can lead to security breaches and compromised infrastructure.

Expert Zone

1

GitLab runners can be configured with different executors (Docker, shell, Kubernetes), and the 'image' keyword only affects Docker executors.

2

Using Docker socket binding instead of Docker-in-Docker can improve performance but increases security risks by exposing the host Docker daemon.

3

Multi-stage builds not only reduce image size but also improve build cache efficiency by separating build dependencies from runtime dependencies.

When NOT to use

Avoid Docker-in-Docker in shared or untrusted runner environments due to security risks; instead, use remote Docker daemons or Kubernetes-based runners. For very simple pipelines, consider shell runners without Docker to reduce complexity.

Production Patterns

In production, teams use dedicated GitLab runners with Docker executor and Docker-in-Docker for image builds, combined with GitLab Container Registry for storing images. Pipelines often include multi-stage builds, caching strategies, and secrets management for credentials. Scaling runners and monitoring resource usage ensures reliable CI/CD performance.

Connections

Kubernetes

Builds-on

Understanding GitLab CI with Docker prepares you for deploying containers to Kubernetes clusters, as both use container images and orchestration concepts.

Continuous Integration (CI) Principles

Same pattern

GitLab CI with Docker is a practical application of CI principles, automating testing and building in isolated environments to improve software quality.

Manufacturing Assembly Lines

Similar process flow

Like an assembly line where each station performs a specific task on a product, GitLab CI pipelines run jobs in sequence or parallel inside containers to build and test software efficiently.

Common Pitfalls

#1 Trying to run Docker build commands without enabling Docker-in-Docker or Docker socket access.

Wrong approach:

script:
- docker build -t myimage .

Correct approach:

services:
- docker:dind
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_TLS_CERTDIR: ""
script:
- docker build -t myimage .

Root cause:

Not understanding that Docker commands require access to a Docker daemon, which is not available by default in CI jobs.

#2 Assuming Docker layer caching works automatically between pipeline runs, leading to slow builds.

Wrong approach:

script:
- docker build -t myimage . # no caching strategy used

Correct approach:

cache:
key: docker-cache
paths:
- .docker/cache/
script:
- docker build --cache-from=myimage:latest -t myimage .

Root cause:

Misunderstanding that each CI job runs in a clean environment without persistent Docker cache.

#3 Running Docker-in-Docker without privileged mode, causing build failures.

Wrong approach:

services:
- name: docker:dind
command: []
# no privileged flag set
script:
- docker build -t myimage .

Correct approach:

services:
- name: docker:dind
privileged: true
script:
- docker build -t myimage .

Root cause:

Not knowing that Docker-in-Docker requires privileged mode to function properly.

Key Takeaways

GitLab CI with Docker automates software workflows by running jobs inside isolated containers, ensuring consistency.

Specifying Docker images in .gitlab-ci.yml controls the environment, but requires the runner to use the Docker executor.

Building Docker images in CI needs Docker-in-Docker or Docker socket access, which must be configured carefully for security.

Caching Docker layers and using multi-stage builds optimize pipeline speed and produce efficient images.

Security and performance trade-offs in Docker-in-Docker setups require expert attention to avoid risks and bottlenecks.

Practice

1. What is the main purpose of using docker:dind service in a GitLab CI pipeline?

easy

A. To send notifications after the pipeline finishes

B. To deploy the application to production automatically

C. To run Docker commands inside the CI job container

D. To store build artifacts permanently

Solution

1. Step 1: Understand the role of docker:dind

   The docker:dind (Docker-in-Docker) service allows Docker commands to run inside a container, which is essential for building and managing Docker images during CI jobs.

2. Step 2: Identify the purpose in GitLab CI context

   In GitLab CI, using docker:dind lets the pipeline build, run, and push Docker images isolated from the host system.

3. Final Answer:

   To run Docker commands inside the CI job container -> Option C

4. Quick Check:

   docker:dind = run Docker commands in CI [OK]

Hint: Remember: dind means Docker runs inside Docker [OK]

Common Mistakes:

• Confusing dind with deployment step
• Thinking dind stores artifacts
• Assuming dind sends notifications

2. Which of the following is the correct way to specify the Docker service in a GitLab CI job to enable Docker commands?

easy

A. services:
- docker:dind

B. services:
- docker:latest

C. services:
- docker:build

D. services:
- docker:run

Solution

1. Step 1: Recall the correct service name for Docker-in-Docker

   The official Docker-in-Docker service image is docker:dind, which must be specified exactly to enable Docker commands in GitLab CI.

2. Step 2: Compare options to find exact match

   Only services: - docker:dind uses the correct service name docker:dind. Other options are invalid or do not exist as official images.

3. Final Answer:

   services: - docker:dind -> Option A

4. Quick Check:

   Correct service name = docker:dind [OK]

Hint: Use exact service name 'docker:dind' for Docker commands [OK]

Common Mistakes:

• Using 'docker:latest' instead of 'docker:dind'
• Adding extra tags like 'build' or 'run'
• Omitting the services section

3. Given this GitLab CI job snippet, what will be the output of the docker ps command inside the job?

job:
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker info
    - docker ps

medium

A. Shows an error: Docker daemon not running

B. Lists running containers inside the dind service (likely empty)

C. Lists containers on the host machine

D. Outputs the Docker version only

Solution

1. Step 1: Understand the environment of the job

   The job uses docker:dind service, which runs a Docker daemon inside the CI environment. The docker ps command connects to this daemon.

2. Step 2: Predict the output of docker ps

   Since the job just started and no containers were run inside the dind daemon, docker ps will list running containers inside the dind environment, which is likely empty.

3. Final Answer:

   Lists running containers inside the dind service (likely empty) -> Option B

4. Quick Check:

   docker ps inside dind = running containers (empty if none) [OK]

Hint: docker ps shows containers inside dind, not host [OK]

Common Mistakes:

• Expecting error about Docker daemon
• Thinking it lists host machine containers
• Confusing docker info output with docker ps

4. You have this GitLab CI job configuration:

build:
  image: docker:latest
  script:
    - docker build -t myapp .
  services:
    - docker:dind

The job fails with error: Cannot connect to the Docker daemon. What is the most likely fix?

medium

A. Add variables:
DOCKER_HOST: tcp://docker:2375 to the job

B. Change image to docker:dind

C. Remove the services section

D. Add docker login before build

Solution

1. Step 1: Identify cause of Docker daemon connection error

   The error means the Docker client inside the job cannot reach the Docker daemon running in the docker:dind service container.

2. Step 2: Fix connection by setting DOCKER_HOST variable

   Setting DOCKER_HOST: tcp://docker:2375 tells the Docker client to connect to the dind service over TCP, enabling communication.

3. Final Answer:

   Add variables: DOCKER_HOST: tcp://docker:2375 to the job -> Option A

4. Quick Check:

   Set DOCKER_HOST to connect client to dind daemon [OK]

Hint: Set DOCKER_HOST to tcp://docker:2375 to fix connection [OK]

Common Mistakes:

• Changing image to docker:dind instead of using service
• Removing services section disables Docker daemon
• Adding docker login does not fix daemon connection

5. You want to build a Docker image and push it to GitLab Container Registry in a GitLab CI job. Which snippet correctly sets up the job with Docker-in-Docker and pushes the image?

build_and_push:
  image: docker:latest
  services:
    - docker:dind
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
  script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
    - docker build -t $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE:latest

hard

A. No docker login command before push

B. Missing DOCKER_HOST variable, so Docker commands fail

C. Uses docker:dind as image instead of service

D. Correct setup with variables and login before build and push

Solution

1. Step 1: Check Docker-in-Docker setup

   The job uses docker:dind as a service and sets DOCKER_HOST and disables TLS cert dir, which is required for Docker commands to connect properly.

2. Step 2: Verify login and push steps

   The job logs into the GitLab Container Registry using CI variables, then builds and pushes the image with correct tags.

3. Final Answer:

   Correct setup with variables and login before build and push -> Option D

4. Quick Check:

   Proper dind setup + login + build + push = Correct setup with variables and login before build and push [OK]

Hint: Always login before push and set DOCKER_HOST for dind [OK]

Common Mistakes:

• Forgetting DOCKER_HOST causes connection errors
• Using docker:dind as image instead of service
• Skipping docker login before push