Docker architecture (client, daemon, registry) - Deep Dive

Overview - Docker architecture (client, daemon, registry)

What is it?

Docker architecture is the way Docker software is organized to build, run, and share containers. It mainly consists of three parts: the Docker client, the Docker daemon, and the Docker registry. The client is the tool you use to tell Docker what to do. The daemon is the background service that does the work. The registry stores container images that you can download or upload.

Why it matters

Without this architecture, managing containers would be confusing and inefficient. The separation allows users to easily create and run containers on their machines while sharing images through registries. It makes container technology simple, scalable, and collaborative, which is why Docker became so popular in software development and deployment.

Where it fits

Before learning Docker architecture, you should understand what containers are and why they are useful. After this, you can learn about Docker commands, container lifecycle, and how to write Dockerfiles to create custom images.

Mental Model

Core Idea

Docker architecture splits container management into client commands, a background service that runs containers, and a storage place for container images.

Think of it like...

Think of Docker like ordering food at a restaurant: the client is you placing the order, the daemon is the kitchen preparing the food, and the registry is the pantry where ingredients (images) are stored and fetched from.

┌─────────────┐       ┌─────────────┐       ┌───────────────┐
│ Docker      │       │ Docker      │       │ Docker        │
│ Client      │──────▶│ Daemon      │──────▶│ Registry      │
│ (You order) │       │ (Kitchen)   │       │ (Pantry)      │
└─────────────┘       └─────────────┘       └───────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Docker Client Role

Concept: The Docker client is the command-line tool or API you use to interact with Docker.

When you type commands like 'docker run' or 'docker pull', you are using the Docker client. It sends your requests to the Docker daemon to perform actions. The client itself does not run containers or manage images; it only tells the daemon what to do.

Result

You can control Docker by typing commands in your terminal or using Docker APIs.

Knowing that the client is just the user interface helps you understand that commands are requests, not the actual work.

2

Foundation

Docker Daemon's Background Work

Concept: The Docker daemon is the background service that manages containers, images, networks, and storage.

The daemon listens for commands from the client and executes them. It creates, runs, and stops containers, builds images, and manages resources. The daemon runs on your machine or server and handles all the heavy lifting.

Result

Containers and images are created and managed by the daemon, invisible to you but essential.

Understanding the daemon as the worker behind the scenes clarifies why Docker commands sometimes need special permissions.

3

Intermediate

Role of Docker Registry

Concept: The Docker registry is a storage and distribution system for Docker images.

Registries hold container images that you can download (pull) or upload (push). Docker Hub is the most popular public registry, but private registries exist too. When you run 'docker pull ubuntu', the client asks the daemon to fetch the image from the registry.

Result

You can share and reuse container images easily across different machines and teams.

Knowing registries are like image libraries explains how Docker supports collaboration and consistent environments.

4

Intermediate

Communication Between Client and Daemon

🤔Before reading on: do you think the Docker client and daemon must run on the same machine? Commit to your answer.

Concept: The Docker client communicates with the daemon using a REST API over sockets or network connections.

By default, the client talks to the daemon on the same machine via a Unix socket. However, the client can also connect to a remote daemon over TCP, allowing control of Docker on other machines. This flexibility enables managing containers on servers from your local computer.

Result

You can control Docker locally or remotely, depending on configuration.

Understanding this communication method reveals how Docker supports distributed container management.

5

Intermediate

Image Lifecycle in Docker Architecture

🤔Before reading on: do you think images are stored on your machine or only in registries? Commit to your answer.

Concept: Docker images are stored locally by the daemon after being pulled from registries and used to create containers.

When you pull an image, the daemon downloads and saves it on your machine. Containers are created from these images. You can also build new images locally and push them to registries for sharing. This lifecycle ensures images are reusable and portable.

Result

Images exist both locally and remotely, enabling fast container creation and sharing.

Knowing where images live helps understand storage needs and how Docker speeds up container startup.

6

Advanced

Daemon Architecture and Container Runtime

🤔Before reading on: do you think the Docker daemon runs containers directly or uses another tool? Commit to your answer.

Concept: The Docker daemon uses a container runtime to create and manage containers, separating concerns for flexibility and security.

Docker originally used its own runtime but now relies on container runtimes like containerd. The daemon manages images and container lifecycle but delegates container execution to the runtime. This modular design allows Docker to support different runtimes and improve stability.

Result

Containers run efficiently and securely through a layered architecture.

Understanding this separation explains Docker's adaptability and why container runtimes are important.

7

Expert

Registry Protocols and Image Distribution

🤔Before reading on: do you think Docker registries store images as single files or layers? Commit to your answer.

Concept: Docker registries store images as layers and use protocols to efficiently distribute and cache these layers.

Images are made of layers representing changes. Registries store these layers separately. When pulling an image, only missing layers are downloaded, saving bandwidth. Registries use HTTP APIs and support authentication and access control. This design enables fast, secure, and scalable image distribution.

Result

Image sharing is optimized for speed and storage efficiency.

Knowing about layers and protocols reveals why Docker images are lightweight and how registries handle millions of images.

Under the Hood

The Docker client sends commands via REST API to the Docker daemon, which runs as a background process. The daemon manages container lifecycle by interacting with the container runtime (like containerd) to create namespaces, cgroups, and filesystems for containers. Images are stored as layered filesystems locally and in registries. Registries communicate over HTTP(S) using a standard API to serve image layers on demand.

Why designed this way?

Docker was designed to separate user commands (client) from container management (daemon) to allow flexibility and security. Using registries for image storage enables sharing and versioning. The layered image model reduces duplication and speeds up downloads. The modular runtime approach allows Docker to evolve with container standards and improve reliability.

┌─────────────┐       ┌─────────────┐       ┌───────────────┐
│ Docker      │       │ Docker      │       │ Docker        │
│ Client      │──────▶│ Daemon      │──────▶│ Registry      │
│ (CLI/API)   │       │ (Background │       │ (Image Store) │
│             │       │  Service)   │       │               │
└─────────────┘       └─────────────┘       └───────────────┘
                         │
                         ▼
                 ┌─────────────────┐
                 │ Container Runtime│
                 │ (containerd)    │
                 └─────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does the Docker client run containers itself? Commit to yes or no.

Common Belief: The Docker client runs containers directly on your machine.

Reality: The Docker client only sends commands; the Docker daemon runs containers.

Why it matters: Thinking the client runs containers can confuse troubleshooting and permissions, since the daemon needs proper rights to manage containers.

Quick: Are Docker images stored only in registries? Commit to yes or no.

Common Belief: Docker images exist only in remote registries like Docker Hub.

Reality: Images are stored locally by the daemon after pulling and can be built locally before pushing to registries.

Why it matters: Misunderstanding this leads to confusion about storage space and why containers can start quickly without downloading images every time.

Quick: Does the Docker daemon run on the client machine only? Commit to yes or no.

Common Belief: The Docker daemon must run on the same machine as the client.

Reality: The client can connect to remote daemons over the network to manage containers on other machines.

Why it matters: This misconception limits understanding of Docker's flexibility in managing containers across servers.

Quick: Are Docker images stored as single files? Commit to yes or no.

Common Belief: Docker images are single large files stored in registries.

Reality: Images are stored as multiple layers, which are shared and reused to save space and bandwidth.

Why it matters: Not knowing about layers can cause confusion about image size and update efficiency.

Expert Zone

1

The Docker daemon uses containerd as a separate process to handle container lifecycle, improving modularity and stability.

2

Docker registries support image signing and content trust to verify image authenticity, which is critical for security.

3

The layered image system allows caching and reusing layers across different images, reducing storage and network usage.

When NOT to use

Docker architecture is not ideal for running full virtual machines or applications requiring heavy GUI support. Alternatives like traditional VMs or specialized container runtimes (e.g., Kata Containers) may be better for strong isolation or hardware access.

Production Patterns

In production, Docker clients often connect to remote daemons on orchestrated clusters (like Kubernetes). Private registries are used for secure image storage. Multi-stage builds optimize image size. Monitoring and logging are integrated with the daemon to track container health.

Connections

Client-Server Architecture

Docker architecture follows the client-server model where the client sends requests and the server (daemon) processes them.

Understanding client-server basics helps grasp how Docker separates user commands from container management.

Version Control Systems (e.g., Git)

Docker registries store image layers similar to how Git stores commits and file changes as layers.

Knowing about version control layering clarifies why Docker images are efficient and easy to update.

Restaurant Kitchen Workflow

The Docker client, daemon, and registry correspond to ordering, cooking, and pantry in a restaurant.

This cross-domain connection helps understand the flow of commands, execution, and resource storage in Docker.

Common Pitfalls

#1 Trying to run Docker commands without the daemon running.

Wrong approach:

docker run hello-world

Correct approach:

sudo systemctl start docker
docker run hello-world

Root cause:

Not realizing the daemon is a background service that must be running to process commands.

#2 Assuming images are downloaded every time a container starts.

Wrong approach:

docker pull ubuntu
# Then immediately docker pull ubuntu again before running container

Correct approach:

docker pull ubuntu
# Run container directly without pulling again if image exists locally
docker run ubuntu

Root cause:

Misunderstanding that images are cached locally by the daemon after the first pull.

#3 Trying to connect Docker client to a remote daemon without configuring TLS or permissions.

Wrong approach:

docker -H tcp://remote-server:2375 run nginx

Correct approach:

docker --tlsverify -H tcp://remote-server:2376 run nginx

Root cause:

Ignoring security requirements for remote daemon connections leads to connection failures or security risks.

Key Takeaways

Docker architecture separates user commands (client), container management (daemon), and image storage (registry) for clarity and flexibility.

The Docker client sends requests but does not run containers; the daemon handles all container operations.

Registries store images as layers, enabling efficient sharing and updating of container images.

The daemon communicates with container runtimes like containerd to run containers securely and efficiently.

Understanding this architecture helps troubleshoot Docker issues and use Docker effectively in local and distributed environments.