Creating and managing volumes in Docker - Mechanics & Internals

Overview - Creating and managing volumes

What is it?

Volumes in Docker are special storage areas that let containers save and share data outside their own temporary space. They help keep data safe even if a container stops or is deleted. You can create, list, inspect, and remove these volumes using Docker commands. Volumes are the preferred way to persist data in Docker environments.

Why it matters

Without volumes, any data created inside a container would disappear when the container stops or is removed, like writing notes on a whiteboard that gets erased. Volumes solve this by providing a permanent place to store data, making applications reliable and stateful. This is crucial for databases, logs, and any data that must survive container restarts or upgrades.

Where it fits

Before learning volumes, you should understand basic Docker containers and images. After mastering volumes, you can explore advanced storage options like bind mounts and storage drivers, and then move on to orchestration tools like Docker Compose or Kubernetes that manage volumes at scale.

Mental Model

Core Idea

Docker volumes are like external hard drives for containers, letting them save and share data safely outside their own temporary space.

Think of it like...

Imagine a container as a laptop that loses all its files when turned off unless you save them on an external USB drive. Docker volumes are like that USB drive, keeping your important files safe no matter what happens to the laptop.

┌───────────────┐       ┌───────────────┐
│   Container   │──────▶│   Volume      │
│  (temporary)  │       │ (persistent)  │
└───────────────┘       └───────────────┘
       ▲                        ▲
       │                        │
  Data inside             Data stored
  container lost          safely in volume
  on stop/removal

Build-Up - 7 Steps

1

Foundation

What is a Docker volume?

Concept: Introduce the basic idea of Docker volumes as persistent storage outside containers.

Docker containers have their own file system that disappears when the container stops. Volumes are special storage areas managed by Docker that live outside the container's life cycle. You create a volume once, and many containers can use it to save or share data.

Result

You understand that volumes keep data safe beyond container life and are managed separately.

Knowing that container storage is temporary explains why volumes are essential for any data you want to keep.

2

Foundation

Creating and listing volumes

Concept: Learn how to create a volume and see existing volumes using Docker commands.

To create a volume, run: docker volume create mydata To list all volumes, run: docker volume ls These commands let you manage volumes independently from containers.

Result

You can create a named volume and verify it exists on your system.

Understanding volume creation as a separate step helps you plan data storage before running containers.

3

Intermediate

Using volumes in containers

🤔Before reading on: do you think mounting a volume replaces the container's entire file system or just adds extra storage? Commit to your answer.

Concept: Learn how to attach a volume to a container to persist data.

Run a container with a volume attached: docker run -d --name mycontainer -v mydata:/app/data busybox sleep 1000 This mounts the volume 'mydata' inside the container at /app/data. Data written there stays in the volume even if the container stops.

Result

Data saved in /app/data inside the container persists on the volume after container removal.

Knowing volumes add storage without replacing the container's file system clarifies how data persistence works.

4

Intermediate

Inspecting and removing volumes

🤔Before reading on: do you think removing a container automatically deletes its volumes? Commit to yes or no.

Concept: Learn how to check volume details and clean up unused volumes.

Inspect a volume: docker volume inspect mydata Remove a volume: docker volume rm mydata Note: Removing a container does NOT delete its volumes by default, so volumes can accumulate if not cleaned.

Result

You can see volume metadata and safely remove volumes when no longer needed.

Understanding volume lifecycle separate from containers prevents storage bloat and data loss.

5

Intermediate

Sharing volumes between containers

Concept: Learn how multiple containers can use the same volume to share data.

Run two containers sharing a volume: docker run -d --name c1 -v sharedvol:/data busybox sleep 1000 docker run -d --name c2 -v sharedvol:/data busybox sleep 1000 Both containers read and write to /data, sharing files through the volume.

Result

Containers can exchange data via the shared volume, enabling collaboration or state sharing.

Knowing volumes enable data sharing helps design multi-container applications that cooperate.

6

Advanced

Volume drivers and custom storage

🤔Before reading on: do you think all volumes store data on the local machine by default? Commit to yes or no.

Concept: Explore how Docker supports different volume drivers for custom storage backends.

By default, volumes store data on the local host. But Docker supports volume drivers to use remote storage, cloud services, or encrypted volumes. Example creating a volume with a driver: docker volume create --driver local mylocalvol Custom drivers can be installed for advanced use cases.

Result

You understand that volumes can use various storage backends beyond local disk.

Knowing about volume drivers opens possibilities for scalable, secure, or distributed storage in production.

7

Expert

Volume lifecycle and orphan volumes

🤔Before reading on: do you think Docker automatically cleans up unused volumes? Commit to yes or no.

Concept: Understand how volumes can become orphaned and how to manage them effectively.

When containers are removed, their volumes remain unless explicitly deleted. Over time, unused volumes accumulate, wasting disk space. Use docker volume prune to remove all unused volumes. Be careful: pruning deletes volumes not used by any container, which may cause data loss if volumes are still needed.

Result

You can prevent disk bloat by cleaning orphan volumes safely.

Understanding volume lifecycle management is critical to maintain healthy Docker environments and avoid hidden storage issues.

Under the Hood

Docker volumes are directories stored on the host machine, managed by Docker in a special location (usually /var/lib/docker/volumes). When a container mounts a volume, Docker links that directory into the container's file system namespace. This linking is done using mount points at the OS level, allowing containers to read and write data directly to the host storage. Volumes are isolated from the container's writable layer, so data persists independently of container life.

Why designed this way?

Volumes were designed to separate data from container life cycles to solve the problem of ephemeral container storage. Early Docker versions used container writable layers for data, which were lost on container removal. Volumes provide a stable, managed, and shareable storage solution. The design balances ease of use, performance, and portability, avoiding direct host path dependencies that can cause security and compatibility issues.

┌───────────────┐       ┌───────────────────────────┐       ┌───────────────┐
│   Container   │──────▶│ Docker Volume Mount Point  │──────▶│ Host Volume   │
│  File System  │       │ (mounts volume directory) │       │ Directory     │
└───────────────┘       └───────────────────────────┘       └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does deleting a container also delete its volumes automatically? Commit to yes or no.

Common Belief: Deleting a container removes all its data including attached volumes.

Reality: Volumes persist even after the container is deleted unless explicitly removed.

Why it matters: Assuming volumes are deleted can cause unexpected data loss or leftover unused volumes consuming disk space.

Quick: Are volumes the same as bind mounts? Commit to yes or no.

Common Belief: Volumes and bind mounts are identical ways to store data for containers.

Reality: Volumes are managed by Docker and stored in Docker's area; bind mounts link to any host directory directly, which can cause portability and security issues.

Why it matters: Confusing these can lead to fragile setups that break when moved or expose sensitive host files.

Quick: Can multiple containers safely write to the same volume at the same time? Commit to yes or no.

Common Belief: Sharing a volume between containers is always safe and causes no conflicts.

Reality: Concurrent writes can cause data corruption unless the application handles synchronization.

Why it matters: Ignoring this can cause subtle bugs and data loss in multi-container setups.

Quick: Are all volumes stored on the local machine by default? Commit to yes or no.

Common Belief: Docker volumes always store data on the local host disk.

Reality: Volumes can use different drivers to store data remotely or in specialized storage systems.

Why it matters: Assuming local storage limits scalability and security options in production.

Expert Zone

1

Volume mount options can control permissions and propagation, affecting container access and data consistency.

2

Docker volumes can be backed up and restored independently, enabling data migration and disaster recovery.

3

Using named volumes improves clarity and management compared to anonymous volumes created automatically by Docker.

When NOT to use

Avoid volumes when you need direct access to specific host files or directories; use bind mounts instead. For ephemeral data that does not need persistence, rely on container writable layers. In distributed systems requiring shared storage across hosts, consider networked storage solutions or orchestration volume plugins.

Production Patterns

In production, volumes are used to store database files, logs, and configuration data. Teams use volume drivers to connect to cloud storage or encrypted volumes for security. Automated cleanup scripts prune unused volumes regularly to save space. Multi-container apps share volumes for stateful services, and volumes are backed up as part of disaster recovery plans.

Connections

File System Mounts

Volumes use OS-level mount points to link storage into containers.

Understanding how operating systems mount file systems helps grasp how Docker volumes integrate storage seamlessly.

Cloud Storage Services

Volume drivers can connect Docker volumes to cloud storage like AWS EBS or Azure Disks.

Knowing cloud storage concepts helps leverage Docker volumes for scalable and durable data in cloud environments.

Database Transaction Logs

Volumes often store database logs to persist state and enable recovery.

Understanding how databases rely on persistent storage clarifies why volumes are critical for stateful applications.

Common Pitfalls

#1 Assuming volumes are deleted with containers, leading to orphaned data or unexpected data loss.

Wrong approach:

docker rm mycontainer
# Expect volume to be deleted automatically

Correct approach:

docker rm mycontainer
docker volume rm myvolume
# Explicitly remove volume when no longer needed

Root cause:

Misunderstanding that volumes have independent lifecycles separate from containers.

#2 Using bind mounts when portability and security are important, causing fragile setups.

Wrong approach:

docker run -v /host/path:/app/data busybox
# Directly mounts host directory

Correct approach:

docker volume create myvol
docker run -v myvol:/app/data busybox
# Uses managed Docker volume

Root cause:

Confusing bind mounts with volumes and not considering environment differences.

#3 Sharing a volume between containers without handling concurrent writes, causing data corruption.

Wrong approach:

docker run -v sharedvol:/data container1
and
docker run -v sharedvol:/data container2
# Both write without coordination

Correct approach:

Use application-level locking or databases designed for concurrent access when sharing volumes.

Root cause:

Ignoring concurrency issues in shared storage leads to subtle bugs.

Key Takeaways

Docker volumes provide persistent storage that outlives containers, essential for saving important data.

Volumes are managed separately from containers, so removing containers does not delete volumes automatically.

Using volumes allows multiple containers to share data, but concurrent access must be managed carefully.

Volume drivers enable flexible storage backends beyond local disks, supporting cloud and specialized storage.

Proper volume lifecycle management prevents disk space waste and data loss in Docker environments.