Sharing volumes between containers in Docker - Deep Dive

Overview - Sharing volumes between containers

What is it?

Sharing volumes between containers means letting multiple Docker containers access the same storage space on your computer or server. This storage space is called a volume. It allows containers to read and write files in the same place, making it easy to share data or keep information safe even if containers stop or restart.

Why it matters

Without shared volumes, containers would keep their data isolated and lose it when they stop. Sharing volumes solves this by providing a common place to store data that lasts beyond a container's life. This is important for things like databases, logs, or apps that need to share files, making your system more reliable and easier to manage.

Where it fits

Before learning about sharing volumes, you should understand basic Docker concepts like containers and images. After this, you can learn about Docker Compose for managing multi-container apps or advanced storage options like bind mounts and volume drivers.

Mental Model

Core Idea

A shared volume is like a common folder on your computer that multiple containers can open and use at the same time to share and save files.

Think of it like...

Imagine several friends working on a group project using the same notebook. Each friend can write notes or read what others wrote in that notebook. The notebook is the shared volume, and the friends are the containers.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│   Container   │       │   Container   │       │   Container   │
│      A        │       │      B        │       │      C        │
└──────┬────────┘       └──────┬────────┘       └──────┬────────┘
       │                       │                       │
       │                       │                       │
       │                       │                       │
       ▼                       ▼                       ▼
  ┌─────────────────────────────────────────────────────────┐
  │                      Shared Volume                      │
  │  (Common storage space accessible by all containers)   │
  └─────────────────────────────────────────────────────────┘

Build-Up - 6 Steps

1

Foundation

Understanding Docker Volumes Basics

Concept: Learn what Docker volumes are and why they exist.

Docker volumes are special storage areas managed by Docker. They let containers save data outside their own temporary storage. This means data stays safe even if the container is deleted or recreated. Volumes live on the host machine and can be reused by containers.

Result

You understand that volumes are persistent storage areas separate from container filesystems.

Knowing volumes keep data safe beyond container life is key to managing persistent data in Docker.

2

Foundation

Creating and Using a Single Volume

Concept: Learn how to create a volume and attach it to one container.

Use the command 'docker volume create mydata' to make a volume named 'mydata'. Then run a container with 'docker run -v mydata:/data alpine' to attach it. Inside the container, files saved to /data go to the volume.

Result

A volume named 'mydata' is created and attached to a container at /data.

Attaching a volume to a container lets you save data persistently in a known location.

3

Intermediate

Sharing One Volume Between Multiple Containers

🤔Before reading on: do you think two containers can write to the same volume at the same time without conflicts? Commit to your answer.

Concept: Learn how to attach the same volume to multiple containers to share data.

Run two containers with the same volume: 'docker run -v mydata:/data alpine' and 'docker run -v mydata:/data alpine'. Both containers can read and write files in /data, sharing information.

Result

Both containers access the same files in the shared volume, allowing data exchange.

Understanding that volumes can be shared enables data collaboration between containers but requires care to avoid conflicts.

4

Intermediate

Using Docker Compose for Volume Sharing

🤔Before reading on: do you think Docker Compose simplifies volume sharing compared to manual commands? Commit to your answer.

Concept: Learn how Docker Compose defines shared volumes for multiple containers in one file.

Create a docker-compose.yml file with a shared volume: version: '3' services: app1: image: alpine volumes: - shared-data:/data app2: image: alpine volumes: - shared-data:/data volumes: shared-data: Run 'docker-compose up' to start both containers sharing 'shared-data'.

Result

Two containers start with a shared volume defined in one easy-to-read file.

Using Docker Compose makes managing shared volumes across containers simpler and more organized.

5

Advanced

Handling Data Consistency and Conflicts

🤔Before reading on: do you think Docker automatically prevents file conflicts when multiple containers write to the same volume? Commit to your answer.

Concept: Learn about risks when multiple containers write to the same files and how to manage them.

When containers write to the same files simultaneously, data can get corrupted or lost. Docker does not lock files. To avoid this, use application-level locks, separate files, or databases designed for concurrency.

Result

You understand that sharing volumes needs careful coordination to keep data safe.

Knowing Docker volumes don't handle file conflicts prevents data corruption in shared storage.

6

Expert

Using Named Volumes with Volume Drivers

🤔Before reading on: do you think all volumes are stored on the local machine by default? Commit to your answer.

Concept: Learn about advanced volume types using drivers to store data on remote or special storage.

Docker supports volume drivers that let volumes live on network storage, cloud services, or encrypted disks. For example, using 'docker volume create --driver rexray/ebs mydata' stores data on AWS EBS. Containers share this volume like a local one.

Result

Volumes can be shared across hosts or use special storage, expanding Docker's flexibility.

Understanding volume drivers unlocks powerful storage options for scalable and secure container data sharing.

Under the Hood

Docker volumes are directories on the host machine managed by Docker. When a container starts with a volume, Docker mounts that directory inside the container's filesystem. Multiple containers mounting the same volume see the same files. Docker uses the host's filesystem permissions and caching. It does not control file locking or concurrency inside the volume.

Why designed this way?

Docker volumes were designed to separate container lifecycle from data lifecycle, so data persists independently. Using host-managed directories allows flexibility and performance. Docker avoids managing file locking to keep volumes simple and fast, leaving concurrency control to applications.

┌───────────────┐       ┌───────────────┐
│   Container   │       │   Container   │
│    Filesystem │       │    Filesystem │
│  ┌─────────┐  │       │  ┌─────────┐  │
│  │ /data   │◄───────►│  │ /data   │  │
└──┴─────────┴──┘       └──┴─────────┴──┘
       ▲                       ▲
       │                       │
       └──────────────┬────────┘
                      │
              ┌─────────────────┐
              │  Host Directory │
              │  (Volume Data)  │
              └─────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think Docker volumes automatically prevent file conflicts when shared? Commit yes or no.

Common Belief: Docker volumes handle file locking and prevent conflicts automatically.

Reality: Docker volumes do not manage file locking or concurrency; applications must handle conflicts.

Why it matters: Assuming automatic locking can cause data corruption when multiple containers write simultaneously.

Quick: Do you think deleting a container deletes its volumes by default? Commit yes or no.

Common Belief: When a container is deleted, its volumes are deleted too.

Reality: Named volumes persist even if containers are removed, keeping data safe.

Why it matters: Deleting volumes unintentionally can cause permanent data loss.

Quick: Can volumes be shared between containers on different hosts by default? Commit yes or no.

Common Belief: Docker volumes can be shared across different machines without extra setup.

Reality: By default, volumes are local to the host machine; sharing across hosts requires special drivers or network storage.

Why it matters: Expecting cross-host sharing without setup leads to broken apps and lost data.

Quick: Do you think bind mounts and volumes are the same? Commit yes or no.

Common Belief: Bind mounts and volumes are identical in Docker.

Reality: Bind mounts link to any host directory and can cause permission issues; volumes are managed by Docker and safer for data.

Why it matters: Confusing them can cause security risks and unexpected behavior.

Expert Zone

1

Named volumes are managed by Docker and can be easily backed up or migrated, unlike bind mounts which depend on host paths.

2

Sharing volumes between containers requires careful application design to avoid race conditions and data corruption.

3

Volume drivers enable advanced storage options like encryption, replication, or cloud storage integration, which are critical for production environments.

When NOT to use

Avoid sharing volumes when containers need strict isolation or when data consistency is critical without application-level locking. Instead, use networked databases, message queues, or object storage services designed for concurrent access.

Production Patterns

In production, shared volumes are often used for logs, configuration files, or cache data. Teams use Docker Compose or Kubernetes Persistent Volumes with access modes to manage sharing. Volume drivers connect containers to cloud storage or distributed filesystems for scalability.

Connections

Network File Systems (NFS)

Shared volumes in Docker are similar to NFS shares that multiple computers mount to access the same files.

Understanding NFS helps grasp how shared storage works across systems and the challenges of concurrent access.

Database Transaction Locks

Both shared volumes and databases face concurrency issues requiring locking or coordination to avoid conflicts.

Knowing how databases handle locks informs better design of applications sharing Docker volumes.

Collaborative Document Editing

Sharing volumes is like multiple users editing a shared document; without coordination, changes can overwrite each other.

This connection highlights the importance of conflict resolution and synchronization in shared data environments.

Common Pitfalls

#1 Assuming data written by one container is instantly visible and safe for others without coordination.

Wrong approach:

docker run -v shared:/data alpine sh -c 'echo hello > /data/file.txt' & docker run -v shared:/data alpine sh -c 'cat /data/file.txt'

Correct approach:

Use application-level locks or wait mechanisms to ensure data consistency before reading shared files.

Root cause:

Misunderstanding that Docker volumes do not provide synchronization or atomic operations.

#2 Deleting containers and expecting volumes to be deleted automatically, causing leftover data or confusion.

Wrong approach:

docker rm mycontainer (expecting volume to be removed)

Correct approach:

Use 'docker volume rm myvolume' explicitly to remove volumes when no longer needed.

Root cause:

Not knowing that volumes persist independently of containers.

#3 Using bind mounts instead of volumes for sharing data without considering permission and portability issues.

Wrong approach:

docker run -v /host/path:/data alpine

Correct approach:

docker volume create shareddata && docker run -v shareddata:/data alpine

Root cause:

Confusing bind mounts with volumes and ignoring Docker's management benefits.

Key Takeaways

Docker volumes provide persistent storage that outlives containers and can be shared between them.

Sharing volumes allows containers to exchange data but requires careful handling to avoid conflicts and corruption.

Docker Compose simplifies managing shared volumes across multiple containers with clear configuration.

Volumes are local to the host by default; sharing across hosts needs special drivers or network storage.

Understanding the limits and behavior of volumes helps design reliable, scalable containerized applications.

Practice

1. What is the main purpose of sharing volumes between Docker containers?

easy

A. To automatically update container images

B. To increase the CPU performance of containers

C. To allow containers to access and modify the same files

D. To isolate containers completely from each other

Solution

1. Step 1: Understand volume sharing concept

   Sharing volumes means multiple containers can read and write the same files stored in a volume.

2. Step 2: Identify the purpose of sharing volumes

   This allows containers to share data easily, like sharing notes or files in a folder between friends.

3. Final Answer:

   To allow containers to access and modify the same files -> Option C

4. Quick Check:

   Sharing volumes = shared files access [OK]

Hint: Sharing volumes means sharing files between containers [OK]

Common Mistakes:

• Thinking volumes improve CPU or performance
• Confusing volume sharing with container isolation
• Assuming volumes update container images automatically

2. Which Docker command syntax correctly shares a volume named shared_data between two containers app1 and app2?

easy

A. docker run --volumes shared_data:/data --name app1 busybox

B. docker run -v shared_data:/data --name app1 busybox

C. docker run -v /shared_data:/data --name app1 busybox

D. docker run -v shared_data --name app1 --link app2 busybox

Solution

1. Step 1: Check volume syntax for sharing

   The correct syntax to share a named volume is -v volume_name:container_path. docker run -v shared_data:/data --name app1 busybox uses -v shared_data:/data which is correct.

2. Step 2: Verify other options

   docker run --volumes shared_data:/data --name app1 busybox uses invalid --volumes flag (should be --volume or -v). docker run -v /shared_data:/data --name app1 busybox uses a host path, not a named volume. docker run -v shared_data --name app1 --link app2 busybox uses incomplete volume syntax (missing :/data) and --link which is unrelated to volume sharing.

3. Final Answer:

   docker run -v shared_data:/data --name app1 busybox -> Option B

4. Quick Check:

   Use -v volume_name:path to share volumes [OK]

Hint: Use -v volume_name:container_path to share volumes [OK]

Common Mistakes:

• Using host paths instead of named volumes
• Confusing --link with volume sharing
• Using incorrect volume syntax

3. Given these commands, what will be the output of docker exec app2 cat /data/message.txt?

docker volume create shared_vol
docker run -d -v shared_vol:/data --name app1 busybox sh -c "echo Hello > /data/message.txt"
docker run -d -v shared_vol:/data --name app2 busybox sleep 1000

medium

A. Empty output

B. Error: file not found

C. Container app2 does not exist

D. Hello

Solution

1. Step 1: Understand volume sharing effect

   Both containers use the same volume shared_vol mounted at /data. Writing in app1 creates the file in the shared volume.

2. Step 2: Check file access from app2

   Since app2 mounts the same volume, it can read the file /data/message.txt created by app1. So the command outputs "Hello".

3. Final Answer:

   Hello -> Option D

4. Quick Check:

   Shared volume files visible in all containers [OK]

Hint: Shared volumes show same files in all containers [OK]

Common Mistakes:

• Assuming files are isolated per container
• Expecting file not found error
• Confusing container names or existence

4. You tried to share a volume between two containers but the second container cannot see the files created by the first. Which of these is the most likely cause?

medium

A. The volume was not named the same in both containers

B. The containers are running different images

C. The containers are on different networks

D. The first container did not expose ports

Solution

1. Step 1: Check volume naming consistency

   For containers to share data, they must use the exact same volume name. Different names create separate volumes.

2. Step 2: Evaluate other options

   Different images or networks do not affect volume sharing. Port exposure is unrelated to file sharing.

3. Final Answer:

   The volume was not named the same in both containers -> Option A

4. Quick Check:

   Same volume name needed to share data [OK]

Hint: Use identical volume names to share data [OK]

Common Mistakes:

• Thinking network or image affects volume sharing
• Confusing port exposure with volumes
• Using different volume names by mistake

5. You want two containers to share a volume but keep their own separate data initially, then merge changes later. Which Docker feature or approach helps achieve this?

hard

A. Use named volumes with different initial content and copy files manually

B. Use a bind mount to a host directory shared by both containers

C. Use Docker networks to share data between containers

D. Run containers with --rm to remove volumes after exit

Solution

1. Step 1: Understand volume sharing with initial separate data

   Named volumes start empty by default. To have separate initial data, create volumes separately and copy files manually.

2. Step 2: Consider merging changes later

   After initial separate data, you can copy or sync files between volumes or containers to merge changes.

3. Step 3: Evaluate other options

   Bind mounts share host directories but do not isolate initial data. Networks do not share files. --rm removes containers and volumes, not helpful here.

4. Final Answer:

   Use named volumes with different initial content and copy files manually -> Option A

5. Quick Check:

   Named volumes + manual copy = separate then merge [OK]

Hint: Start with separate volumes, copy files to merge later [OK]

Common Mistakes:

• Assuming networks share files
• Using --rm which deletes volumes
• Thinking bind mounts isolate initial data