Copying files to and from containers in Docker - Deep Dive

Overview - Copying files to and from containers

What is it?

Copying files to and from containers means moving files between your computer and a running Docker container. This lets you add files into a container or get files out without rebuilding the container. It is useful for sharing data, logs, or configuration between your host and container.

Why it matters

Without the ability to copy files, you would have to rebuild containers every time you want to change or retrieve files, which slows down development and debugging. Copying files makes working with containers flexible and efficient, saving time and effort.

Where it fits

You should know basic Docker commands and how containers work before learning this. After this, you can explore Docker volumes for persistent storage and Docker networking for container communication.

Mental Model

Core Idea

Copying files to and from containers is like moving files between two computers connected by a cable, allowing easy sharing without changing the computers themselves.

Think of it like...

Imagine your container is a USB drive plugged into your computer. You can drag files onto it or copy files off it without changing the drive’s contents permanently.

Host Machine
  │
  ├── docker cp <source> <container>:<destination>  (copy to container)
  └── docker cp <container>:<source> <destination>  (copy from container)

Container
  └── Filesystem

Commands move files across this boundary.

Build-Up - 7 Steps

1

Foundation

Understanding Docker Containers

Concept: Learn what a Docker container is and how it runs isolated from your host system.

A Docker container is like a mini-computer running inside your main computer. It has its own file system and processes but shares the host's kernel. You can start, stop, and interact with containers using Docker commands.

Result

You understand that containers have their own files separate from your host.

Knowing containers have isolated filesystems explains why copying files is needed to move data in or out.

2

Foundation

Basic Docker CLI Commands

Concept: Familiarize with Docker commands to list and manage containers.

Use 'docker ps' to see running containers and 'docker exec' to run commands inside a container. For example, 'docker exec -it ls /' lists files inside the container root.

Result

You can identify containers and explore their filesystems.

Being able to access container files helps you know what files you might want to copy.

3

Intermediate

Copying Files Into Containers

🤔Before reading on: do you think copying files into a container changes the container image or just the running container? Commit to your answer.

Concept: Learn how to copy files from your host into a running container using 'docker cp'.

The command 'docker cp myfile.txt :/app/' copies 'myfile.txt' from your host into the '/app/' folder inside the container. This changes the container's filesystem but not the original image.

Result

The file appears inside the container at the specified path.

Understanding that copying files affects only the running container prevents confusion about image immutability.

4

Intermediate

Copying Files Out of Containers

🤔Before reading on: do you think you can copy files from a stopped container? Commit to your answer.

Concept: Learn how to copy files from a container back to your host.

Use 'docker cp :/var/log/app.log ./app.log' to copy a log file from the container to your current host directory. This works even if the container is stopped.

Result

The file from the container is now on your host machine.

Knowing you can copy files from stopped containers helps in debugging and data retrieval.

5

Intermediate

Copying Directories and Permissions

Concept: Learn how to copy entire folders and preserve file permissions.

You can copy directories by specifying folder paths, e.g., 'docker cp ./config :/app/'. Docker preserves file permissions and metadata during copy, which is important for executables or config files.

Result

The whole directory and its contents appear inside the container with correct permissions.

Preserving permissions avoids runtime errors caused by wrong file access inside containers.

6

Advanced

Copying Files vs. Using Volumes

🤔Before reading on: do you think copying files is better than volumes for persistent data? Commit to your answer.

Concept: Understand when to use file copying and when to use Docker volumes for sharing data.

Copying files is good for one-time transfers or small changes. Volumes are better for ongoing data sharing and persistence across container restarts. Volumes mount host directories or managed storage directly into containers.

Result

You can choose the right method for your use case.

Knowing the tradeoffs helps optimize container data management and avoid data loss.

7

Expert

Copy Command Internals and Limitations

🤔Before reading on: do you think 'docker cp' uses network protocols or direct filesystem access? Commit to your answer.

Concept: Learn how 'docker cp' works under the hood and its limitations.

'docker cp' uses the Docker API to archive files inside the container and transfer them over the Docker daemon connection. It does not use network protocols like SSH. It cannot copy files from containers running on remote hosts without extra setup.

Result

You understand why 'docker cp' is fast but limited to local Docker hosts.

Knowing the mechanism explains why copying large files can be slow and why remote copying needs other tools.

Under the Hood

'docker cp' works by creating a tar archive of the source files inside the container or host, then streams this archive through the Docker daemon to the destination. The daemon extracts the archive at the target location, preserving file metadata. This avoids mounting filesystems or using network protocols.

Why designed this way?

This design keeps file copying simple and secure by using Docker's existing API and daemon communication. It avoids exposing container internals over the network and works consistently across platforms. Alternatives like SSH would require extra setup and security considerations.

Host Filesystem
  │
  ├─> docker cp command
  │
Docker Daemon ── Streams tar archive ──> Container Filesystem

Copy direction can be host→container or container→host using this archive stream.

Myth Busters - 4 Common Misconceptions

Quick: Does copying files into a container change the original Docker image? Commit yes or no.

Common Belief: Copying files into a container modifies the Docker image permanently.

Reality: Copying files only changes the running container's filesystem, not the original image used to create it.

Why it matters: Believing this causes confusion when changes disappear after container removal, leading to wasted time debugging.

Quick: Can you copy files from a container that is stopped? Commit yes or no.

Common Belief: You cannot copy files from a stopped container because it is not running.

Reality: You can copy files from stopped containers since their filesystem still exists on disk.

Why it matters: This misconception limits your ability to retrieve logs or data from stopped containers, hindering troubleshooting.

Quick: Does 'docker cp' use network protocols like SSH to transfer files? Commit yes or no.

Common Belief: 'docker cp' uses SSH or similar network protocols to copy files.

Reality: 'docker cp' uses Docker daemon's API and streams tar archives internally without network protocols.

Why it matters: Misunderstanding this leads to wrong assumptions about security and performance of file copying.

Quick: Is copying files always better than using volumes for sharing data? Commit yes or no.

Common Belief: Copying files is always the best way to share data with containers.

Reality: Volumes are better for persistent and ongoing data sharing; copying is for one-time transfers.

Why it matters: Using copying instead of volumes can cause data loss and inefficient workflows.

Expert Zone

1

Copying files does not trigger container rebuilds, so changes are ephemeral unless committed to a new image.

2

File permissions and ownership inside containers can differ from the host, causing subtle bugs if not handled carefully.

3

Copying large files can be slow because 'docker cp' streams archives through the Docker daemon, which can be a bottleneck.

When NOT to use

Avoid using 'docker cp' for frequent or large data transfers; instead, use Docker volumes or bind mounts for persistent and efficient sharing. For remote hosts, use SSH or dedicated file transfer tools.

Production Patterns

In production, 'docker cp' is mainly used for quick debugging or extracting logs. Persistent data is managed with volumes or external storage. Automated pipelines avoid copying files manually by baking assets into images or using shared storage.

Connections

Docker Volumes

Alternative method for sharing data with containers

Understanding copying files clarifies when volumes are more efficient for persistent data.

SSH File Transfer

Different protocol for remote file copying

Knowing 'docker cp' does not use SSH highlights the need for other tools when working with remote containers.

File System Mounting (Operating Systems)

Underlying concept of sharing filesystems between environments

Recognizing how mounting works helps understand why copying files is sometimes necessary instead of direct access.

Common Pitfalls

#1 Trying to copy files into a container using the container name but misspelling it.

Wrong approach:

docker cp myfile.txt contaner123:/app/

Correct approach:

docker cp myfile.txt container123:/app/

Root cause:

Typos in container names cause Docker to fail finding the container.

#2 Copying files to a container path that does not exist without creating it first.

Wrong approach:

docker cp config.json container123:/nonexistent/path/

Correct approach:

docker exec container123 mkdir -p /nonexistent/path && docker cp config.json container123:/nonexistent/path/

Root cause:

Docker does not create destination directories automatically during copy.

#3 Assuming copied files persist after container removal without committing to an image or using volumes.

Wrong approach:

docker cp myfile.txt container123:/app/ # then docker rm container123

Correct approach:

docker commit container123 newimage:tag # or use volumes for persistence

Root cause:

Containers are ephemeral; copied files vanish when container is deleted unless saved.

Key Takeaways

Copying files to and from containers moves data between your host and container without rebuilding images.

The 'docker cp' command streams files as archives through the Docker daemon, working even with stopped containers.

Copying affects only the running container's filesystem, not the original image, so changes are temporary unless saved.

For persistent or frequent data sharing, Docker volumes are a better choice than copying files.

Understanding the limitations and mechanism of copying files helps avoid common mistakes and improves container workflows.