Copying files to and from containers in Docker - Deep Dive

Overview - Copying files to and from containers

What is it?

Copying files to and from containers means moving files between your computer and a running Docker container. This lets you add files into a container or get files out without rebuilding the container. It is useful for sharing data, logs, or configuration between your host and container.

Why it matters

Without the ability to copy files, you would have to rebuild containers every time you want to change or retrieve files, which slows down development and debugging. Copying files makes working with containers flexible and efficient, saving time and effort.

Where it fits

You should know basic Docker commands and how containers work before learning this. After this, you can explore Docker volumes for persistent storage and Docker networking for container communication.

Mental Model

Core Idea

Copying files to and from containers is like moving files between two computers connected by a cable, allowing easy sharing without changing the computers themselves.

Think of it like...

Imagine your container is a USB drive plugged into your computer. You can drag files onto it or copy files off it without changing the drive’s contents permanently.

Host Machine
  │
  ├── docker cp <source> <container>:<destination>  (copy to container)
  └── docker cp <container>:<source> <destination>  (copy from container)

Container
  └── Filesystem

Commands move files across this boundary.

Build-Up - 7 Steps

1

Foundation

Understanding Docker Containers

Concept: Learn what a Docker container is and how it runs isolated from your host system.

A Docker container is like a mini-computer running inside your main computer. It has its own file system and processes but shares the host's kernel. You can start, stop, and interact with containers using Docker commands.

Result

You understand that containers have their own files separate from your host.

Knowing containers have isolated filesystems explains why copying files is needed to move data in or out.

2

Foundation

Basic Docker CLI Commands

Concept: Familiarize with Docker commands to list and manage containers.

Use 'docker ps' to see running containers and 'docker exec' to run commands inside a container. For example, 'docker exec -it ls /' lists files inside the container root.

Result

You can identify containers and explore their filesystems.

Being able to access container files helps you know what files you might want to copy.

3

Intermediate

Copying Files Into Containers

🤔Before reading on: do you think copying files into a container changes the container image or just the running container? Commit to your answer.

Concept: Learn how to copy files from your host into a running container using 'docker cp'.

The command 'docker cp myfile.txt :/app/' copies 'myfile.txt' from your host into the '/app/' folder inside the container. This changes the container's filesystem but not the original image.

Result

The file appears inside the container at the specified path.

Understanding that copying files affects only the running container prevents confusion about image immutability.

4

Intermediate

Copying Files Out of Containers

🤔Before reading on: do you think you can copy files from a stopped container? Commit to your answer.

Concept: Learn how to copy files from a container back to your host.

Use 'docker cp :/var/log/app.log ./app.log' to copy a log file from the container to your current host directory. This works even if the container is stopped.

Result

The file from the container is now on your host machine.

Knowing you can copy files from stopped containers helps in debugging and data retrieval.

5

Intermediate

Copying Directories and Permissions

Concept: Learn how to copy entire folders and preserve file permissions.

You can copy directories by specifying folder paths, e.g., 'docker cp ./config :/app/'. Docker preserves file permissions and metadata during copy, which is important for executables or config files.

Result

The whole directory and its contents appear inside the container with correct permissions.

Preserving permissions avoids runtime errors caused by wrong file access inside containers.

6

Advanced

Copying Files vs. Using Volumes

🤔Before reading on: do you think copying files is better than volumes for persistent data? Commit to your answer.

Concept: Understand when to use file copying and when to use Docker volumes for sharing data.

Copying files is good for one-time transfers or small changes. Volumes are better for ongoing data sharing and persistence across container restarts. Volumes mount host directories or managed storage directly into containers.

Result

You can choose the right method for your use case.

Knowing the tradeoffs helps optimize container data management and avoid data loss.

7

Expert

Copy Command Internals and Limitations

🤔Before reading on: do you think 'docker cp' uses network protocols or direct filesystem access? Commit to your answer.

Concept: Learn how 'docker cp' works under the hood and its limitations.

'docker cp' uses the Docker API to archive files inside the container and transfer them over the Docker daemon connection. It does not use network protocols like SSH. It cannot copy files from containers running on remote hosts without extra setup.

Result

You understand why 'docker cp' is fast but limited to local Docker hosts.

Knowing the mechanism explains why copying large files can be slow and why remote copying needs other tools.

Under the Hood

'docker cp' works by creating a tar archive of the source files inside the container or host, then streams this archive through the Docker daemon to the destination. The daemon extracts the archive at the target location, preserving file metadata. This avoids mounting filesystems or using network protocols.

Why designed this way?

This design keeps file copying simple and secure by using Docker's existing API and daemon communication. It avoids exposing container internals over the network and works consistently across platforms. Alternatives like SSH would require extra setup and security considerations.

Host Filesystem
  │
  ├─> docker cp command
  │
Docker Daemon ── Streams tar archive ──> Container Filesystem

Copy direction can be host→container or container→host using this archive stream.

Myth Busters - 4 Common Misconceptions

Quick: Does copying files into a container change the original Docker image? Commit yes or no.

Common Belief: Copying files into a container modifies the Docker image permanently.

Reality: Copying files only changes the running container's filesystem, not the original image used to create it.

Why it matters: Believing this causes confusion when changes disappear after container removal, leading to wasted time debugging.

Quick: Can you copy files from a container that is stopped? Commit yes or no.

Common Belief: You cannot copy files from a stopped container because it is not running.

Reality: You can copy files from stopped containers since their filesystem still exists on disk.

Why it matters: This misconception limits your ability to retrieve logs or data from stopped containers, hindering troubleshooting.

Quick: Does 'docker cp' use network protocols like SSH to transfer files? Commit yes or no.

Common Belief: 'docker cp' uses SSH or similar network protocols to copy files.

Reality: 'docker cp' uses Docker daemon's API and streams tar archives internally without network protocols.

Why it matters: Misunderstanding this leads to wrong assumptions about security and performance of file copying.

Quick: Is copying files always better than using volumes for sharing data? Commit yes or no.

Common Belief: Copying files is always the best way to share data with containers.

Reality: Volumes are better for persistent and ongoing data sharing; copying is for one-time transfers.

Why it matters: Using copying instead of volumes can cause data loss and inefficient workflows.

Expert Zone

1

Copying files does not trigger container rebuilds, so changes are ephemeral unless committed to a new image.

2

File permissions and ownership inside containers can differ from the host, causing subtle bugs if not handled carefully.

3

Copying large files can be slow because 'docker cp' streams archives through the Docker daemon, which can be a bottleneck.

When NOT to use

Avoid using 'docker cp' for frequent or large data transfers; instead, use Docker volumes or bind mounts for persistent and efficient sharing. For remote hosts, use SSH or dedicated file transfer tools.

Production Patterns

In production, 'docker cp' is mainly used for quick debugging or extracting logs. Persistent data is managed with volumes or external storage. Automated pipelines avoid copying files manually by baking assets into images or using shared storage.

Connections

Docker Volumes

Alternative method for sharing data with containers

Understanding copying files clarifies when volumes are more efficient for persistent data.

SSH File Transfer

Different protocol for remote file copying

Knowing 'docker cp' does not use SSH highlights the need for other tools when working with remote containers.

File System Mounting (Operating Systems)

Underlying concept of sharing filesystems between environments

Recognizing how mounting works helps understand why copying files is sometimes necessary instead of direct access.

Common Pitfalls

#1 Trying to copy files into a container using the container name but misspelling it.

Wrong approach:

docker cp myfile.txt contaner123:/app/

Correct approach:

docker cp myfile.txt container123:/app/

Root cause:

Typos in container names cause Docker to fail finding the container.

#2 Copying files to a container path that does not exist without creating it first.

Wrong approach:

docker cp config.json container123:/nonexistent/path/

Correct approach:

docker exec container123 mkdir -p /nonexistent/path && docker cp config.json container123:/nonexistent/path/

Root cause:

Docker does not create destination directories automatically during copy.

#3 Assuming copied files persist after container removal without committing to an image or using volumes.

Wrong approach:

docker cp myfile.txt container123:/app/ # then docker rm container123

Correct approach:

docker commit container123 newimage:tag # or use volumes for persistence

Root cause:

Containers are ephemeral; copied files vanish when container is deleted unless saved.

Key Takeaways

Copying files to and from containers moves data between your host and container without rebuilding images.

The 'docker cp' command streams files as archives through the Docker daemon, working even with stopped containers.

Copying affects only the running container's filesystem, not the original image, so changes are temporary unless saved.

For persistent or frequent data sharing, Docker volumes are a better choice than copying files.

Understanding the limitations and mechanism of copying files helps avoid common mistakes and improves container workflows.

Practice

1. What does the docker cp command do?

easy

A. Copies files or folders between your computer and a Docker container

B. Copies Docker images between registries

C. Copies environment variables from a container

D. Copies container logs to a file

Solution

1. Step 1: Understand the purpose of docker cp

   The docker cp command is designed to transfer files or folders between the host machine and a Docker container.

2. Step 2: Differentiate from other Docker commands

   Other options relate to images, environment variables, or logs, which are not handled by docker cp.

3. Final Answer:

   Copies files or folders between your computer and a Docker container -> Option A

4. Quick Check:

   docker cp = copy files/folders [OK]

Hint: Remember: cp means copy files between host and container [OK]

Common Mistakes:

• Confusing docker cp with image transfer commands
• Thinking it copies logs or environment variables
• Assuming it copies container settings

2. Which of the following is the correct syntax to copy a file named app.log from your computer to a container named webapp at path /var/log/?

easy

A. docker cp app.log webapp:/var/log/

B. docker cp webapp:/var/log/ app.log

C. docker copy app.log webapp:/var/log/

D. docker cp /var/log/ app.log webapp

Solution

1. Step 1: Identify source and destination in docker cp

   The syntax is docker cp [source] [destination]. To copy from host to container, source is local file, destination is container:path.

2. Step 2: Check each option's order and command correctness

   docker cp app.log webapp:/var/log/ correctly uses docker cp app.log webapp:/var/log/. docker cp webapp:/var/log/ app.log reverses source and destination. docker copy app.log webapp:/var/log/ uses wrong command docker copy. docker cp /var/log/ app.log webapp has wrong order and extra arguments.

3. Final Answer:

   docker cp app.log webapp:/var/log/ -> Option A

4. Quick Check:

   docker cp source destination [OK]

Hint: Source first, then destination with container:path [OK]

Common Mistakes:

• Swapping source and destination paths
• Using wrong command name like docker copy
• Missing container name or colon separator

3. Given a container named db with a file /data/dump.sql, what will be the result of running docker cp db:/data/dump.sql ./backup.sql?

medium

A. Throws an error because the destination must be a directory

B. Copies the file dump.sql from the container to the current directory as backup.sql

C. Copies the file backup.sql from the current directory to the container at /data/dump.sql

D. Copies the entire /data directory from the container to ./backup.sql

Solution

1. Step 1: Analyze the command syntax

   The command copies from container db path /data/dump.sql to local path ./backup.sql. This means copying a single file from container to host with a new name.

2. Step 2: Understand the effect on local filesystem

   The file dump.sql inside the container is copied and saved locally as backup.sql in the current directory.

3. Final Answer:

   Copies the file dump.sql from the container to the current directory as backup.sql -> Option B

4. Quick Check:

   docker cp container:file localfile [OK]

Hint: Container source first, local destination second [OK]

Common Mistakes:

• Confusing source and destination order
• Assuming destination must be a directory
• Thinking it copies directories when given a file

4. You run docker cp mycontainer:/app/config.json ./config.json but get an error: no such file or directory. What is the most likely cause?

medium

A. You forgot to specify the container ID instead of name

B. The destination path ./config.json is invalid on your host

C. The file /app/config.json does not exist inside the container

D. You need to run the command inside the container shell

Solution

1. Step 1: Understand the error message

   The error "no such file or directory" usually means the source file inside the container does not exist at the specified path.

2. Step 2: Evaluate other options

   Container name or ID can be used interchangeably if correct. Destination path on host is usually valid if relative path is used. Running inside container shell is not needed for docker cp.

3. Final Answer:

   The file /app/config.json does not exist inside the container -> Option C

4. Quick Check:

   Source file missing = error [OK]

Hint: Check source file path inside container first [OK]

Common Mistakes:

• Assuming destination path causes error
• Thinking container name must be ID only
• Trying to run docker cp inside container shell

5. You want to copy a folder named logs from your host to a running container appserver at path /var/app/. Which command correctly copies the entire folder including its contents?

hard

A. docker cp logs/* appserver:/var/app/

B. docker cp logs/ appserver:/var/app/

C. docker cp logs appserver:/var/app/

D. docker cp logs appserver:/var/app/logs

Solution

1. Step 1: Understand folder copy behavior

   To copy a folder and its contents, specify the folder as source and the full destination path including the folder name if you want it placed inside the container.

2. Step 2: Analyze each option

   docker cp logs appserver:/var/app/logs copies the entire logs folder into /var/app/logs, preserving folder structure. docker cp logs/ appserver:/var/app/ with trailing slash copies contents of logs into /var/app/ but does not create logs folder inside container. docker cp logs appserver:/var/app/ copies contents of logs into /var/app/ without creating logs folder. docker cp logs/* appserver:/var/app/ copies only files inside logs, not subfolders.

3. Final Answer:

   docker cp logs appserver:/var/app/logs -> Option D

4. Quick Check:

   Include folder name in destination to copy folder [OK]

Hint: Add folder name in destination path to copy folder [OK]

Common Mistakes:

• Using trailing slash and losing folder structure
• Copying only files inside folder, not subfolders
• Not specifying full destination path including folder name