Docker-based test execution in PyTest - Framework Patterns

Framework Mode - Docker-based test execution

Folder Structure

my-pytest-project/
├── tests/
│   ├── test_example.py
│   └── __init__.py
├── src/
│   └── app_code.py
├── docker/
│   ├── Dockerfile
│   └── entrypoint.sh
├── pytest.ini
├── requirements.txt
└── README.md

This structure keeps tests in tests/, source code in src/, and Docker files in docker/.

Test Framework Layers

• Test Layer: tests/ folder with pytest test files.
• Application Layer: src/ folder with application code.
• Docker Layer: docker/Dockerfile and entrypoint.sh to build and run tests inside a container.
• Configuration Layer: pytest.ini for pytest settings and requirements.txt for dependencies.
• Utilities Layer: Helper scripts or fixtures can be added inside tests/ or a separate utils/ folder if needed.

Configuration Patterns

• Environment Variables: Use environment variables in Docker to pass settings like API URLs or credentials.
• pytest.ini: Configure pytest options such as markers, test paths, and logging.
• requirements.txt: List all Python dependencies to install inside the Docker container.
• Dockerfile: Defines the Python environment, installs dependencies, copies source and tests, and runs pytest.
• Entrypoint Script: entrypoint.sh can set environment variables and execute pytest with arguments.

Test Reporting and CI/CD Integration

• Test Reports: Use pytest plugins like pytest-html or pytest-junitxml to generate reports inside the container.
• Volume Mounts: Mount host folders to container to save reports outside the container.
• CI/CD: Integrate Docker test runs in pipelines (GitHub Actions, Jenkins, GitLab CI) by running docker build and docker run commands.
• Exit Codes: Docker container exits with pytest exit code to indicate pass/fail status to CI.

Best Practices

1. Isolate Tests: Run tests inside Docker to ensure consistent environment across machines.
2. Keep Dockerfile Simple: Use lightweight base images (e.g., python:3.12-slim) and layer caching.
3. Use Entrypoint Script: Manage environment setup and test execution cleanly.
4. Separate Config: Use environment variables and config files instead of hardcoding values.
5. Save Artifacts: Mount volumes to keep logs and reports after container finishes.

Self Check

Where in this folder structure would you add a new pytest fixture to share setup code for multiple tests?

Key Result

Use Docker to run pytest tests in a consistent, isolated environment with clear folder structure and config.

Practice

1. What is the main benefit of running pytest tests inside a Docker container?

easy

A. It ensures tests run in a clean, consistent environment every time.

B. It makes tests run faster than on the host machine.

C. It automatically fixes failing tests.

D. It allows tests to run without writing any test code.

Solution

1. Step 1: Understand Docker container purpose

   Docker containers provide isolated environments that are the same every time they run.

2. Step 2: Connect to pytest benefits

   Running pytest inside Docker avoids environment differences causing test failures.

3. Final Answer:

   It ensures tests run in a clean, consistent environment every time. -> Option A

4. Quick Check:

   Docker isolation = consistent test environment [OK]

Hint: Docker isolates environment to avoid setup issues [OK]

Common Mistakes:

• Thinking Docker speeds up tests automatically
• Believing Docker fixes test code errors
• Assuming tests run without writing code

2. Which of the following is the correct way to run pytest inside a Docker container with your current directory shared?

easy

A. docker run -p 8080:80 python:3.12 pytest

B. docker run -v $(pwd):/app -w /app python:3.12 pytest

C. docker build -t pytest-image . && docker run pytest-image

D. docker run --rm python:3.12 pytest /app

Solution

1. Step 1: Identify volume sharing syntax

   The -v $(pwd):/app option shares the current directory to /app inside the container.

2. Step 2: Check working directory and command

   -w /app sets the working directory, and pytest runs tests there.

3. Final Answer:

   docker run -v $(pwd):/app -w /app python:3.12 pytest -> Option B

4. Quick Check:

   Volume + workdir + pytest = correct run command [OK]

Hint: Use -v for volume and -w for working directory [OK]

Common Mistakes:

• Using -p instead of -v for volume
• Not setting working directory with -w
• Running pytest without mounting code

3. Given this Docker command:

docker run --rm -v $(pwd):/tests -w /tests python:3.12 pytest test_sample.py

What will happen if test_sample.py contains one passing and one failing test?

medium

A. Pytest will run only the passing test and ignore failures.

B. Pytest will skip the failing test and exit with zero code.

C. Docker will fail to start because of volume issues.

D. Pytest will run both tests and exit with a non-zero code indicating failure.

Solution

1. Step 1: Understand pytest behavior on failures

   Pytest runs all tests and returns a non-zero exit code if any test fails.

2. Step 2: Confirm Docker command mounts tests correctly

   The volume mounts current directory to /tests, so test_sample.py is accessible and runs.

3. Final Answer:

   Pytest will run both tests and exit with a non-zero code indicating failure. -> Option D

4. Quick Check:

   Failing test causes pytest exit code != 0 [OK]

Hint: Pytest exit code shows if tests passed or failed [OK]

Common Mistakes:

• Assuming pytest skips failing tests
• Thinking Docker volume mount fails here
• Believing pytest ignores failures

4. You run this command:

docker run -v $(pwd):/app python:3.12 pytest

But pytest does not find any tests. What is the most likely cause?

medium

A. You did not set the working directory inside the container.

B. The Python image does not have pytest installed.

C. Docker volume syntax is incorrect.

D. Tests must be named test_docker.py to run.

Solution

1. Step 1: Check working directory usage

   Without -w /app, pytest runs in root, not where tests are mounted.

2. Step 2: Understand pytest test discovery

   Pytest looks for tests in current directory; wrong directory means no tests found.

3. Final Answer:

   You did not set the working directory inside the container. -> Option A

4. Quick Check:

   Working directory missing = no tests found [OK]

Hint: Always set -w to your mounted test folder [OK]

Common Mistakes:

• Assuming pytest is missing in python image by default
• Blaming volume syntax without error
• Thinking test file names must be exact

5. You want to run pytest inside Docker and save the test report to your host machine. Which command correctly achieves this?

hard

A. docker run -v $(pwd):/app -w /app python:3.12 pytest --junitxml=/tmp/report.xml

B. docker run python:3.12 pytest --junitxml=/app/report.xml

C. docker run -v $(pwd):/app -w /app python:3.12 pytest --junitxml=/app/report.xml

D. docker run -v $(pwd):/app python:3.12 pytest --junitxml=report.xml

Solution

1. Step 1: Ensure volume mount and working directory

   Mount current directory to /app and set working directory to /app so files are accessible.

2. Step 2: Specify report path inside container

   Use --junitxml=/app/report.xml to save report inside mounted folder, making it visible on host.

3. Final Answer:

   docker run -v $(pwd):/app -w /app python:3.12 pytest --junitxml=/app/report.xml -> Option C

4. Quick Check:

   Volume + workdir + report path inside mount = correct [OK]

Hint: Set -w and use full path for report inside mounted folder [OK]

Common Mistakes:

• Not setting working directory with -w
• Using relative path without mount inside container
• Omitting volume mount so report is lost