Continuous testing in CI/CD in Testing Fundamentals - Framework Patterns

Framework Mode - Continuous testing in CI/CD

Folder Structure

  project-root/
  ├── src/                  # Application source code
  ├── tests/                # Automated tests
  │   ├── unit/             # Unit tests
  │   ├── integration/      # Integration tests
  │   ├── e2e/              # End-to-end tests
  │   └── fixtures/         # Test data and mocks
  ├── ci/                   # CI/CD pipeline scripts and configs
  ├── config/               # Environment and test configurations
  │   ├── dev.env           # Development environment variables
  │   ├── staging.env       # Staging environment variables
  │   └── prod.env          # Production environment variables
  ├── reports/              # Test reports and logs
  ├── utils/                # Helper functions and utilities
  ├── package.json          # Project dependencies and scripts (example for JS)
  └── README.md             # Project documentation
  

Test Framework Layers

• Test Scripts Layer: Contains automated tests (unit, integration, e2e) that verify application behavior.
• Test Data & Fixtures Layer: Provides mock data and setup needed for tests to run consistently.
• Utilities Layer: Helper functions for common tasks like API calls, data generation, or custom assertions.
• Configuration Layer: Manages environment variables, test settings, and credentials for different deployment stages.
• CI/CD Pipeline Layer: Automates test execution on code changes, integrates with version control and deployment tools.
• Reporting Layer: Collects and presents test results clearly for developers and stakeholders.

Configuration Patterns

• Environment Files: Use separate files for dev, staging, and production to store URLs, credentials, and flags.
• Parameterization: Pass environment and browser info as parameters or environment variables to tests.
• Secrets Management: Store sensitive data securely outside the codebase, e.g., CI secrets or vaults.
• Config Loader Utility: Centralize reading and applying configuration to avoid duplication.
• Version Control: Keep config files (except secrets) under version control for traceability.

Test Reporting and CI/CD Integration

• Automated Test Runs: Trigger tests automatically on code commits or pull requests via CI tools (e.g., Jenkins, GitHub Actions).
• Clear Reports: Generate readable reports (HTML, JUnit XML) showing pass/fail status and error details.
• Notifications: Send alerts on test failures to developers via email or chat tools.
• Test Coverage: Include coverage reports to measure how much code is tested.
• Gatekeeping: Use test results to block deployments if critical tests fail, ensuring quality before release.

Framework Design Principles

1. Automate Everything: Tests should run automatically on every code change without manual steps.
2. Keep Tests Fast and Reliable: Fast tests give quick feedback; reliable tests avoid false failures.
3. Isolate Tests: Tests should not depend on each other or external state to avoid flaky results.
4. Use Clear Naming: Name tests and folders clearly to understand their purpose at a glance.
5. Integrate Early: Add tests to CI/CD pipelines early to catch issues sooner and improve confidence.

Self Check

Where in this folder structure would you add a new end-to-end test for the user login feature?

Key Result

Continuous testing in CI/CD means automating tests to run on every code change, integrated tightly with deployment pipelines for fast feedback and quality assurance.