Visual regression testing concept in Cypress - Framework Patterns

Framework Mode - Visual regression testing concept

Folder Structure

cypress-project/
├── cypress/
│   ├── e2e/
│   │   ├── visual/
│   │   │   └── homepage_visual_spec.cy.js
│   │   └── functional/
│   │       └── login_spec.cy.js
│   ├── fixtures/
│   │   └── testData.json
│   ├── support/
│   │   ├── commands.js
│   │   ├── e2e.js
│   │   └── visualCommands.js
├── cypress.config.js
├── package.json
└── reports/
    └── visual-regression/
        └── baseline/
        └── diffs/
        └── latest/

Test Framework Layers

• Test Specs (cypress/e2e/visual/): Contains visual regression test scripts that capture and compare screenshots.
• Support Layer (cypress/support/): Custom commands and utilities for visual testing, like screenshot capture and comparison helpers.
• Fixtures (cypress/fixtures/): Static test data used in tests, e.g., URLs or user info.
• Configuration (cypress.config.js): Defines environment settings, viewport sizes, and plugins for visual testing.
• Reports (reports/visual-regression/): Stores baseline images, latest screenshots, and difference images for review.

Configuration Patterns

• Environment Variables: Use cypress.config.js to set base URLs and environment-specific settings (e.g., staging, production).
• Viewport Sizes: Define multiple viewport sizes in config or tests to test responsiveness visually.
• Baseline Management: Store baseline screenshots in reports/visual-regression/baseline/. Update baselines only after manual approval.
• Credentials: Use environment variables or cypress.env.json (ignored by VCS) to securely store sensitive data.
• Plugins: Integrate visual testing plugins like cypress-image-snapshot in cypress.config.js for screenshot comparison.

Test Reporting and CI/CD Integration

• Visual Test Reports: Generate reports showing passed/failed visual tests with side-by-side image diffs.
• CI/CD Integration: Run visual tests automatically on pull requests or merges using CI tools (GitHub Actions, Jenkins).
• Baseline Approval Workflow: Fail tests if visual differences exceed threshold; require manual baseline update to accept changes.
• Artifacts Storage: Save screenshots and diff images as build artifacts for team review.

Best Practices

• Keep Baselines Stable: Only update baseline images after confirming UI changes are intentional.
• Use Explicit Viewports: Test multiple screen sizes to catch responsive design issues visually.
• Isolate Visual Tests: Separate visual regression tests from functional tests for clarity and maintenance.
• Use Thresholds: Set pixel difference thresholds to avoid false positives from minor rendering differences.
• Automate in CI: Integrate visual tests in CI pipelines to catch UI regressions early.

Self Check

Where in this folder structure would you add a new visual regression test for the user profile page?

• a) cypress/e2e/visual/
• b) cypress/support/
• c) reports/visual-regression/
• d) cypress/fixtures/

Correct answer: a) cypress/e2e/visual/

Key Result

Organize visual regression tests in dedicated folders with baseline image management and CI integration for reliable UI change detection.

Practice

1. What is the main purpose of visual regression testing in Cypress?

easy

A. To check if the webpage links are working

B. To test the speed of webpage loading

C. To verify the correctness of backend data processing

D. To check if the webpage appearance changes unexpectedly after updates

Solution

1. Step 1: Understand visual regression testing goal

   Visual regression testing compares screenshots to detect visual changes.

2. Step 2: Identify the main purpose in Cypress context

   Cypress uses visual regression to ensure UI looks the same after code changes.

3. Final Answer:

   To check if the webpage appearance changes unexpectedly after updates -> Option D

4. Quick Check:

   Visual regression = check appearance changes [OK]

Hint: Visual regression tests page look, not speed or links [OK]

Common Mistakes:

• Confusing visual regression with performance testing
• Thinking it tests backend logic
• Assuming it checks link functionality

2. Which Cypress command is used to take a snapshot for visual regression testing?

easy

A. cy.get()

B. cy.visit()

C. cy.matchImageSnapshot()

D. cy.click()

Solution

1. Step 1: Identify commands related to visual regression

   cy.matchImageSnapshot() is designed to compare screenshots for visual testing.

2. Step 2: Differentiate from other Cypress commands

   cy.visit() loads pages, cy.get() selects elements, cy.click() simulates clicks.

3. Final Answer:

   cy.matchImageSnapshot() -> Option C

4. Quick Check:

   Snapshot command = cy.matchImageSnapshot() [OK]

Hint: Snapshot command has 'matchImage' in its name [OK]

Common Mistakes:

• Choosing cy.visit() which loads pages
• Confusing cy.get() with snapshot command
• Selecting cy.click() which triggers clicks

3. What will happen if the following Cypress test runs and the page appearance has changed since the last snapshot?

cy.visit('https://example.com')
cy.matchImageSnapshot('homepage')

medium

A. The test fails because the new screenshot differs from the saved snapshot

B. The test reloads the page automatically

C. The test passes because it always ignores visual differences

D. The test skips the snapshot comparison

Solution

1. Step 1: Understand snapshot comparison behavior

   cy.matchImageSnapshot compares current screenshot to saved snapshot and fails if different.

2. Step 2: Apply this to changed page appearance

   If the page looks different, the test detects difference and fails.

3. Final Answer:

   The test fails because the new screenshot differs from the saved snapshot -> Option A

4. Quick Check:

   Visual difference causes test failure [OK]

Hint: Changed appearance means snapshot test fails [OK]

Common Mistakes:

• Thinking test always passes regardless of changes
• Assuming test reloads page automatically
• Believing snapshot comparison is skipped

4. Identify the error in this Cypress visual regression test code:

cy.matchImageSnapshot('homepage')
cy.visit('https://example.com')

medium

A. cy.visit() should be called before cy.matchImageSnapshot()

B. cy.matchImageSnapshot() cannot take a string argument

C. cy.visit() is not a valid Cypress command

D. The test is missing cy.get() before snapshot

Solution

1. Step 1: Check command order for visual testing

   Page must be loaded with cy.visit() before taking a snapshot.

2. Step 2: Verify argument and command validity

   cy.matchImageSnapshot() accepts a string name; cy.visit() is valid; cy.get() not required here.

3. Final Answer:

   cy.visit() should be called before cy.matchImageSnapshot() -> Option A

4. Quick Check:

   Visit page first, then snapshot [OK]

Hint: Always visit page before snapshot command [OK]

Common Mistakes:

• Calling snapshot before loading page
• Thinking cy.matchImageSnapshot() disallows string names
• Assuming cy.visit() is invalid
• Believing cy.get() is mandatory before snapshot

5. You want to add visual regression testing for a login page that changes its welcome message based on the user. How should you handle this dynamic content in Cypress visual regression tests?

hard

A. Use a custom snapshot name for each user to save separate images

B. Ignore the dynamic area by masking or hiding it before snapshot

C. Remove cy.visit() to avoid loading the page

D. Run cy.matchImageSnapshot() twice to average results

Solution

1. Step 1: Understand problem with dynamic content

   Dynamic text changes cause snapshot differences and false failures.

2. Step 2: Choose best practice to handle dynamic parts

   Masking or hiding dynamic areas before snapshot avoids false positives.

3. Step 3: Evaluate other options

   Custom names create many snapshots; removing cy.visit() breaks test; averaging snapshots is invalid.

4. Final Answer:

   Ignore the dynamic area by masking or hiding it before snapshot -> Option B

5. Quick Check:

   Mask dynamic content to stabilize snapshots [OK]

Hint: Mask or hide changing parts before snapshot [OK]

Common Mistakes:

• Saving separate snapshots for each user causes clutter
• Skipping page load breaks test
• Trying to average snapshots is not supported