Why Cypress auto-retries reduce flakiness - Framework Benefits

Framework Mode - Why Cypress auto-retries reduce flakiness

Folder Structure of a Cypress Test Project

cypress/
├── e2e/                  # Test files (end-to-end tests)
│   ├── login.cy.js       # Example test for login
│   └── dashboard.cy.js   # Example test for dashboard
├── fixtures/             # Test data files (JSON, etc.)
│   └── users.json
├── support/              # Custom commands and reusable code
│   ├── commands.js       # Custom Cypress commands
│   └── e2e.js            # Support file loaded before tests
cypress.config.js         # Cypress configuration file
package.json              # Project dependencies and scripts

Test Framework Layers in Cypress

• Test Layer (cypress/e2e): Contains test scripts that describe user actions and assertions.
• Support Layer (cypress/support): Holds reusable commands and setup code to keep tests clean.
• Fixtures Layer (cypress/fixtures): Stores test data like user info or mock responses.
• Configuration Layer (cypress.config.js): Defines environment settings, base URLs, and browser options.
• Driver Layer: Cypress itself acts as the driver controlling the browser and retrying commands automatically.

Configuration Patterns for Cypress Auto-Retries

Cypress automatically retries commands and assertions until they pass or a timeout is reached. You can configure retry behavior in cypress.config.js or test-specific options.

// cypress.config.js
export default {
  e2e: {
    baseUrl: 'https://example.com',
    defaultCommandTimeout: 4000,  // Time Cypress waits before failing a command
    retries: {
      runMode: 2,    // Retry failed tests twice in CI runs
      openMode: 1    // Retry once when running locally
    }
  }
}

This setup helps tests recover from temporary delays or slow loading elements, reducing false failures.

Test Reporting and CI/CD Integration

• Test Reports: Cypress generates detailed HTML reports showing which tests passed, failed, and retried.
• CI/CD Integration: Configure your CI pipeline (GitHub Actions, Jenkins, etc.) to run Cypress tests with retries enabled to reduce flaky failures.
• Flakiness Reduction: Auto-retries help avoid flaky test failures caused by network delays or slow UI rendering.

Best Practices for Using Cypress Auto-Retries

1. Use Explicit Assertions: Write clear assertions so Cypress knows exactly what to retry.
2. Set Reasonable Timeouts: Adjust defaultCommandTimeout to balance test speed and reliability.
3. Limit Retries: Use retries sparingly to avoid hiding real bugs.
4. Use Custom Commands: Encapsulate retry logic in commands for reuse and clarity.
5. Monitor Flaky Tests: Track tests that often retry and investigate root causes.

Self-Check Question

In the folder structure shown, where would you add a new custom command that implements a retryable login action?

Key Result

Cypress auto-retries commands and assertions to reduce flaky test failures caused by temporary delays.

Practice

1. Why does Cypress automatically retry commands during test execution?

easy

A. To wait for elements to become ready and reduce test flakiness

B. To slow down tests for better readability

C. To randomly change test data during execution

D. To skip tests that are likely to fail

Solution

1. Step 1: Understand Cypress command retries

   Cypress retries commands automatically to wait for elements to appear or become ready before continuing.

2. Step 2: Connect retries to test stability

   This retry behavior helps reduce flaky tests caused by timing issues or slow page loads.

3. Final Answer:

   To wait for elements to become ready and reduce test flakiness -> Option A

4. Quick Check:

   Auto-retries improve stability = B [OK]

Hint: Auto-retry means waiting for elements, not slowing tests [OK]

Common Mistakes:

• Thinking retries slow down tests intentionally
• Believing retries change test data
• Assuming retries skip tests

2. Which of the following is the correct Cypress command syntax that benefits from auto-retries?

easy

A. cy.click('.btn')

B. cy.waitFor('.btn').click()

C. cy.find('.btn').click()

D. cy.get('.btn').click()

Solution

1. Step 1: Identify valid Cypress command syntax

   The correct command to get an element and click it is cy.get('.btn').click().

2. Step 2: Understand auto-retry usage

   cy.get() automatically retries until the element appears, making tests less flaky.

3. Final Answer:

   cy.get('.btn').click() -> Option D

4. Quick Check:

   Correct command with auto-retry = A [OK]

Hint: Use cy.get() for auto-retry, not cy.waitFor or cy.click [OK]

Common Mistakes:

• Using non-existent commands like cy.waitFor
• Confusing cy.find with cy.get
• Trying to click directly with cy.click('.btn')

3. Consider this Cypress test snippet:

cy.get('#submit').click()
cy.get('.result').should('contain', 'Success')

What happens if the '.result' element appears after 2 seconds?

medium

A. Test fails immediately because element is not found

B. Cypress retries and waits up to default timeout, then passes if element appears

C. Test passes without checking the '.result' element

D. Test throws a syntax error

Solution

1. Step 1: Analyze Cypress auto-retry behavior on assertions

   Cypress retries cy.get('.result') and the assertion until the element appears or timeout expires.

2. Step 2: Understand timing and test outcome

   If '.result' appears within the default timeout (usually 4 seconds), the test passes after retrying.

3. Final Answer:

   Cypress retries and waits up to default timeout, then passes if element appears -> Option B

4. Quick Check:

   Auto-retry waits for element = D [OK]

Hint: Cypress waits and retries assertions before failing [OK]

Common Mistakes:

• Assuming immediate failure if element not found instantly
• Thinking test passes without checking element
• Expecting syntax errors from valid code

4. You wrote this Cypress test:

cy.get('.list-item').click()
cy.get('.detail').should('be.visible')

But the test sometimes fails because '.detail' is not visible immediately. What is the best fix?

medium

A. Add explicit wait like cy.wait(5000) before checking '.detail'

B. Rely on Cypress auto-retry by keeping the code as is

C. Use cy.get('.detail', { timeout: 10000 }).should('be.visible')

D. Replace cy.get with cy.find to speed up test

Solution

1. Step 1: Identify problem with visibility timing

   The '.detail' element is not visible immediately, causing flaky failures.

2. Step 2: Use timeout option to extend auto-retry wait

   Adding { timeout: 10000 } to cy.get() lets Cypress retry longer before failing.

3. Final Answer:

   Use cy.get('.detail', { timeout: 10000 }).should('be.visible') -> Option C

4. Quick Check:

   Extend timeout for retries = A [OK]

Hint: Extend timeout in cy.get instead of adding fixed waits [OK]

Common Mistakes:

• Adding fixed waits which slow tests unnecessarily
• Ignoring retries and expecting instant visibility
• Using cy.find incorrectly

5. You want to test a button that appears only after a network call finishes. Which approach best uses Cypress auto-retries to reduce flakiness?

hard

A. Use cy.get('button').should('be.visible') without extra waits

B. Add cy.wait(10000) before checking the button

C. Use setTimeout in test to delay checking button

D. Manually refresh the page until button appears

Solution

1. Step 1: Understand Cypress auto-retry for dynamic elements

   Cypress automatically retries cy.get() and assertions until the button appears or timeout.

2. Step 2: Avoid fixed waits or manual delays

   Using fixed waits or manual delays slows tests and can cause flakiness if timing changes.

3. Final Answer:

   Use cy.get('button').should('be.visible') without extra waits -> Option A

4. Quick Check:

   Rely on auto-retry, avoid fixed waits = C [OK]

Hint: Trust Cypress retries instead of fixed waits or manual delays [OK]

Common Mistakes:

• Adding long fixed waits that slow tests
• Using setTimeout which is not Cypress-friendly
• Refreshing page manually instead of waiting