Test prioritization in Testing Fundamentals - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why prioritize tests in a large project?

Imagine you have a big software project with thousands of tests but limited time to run them all before a release. Why is test prioritization important in this situation?

A To run tests that take the longest time first to finish testing faster.

B To run tests randomly so every test has an equal chance to run.

C To run only the oldest tests because they are more reliable.

D To run the tests that are most likely to find bugs first, saving time and improving quality.

💡 Hint

Think about which tests give the best value early.

Prioritizing tests helps find bugs earlier by running the most important or risky tests first, which saves time and improves software quality.

❓ Predict Output

intermediate

Output of prioritized test execution order

Given the following test cases with priorities, what is the order of test execution?

Testing Fundamentals

tests = [
  {'name': 'LoginTest', 'priority': 2},
  {'name': 'PaymentTest', 'priority': 1},
  {'name': 'SignupTest', 'priority': 3},
  {'name': 'ProfileTest', 'priority': 2}
]

# Tests run in ascending order of priority (1 is highest)
execution_order = [test['name'] for test in sorted(tests, key=lambda x: x['priority'])]
print(execution_order)

A ['SignupTest', 'LoginTest', 'ProfileTest', 'PaymentTest']

B ['LoginTest', 'ProfileTest', 'PaymentTest', 'SignupTest']

C ['PaymentTest', 'LoginTest', 'ProfileTest', 'SignupTest']

D ['ProfileTest', 'LoginTest', 'SignupTest', 'PaymentTest']

💡 Hint

Lower priority number means higher priority.

The tests are sorted by their priority number ascending: 1 (PaymentTest), 2 (LoginTest, ProfileTest), 3 (SignupTest).

❓ assertion

advanced

Correct assertion for test priority filtering

You want to write an assertion to check that only tests with priority 1 or 2 are selected for execution. Which assertion is correct?

Testing Fundamentals

selected_tests = [
  {'name': 'LoginTest', 'priority': 2},
  {'name': 'PaymentTest', 'priority': 1},
  {'name': 'SignupTest', 'priority': 3}
]

# Which assertion below correctly verifies all selected tests have priority 1 or 2?

A assert all(test['priority'] in [1, 2] for test in selected_tests)

B assert all(test['priority'] == 1 or 2 for test in selected_tests)

C assert any(test['priority'] == 1 or 2 for test in selected_tests)

D assert all(test['priority'] == 1 and test['priority'] == 2 for test in selected_tests)

💡 Hint

Use all() and check membership with in.

Option A correctly checks that every test's priority is either 1 or 2. Option A uses any() which only requires one test to match. Option A and D have logical errors.

🔧 Debug

advanced

Identify the bug in test prioritization code

What is the bug in this Python code that tries to prioritize tests by priority but fails?

Testing Fundamentals

tests = [
  {'name': 'TestA', 'priority': 3},
  {'name': 'TestB', 'priority': 1},
  {'name': 'TestC', 'priority': 2}
]

# Attempt to sort tests by priority
sorted_tests = sorted(tests, key=lambda test: test.priority)
print([t['name'] for t in sorted_tests])

A The sorted function is missing the reverse=True parameter to sort descending.

B The code uses test.priority instead of test['priority'], causing an AttributeError.

C The lambda function should return test['name'] instead of test.priority.

D The print statement should use sorted_tests.name instead of list comprehension.

💡 Hint

Check how dictionary keys are accessed in Python.

Dictionaries use bracket notation with keys as strings, not dot notation. Using test.priority causes an AttributeError.

❓ framework

expert

Best practice for integrating test prioritization in CI pipeline

You want to integrate test prioritization in your Continuous Integration (CI) pipeline to run critical tests first. Which approach is best?

A Tag tests with priority labels and configure the CI to run high-priority tags first, then others later.

B Run all tests in random order every time to avoid bias.

C Run only the oldest tests in the suite to ensure stability.

D Run tests manually outside the CI pipeline to control order.

💡 Hint

Think about automation and maintainability in CI.

Tagging tests by priority and configuring CI to run high-priority tests first is a scalable and automated way to prioritize tests in CI pipelines.

Practice

1. What is the main goal of test prioritization in software testing?

easy

A. To run the most important tests first to find bugs faster

B. To run all tests in alphabetical order

C. To skip tests that are easy to pass

D. To run tests only after the software is released

Solution

1. Step 1: Understand the purpose of test prioritization

   Test prioritization means deciding which tests to run first based on their importance or likelihood to find bugs.

2. Step 2: Identify the correct goal

   The goal is to save time and find bugs faster by running important tests first.

3. Final Answer:

   To run the most important tests first to find bugs faster -> Option A

4. Quick Check:

   Test prioritization = run important tests first [OK]

Hint: Remember: prioritize tests that catch bugs quickly [OK]

Common Mistakes:

• Thinking test order is random
• Believing all tests have equal priority
• Skipping tests without reason

2. Which of the following is a good factor to consider when prioritizing tests?

easy

A. The number of testers available

B. The color of the test case document

C. The font size used in test descriptions

D. Recent changes in the code

Solution

1. Step 1: Identify relevant factors for test prioritization

   Good factors include recent code changes, past failures, and test importance.

2. Step 2: Choose the factor related to code changes

   Recent changes in code can introduce new bugs, so tests covering those areas should run first.

3. Final Answer:

   Recent changes in the code -> Option D

4. Quick Check:

   Prioritize tests by recent code changes [OK]

Hint: Focus on code changes, not irrelevant details [OK]

Common Mistakes:

• Choosing unrelated factors like document color
• Ignoring recent code updates
• Confusing tester count with test priority

3. Given the following test cases and their last failure dates, which test should be run first based on test prioritization?

Test A: failed 2 days ago
Test B: passed last run
Test C: failed 10 days ago
Test D: never failed

medium

A. Test B

B. Test D

C. Test A

D. Test C

Solution

1. Step 1: Analyze failure recency

   Tests that failed recently are more likely to find bugs again, so they get higher priority.

2. Step 2: Compare failure dates

   Test A failed 2 days ago, which is more recent than Test C (10 days ago). Tests B and D did not fail recently.

3. Final Answer:

   Test A -> Option C

4. Quick Check:

   Run tests with recent failures first [OK]

Hint: Pick tests with the most recent failures first [OK]

Common Mistakes:

• Choosing tests that never failed
• Ignoring failure dates
• Picking tests that passed last run

4. You have a test suite with 4 tests. The priority order should be based on recent code changes and past failures. The current order is:

1. Test X (no recent changes, no failures)
2. Test Y (recent changes, no failures)
3. Test Z (no recent changes, failed last run)
4. Test W (recent changes, failed last run)

Which test should be run first to follow best test prioritization?

medium

A. Test W

B. Test X

C. Test Z

D. Test Y

Solution

1. Step 1: Combine recent changes and past failures

   Tests with both recent changes and past failures have highest priority.

2. Step 2: Identify test with both factors

   Test W has recent changes and failed last run, so it should run first.

3. Final Answer:

   Test W -> Option A

4. Quick Check:

   Prioritize tests with recent changes and failures [OK]

Hint: Pick tests with both recent changes and failures first [OK]

Common Mistakes:

• Ignoring one of the prioritization factors
• Running tests without recent changes first
• Choosing tests that never failed

5. A team has 5 tests with these details:

Test 1: covers critical feature, no recent changes, failed last run
Test 2: covers minor feature, recent changes, passed last run
Test 3: covers critical feature, recent changes, passed last run
Test 4: covers minor feature, no recent changes, never failed
Test 5: covers critical feature, recent changes, failed last run

Which order best follows test prioritization principles?

hard

A. Test 1, Test 5, Test 3, Test 2, Test 4

B. Test 5, Test 1, Test 3, Test 2, Test 4

C. Test 5, Test 3, Test 1, Test 2, Test 4

D. Test 4, Test 2, Test 3, Test 1, Test 5

Solution

1. Step 1: Identify highest priority tests

   Tests covering critical features with recent changes and failures have highest priority. Test 5 fits all three.

2. Step 2: Order remaining tests by priority

   Next is Test 1 (critical, failed but no recent changes), then Test 3 (critical, recent changes, passed), followed by Test 2 and Test 4 (minor features, lower priority).

3. Final Answer:

   Test 5, Test 1, Test 3, Test 2, Test 4 -> Option B

4. Quick Check:

   Prioritize critical, recent changes, failures first [OK]

Hint: Order by criticality, recent changes, then failures [OK]

Common Mistakes:

• Ignoring feature criticality
• Not combining multiple factors
• Running low priority tests first