Smoke testing and sanity testing in Testing Fundamentals - Test Execution Trace

Test Overview

This test performs a smoke test to check if the main features of the application load correctly. It then performs a sanity test to verify a specific feature works after a minor change.

Test Code - Selenium with unittest

Testing Fundamentals

import unittest
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class SmokeSanityTest(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Chrome()
        self.driver.get('https://example.com')

    def test_smoke_and_sanity(self):
        driver = self.driver
        # Smoke Test: Check homepage title
        self.assertIn('Example Domain', driver.title)

        # Smoke Test: Check main heading is present
        heading = WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.TAG_NAME, 'h1'))
        )
        self.assertEqual(heading.text, 'Example Domain')

        # Sanity Test: Click More Information link and verify URL
        more_info_link = driver.find_element(By.CSS_SELECTOR, 'a')
        more_info_link.click()

        WebDriverWait(driver, 10).until(
            EC.url_contains('iana.org/domains/example')
        )
        self.assertIn('iana.org/domains/example', driver.current_url)

    def tearDown(self):
        self.driver.quit()

if __name__ == '__main__':
    unittest.main()

Execution Trace - 6 Steps

Step	Action	System State	Assertion	Result
1	Test starts and browser opens Chrome	Chrome browser window opens at https://example.com homepage	—	PASS
2	Check homepage title contains 'Example Domain'	Browser shows homepage with title 'Example Domain'	Assert 'Example Domain' in page title	PASS
3	Wait for main heading <h1> to be present	Page loaded with visible main heading	Assert heading text equals 'Example Domain'	PASS
4	Find and click the 'More Information' link	User clicks link on homepage	—	PASS
5	Wait until URL contains 'iana.org/domains/example'	Browser navigates to IANA example domains page	Assert current URL contains 'iana.org/domains/example'	PASS
6	Test ends and browser closes	Browser window closes	—	PASS

Failure Scenario

Failing Condition: If the main heading <h1> is missing or text is incorrect

Execution Trace Quiz - 3 Questions

What does the smoke test verify in this test?

A The 'More Information' link navigates correctly

B The homepage title and main heading are present

C The browser can open and close

D The URL contains 'iana.org/domains/example'

Key Result

Perform smoke tests to quickly check if the main parts of the app load, then use sanity tests to verify specific features after changes.

Practice

1. What is the main purpose of smoke testing in software testing?

easy

A. To perform security testing on the application

B. To quickly check if the main features of a new build work

C. To verify the user interface design

D. To test detailed functionality of all features

Solution

1. Step 1: Understand smoke testing purpose

   Smoke testing is done to quickly verify that the main features of a new software build are working and the build is stable enough for further testing.

2. Step 2: Compare options with smoke testing definition

   To quickly check if the main features of a new build work matches this purpose exactly, while others describe different testing types.

3. Final Answer:

   To quickly check if the main features of a new build work -> Option B

4. Quick Check:

   Smoke testing = Quick main feature check [OK]

Hint: Smoke test = quick main feature check after build [OK]

Common Mistakes:

• Confusing smoke testing with detailed testing
• Thinking smoke testing checks UI design
• Mixing smoke testing with security testing

2. Which of the following is the correct description of sanity testing?

easy

A. Checking specific functionality after small changes or bug fixes

B. Testing all features in detail after a major release

C. Performing load testing to check performance

D. Running automated tests for regression

Solution

1. Step 1: Define sanity testing

   Sanity testing is a quick check of specific functionality or bug fixes after small changes to ensure they work as expected.

2. Step 2: Match options to sanity testing

   Checking specific functionality after small changes or bug fixes correctly describes this focused testing, while others describe different testing types.

3. Final Answer:

   Checking specific functionality after small changes or bug fixes -> Option A

4. Quick Check:

   Sanity testing = Focused check after small changes [OK]

Hint: Sanity test = check specific fixes after small changes [OK]

Common Mistakes:

• Confusing sanity testing with full regression testing
• Thinking sanity testing covers all features
• Mixing sanity testing with performance testing

3. Consider this scenario: A new build is deployed with multiple new features and bug fixes. Which testing should be done first to decide if detailed testing can proceed?

medium

A. Sanity testing on the fixed bugs only

B. Full regression testing on all features

C. Smoke testing on main features of the build

D. Performance testing under load

Solution

1. Step 1: Identify testing purpose after new build

   After a new build with many changes, smoke testing is done first to quickly check main features and decide if the build is stable for further testing.

2. Step 2: Evaluate options for initial testing

   Smoke testing on main features of the build matches this initial check. Sanity testing (Sanity testing on the fixed bugs only) is for small changes, not full builds. Full regression (Full regression testing on all features) and performance (Performance testing under load) come later.

3. Final Answer:

   Smoke testing on main features of the build -> Option C

4. Quick Check:

   New build? Start with smoke testing [OK]

Hint: New build? Start with smoke testing main features [OK]

Common Mistakes:

• Starting with sanity testing on big builds
• Skipping smoke testing and doing full regression immediately
• Confusing performance testing as first step

4. A tester runs sanity testing after a small bug fix but finds unrelated features failing. What is the likely cause?

medium

A. Sanity testing scope was too narrow, missing regression tests

B. Smoke testing was done incorrectly

C. The bug fix introduced new unrelated bugs

D. Performance testing was skipped

Solution

1. Step 1: Understand sanity testing scope

   Sanity testing focuses on verifying specific fixes or features, not the entire application.

2. Step 2: Analyze failure of unrelated features

   If unrelated features fail, it means sanity testing missed broader checks. Full regression testing is needed to catch such issues.

3. Final Answer:

   Sanity testing scope was too narrow, missing regression tests -> Option A

4. Quick Check:

   Unrelated failures after sanity test? Need regression [OK]

Hint: Sanity test checks fixes only; unrelated fails need regression [OK]

Common Mistakes:

• Blaming smoke testing for unrelated failures
• Assuming bug fix always causes unrelated failures
• Ignoring need for regression testing

5. A software team wants to improve testing efficiency after frequent small updates. Which approach best combines smoke and sanity testing?

hard

A. Perform performance testing before smoke or sanity testing

B. Skip smoke testing and run full regression after each update

C. Only run sanity testing on all features after every update

D. Run smoke testing on main features after every build, then sanity testing on specific fixes

Solution

1. Step 1: Understand testing goals for frequent small updates

   To improve efficiency, quick checks are needed to catch major issues early, then focused checks on fixes.

2. Step 2: Evaluate combined approach of smoke and sanity testing

   Running smoke testing on main features ensures build stability, followed by sanity testing on specific fixes ensures correctness of changes.

3. Step 3: Eliminate other options

   Skipping smoke testing or running full regression every time is inefficient. Performance testing is unrelated to this goal.

4. Final Answer:

   Run smoke testing on main features after every build, then sanity testing on specific fixes -> Option D

5. Quick Check:

   Smoke then sanity for efficient small update testing [OK]

Hint: Smoke test main features, then sanity test fixes for efficiency [OK]

Common Mistakes:

• Skipping smoke testing for faster results
• Running full regression after every small update
• Confusing performance testing with smoke/sanity