System testing in Testing Fundamentals - Framework Patterns

Framework Mode - System testing

Folder Structure for System Testing Project

system-testing-project/
├── config/
│   ├── environments/
│   │   ├── dev.env
│   │   ├── test.env
│   │   └── prod.env
│   └── config.yaml
├── tests/
│   ├── system/
│   │   ├── test_user_login.py
│   │   ├── test_payment_flow.py
│   │   └── test_data_integrity.py
│   └── integration/
├── utilities/
│   ├── logger.py
│   ├── data_loader.py
│   └── api_client.py
├── reports/
│   └── latest_report.html
├── conftest.py
└── pytest.ini

Test Framework Layers in System Testing

• Configuration Layer: Holds environment settings, credentials, and test parameters.
• Utilities Layer: Contains helper functions like logging, data loading, and API clients.
• Test Layer: Contains system test cases that verify the complete integrated system behavior.
• Reporting Layer: Generates test execution reports for analysis and feedback.

Configuration Patterns for System Testing

• Environment Files: Separate files for dev, test, and production environments to switch settings easily.
• Central Config File: A YAML or JSON file to define global settings like timeouts and base URLs.
• Secure Credential Storage: Use environment variables or encrypted files to keep sensitive data safe.
• Parameterization: Use fixtures or config parsers to inject different data sets or environment variables into tests.

Test Reporting and CI/CD Integration

• HTML Reports: Generate clear, easy-to-read reports after test runs showing pass/fail status and logs.
• CI/CD Integration: Connect tests to pipelines (e.g., Jenkins, GitHub Actions) to run system tests automatically on code changes.
• Notifications: Configure email or chat alerts for test failures to quickly inform the team.
• Historical Data: Store reports to track test results over time and identify trends.

Best Practices for System Testing Framework

1. Test Full System Behavior: Focus on end-to-end flows covering all integrated components.
2. Isolate Environment Setup: Use dedicated test environments to avoid affecting real users or data.
3. Use Clear Naming: Name tests and files to reflect the system feature or flow they verify.
4. Keep Tests Independent: Each test should run alone without relying on others to avoid false failures.
5. Automate Reporting: Always generate and review reports to catch issues early and improve quality.

Self Check Question

Where in this folder structure would you add a new system test for the user registration process?

Key Result

System testing frameworks verify the complete integrated system using organized layers and clear environment configurations.

Practice

1. What is the main purpose of system testing in software development?

easy

A. To write code for new features

B. To test individual units or components separately

C. To test the complete software as a whole to verify it meets requirements

D. To deploy the software to users

Solution

1. Step 1: Understand system testing scope

   System testing tests the entire software system together, not parts separately.

2. Step 2: Compare options with system testing goal

   Only To test the complete software as a whole to verify it meets requirements describes testing the complete software to check requirements.

3. Final Answer:

   To test the complete software as a whole to verify it meets requirements -> Option C

4. Quick Check:

   System testing = whole software check [OK]

Hint: System testing checks the whole software, not parts [OK]

Common Mistakes:

• Confusing system testing with unit testing
• Thinking system testing is done before integration testing
• Believing system testing involves coding or deployment

2. Which of the following is the correct sequence of testing phases including system testing?

easy

A. Unit testing -> Integration testing -> System testing -> Acceptance testing

B. System testing -> Unit testing -> Integration testing -> Acceptance testing

C. Integration testing -> Unit testing -> System testing -> Acceptance testing

D. Acceptance testing -> System testing -> Integration testing -> Unit testing

Solution

1. Step 1: Recall testing order

   Testing starts with unit testing, then integration, then system, and finally acceptance testing.

2. Step 2: Match sequence with options

   Only Unit testing -> Integration testing -> System testing -> Acceptance testing matches the correct order.

3. Final Answer:

   Unit testing -> Integration testing -> System testing -> Acceptance testing -> Option A

4. Quick Check:

   Testing order = Unit -> Integration -> System -> Acceptance [OK]

Hint: Remember testing flows from small parts to whole system [OK]

Common Mistakes:

• Mixing up the order of integration and system testing
• Thinking system testing happens before unit testing
• Confusing acceptance testing with system testing

3. Consider this test scenario during system testing:
Test if the login page accepts valid username and password and redirects to the dashboard.
What is the expected result if the system works correctly?

medium

A. The login page shows an error message for valid credentials

B. The user is redirected to the dashboard after successful login

C. The system crashes after submitting credentials

D. The login page reloads without any message

Solution

1. Step 1: Understand the test scenario

   The test checks if valid login credentials allow access to the dashboard.

2. Step 2: Identify correct expected behavior

   On valid login, the user should be redirected to the dashboard, which is The user is redirected to the dashboard after successful login.

3. Final Answer:

   The user is redirected to the dashboard after successful login -> Option B

4. Quick Check:

   Valid login -> dashboard redirect [OK]

Hint: Valid login should lead to dashboard, not errors [OK]

Common Mistakes:

• Expecting error messages for valid inputs
• Confusing page reload with successful login
• Ignoring redirection behavior in system testing

4. During system testing, a tester finds that the software crashes when clicking the 'Submit' button on a form. What should the tester do next?

medium

A. Ignore the crash and continue testing other features

B. Restart the software and try again without reporting

C. Fix the code themselves immediately

D. Report the defect with steps to reproduce and environment details

Solution

1. Step 1: Identify correct tester action

   Testers should report defects with clear reproduction steps and environment info.

2. Step 2: Evaluate options

   Only Report the defect with steps to reproduce and environment details describes proper defect reporting. Others ignore or bypass the issue.

3. Final Answer:

   Report the defect with steps to reproduce and environment details -> Option D

4. Quick Check:

   Tester reports bugs properly [OK]

Hint: Always report bugs with clear details, never ignore [OK]

Common Mistakes:

• Ignoring critical crashes during testing
• Trying to fix code without developer role
• Not providing reproduction steps in bug reports

5. A software system has passed unit and integration tests. During system testing, it fails to handle simultaneous user logins correctly, causing data conflicts. What is the best approach to solve this issue?

hard

A. Add concurrency control mechanisms and retest the system

B. Skip system testing and release the software anyway

C. Only fix the user interface without changing backend logic

D. Reduce the number of users allowed to login simultaneously

Solution

1. Step 1: Analyze the problem cause

   Data conflicts on simultaneous logins indicate concurrency issues in backend logic.

2. Step 2: Choose appropriate fix

   Adding concurrency control (like locks or transactions) addresses the root cause and requires retesting.

3. Step 3: Evaluate other options

   Skipping testing or only fixing UI won't solve backend conflicts; limiting users is impractical.

4. Final Answer:

   Add concurrency control mechanisms and retest the system -> Option A

5. Quick Check:

   Fix concurrency bugs before release [OK]

Hint: Fix backend concurrency issues, then retest system [OK]

Common Mistakes:

• Ignoring concurrency problems during system testing
• Trying to fix only UI for backend issues
• Releasing software without fixing critical bugs