Instrumented tests in Android Kotlin - Build, Publish & Deploy

Build & Publish - Instrumented tests

Performance Impact of Instrumented Tests

Instrumented tests run on real or virtual Android devices. They do not affect your app's runtime performance for users because they run separately from the app's production code.

However, running many or complex instrumented tests can consume significant CPU and memory on the test device or emulator, which may slow down test execution and drain battery faster during testing.

Instrumented tests do not impact your app's frame rate or memory usage in production.

💻

Optimizing Instrumented Tests for Faster Execution

• Keep tests focused and small to reduce execution time.
• Use AndroidJUnitRunner efficiently and avoid unnecessary setup or teardown steps.
• Run tests on fast emulators or physical devices with good performance.
• Use adb shell am instrument commands with filters to run only relevant tests.
• Mock external dependencies to avoid slow network or database operations.
• Parallelize tests when possible using Gradle test options.

Impact on App Bundle Size and Startup Time

Instrumented tests are packaged separately from your app's release APK or AAB. They do not increase your app's bundle size or affect startup time for end users.

Test APKs include test code and dependencies, which increase the size of the test package but this is only relevant during development and CI pipelines.

iOS vs Android Differences for Instrumented Tests

Android instrumented tests run on devices or emulators using AndroidJUnitRunner and access app context.

iOS uses XCTest framework for UI and integration tests, running on simulators or devices with Xcode tools.

Android tests require androidTest source sets and Gradle configuration, while iOS tests are integrated into Xcode projects.

Both platforms separate test code from production code to avoid impacting app performance.

Store Review Guidelines and Requirements

• Instrumented tests are for development only and are not included in app store submissions.
• Ensure no test code or debug flags remain in your release APK or AAB to comply with Google Play policies.
• Remove or disable any test-only permissions or features before publishing.
• Follow Google Play's security and privacy guidelines to avoid exposing test endpoints or data.

Self-Check: Your App Takes 5 Seconds to Load This Screen. What's Likely Wrong?

If your instrumented test runs slowly or your app screen loads slowly during testing, possible issues include:

• Tests performing heavy UI operations or waiting unnecessarily.
• Network calls or database queries not mocked, causing delays.
• Running tests on slow emulators or overloaded devices.
• Excessive setup or teardown steps in tests.

Optimizing test code and environment can improve speed and reliability.

Key Result

Instrumented tests run separately from your app and do not affect user performance or app size. Optimize tests for speed by keeping them small, mocking dependencies, and running on fast devices. Ensure no test code is included in release builds to meet store guidelines.

Practice

1. What is the main purpose of instrumented tests in Android development?

easy

A. To test only the backend logic without UI

B. To write unit tests that run on the local JVM

C. To run tests on real devices or emulators to check app behavior

D. To create UI layouts for the app

Solution

1. Step 1: Understand what instrumented tests do

   Instrumented tests run on actual devices or emulators to test app behavior in a real environment.

2. Step 2: Compare with other test types

   Unit tests run locally on JVM and do not test device features or UI on real devices.

3. Final Answer:

   To run tests on real devices or emulators to check app behavior -> Option C

4. Quick Check:

   Instrumented tests = run on device/emulator [OK]

Hint: Instrumented tests run on devices, not just local JVM [OK]

Common Mistakes:

• Confusing instrumented tests with unit tests
• Thinking instrumented tests only test backend logic
• Assuming instrumented tests create UI layouts

2. Which annotation is required to run an instrumented test with AndroidJUnit4?

easy

A. @RunWith(AndroidJUnit4::class)

B. @Ignore

C. @Before

D. @Test

Solution

1. Step 1: Identify the runner annotation

   Instrumented tests need the @RunWith(AndroidJUnit4::class) annotation to specify the test runner.

2. Step 2: Differentiate from other annotations

   @Test marks test methods, @Before runs setup code, @Ignore skips tests; none specify the runner.

3. Final Answer:

   @RunWith(AndroidJUnit4::class) -> Option A

4. Quick Check:

   Runner annotation = @RunWith(AndroidJUnit4::class) [OK]

Hint: Use @RunWith(AndroidJUnit4::class) to set test runner [OK]

Common Mistakes:

• Using @Test instead of @RunWith for runner
• Confusing @Before with runner annotation
• Forgetting to add any runner annotation

3. Given this instrumented test code snippet, what will be the output?

val appContext = InstrumentationRegistry.getInstrumentation().targetContext
val packageName = appContext.packageName
println(packageName)

medium

A. An empty string

B. The package name of the app under test

C. The package name of the test app

D. A runtime exception

Solution

1. Step 1: Understand InstrumentationRegistry usage

   InstrumentationRegistry.getInstrumentation().targetContext returns the context of the app under test, not the test app.

2. Step 2: Extract package name from app context

   appContext.packageName gives the package name of the app being tested, which is printed.

3. Final Answer:

   The package name of the app under test -> Option B

4. Quick Check:

   targetContext.packageName = app package [OK]

Hint: targetContext is app under test, not test app [OK]

Common Mistakes:

• Confusing targetContext with testContext
• Expecting empty string instead of package name
• Assuming runtime exception without cause

4. What is wrong with this instrumented test code?

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {
  @Test
  fun useAppContext() {
    val context = InstrumentationRegistry.getContext()
    assertEquals("com.example.app", context.packageName)
  }
}

medium

A. InstrumentationRegistry.getContext() is deprecated; should use getInstrumentation().targetContext

B. Missing @Before annotation for setup

C. assertEquals syntax is incorrect

D. Missing @RunWith annotation

Solution

1. Step 1: Check InstrumentationRegistry usage

   InstrumentationRegistry.getContext() is deprecated and does not return the app context; getInstrumentation().targetContext should be used.

2. Step 2: Verify other parts of the code

   @RunWith is present, assertEquals syntax is correct, and @Before is not required here.

3. Final Answer:

   InstrumentationRegistry.getContext() is deprecated; should use getInstrumentation().targetContext -> Option A

4. Quick Check:

   Use getInstrumentation().targetContext, not getContext() [OK]

Hint: Use getInstrumentation().targetContext, not getContext() [OK]

Common Mistakes:

• Using deprecated getContext() method
• Assuming @Before is mandatory for all tests
• Thinking assertEquals syntax is wrong

5. You want to write an instrumented test that verifies a button click updates a TextView on a real device. Which steps should you follow to correctly implement this test?

hard

A. Use @RunWith(AndroidJUnit4::class), get test context, launch activity, perform click, check TextView text

B. Write a unit test with @Test, mock the button click, and check TextView text without device

C. Write a UI test without annotations, directly call button click method and check TextView

D. Use @RunWith(AndroidJUnit4::class), get app context with InstrumentationRegistry.getInstrumentation().targetContext, launch activity, perform click, check TextView text

Solution

1. Step 1: Setup instrumented test with correct runner and context

   Use @RunWith(AndroidJUnit4::class) and get app context via InstrumentationRegistry.getInstrumentation().targetContext to access real app environment.

2. Step 2: Launch activity, perform UI action, and verify result

   Launch the activity, simulate button click, and check if TextView text updates as expected on device/emulator.

3. Final Answer:

   Use @RunWith(AndroidJUnit4::class), get app context with InstrumentationRegistry.getInstrumentation().targetContext, launch activity, perform click, check TextView text -> Option D

4. Quick Check:

   Instrumented UI test = runner + app context + UI actions [OK]

Hint: Instrumented UI tests need real context and runner [OK]

Common Mistakes:

• Using test context instead of app context
• Writing unit tests for UI behavior
• Skipping @RunWith annotation