Qa Testing Android

---
name: qa-testing-android
description: "Designs Android testing with Espresso, UI Automator, Robolectric, and Compose. Use when planning device matrices, emulator CI, Compose test migration, screenshots, or flakes."
compatibility: Portable core. Works on Claude Code and Codex.
version: "1.2"
last_validated: 2026-07-11
---

# QA Testing (Android)

Android testing automation with Espresso, UI Automator, Compose Testing, screenshot tests, and adaptive UI validation.

**Core References**: [Android Testing Docs](https://developer.android.com/training/testing), [Build-Managed Devices](https://developer.android.com/studio/test/managed-devices), [Compose Testing](https://developer.android.com/develop/ui/compose/testing), [UI Automator](https://developer.android.com/training/testing/other-components/ui-automator), [Screenshot Testing](https://developer.android.com/training/testing/ui-tests/screenshot), [Accessibility Checking](https://developer.android.com/training/testing/espresso/accessibility-checking)

## Quick Reference

| Task | Command |
|------|---------|
| List emulators | `emulator -list-avds` |
| Start emulator | `emulator @<avd_name>` |
| List devices | `adb devices` |
| Install APK | `adb install -r <path-to-apk>` |
| Run unit tests | `./gradlew test` |
| Run instrumented tests (connected) | `./gradlew connectedAndroidTest` |
| Run instrumented tests (GMD) | `./gradlew <device><variant>AndroidTest` |
| Run screenshot tests | `./gradlew validateDebugScreenshotTest` |
| List GMD tasks | `./gradlew tasks --all | rg -n "AndroidTest|Group|ManagedDevices"` |
| Clear app data | `adb shell pm clear <applicationId>` |

## Platform Versions

Look up current versions before pinning; this table keeps only the compatibility rules.

| Component | Where to check the current version | Stable notes |
|-----------|-----------------------------------|--------------|
| Android / API level | Android developer release notes | The newest OS version is not automatically the target-API requirement; see "Google Play Target API Policy" below |
| AGP | https://developer.android.com/build/releases/about-agp | Select the wrapper/JDK from the compatibility table for the installed AGP minor; major-version matching alone is insufficient |
| Robolectric | https://github.com/robolectric/robolectric/releases | Robolectric 4.17 added SDK 37 support. JDK 21 is required for SDK 36+ |
| UI Automator | https://developer.android.com/jetpack/androidx/releases/test-uiautomator | The 2.4 API boundary adds the `uiAutomator {}` DSL; select the project-approved release |
| Compose Preview Screenshot Testing | Android Studio / AGP release notes | Check plugin vs AGP test-suite setup and CLI vs IDE requirements separately; see screenshot-testing.md |
| ATD images | Emulator system-image list | Check supported API images in the managed-device guide and SDK Manager; ATD disables hardware rendering |
| Maestro | Maestro release notes | New API-level support can reach the local CLI before Maestro Cloud; check before targeting a new Android version |

### Google Play Target API Policy (check before every release cutover)

Before choosing the release matrix, read the [target-API requirements](https://developer.android.com/google/play/requirements/target-sdk)
for the app's form factor, submission type, deadline and extension status. New submissions and
existing-app availability have different rules; private apps and Wear/TV/Automotive/XR can have
exceptions. Record the applicable requirement and verify the app's `targetSdk` in the built artifact.

## Quick Start

- Prefer build-managed devices (GMD) for CI, using ATD images where the suite and supported API levels allow it; use `connectedAndroidTest` for local ad-hoc runs.
- Select an ATD API from the supported image list; use standard images when the required API or rendering path is unavailable.
- Enable Android Test Orchestrator and `clearPackageData` for instrumented suites that need strong isolation.
- Disable animations in Gradle, not with ad-hoc ADB steps inside the test flow.
- Treat screenshot tests as a separate fast regression layer: Compose Preview Screenshot Testing for `@Preview` coverage, Paparazzi or Roborazzi for JVM rendering, device-based snapshots only when hardware fidelity matters.
- Cover adaptive layouts explicitly: compact and expanded layouts, tablet or foldable paths when supported, and orientation changes where the product depends on them.
- Add accessibility checks to existing UI tests early instead of treating accessibility as a manual-only phase.

Recommended Gradle defaults for stable instrumented tests:

```kotlin
android {
    testOptions {
        animationsDisabled = true
        execution = "ANDROIDX_TEST_ORCHESTRATOR"

        emulatorSnapshots {
            enableForTestFailures = true
            maxSnapshotsForTestFailures = 2
        }
    }
}

dependencies {
    androidTestUtil(libs.androidx.test.orchestrator)
}

```

If you rely on test isolation between instrumented tests, also set runner args such as `clearPackageData=true` in your Gradle or CI wiring.

## Inputs to Gather

- UI stack: Views, Compose, or mixed
- Test layer: unit, Robolectric, instrumented UI, screenshot, UI Automator, macrobenchmark
- CI target: PR gate vs nightly vs release; managed emulator vs device farm
- Device matrix: min supported API, target API, screen classes, locales, dark mode, form factors
- Flake symptoms: timeouts, missing nodes, idling or sync issues, renderer-only diffs, device-only failures
- Accessibility requirements: touch targets, labels, contrast, TalkBack-critical flows
- App seams: DI hooks for fakes, feature flags, test accounts, deterministic clocks, image loaders

## Testing Layers

| Layer | Framework | Scope |
|-------|-----------|-------|
| Unit | JUnit + Mockito | JVM, no Android |
| Unit (Android) | Robolectric | JVM, simulated framework |
| UI (Views) | Espresso | Instrumented |
| UI (Compose) | Compose Testing | Instrumented; JVM with explicit Robolectric setup |
| Adaptive UI | Espresso Device API + `DeviceConfigurationOverride` | Instrumented or host-assisted |
| Screenshot | Compose Preview Screenshot Testing, Paparazzi, Roborazzi | JVM or instrumented |
| System | UI Automator | Cross-app, system UI, benchmarking drivers |

## Core Principles (Stability)

### Device Matrix

- Default: emulators for PR gates; real devices or device farms for release-critical journeys.
- Cover at least min supported API and target API, then add tablet or foldable coverage if the product exposes expanded layouts.
- Model adaptive coverage by layout behavior, not by device marketing names alone.

### Flake Control

- Prefer `testOptions { animationsDisabled = true }` for instrumented tests.
- Use Android Test Orchestrator when shared app state or process crashes cause cross-test leakage.
- Use IdlingResources, Compose synchronization, `waitUntil`, or UI Automator conditions instead of sleeps.
- Mock network with `MockWebServer` or DI fakes; avoid live backends in CI.
- Keep screenshot tests deterministic: fixed fonts, locale, clocks, network images, and seeded data.
- Use test-failure snapshots or screenshots in CI so failures have artifacts, not just stack traces.

### Accessibility And Adaptive Coverage

- Run accessibility checks in the same suite as high-value Espresso or Compose journeys.
- Validate compact and expanded layout states intentionally; do not assume a phone-only test matrix covers responsive layouts.
- Prefer stable selectors: `withId()` for Views, `testTag` for Compose, and resource-id or content descriptions for UI Automator.

## Expert Judgment (What A Checklist Misses)

### Emulator vs. Physical Device — the actual decision

A checklist says "use emulators for CI, real devices for release." The judgment call is *which* real-device signals are worth paying for:

- Emulators (ideally build-managed, ATD where hardware fidelity is not required) are correct for PR gates because they are deterministic, disposable, and free of thermal/battery/OEM noise that would otherwise get misattributed to app bugs.
- Physical devices or a device farm earn their cost only for journeys where OEM or hardware behavior is the actual risk: camera capture, biometric auth, NFC/BLE, background-work reliability under real Doze/App Standby, foldable hinge/posture transitions, and payment SDKs that behave differently under real Play Integrity checks. If none of those are in the flow, a device farm run is theater, not signal.
- A common anti-pattern is running the *entire* regression suite nightly on a device farm "for confidence." This burns budget without changing the failure mode you'd actually catch — most of what fails there also fails, faster and cheaper, on a managed emulator. Reserve device-farm minutes for the handful of hardware-coupled journeys above and gate them separately from the general regression suite.
- Emulator system-image choice is itself a judgment call, not just "google vs aosp": for services-dependent flows, verify the image supplies the required services and distribution channel; anything asserting on hardware-rendered pixels needs a non-ATD image; use ATD where its supported API and reduced system surface fit the journey, otherwise a standard image. Google API images do not prove Play Store/distribution-dependent behavior; check that channel separately.

### Android Flake Taxonomy (diagnose by category, not by symptom)

Generic "flaky test" triage wastes time re-running instead of classifying. Android UI-test flakiness clusters into a small number of root-cause families — identify which one you're looking at before reaching for retries:

1. **Synchronization gaps** — an async operation Espresso/Compose does not know how to wait for. Signature: passes on a fast local machine, fails under CI load; `NoMatchingViewException` or an assertion firing against a stale loading state. Root cause is almost always an unregistered `IdlingResource` (Espresso) or a missing `waitUntil`/synchronized `TestDispatcher` (Compose/coroutines) — see `references/espresso-patterns.md` for the registration trap.
2. **Animation/transition timing** — window or property animations still running when Espresso samples the view tree. Prefer the Gradle-level `testOptions { animationsDisabled = true }` for CI. Only fall back to the per-test adb-settings `DisableAnimationsRule` pattern in `references/espresso-patterns.md` when the Gradle flag is unavailable (for example, a local module or Gradle version that predates it) — label it "legacy" in your own suite, since the two mechanisms can silently disagree if both are present.
3. **Semantics-tree merging (Compose-specific)** — a child `testTag` inside a Material component (e.g. the `Text` inside a `Button`) is invisible to `onNodeWithTag` because merged descendants collapse into the parent node; or a multi-value assertion like `assertTextEquals("Save")` fails against a merged node's full Text list. Text finders such as `onNodeWithText` still match the merged parent by default — this is not a "finds nothing" bug, and not a timing issue at all; do not "fix" it with a wait loop. See the corrected merged-tree trap in `references/compose-testing.md`. A v1-to-v2 migration can instead expose queued coroutine work; classify that under synchronization gaps (#1), using the migration section in `references/compose-testing.md`.
4. **State leakage across tests** — a prior test left SharedPreferences, a DB row, or a singleton in a state the next test doesn't expect. Symptom is order-dependent failures that disappear when the failing test is run alone. Fix with Test Orchestrator + `clearPackageData`, not by reordering tests.
5. **Doze / App Standby / background restrictions** — tests that rely on background work (WorkManager, FCM, foreground services) can behave differently under device power-management states than in a freshly booted emulator. This shows up almost exclusively on physical devices or long-running CI machines, rarely on ephemeral managed devices — if a background-work test is flaky only in one environment, suspect power-management state before suspecting the test.
6. **Renderer/fidelity mismatches** — screenshot diffs that are real but meaningless: ATD disabling hardware rendering, GPU driver differences between CI and local, or font substitution differences across OS images. Distinguish this from an actual visual regression by re-running the same golden generation pipeline on the same image type used to record it.
7. **Non-deterministic test data or clocks** — live network calls, real `System.currentTimeMillis()`, or unseeded random data leaking into assertions or screenshot goldens. Always mockable; the fact that it's still happening usually means a fake wasn't reused when a new screen was added.

When a test is flaky, name which of the seven it is before touching the test — the fix for #1 (register/await) will not touch #4 (isolate/clear state), and applying #2's fix (disable animations) to a #5 issue (Doze) does nothing.

### Test Pyramid Reality For Compose Apps

The classic pyramid (many unit tests, some integration, few E2E) still holds, but Compose changes where the *middle* layer sits:

- `createComposeRule()` still hosts your content in a `ComponentActivity` under the hood and runs instrumented on a device or emulator unless the module also runs it under Robolectric in `test/` — it is not automatically device-free. What it *does* avoid is your app's own `Activity`, DI graph, and navigation. Prefer it over `createAndroidComposeRule<Activity>()` whenever the composable doesn't need real navigation, DI graph, or activity lifecycle, and run it under Robolectric (Roborazzi or a `@RunWith(AndroidJUnit4)` + `isIncludeAndroidResources` JVM test) when you actually want a fast, unit-test-like run. See `references/compose-testing.md` for the setup.
- This shifts a large share of what used to be slow instrumented Espresso coverage down into fast, parallelizable JVM-adjacent tests — but only if ViewModels are tested independently (Turbine/MockK against `StateFlow`, not through the UI). A Compose app that only has "instrumented Compose tests that also exercise the ViewModel" has recreated the old inverted pyramid with new tools.
- Screenshot tests (Compose Preview Screenshot Testing / Paparazzi / Roborazzi) form a *separate, parallel* layer, not a replacement for either unit or instrumented behavioral tests — they catch visual regressions that behavioral assertions cannot see (spacing, overlap, dark-mode contrast) but say nothing about correctness of state transitions.
- Reserve UI Automator and Maestro-style E2E flows for the top of the pyramid: a handful of true cross-app or full-stack smoke journeys (onboarding, checkout, deep-link entry), run on nightly/release pipelines, not PR gates.

### Device / API Fragmentation Strategy

- Always cover the app's actual min supported API and its current target API — these two are non-negotiable regardless of market share.
- Beyond that pair, prioritize by *behavioral risk*, not by device popularity: API levels that changed permission models, background-execution limits, or storage scoping (e.g., scoped storage, notification runtime permission, foreground-service type declarations) deserve dedicated coverage on the OS version that introduced the change, even if that OS version is not the most common one in your install base.
- Treat any specific device/API market-share number as **unverified unless sourced from your own analytics** (Play Console's own device catalog and Android Vitals for your app) — public aggregate stats go stale within months and vary enormously by app category and geography. Do not hardcode "X% of users are on API N" into test-planning docs; pull it fresh from the app's own Play Console dashboard each planning cycle.
- Foldable/tablet/expanded-layout coverage is now a fragmentation axis in its own right, independent of API level — a phone-only test matrix increasingly under-covers real usage on large-screen and foldable devices even when API coverage looks complete.

### When To Invest In Screenshot Testing

Screenshot testing has real setup and maintenance cost (goldens go stale, false positives from font/renderer drift, review burden on every intentional UI change). It is worth that cost when:

- The team ships UI changes frequently enough that manual visual review does not scale (multiple PRs/day touching shared components).
- The app has a design system or component library where a single regression fans out across many screens — screenshot tests catch that fan-out cheaply; behavioral tests do not.
- Dark mode, large font scale, or RTL layouts are supported and have historically regressed silently.

It is a poor early investment for a small team still iterating rapidly on visual design — churn in intentional goldens will dominate signal from real regressions. Start with host-side tools (Compose Preview Screenshot Testing, Paparazzi, Roborazzi) for cheap iteration; only add device-based snapshot testing (Shot, or ATD-excluded device runs) once a specific rendering-sensitive surface (WebView, Maps, camera preview, custom `Canvas`/GPU work) has already caused a shipped visual bug that host-side rendering could not have caught.

## Writing Tests

- Espresso (Views): open `references/espresso-patterns.md`
- Compose: open `references/compose-testing.md`
- UI Automator (system or cross-app): open `references/uiautomator.md`
- Screenshot tests: open `references/screenshot-testing.md`
- Adaptive screen-size testing: open `references/adaptive-screen-testing.md`
- Accessibility checks: open `references/accessibility-checks.md`

### Compose Test v2 Migration

The v1 Compose test APIs (`androidx.compose.ui.test.junit4.createComposeRule`) are deprecated. v2
(`...junit4.v2.createComposeRule`, `...test.v2.runComposeUiTest`, from `ui-test-junit4`
1.11.0-alpha03+) switches the default dispatcher from `UnconfinedTestDispatcher` to
`StandardTestDispatcher` and forces touch input mode by default. Standard UI assertions already synchronize; direct state reads and manual clock control can need
`waitForIdle()`/`runOnIdle {}` or scheduler advancement (taxonomy item #1). See `references/compose-testing.md` for the full migration guide,
failure-class breakdown, and code samples.

## Workflow

### Add a New UI Test (Instrumented)

- Pick the narrowest framework that can observe the behavior: Espresso or Compose first, UI Automator at the system boundary.
- Add stable selectors: View `id`, Compose `Modifier.testTag`, system `resource-id` or content description.
- Control externals: fake network, deterministic data, fixed time and locale, and stable image sources.
- Add waits through framework synchronization or explicit conditions; avoid `Thread.sleep()`.
- Run locally with `connectedAndroidTest` or a single managed-device task before widening the matrix.

### Add Screenshot Coverage

- Use Compose Preview Screenshot Testing for `@Preview`-driven Compose UI states.
- Use Paparazzi or Roborazzi when you need JVM-fast coverage across Compose and legacy Views.
- Keep goldens small, reviewable, and tied to stable UI states instead of whole-screen snapshots of volatile content.
- Capture HTML or diff artifacts in CI on every failure.

### Diagnose a Flaky Instrumented Test

- Reproduce locally or on one managed device before widening the matrix.
- Remove nondeterminism: network, locale, clock, feature flags, image loading, renderer differences.
- Replace sleeps with idling, `waitUntil`, `onElement`, or watcher-based synchronization.
- Capture logcat, screenshot, screen recording, and any managed-device failure snapshot.
- If still flaky, isolate app state further with Orchestrator or runner args and bisect the interaction steps.

### Add A CI Gate (Preferred: Build-Managed Devices)

- Configure build-managed devices plus ATD images when hardware rendering is not required and the API level is supported by the image.
- Add the GitHub Actions GPU flag when your runner lacks hardware rendering support.
- Keep PR gates small and deterministic; expand device groups or sharding on nightly and release pipelines.
- Upload reports from `build/reports/androidTests/`, screenshot reports, diff images, logcat, and managed-device outputs.
- Report build, install, launch, interaction, and distribution evidence separately. Record APK/AAB identity, variant, device/API, locale, exact test task, and artifacts; a Gradle build proves packaging, while only an outcome assertion on the named device proves the exercised journey.
- Keep notifications, billing, deep links, background work, and OEM-specific flows unverified until the relevant real-device or distribution-channel check runs.

## ADB Commands (Triage)

```bash
# Screenshot
adb exec-out screencap -p > screenshot.png

# Screen recording
adb shell screenrecord /sdcard/demo.mp4

# Pull managed-device test artifacts after a local failure
adb pull /sdcard/Android/media ./device-artifacts
```

## CI Integration

Preferred: build-managed devices. See `references/gradle-managed-devices.md` and `references/android-ci-optimization.md`.

```yaml
# .github/workflows/android.yml
name: Android CI
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6  # or later — verify at actions/checkout releases
      - uses: actions/setup-java@v5  # or later
        with:
          java-version: '21'  # SDK 36 unit tests need JDK 21
          distribution: 'temurin'
      - uses: gradle/actions/setup-gradle@v5  # or later
      - run: ./gradlew testDebugUnitTest <device><api>DebugAndroidTest -Pandroid.testoptions.manageddevices.emulator.gpu=swiftshader_indirect
```

## Navigation

The reference guides are intentionally large; search within them instead of loading everything:

- `rg -n "^## " skills/universal/qa-testing-android/references/compose-testing.md`
- `rg -n "Idling|waitUntil|Synchronization" skills/universal/qa-testing-android/references/compose-testing.md`
- `rg -n "DisplaySizeRule|DeviceConfigurationOverride|fold" skills/universal/qa-testing-android/references/adaptive-screen-testing.md`
- `rg -n "PreviewTest|Paparazzi|Roborazzi|Shot" skills/universal/qa-testing-android/references/screenshot-testing.md`

## Do / Avoid

### Do

- Prefer build-managed devices plus ATD images for stable CI
- Use Android Test Orchestrator only where isolation solves a real state-leak or crash problem
- Add screenshot and accessibility checks to high-value UI flows
- Run a small matrix on PRs and widen via groups or shards on nightly or release pipelines
- Use a `MainDispatcherRule` + `TestDispatcher` (`runTest { }`, `advanceUntilIdle()`, `kotlinx-coroutines-test`) for every ViewModel and `StateFlow` test — choose eager `UnconfinedTestDispatcher` only when that scheduling behavior is intentional, because it can mask ordering races
- Run a release-variant smoke test in CI that exercises at least one `@Serializable` endpoint per data class, so R8 full-mode stripping of kotlinx-serialization `$serializer` classes is caught before release (see `software-android-runtime-debugging/references/proguard-r8-triage.md`)
- For Compose tests, assert on semantic state — `onNodeWithTag(...).assertTextEquals(...)` — not on reference equality of UI state objects, because Strong Skipping Mode means the UI may or may not receive the same instance across emissions

### Avoid

- `Thread.sleep()` for synchronization
- Tests that depend on live backends or time-sensitive external content
- Flaky selectors such as localized text or positional-only nodes
- Device-wide screenshot assertions on ATD when hardware-rendered fidelity is required
- Asserting `StateFlow<UiState>` reference equality across emissions (`assertThat(state).isSameInstanceAs(previous)`) — new `data class` instances from `copy()` have different references but structurally equal content; test semantic equality, not identity
- Running serialization-dependent tests only on debug builds — R8 is off in debug, so debug-only suites cannot catch serialization keep-rule failures

## Resources

| Resource | Purpose |
|----------|---------|
| [references/espresso-patterns.md](references/espresso-patterns.md) | Espresso matchers and actions |
| [references/compose-testing.md](references/compose-testing.md) | Compose testing guide |
| [references/uiautomator.md](references/uiautomator.md) | UI Automator patterns for system UI and benchmarking |
| [references/gradle-managed-devices.md](references/gradle-managed-devices.md) | Managed device setup and CI |
| [references/screenshot-testing.md](references/screenshot-testing.md) | Visual regression testing |
| [references/adaptive-screen-testing.md](references/adaptive-screen-testing.md) | Screen-size and foldable coverage |
| [references/accessibility-checks.md](references/accessibility-checks.md) | Accessibility checks for Espresso and Compose |
| [references/test-orchestrator-patterns.md](references/test-orchestrator-patterns.md) | AndroidX Test Orchestrator patterns |
| [references/android-ci-optimization.md](references/android-ci-optimization.md) | CI pipeline optimization |
| [references/modern-test-tooling.md](references/modern-test-tooling.md) | JUnit 5, MockK, Turbine, Robolectric, Maestro |
| [data/sources.json](data/sources.json) | Curated external sources |

## Templates

| Template | Purpose |
|----------|---------|
| [assets/template-android-test-checklist.md](assets/template-android-test-checklist.md) | Stability checklist |

## Related Skills

| Skill | Purpose |
|-------|---------|
| [software-mobile](../software-mobile/SKILL.md) | Android development |
| [qa-testing-strategy](../qa-testing-strategy/SKILL.md) | Test strategy |
| [qa-testing-mobile](../qa-testing-mobile/SKILL.md) | Cross-platform mobile |
| [software-android-native](../software-android-native/SKILL.md) | Native Android implementation and agent workflows |
| [software-android-runtime-debugging](../software-android-runtime-debugging/SKILL.md) | Build/install/launch proof and stale-build triage |
| [software-android-design](../software-android-design/SKILL.md) | Native Android visual design and Material 3 review |

## Learnings Loop

When prior decisions or pitfalls are relevant, consult `learnings.consolidated.md` if present; use `learnings.md` only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to `learnings.md` via `agents-skills-feedback-loop/scripts/append_learning.py`. Do not modify `SKILL.md` itself.