Android · Testing
Testing Android Apps: The Test Pyramid, JUnit, UI Automation and TDD
Every Android team says it wants tests. Fewer can say which tests to write, where they should run, and which of the many libraries to pick. This guide walks the whole map in order: the test pyramid, JUnit 4 and 5, the libraries that sit around JUnit, instrumentation and UI tests, Appium and other UI automation tools, how to keep UI tests from being flaky, and what test-driven development (TDD) really gives you. Every idea has a small Kotlin example.
- Write many fast unit tests, some integration tests and few end-to-end UI tests. That shape is the test pyramid.
- Unit tests run on your computer’s JVM (
src/test). Instrumented tests run on a device or emulator (src/androidTest). - JUnit is the base. Around it you add an assertion library, a mocking library, coroutine and Flow helpers, and (when needed) Robolectric.
- Espresso and Compose tests drive your own app’s UI. UI Automator and Appium drive the whole device, from the outside.
- UI tests get flaky from animations, timing, shared state and real networks. Each cause has a known fix.
- TDD (red, green, refactor) pushes you toward small, decoupled code. It costs time at first and does not suit every kind of work.
- A starter stack and a four-step path are at the end.
Contents
- Why test at all
- The test pyramid
- Where Android tests live: local and instrumented
- Unit tests with JUnit 4 and 5
- The library map around JUnit
- Fakes, stubs and mocks
- Testing coroutines, Flow and ViewModels
- Instrumentation tests: Espresso, Compose and UI Automator
- UI automation and Appium
- Keeping UI tests reliable
- TDD: red, green, refactor
- A practical test strategy
- Common mistakes
- Takeaways, a starter path and glossary
1. Why test at all
A test is a small program that checks your program. You run it, it either passes or fails, and nobody has to look at the screen. That one property is why tests are worth the effort: they turn “I think this still works” into “I know this still works”, in seconds, every time you change something.
Good tests give you four things. They catch regressions, which are bugs in behaviour that used to work. They let you refactor with confidence, because a red test tells you immediately when a cleanup broke something. They act as executable documentation of what the code is meant to do. And they shorten code review, because a reviewer can read the tests to see the intent.
Analogy. Tests are the safety net under a trapeze artist. The artist does not perform better because of the net. They perform bolder, because a fall is no longer a disaster. Teams with good tests change old code that other teams are afraid to touch.
Tests are also code, and code has a cost. A slow, brittle test suite is worse than none, because people stop trusting it and start ignoring red builds. The goal is never “100% coverage”. The goal is the most confidence per minute of writing, running and maintaining.
2. The test pyramid
The test pyramid is a picture of how many tests to write at each level. Small, fast tests that check one thing go at the bottom, and there should be many of them. Big, slow tests that drive the whole app go at the top, and there should be few.
| Layer | What it checks | Speed | Cost to maintain | Flaky? |
|---|---|---|---|---|
| Unit | One class or function, with its collaborators replaced by fakes | Milliseconds | Low | Almost never |
| Integration | Several real pieces together: a repository with a real database, a service with a fake server | Seconds | Medium | Sometimes |
| UI / end-to-end | The app on a device, driven like a user, from tap to result | Tens of seconds to minutes | High | Often, unless managed |
Why a pyramid and not a rectangle
- Feedback time. A thousand unit tests can finish before you let go of the save shortcut. A thousand UI tests take hours, so people run them rarely, and bugs live longer.
- Pinpointing. When a unit test fails, it names the class. When an end-to-end test fails, the cause could be the UI, the network, the emulator, the test data or the test itself.
- Cost. High-level tests break whenever a button moves. Low-level tests break only when the behaviour they describe changes.
- Combinations. A form with five fields and three states per field has 243 combinations. Testing each one through the UI is absurd. Testing the validation logic with unit tests is trivial.
A common rule of thumb, often credited to Google’s testing guidance, is roughly 70% small, 20% medium and 10% large tests. Treat it as a starting point, not a quota.
Analogy. Building a car: you bench-test each part first (unit), then check that the engine and gearbox work together (integration), and only then take it for a test drive (end to end). If the engine cannot be bench-tested, every fault turns into a long drive with a mechanic.
The shapes to avoid and the other shapes
- The ice-cream cone. The pyramid upside down: lots of manual and UI testing, a few unit tests. It is slow, expensive and fragile. Many teams end up here by accident because UI tests are the easiest to start with.
- The testing trophy (popular in web teams) puts the most value on integration tests. The honeycomb (popular for microservices) does the same. Both share the idea that the right shape depends on your system. For Android apps with lots of logic in ViewModels and use cases, the pyramid still fits well.
3. Where Android tests live: local and instrumented
Android splits tests into two homes, and picking the right one is the first real decision.
| Local tests | Instrumented tests | |
|---|---|---|
| Folder | src/test | src/androidTest |
| Runs on | The JVM on your computer or CI machine | A real device or an emulator |
| Speed | Very fast, no install step | Slow: build, install, boot or connect a device |
| Android framework | Not real. Calls hit stubs (or Robolectric supplies a simulation) | The real framework, real UI, real database |
| Runner | JUnit | AndroidJUnitRunner |
| Gradle task | ./gradlew testDebugUnitTest | ./gradlew connectedDebugAndroidTest |
| Best for | Business logic, ViewModels, mappers, validators, repositories with fakes | UI, navigation, database, permissions, anything that needs the real OS |
Google also names tests by size: small (one class, no I/O, runs anywhere), medium (a few components together) and large (the whole app, end to end). The annotations @SmallTest, @MediumTest and @LargeTest from androidx.test.filters let you run a size on its own. They are the pyramid’s three layers with different names.
The “not mocked” error
Local tests compile against a stripped android.jar whose methods throw. If your logic calls Log.d(...) or TextUtils.isEmpty(...), the test fails with “Method ... not mocked”. You have three ways out:
- Keep Android out of your logic. This is the best fix: use plain Kotlin types in ViewModels, use cases and mappers, and wrap the framework behind an interface.
- Use Robolectric, which supplies a working simulation of the framework on the JVM.
- Move the test to
androidTestand run it on a device.
The shortcut testOptions { unitTests.isReturnDefaultValues = true } makes the stubs return zero or null instead of throwing. It hides the problem rather than fixing it, so use it only as a stop-gap.
4. Unit tests with JUnit 4 and 5
JUnit is the standard test framework for Java and Kotlin. It finds methods marked @Test, runs each one, and reports pass or fail. Every other tool in this post builds on it or plugs into it.
The anatomy of a test: Arrange, Act, Assert
Every good unit test has three steps. Arrange the objects and inputs. Act by calling the one thing under test. Assert that the result is what you expected. The same idea is called Given, When, Then in behaviour-driven development.
Our running example is a small price rule: orders of 5,000 or more get 10% off.
class DiscountCalculator {
fun priceAfterDiscount(price: Int): Int {
require(price >= 0) { "price must not be negative" }
return if (price >= 5_000) price - price / 10 else price
}
}
JUnit 4
JUnit 4 (current release 4.13.2) is the default on Android and the base for AndroidX Test.
// build.gradle.kts
dependencies {
testImplementation("junit:junit:4.13.2")
}
// src/test/.../DiscountCalculatorTest.kt
class DiscountCalculatorTest {
private lateinit var calculator: DiscountCalculator
@Before
fun setUp() {
calculator = DiscountCalculator() // Arrange (runs before every test)
}
@Test
fun `no discount below the threshold`() {
assertEquals(4_000, calculator.priceAfterDiscount(4_000))
}
@Test
fun `10 percent off at 5000 or more`() {
assertEquals(9_000, calculator.priceAfterDiscount(10_000))
}
@Test(expected = IllegalArgumentException::class)
fun `negative price is rejected`() {
calculator.priceAfterDiscount(-1)
}
}
Kotlin lets you name a test with spaces by using backticks, which makes failure reports read like sentences. This works well in local tests. Older Android versions cannot load method names with spaces in instrumented tests, so use camelCase there.
| JUnit 4 annotation | When it runs or what it does |
|---|---|
@Test | Marks a test method. expected = and timeout = are options. |
@Before / @After | Before and after every test. Use for setup and cleanup. |
@BeforeClass / @AfterClass | Once for the whole class (static, so @JvmStatic in Kotlin). |
@Ignore | Skips a test. Leave a reason, and do not let skipped tests pile up. |
@Rule | Reusable setup and teardown: TemporaryFolder, TestWatcher, InstantTaskExecutorRule, ActivityScenarioRule. |
@RunWith | Chooses a different runner: AndroidJUnit4, Parameterized, Suite. |
@Category | Groups tests so you can run a subset. |
JUnit 5 (Jupiter)
JUnit 5 is a rewrite with a cleaner API and a more flexible extension model. Its pieces are Jupiter (the new API and engine), Vintage (runs old JUnit 3 and 4 tests on the new platform) and the Platform itself. Local tests can use it directly with one Gradle line.
// build.gradle.kts
android {
testOptions {
unitTests.all { it.useJUnitPlatform() } // run local tests on JUnit 5
}
}
dependencies {
testImplementation("org.junit.jupiter:junit-jupiter:<version>")
// Optional: keep old JUnit 4 tests running in the same module
testRuntimeOnly("org.junit.vintage:junit-vintage-engine:<version>")
}
// src/test/.../DiscountCalculatorTest.kt
class DiscountCalculatorTest {
private val calculator = DiscountCalculator() // a new instance is created for every test
@Test
@DisplayName("10% off at 5,000 or more")
fun tenPercentOff() {
assertEquals(9_000, calculator.priceAfterDiscount(10_000))
}
@ParameterizedTest(name = "{0} becomes {1}")
@CsvSource("0,0", "4999,4999", "5000,4500", "10000,9000")
fun `discount table`(price: Int, expected: Int) {
assertEquals(expected, calculator.priceAfterDiscount(price))
}
@Test
fun `negative price is rejected`() {
assertThrows<IllegalArgumentException> { calculator.priceAfterDiscount(-1) }
}
}
The <version> placeholder stands for the latest release shown on the project’s page (each library is linked from the tables in section 5). Library versions change often, so take them from the source rather than from a blog post.
| JUnit 4 | JUnit 5 (Jupiter) | |
|---|---|---|
| Setup / teardown | @Before, @After | @BeforeEach, @AfterEach, @BeforeAll, @AfterAll |
| Extending | Runners and rules | One model: @ExtendWith |
| Parameterised tests | Separate Parameterized runner | Built in: @ParameterizedTest with @ValueSource, @CsvSource, @MethodSource |
| Grouping | @Category | @Tag, plus @Nested classes |
| Other features | Basic | @DisplayName, @RepeatedTest, @TestFactory (dynamic tests), assertAll, assertTimeout |
| Local tests on Android | Works out of the box | Works with useJUnitPlatform() |
| Instrumented tests on Android | Native: AndroidX Test is built on JUnit 4 | Needs a community plugin, android-junit5 (by mannodermaus) |
Most Android projects use JUnit 4 because every AndroidX Test tool is built on it. Pure-JVM modules (domain and data layers) are a natural place for JUnit 5. Using both in one project is fine: Vintage runs the old tests, Jupiter runs the new ones. Do not switch an existing project over for its own sake.
Assertions
JUnit ships with basic assertions: assertEquals, assertTrue, assertFalse, assertNull, assertNotNull, assertSame, assertArrayEquals and assertThrows. They are enough to start. Dedicated assertion libraries (next section) give you more readable failure messages and richer checks on lists, maps and objects.
5. The library map around JUnit
JUnit runs tests. Everything else (comparing values nicely, replacing collaborators, handling coroutines, simulating Android, driving the UI) comes from libraries around it. This is the map, grouped by job. Pick one from most groups; you do not need them all. Every library name below links to its official page, which is also where you will find the current version.
A. Test frameworks and runners
| Library | Artifact or plugin | What it is |
|---|---|---|
| JUnit 4 | junit:junit | The default on Android. Rules, runners, the base for AndroidX Test. |
| JUnit 5 Jupiter | org.junit.jupiter:junit-jupiter | New API and engine. Add junit-jupiter-params for parameterised tests (the aggregate artifact includes it). |
| JUnit Vintage | org.junit.vintage:junit-vintage-engine | Runs JUnit 3 and 4 tests on the JUnit 5 platform. |
| android-junit5 | Gradle plugin de.mannodermaus.android-junit5 | JUnit 5 for instrumented tests. |
| Kotest | io.kotest:kotest-runner-junit5 | A Kotlin-first framework with many test styles (describe/it, should, behaviour spec), data-driven tests, its own assertions and property testing. |
| kotlin.test | org.jetbrains.kotlin:kotlin-test | Kotlin’s own small assertion API (assertEquals, assertFailsWith) that works on top of JUnit and is also used in multiplatform code. |
| TestNG | org.testng:testng | An older alternative to JUnit. Rare on Android. |
B. Assertion libraries
| Library | Artifact | Style |
|---|---|---|
| Google Truth | com.google.truth:truth | assertThat(list).containsExactly("a", "b"). Very readable failure messages. The usual choice in Google and AndroidX code. |
| AssertJ | org.assertj:assertj-core | Fluent assertThat(x).isEqualTo(y) with a huge set of checks. |
| Hamcrest | org.hamcrest:hamcrest | Matchers: assertThat(x, equalTo(y)). Used by Espresso’s view matchers. |
| Kluent | org.amshove.kluent:kluent | Kotlin infix style: x shouldBeEqualTo y. |
| Strikt | io.strikt:strikt-core | Kotlin assertions: expectThat(x).isEqualTo(y), with chained property checks. |
| Kotest assertions | io.kotest:kotest-assertions-core | x shouldBe y, usable without the Kotest runner. |
C. Mocking and test doubles
| Library | Artifact | Notes |
|---|---|---|
| MockK | io.mockk:mockk and mockk-android | Mocking written for Kotlin. Handles final classes, suspend functions (coEvery, coVerify), objects and extension functions. The usual choice for Kotlin projects. |
| Mockito | org.mockito:mockito-core and mockito-android | The long-standing Java standard. From Mockito 5 the inline mock maker is the default, so Kotlin’s final classes can be mocked. |
| mockito-kotlin | org.mockito.kotlin:mockito-kotlin | Kotlin-friendly helpers on top of Mockito (mock<T>(), whenever, argumentCaptor). |
| PowerMock | — | Legacy way to mock statics and finals. Avoid it in new code; MockK and Mockito 5 cover the need. |
| Hand-written fakes | none | A small in-memory class that implements your interface. Often the best double (see section 6). |
D. Coroutines, Flow and architecture components
| Library | Artifact | Use |
|---|---|---|
| kotlinx-coroutines-test | org.jetbrains.kotlinx:kotlinx-coroutines-test | runTest, TestScope, StandardTestDispatcher, UnconfinedTestDispatcher, Dispatchers.setMain. Delays run in virtual time, so a 10-second wait costs nothing. |
| Turbine | app.cash.turbine:turbine | Test a Flow by awaiting its items one at a time: flow.test { awaitItem() }. |
| Arch core testing | androidx.arch.core:core-testing | InstantTaskExecutorRule makes LiveData run immediately in tests. |
| Lifecycle testing | androidx.lifecycle:lifecycle-runtime-testing | TestLifecycleOwner to move a lifecycle through its states. |
| WorkManager testing | androidx.work:work-testing | Run workers synchronously and control constraints. |
| Navigation testing | androidx.navigation:navigation-testing | TestNavHostController to check destinations. |
E. Android on the JVM
| Library | Artifact | Use |
|---|---|---|
| Robolectric | org.robolectric:robolectric | Runs the Android framework on the JVM, so activities, views, resources and many system services work in a fast local test. Also runs Compose UI tests. |
| AndroidX Test core | androidx.test:core, androidx.test.ext:junit | ApplicationProvider, ActivityScenario, AndroidJUnit4. The same test can run on Robolectric or on a device. |
F. Network, database and dependency injection
| Library | Artifact | Use |
|---|---|---|
| MockWebServer | com.squareup.okhttp3:mockwebserver | A real local HTTP server that returns scripted responses. Tests your real Retrofit or OkHttp stack end to end, including errors and timeouts. |
| Retrofit Mock | com.squareup.retrofit2:retrofit-mock | Fake Retrofit services with simulated delay and failure. |
| Ktor MockEngine | io.ktor:ktor-client-mock | The same idea for Ktor clients. |
| WireMock | org.wiremock:wiremock | A heavier, more featureful HTTP stub server. |
| Room testing | androidx.room:room-testing | Migration testing, plus in-memory databases for DAO tests. |
| Hilt testing | com.google.dagger:hilt-android-testing | @HiltAndroidTest, @TestInstallIn to swap real modules for fakes. |
| Koin test | io.insert-koin:koin-test (and the -junit4 / -junit5 modules) | Check your module graph and swap definitions. |
G. UI and instrumented testing
| Library | Artifact | Use |
|---|---|---|
| Espresso | androidx.test.espresso:espresso-core | Drives your own app’s View UI. Add espresso-contrib (RecyclerView, drawers), espresso-intents (intent checks), espresso-idling-resource, espresso-web. |
| Compose UI test | androidx.compose.ui:ui-test-junit4 and ui-test-manifest | Finds and drives Compose nodes through the semantics tree. |
| UI Automator | androidx.test.uiautomator:uiautomator | Drives the whole device, including other apps and system UI. |
| AndroidX Test runner and rules | androidx.test:runner, androidx.test:rules | AndroidJUnitRunner, GrantPermissionRule, ServiceTestRule. |
| Test Orchestrator | androidx.test:orchestrator | Runs each test in its own process so one crash or leaked state cannot spoil the next. |
| Fragment testing | androidx.fragment:fragment-testing | launchFragmentInContainer to test a fragment on its own. |
| Barista, Kakao, Kaspresso | community libraries | Friendlier DSLs on top of Espresso. Kaspresso (from Kaspersky) also adds retries, logging and screenshots. |
| Accessibility checks | Espresso AccessibilityChecks, Compose accessibility checks | Fail the test when a touch target is too small or a label is missing. |
H. Screenshot (snapshot) testing
| Library | Plugin or artifact | Notes |
|---|---|---|
| Paparazzi | app.cash.paparazzi | Renders views and composables on the JVM without a device, then compares to saved images. Fast. |
| Roborazzi | io.github.takahirom.roborazzi | Screenshot tests on Robolectric. |
| Shot | com.karumi.shot | Screenshot tests on a device, with a Gradle workflow to record and verify. |
| Compose Preview Screenshot Testing | Android Gradle plugin tooling | Google’s own approach, which turns @Preview composables into screenshot tests. Still evolving, so check the current docs. |
I. Coverage, property tests, BDD and contracts
| Library | Artifact or plugin | Use |
|---|---|---|
| JaCoCo | Gradle jacoco plugin | Coverage reports for JVM and Android code. |
| Kover | org.jetbrains.kotlinx.kover | JetBrains’ Kotlin-aware coverage tool. |
| jqwik, Kotest property | net.jqwik:jqwik, kotest-property | Property-based testing: state a rule, and the library tries hundreds of generated inputs. |
| PIT (Pitest) | Gradle PIT plugin | Mutation testing: it changes your code on purpose and checks that a test fails. |
| Cucumber Android | io.cucumber:cucumber-android | Behaviour-driven tests written in plain-language Gherkin. |
| Pact | au.com.dius.pact | Contract tests between the app and its backend. |
J. Devices and clouds
| Option | What it gives you |
|---|---|
| Android Emulator and Gradle managed devices | Emulators created and torn down by Gradle itself, so the setup is the same on every machine. |
| Firebase Test Lab | Real and virtual devices in Google’s data centres, with videos and logs. |
| AWS Device Farm, BrowserStack, Sauce Labs | Large pools of real devices you can run Espresso or Appium tests on. |
- Local: JUnit 4 (or 5 for pure modules), MockK, Truth (or Kluent),
kotlinx-coroutines-test, Turbine. - Integration: MockWebServer, an in-memory Room database, Hilt testing. Add Robolectric only when you need the framework on the JVM.
- UI: Compose UI test or Espresso for your own screens, a few flows.
- Visual: Paparazzi or Roborazzi for screenshots.
- Quality: Kover or JaCoCo for coverage.
6. Fakes, stubs and mocks
A unit test should check one class, so everything that class depends on (a database, a server, the clock) is replaced with a stand-in. The stand-ins are called test doubles, and there are five kinds.
| Double | What it does |
|---|---|
| Dummy | Fills a parameter that the test never uses. |
| Stub | Returns fixed answers: “when asked for user 7, return Ana”. |
| Fake | A working but simple implementation, such as a repository backed by a list in memory. |
| Spy | Wraps a real object and records how it was called. |
| Mock | Records calls and lets you verify them: “the repository was asked to save exactly once”. |
Here is the same login test written both ways. The service under test is the interface UserRepository.
interface UserRepository {
suspend fun login(email: String, password: String): Result<User>
}
// 1) A fake: a tiny real implementation. Reusable across many tests.
class FakeUserRepository : UserRepository {
var failWith: Throwable? = null
override suspend fun login(email: String, password: String): Result<User> =
failWith?.let { Result.failure(it) } ?: Result.success(User(email))
}
// 2) A mock: a recorded stand-in, scripted per test.
val repo = mockk<UserRepository>()
coEvery { repo.login("a@b.com", "bad") } returns Result.failure(IllegalArgumentException())
// ... run the code ...
coVerify(exactly = 1) { repo.login("a@b.com", "bad") }
- Prefer fakes for things with state (repositories, data sources). They survive refactors because they test behaviour, not call order.
- Use mocks at the edges, when the interaction itself is the thing you care about (an analytics event was sent).
- Do not mock what you do not own. Wrap the third-party class behind your own interface, and double that.
- Do not mock simple data classes. Create real ones.
- Inject dependencies through the constructor. A class that builds its own collaborators cannot be tested in isolation.
7. Testing coroutines, Flow and ViewModels
Most Android logic is asynchronous, and asynchronous tests are where beginners get stuck. The fix is to control time and threads. kotlinx-coroutines-test gives you runTest, which runs a coroutine on a virtual clock: a delay(5_000) completes instantly, and the test is deterministic.
Replace the Main dispatcher
A ViewModel launches work on Dispatchers.Main, which does not exist in a local test. Replace it with a test dispatcher using a rule you write once:
class MainDispatcherRule(
private val dispatcher: TestDispatcher = UnconfinedTestDispatcher()
) : TestWatcher() {
override fun starting(description: Description) = Dispatchers.setMain(dispatcher)
override fun finished(description: Description) = Dispatchers.resetMain()
}
A ViewModel test
data class LoginState(
val loading: Boolean = false,
val loggedIn: Boolean = false,
val error: String? = null
)
class LoginViewModel(private val repo: UserRepository) : ViewModel() {
private val _state = MutableStateFlow(LoginState())
val state: StateFlow<LoginState> = _state.asStateFlow()
fun onLoginClicked(email: String, password: String) {
viewModelScope.launch {
_state.update { it.copy(loading = true, error = null) }
repo.login(email, password)
.onSuccess { _state.update { s -> s.copy(loading = false, loggedIn = true) } }
.onFailure { _state.update { s -> s.copy(loading = false, error = "Wrong email or password") } }
}
}
}
class LoginViewModelTest {
@get:Rule val mainRule = MainDispatcherRule()
private val repo = FakeUserRepository()
@Test
fun `wrong password shows an error and stops loading`() = runTest {
repo.failWith = IllegalArgumentException()
val viewModel = LoginViewModel(repo)
viewModel.onLoginClicked("a@b.com", "bad")
val state = viewModel.state.value
assertEquals("Wrong email or password", state.error)
assertFalse(state.loading)
assertFalse(state.loggedIn)
}
}
Testing a Flow with Turbine
A Flow emits several values over time. Turbine collects them into a queue and lets you read them in order, with a timeout so a test cannot hang forever.
@Test
fun `emits loading, then the result, then completes`() = runTest {
val flow = flow { emit("loading"); delay(100); emit("done") }
flow.test {
assertEquals("loading", awaitItem())
assertEquals("done", awaitItem())
awaitComplete()
}
}
Two traps. A StateFlow is conflated: if it changes twice before you collect, you see only the last value, so check the final state or use an unconfined dispatcher when you need every step. And LiveData needs InstantTaskExecutorRule in local tests, otherwise it throws because it expects the Android main looper.
Pass the dispatcher into your classes (class Repo(private val io: CoroutineDispatcher)) rather than hard-coding Dispatchers.IO. A test can then supply its own and run everything on one thread.
8. Instrumentation tests: Espresso, Compose and UI Automator
An instrumentation test runs on a real device or emulator. Gradle builds your app and a second “test APK”, installs both, and starts AndroidJUnitRunner. The test process runs in the same process as your app, so it can reach into your code, start activities and read the live UI.
// build.gradle.kts
android {
defaultConfig {
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArguments["clearPackageData"] = "true" // wipe app data between tests (with the orchestrator)
}
testOptions {
animationsDisabled = true // no animations while testing
execution = "ANDROIDX_TEST_ORCHESTRATOR" // each test in a fresh process
}
}
dependencies {
androidTestImplementation("androidx.test.ext:junit:<version>")
androidTestImplementation("androidx.test.espresso:espresso-core:<version>")
androidTestImplementation("androidx.compose.ui:ui-test-junit4:<version>")
debugImplementation("androidx.compose.ui:ui-test-manifest:<version>")
androidTestUtil("androidx.test:orchestrator:<version>")
}
Espresso (Views)
Espresso’s trick is built-in synchronisation. Before each action it waits until the main thread is idle and background work it knows about (AsyncTask, its own idling resources) is finished. That is why Espresso tests are far more stable than tests that sleep. A test has three parts: find a view (onView(withId(...))), act on it (perform(click())), assert (check(matches(isDisplayed()))).
@RunWith(AndroidJUnit4::class)
class LoginScreenTest {
@get:Rule val rule = ActivityScenarioRule(LoginActivity::class.java)
@Test
fun wrongPasswordShowsError() {
onView(withId(R.id.email)).perform(typeText("a@b.com"), closeSoftKeyboard())
onView(withId(R.id.password)).perform(typeText("bad"), closeSoftKeyboard())
onView(withId(R.id.loginButton)).perform(click())
onView(withText("Wrong email or password")).check(matches(isDisplayed()))
}
}
- Lists: use
RecyclerViewActionsfromespresso-contribto scroll to or click an item. - Intents:
espresso-intentslets you checkintended(hasComponent(...))and stub the result of another app. - Background work: register an
IdlingResourceso Espresso waits for your custom thread pool or network client.
Compose UI tests
Compose has no Views, so tests use the semantics tree: the description of the UI that accessibility services also read. You find nodes, perform actions and make assertions. Compose’s test framework synchronises with recomposition automatically.
class LoginScreenComposeTest {
@get:Rule val rule = createAndroidComposeRule<MainActivity>()
@Test
fun wrongPasswordShowsError() {
rule.onNodeWithTag("email").performTextInput("a@b.com")
rule.onNodeWithTag("password").performTextInput("bad")
rule.onNodeWithText("Log in").performClick()
rule.onNodeWithText("Wrong email or password").assertIsDisplayed()
}
}
To test one composable on its own, use createComposeRule() and call rule.setContent { LoginScreen(...) }. That is faster than launching the whole activity, and it can run on Robolectric as well. Add Modifier.testTag("email") to nodes that have no unique visible text. For waits that are not tied to recomposition, use rule.waitUntil { ... }.
UI Automator
Espresso and Compose tests can see only your app. UI Automator sees the whole screen, so it can press Home, open the notification shade, grant a system permission dialog, or switch between apps.
val device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation())
device.openNotification()
device.wait(Until.hasObject(By.text("Order shipped")), 5_000)
device.findObject(By.text("Order shipped")).click()
Choosing between them
| Tool | Sees | Runs | Best for |
|---|---|---|---|
| Espresso | Your app’s Views | Device or emulator (or Robolectric) | Screens and flows in a View-based app |
| Compose UI test | Your app’s composables | Device, emulator or Robolectric | Screens and flows in a Compose app |
| UI Automator | Any app and the system UI | Device or emulator | Permissions, notifications, deep links, multi-app flows |
| Robolectric | Your app, simulated | JVM, no device | Fast UI and framework tests where full fidelity is not needed |
These tools combine well: a Compose test for the screen, UI Automator for the permission dialog that appears on first launch.
9. UI automation and Appium
UI automation means a program drives the app’s user interface the way a person would: tap, type, scroll, read the screen. Espresso is UI automation from the inside. Appium is UI automation from the outside.
Appium is an open-source tool that exposes a standard web-automation protocol (the W3C WebDriver protocol, the same one Selenium uses) for mobile apps. You write a script in almost any language, Appium translates each command into a call to a platform driver, and the driver operates the device. It sees the app as a black box, with no access to your source code.
What makes Appium different
- Cross-platform. The UiAutomator2 driver covers Android and the XCUITest driver covers iOS, with the same client API. There are also Espresso, Flutter and other drivers, and Appium 2 installs drivers as plugins.
- Any language. Official clients exist for Java (usable from Kotlin), Python, JavaScript, Ruby and C#.
- Black-box. You test the installed APK, even a release build, the way a customer would. You cannot reach into the app’s internals.
- Real devices and clouds. Device farms like BrowserStack, Sauce Labs and AWS Device Farm speak Appium natively.
- Appium Inspector. A desktop app where you click an element on a live screen and see its locators.
Set it up
# You need Node.js, a JDK and the Android SDK (ANDROID_HOME set)
npm install -g appium
appium driver install uiautomator2
appium driver list --installed
appium # starts the server at http://127.0.0.1:4723
Write a test (Kotlin, JUnit and the Java client)
dependencies {
testImplementation("io.appium:java-client:<version>")
testImplementation("org.seleniumhq.selenium:selenium-java:<version>")
}
class LoginAppiumTest {
private lateinit var driver: AndroidDriver
@Before
fun start() {
val options = UiAutomator2Options()
.setDeviceName("Android Emulator")
.setApp("/path/to/app-debug.apk")
driver = AndroidDriver(URI("http://127.0.0.1:4723").toURL(), options)
}
@Test
fun wrongPasswordShowsError() {
driver.findElement(AppiumBy.accessibilityId("email")).sendKeys("a@b.com")
driver.findElement(AppiumBy.accessibilityId("password")).sendKeys("bad")
driver.findElement(AppiumBy.accessibilityId("loginButton")).click()
val error = WebDriverWait(driver, Duration.ofSeconds(5)).until(
ExpectedConditions.visibilityOfElementLocated(AppiumBy.accessibilityId("error"))
)
assertEquals("Wrong email or password", error.text)
}
@After
fun stop() = driver.quit()
}
Locators: how Appium finds an element
| Locator | Example | Verdict |
|---|---|---|
| Accessibility id | AppiumBy.accessibilityId("loginButton") | Best. Uses the content description, so it also improves accessibility, and it works across platforms. |
| Resource id | AppiumBy.id("com.example:id/email") | Good for Views. Stable as long as ids are not renamed. |
| UiSelector | AppiumBy.androidUIAutomator("new UiSelector().text(\"Log in\")") | Powerful, Android-only. Text-based selectors break when copy or language changes. |
| XPath | AppiumBy.xpath("//*[@text='Log in']") | Last resort. Slow and brittle. |
Compose tip. By default, Appium cannot see a composable’s testTag. Turn it into a resource id once, near the root of your UI, and AppiumBy.id("email") will find Modifier.testTag("email"):
@OptIn(ExperimentalComposeUiApi::class)
@Composable
fun AppRoot() {
Box(Modifier.semantics { testTagsAsResourceId = true }) {
// your screens
}
}
Appium or Espresso?
| Espresso / Compose test | Appium | |
|---|---|---|
| Position | Inside the app process | Outside, over the network |
| Speed and stability | Faster. Automatic synchronisation with the UI thread | Slower. You write explicit waits |
| Language | Kotlin or Java only | Almost any |
| Platforms | Android only | Android, iOS and more, with shared test code |
| Access to app code | Yes: fakes, DI, internal state | No: only what the user sees |
| Who writes it | Developers | Developers or a separate QA team |
| Best for | Many screen-level tests that run on every pull request | A small set of release-level journeys, or one test suite for two platforms |
Neither replaces the other. A common split is Espresso or Compose tests for most UI checks, and a short Appium suite for the handful of flows the business cannot afford to break (sign-up, purchase, login).
Other UI automation tools
| Tool | Notes |
|---|---|
| Maestro | Flows written in simple YAML, with built-in waiting. Android, iOS and web. Very quick to start. Example below. |
| Detox | Gray-box end-to-end testing for React Native. |
| Kaspresso, Kakao, Barista | Wrappers that make Espresso code shorter and add retries or screenshots. |
| Katalon, TestComplete and similar | Commercial suites with record-and-replay and reporting. |
| Robotium, Calabash, Selendroid | Older tools. Largely replaced by Espresso, UI Automator and Appium. |
# login.yaml (run with: maestro test login.yaml)
appId: com.example.app
---
- launchApp
- tapOn: "Email"
- inputText: "a@b.com"
- tapOn: "Password"
- inputText: "bad"
- tapOn: "Log in"
- assertVisible: "Wrong email or password"
10. Keeping UI tests reliable
A flaky test passes and fails on the same code. Flaky tests are poison: a red build that people rerun until it turns green teaches them to ignore red. Almost every flaky UI test comes from one of a few causes.
| Cause | Fix |
|---|---|
| Animations and transitions | Turn them off for tests: testOptions.animationsDisabled = true, or set the three developer-option animation scales to 0 on CI devices. |
| Timing: acting before the UI is ready | Use the framework’s sync (Espresso, Compose), an IdlingResource for custom async work, or an explicit wait in Appium. Never use Thread.sleep. |
| Shared state between tests | Use Test Orchestrator with clearPackageData, create fresh data per test, and never let one test depend on another. |
| Real network and real backends | Point the app at MockWebServer or a staging stub so responses are the same every run. |
| Test data and accounts | Create the data you need inside the test (an API call or a database insert), not by hand beforehand. |
| Slow or unhealthy emulators | Use managed devices or a fixed image, enough RAM, hardware acceleration, and a cold start per job. |
| Screen size and OS differences | Pick locators that do not depend on position. Run on a small matrix, not on every device. |
The robot pattern
If the test file knows every id and tag, one UI change breaks fifty tests. Put those details in a small helper class (a robot or page object) so tests read like the story and a UI change is fixed in one place.
class LoginRobot(private val rule: ComposeTestRule) {
fun typeEmail(value: String) = apply { rule.onNodeWithTag("email").performTextInput(value) }
fun typePassword(value: String) = apply { rule.onNodeWithTag("password").performTextInput(value) }
fun tapLogin() = apply { rule.onNodeWithText("Log in").performClick() }
fun assertError(message: String) = apply { rule.onNodeWithText(message).assertIsDisplayed() }
}
@Test
fun wrongPasswordShowsError() {
LoginRobot(rule)
.typeEmail("a@b.com")
.typePassword("bad")
.tapLogin()
.assertError("Wrong email or password")
}
Run them in continuous integration
UI tests that no one runs are decoration. Run unit tests on every pull request, because they take seconds. Run a small UI suite on every pull request too if it stays under about ten minutes, and the full suite nightly or before release.
# .github/workflows/ui-tests.yml (excerpt)
- uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 34
arch: x86_64
script: ./gradlew connectedDebugAndroidTest
The snippet uses the android-emulator-runner action. On Linux runners the emulator needs hardware acceleration (KVM) to be usable. Keep a rule for flaky tests: quarantine them quickly so they stop blocking others, fix them within a set time, and track how many reruns the pipeline performs. One automatic retry is a reasonable safety net. Three retries hide a real problem.
11. TDD: red, green, refactor
Test-driven development turns the usual order around. You write a failing test first, then write just enough code to make it pass, then clean up. You repeat this in very small steps, each a minute or two long.
Robert C. Martin (“Uncle Bob”) states it as three rules: do not write production code until you have a failing test; do not write more of a test than is enough to fail (a compile error counts); do not write more production code than is enough to pass the failing test.
A walk-through
We want the discount rule from section 4. We build it in three laps.
Lap 1, Red. Write the first test. It does not even compile, because DiscountCalculator does not exist yet. Create it with a body that fails.
@Test fun `10 percent off at 5000 or more`() {
assertEquals(9_000, DiscountCalculator().priceAfterDiscount(10_000))
}
class DiscountCalculator {
fun priceAfterDiscount(price: Int): Int = TODO() // test is red
}
Lap 1, Green. Write the simplest code that passes. Not the final code, just the simplest.
fun priceAfterDiscount(price: Int): Int = price - price / 10 // green
Lap 2, Red. Add the next rule as a new test. Small orders must not get a discount.
@Test fun `no discount below the threshold`() {
assertEquals(4_000, DiscountCalculator().priceAfterDiscount(4_000)) // red: returns 3,600
}
Lap 2, Green. The failing test forces the condition into existence.
fun priceAfterDiscount(price: Int): Int =
if (price >= 5_000) price - price / 10 else price // green
Lap 3, Red and Green. Add a test for negative prices that expects IllegalArgumentException, then add require(price >= 0).
Refactor. With three green tests as a net, tidy the code: name the magic numbers and keep the tests passing.
class DiscountCalculator {
private companion object {
const val THRESHOLD = 5_000
const val PERCENT_OFF = 10
}
fun priceAfterDiscount(price: Int): Int {
require(price >= 0) { "price must not be negative" }
return if (price >= THRESHOLD) price - price * PERCENT_OFF / 100 else price
}
}
The benefits of TDD
| Benefit | Why it happens |
|---|---|
| Better design | Code that is hard to test is hard to use. Writing the test first forces small classes, clear interfaces and injected dependencies. |
| Fewer bugs reach later stages | Every behaviour has a test before it exists, so gaps show up while you are still in the code. |
| A safety net for change | You can refactor freely, because a broken behaviour turns a test red within seconds. |
| Tests as documentation | Test names describe what the code does, and they cannot go out of date without failing. |
| Less over-building | You write only what the next failing test needs, which keeps unused features out. |
| Less time in the debugger | Steps are tiny, so when something breaks, the cause is in the last few lines. |
| Useful coverage by default | Every line exists because a test asked for it. |
| Confidence and focus | One small goal at a time lowers the stress of big changes. |
| Easier reviews and onboarding | A new teammate can read the tests to learn what the code promises. |
The costs, honestly
- It is slower at first. Expect a dip in speed while the habit forms. An often-cited study of four industrial teams reported 40% to 90% fewer defects with 15% to 35% more initial effort. Other studies found smaller or mixed effects, so treat TDD as a good bet, not a guarantee.
- It has a learning curve. Writing a good first test needs you to understand the requirement and the design in advance.
- Bad tests give false confidence. A test suite that checks implementation details breaks on every refactor and slows you down.
- Over-mocking couples tests to code. If a test mirrors the production code line by line, it protects nothing.
- It suits some work more than other work. Logic with clear rules is a great fit. Exploratory work, quick prototypes and pixel-level UI are not.
- Tests need upkeep. They are code you now own.
Where TDD works best on Android
- Great fit: use cases, validators, mappers, reducers, ViewModels, repositories with fakes, parsers, anything with rules and edge cases.
- Good fit with care: network and database layers, using MockWebServer and in-memory Room.
- Poor fit: the visual design of a screen. Build it, then add a screenshot test or a short UI test after.
- Make it possible: keep the Android framework at the edges. Put logic in plain Kotlin classes behind interfaces and let Activities and Fragments stay thin (the “humble object” idea).
Cousins of TDD
| Name | What it adds |
|---|---|
| ATDD | Acceptance-test-driven development. The first failing test is a high-level acceptance test agreed with the product owner, and unit-level TDD fills in the details. |
| BDD | Behaviour-driven development. Tests are written as Given, When, Then sentences in the language of the business, often with Cucumber or Kotest specs. |
| Test-after | Write the code, then the test. It is better than no tests, but the design pressure is lost. |
| Classic vs mockist TDD | The “Chicago” school uses real collaborators and fakes. The “London” school mocks each collaborator and verifies interactions. Most Android teams lean classic. |
For how testing fits into architecture (why MVVM and clean layers make tests cheap), see Android architecture, part 8: testing and proof.
12. A practical test strategy
Put each kind of check at the lowest layer that can catch it. A good first draft for a typical app looks like this:
| What you are testing | Test type | Typical tools |
|---|---|---|
| Business rules, validators, mappers | Local unit | JUnit, Truth or Kluent |
| ViewModels, use cases, reducers | Local unit | JUnit, MockK or fakes, coroutines-test, Turbine |
| Repository with its data sources | Local integration | Fakes, MockWebServer, in-memory Room (Robolectric if needed) |
| DAO queries and migrations | Instrumented or Robolectric | Room testing, MigrationTestHelper |
| One screen’s behaviour | UI test with a fake backend | Compose UI test or Espresso, Hilt test modules |
| How a screen looks, in light and dark, large font | Screenshot test | Paparazzi, Roborazzi or Shot |
| Permissions, notifications, other apps | Instrumented | UI Automator |
| The 5 to 10 flows that must never break | End to end | Appium or Maestro on release builds or a device cloud |
Folder layout and naming
app/
src/
main/ production code
test/ local tests (JVM) <- most of your tests
androidTest/ instrumented tests <- fewer, slower
debug/ test-only helpers such as the Compose test manifest
- Name the test class after what it tests:
LoginViewModelTest. - Name the test after the behaviour, not the method:
wrong password shows an error, nottestLogin2. - Follow F.I.R.S.T: tests should be Fast, Independent, Repeatable, Self-validating and Timely.
What to do with coverage
Coverage tools (JaCoCo, Kover) tell you which lines your tests ran, not whether they checked anything. Use the report to find untested code, especially on risky branches. Do not set “90% or the build fails” as a goal for everyone, because people then write assertion-free tests to hit the number. A better pairing is a modest coverage floor plus occasional mutation testing with PIT, which proves your tests actually fail when the code is wrong.
Other kinds of test worth knowing
- Performance: Jetpack Macrobenchmark (startup, scrolling) and Microbenchmark (hot code).
- Accessibility: automated checks in Espresso and Compose, plus a manual TalkBack pass.
- Contract: Pact, so the app and its API agree on the shape of data.
- Exploratory and manual: a person looking for what nobody thought to automate. It is still valuable, just not as a regression plan.
13. Common mistakes
| Mistake | Better |
|---|---|
| The ice-cream cone: lots of UI tests, few unit tests | Move checks down. Test logic in ViewModels and use cases, and keep UI tests for flows. |
| Testing implementation, not behaviour | Assert what the user or caller can observe. Do not verify private calls and call order unless they matter. |
| Mocking everything | Use real simple objects and fakes. Mock only the boundary. |
Thread.sleep in UI tests | Use idling resources, Compose sync, or explicit waits on a condition. |
| Tests that depend on each other or on order | Make each test set up and clean up its own state. |
| One giant test with twenty assertions | One behaviour per test, so a failure points to a cause. |
| Ignoring or skipping failing tests | Fix or delete. A permanently skipped test is a lie in the suite. |
| Chasing a coverage number | Measure risk covered, not lines touched. |
| Testing the framework | Do not test that TextView.setText works. Test your own logic. |
| Production code that reads time, randomness or singletons directly | Inject a clock, a random source and dependencies so tests control them. |
| Assertion-free tests | A test that cannot fail proves nothing. Make it red once on purpose to check it can. |
14. Takeaways, a starter path and glossary
- Shape the suite like a pyramid: many unit tests, some integration tests, few end-to-end tests.
- Keep logic in plain Kotlin so it runs as a fast local test. Reserve the device for what truly needs it.
- JUnit is the base. Add one assertion library, one mocking library and the coroutine helpers.
- Espresso and Compose tests drive your own app. UI Automator and Appium drive the device from outside.
- Fix flakiness at the cause: animations, timing, shared state and real networks.
- TDD improves design and gives a safety net, at the cost of early speed. Use it where rules are clear.
A four-step path to start
- This week: write five JUnit tests for one plain Kotlin class, such as a validator or a mapper. Run them from Gradle.
- Next: test one ViewModel with a fake repository,
runTestand a main dispatcher rule. - Then: add one Compose or Espresso test for your app’s most important happy path, with a fake backend.
- Finally: run the unit tests on every pull request in CI, and the UI test on a schedule. Once that is steady, add screenshot tests and a short Appium or Maestro flow if you need them.
| Term | In plain words |
|---|---|
| Unit test | A fast test of one small piece of code in isolation. |
| Integration test | A test of several real pieces working together. |
| End-to-end (E2E) test | A test that drives the whole app from the user’s point of view. |
| Instrumented test | A test that runs on a device or emulator, in the app’s process. |
| Test double | Any stand-in used in place of a real dependency in a test. |
| Fake | A simple working implementation, such as an in-memory repository. |
| Mock | A stand-in that records calls so you can verify them. |
| Flaky test | A test that sometimes passes and sometimes fails on the same code. |
| Idling resource | A signal that tells Espresso the app is still busy and it should wait. |
| Semantics tree | The description of a Compose UI used by tests and accessibility services. |
| Robot / page object | A helper class that hides locators so tests read like a story. |
| WebDriver | The standard protocol for driving a UI from outside, used by Selenium and Appium. |
| TDD | Test-driven development: write a failing test, make it pass, then refactor. |
| Regression | A bug in something that used to work. |
| Coverage | The share of code that tests ran. It does not prove the tests check anything. |
| Mutation testing | Changing the code on purpose to see whether a test notices. |
| Snapshot / screenshot test | A test that compares a rendered screen to a saved image. |
Links and further reading
- Android Developers: Test your app
- Fundamentals of testing Android apps
- Local tests · Instrumented tests
- Espresso · Testing your Compose layout · UI Automator
- JUnit 5 user guide · JUnit 4 · android-junit5
- MockK · Mockito · Truth · Turbine · Kotest
- Robolectric · Paparazzi · Kover · JaCoCo
- Appium documentation · Maestro · Firebase Test Lab