Using FailTestOnLeakRunListener fails to report back failures when using Android Test Orchestrator. #1046
Labels
Milestone
Comments
|
We should look at how failure reporting is implemented in the orchestrator. |
|
Looks like orchestrator is just an APK: https://dl.google.com/dl/android/maven2/com/android/support/test/orchestrator/1.0.2/orchestrator-1.0.2.pom The pom.xml has a dependency on Which is also an APK and has no dependency. |
|
We'll ship 1.6 as is for now, with the known issue that it doesn't work with the Android Test orchestrator, until we figure out exactly works / how it differs. |
|
|
|
Refiled the bug since the original was closed. Please star the issue here: https://issuetracker.google.com/issues/154639612 |
|
btw I started some exploration here: #1758 |
pyricau
added a commit
that referenced
this issue
Jun 4, 2020
With this change, FailTestOnLeakRunListener now hooks into AndroidJUnitRunner when a test run starts to detect if the tests are run by Android Test Orchestrator, in which case we take over the binder callback to send optionally send an extra failure when a leak is detected in test. This works whether newRunListenerMode is false or true (ie whether OrchestratedInstrumentationListener runs before or after FailTestOnLeakRunListener). When a test finishes without any failure, we intercept any "test finished" message, check for memory leaks, optionally send a "test failure" message then send the "test finished message". This intercepting is done assuming a single thread calls both listeners (see `org.junit.runner.notification.RunNotifier#wrapIfNotThreadSafe`). As before with `Instrumentation.sendStatus()`, the current approach is tied to the test runner implementation details and might have to change to adapt to new `androidx.test:runner` versions. Other change: FailTestOnLeakRunListener will wait up to 2 seconds for all activities to be destroyed before running leak detection, and otherwise proceed. Fixes #1046
pyricau
added a commit
that referenced
this issue
Jun 4, 2020
With this change, FailTestOnLeakRunListener now hooks into AndroidJUnitRunner when a test run starts to detect if the tests are run by Android Test Orchestrator, in which case we take over the binder callback to send optionally send an extra failure when a leak is detected in test. This works whether newRunListenerMode is false or true (ie whether OrchestratedInstrumentationListener runs before or after FailTestOnLeakRunListener). When a test finishes without any failure, we intercept any "test finished" message, check for memory leaks, optionally send a "test failure" message then send the "test finished message". This intercepting is done assuming a single thread calls both listeners (see `org.junit.runner.notification.RunNotifier#wrapIfNotThreadSafe`). As before with `Instrumentation.sendStatus()`, the current approach is tied to the test runner implementation details and might have to change to adapt to new `androidx.test:runner` versions. Other change: FailTestOnLeakRunListener will wait up to 2 seconds for all activities to be destroyed before running leak detection, and otherwise proceed. Fixes #1046
|
Finally got something ready! #1849 |
pyricau
added a commit
that referenced
this issue
Jun 5, 2020
With this change, FailTestOnLeakRunListener now hooks into AndroidJUnitRunner when a test run starts to detect if the tests are run by Android Test Orchestrator, in which case we take over the binder callback to send optionally send an extra failure when a leak is detected in test. This works whether newRunListenerMode is false or true (ie whether OrchestratedInstrumentationListener runs before or after FailTestOnLeakRunListener). When a test finishes without any failure, we intercept any "test finished" message, check for memory leaks, optionally send a "test failure" message then send the "test finished message". This intercepting is done assuming a single thread calls both listeners (see `org.junit.runner.notification.RunNotifier#wrapIfNotThreadSafe`). As before with `Instrumentation.sendStatus()`, the current approach is tied to the test runner implementation details and might have to change to adapt to new `androidx.test:runner` versions. Other change: FailTestOnLeakRunListener will wait up to 2 seconds for all activities to be destroyed before running leak detection, and otherwise proceed. Fixes #1046
pyricau
added a commit
that referenced
this issue
Jun 8, 2020
With this change, FailTestOnLeakRunListener now hooks into AndroidJUnitRunner when a test run starts to detect if the tests are run by Android Test Orchestrator, in which case we take over the binder callback to send optionally send an extra failure when a leak is detected in test. This works whether newRunListenerMode is false or true (ie whether OrchestratedInstrumentationListener runs before or after FailTestOnLeakRunListener). When a test finishes without any failure, we intercept any "test finished" message, check for memory leaks, optionally send a "test failure" message then send the "test finished message". This intercepting is done assuming a single thread calls both listeners (see `org.junit.runner.notification.RunNotifier#wrapIfNotThreadSafe`). As before with `Instrumentation.sendStatus()`, the current approach is tied to the test runner implementation details and might have to change to adapt to new `androidx.test:runner` versions. Other change: FailTestOnLeakRunListener will wait up to 2 seconds for all activities to be destroyed before running leak detection, and otherwise proceed. Fixes #1046
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When using LeakCanary with Android Test Orchestrator and FailTestOnLeakRunListener, the RunListener fails to report the failure back to the Test Orchestrator even though it finds the leak.
Here's a sample. Run TuPeuxPasTest:
https://github.com/runningcode/leakcanary/tree/no/test-orchestrator
This is actually a bug with Android Test Orchestrator, here is a sample repro:
https://github.com/runningcode/TestOrchBug
And filed this bug with android:
https://issuetracker.google.com/issues/110978490Reopened here: https://issuetracker.google.com/issues/154639612
The text was updated successfully, but these errors were encountered: