Skip to content

Demonstration of 2 bugs reproducible using UITest 2.2.7

Pre-release
Pre-release
Compare
Choose a tag to compare
@King-of-Spades King-of-Spades released this 08 Nov 01:21
1.B1-BugDemo
4bd693d

Overview

This is a snapshot to reproduce two specific bugs more consistently. It's not recommended to use this version for everyday use.

Bug 1: App is not restarted between local tests on iOS

Behavior on iOS simulator

After initially running some tests with an app on the iOS simulator, if you start a new test run; the app is not automatically restarted as it normally would be in the default AppDataMode.

This can cause inconsistent test results, since queries in the 1st test executed will pass or fail based on the non-restarted app; rather than when the app has been restarted as will occur for subsequent tests, except for the very last test executed.

Xamarin.UITest will not restart the app on the iOS simulator for the last test executed. I suspect this may be intentional; so that a user viewing the simulator & completed tests will be able to directly see the final state of the app.

This doesn't directly interfere with testing, but since the 1st test doesn't reset it can cause the next test executed to have unexpected behavior due to this bug.

Behavior on iOS device

(Note: I have not explicitly reproduced this because I haven't had the chance to test this on a physical device, but I suspect the behavior is closely related or caused by the same issue we see in the simulator demo noted above)

Reported steps to reproduce:

  1. Run a test
  2. The app starts
  3. The test runs and passes, but an error shows in the output window.
  4. The app DOES NOT close
  5. Run a new test
  6. the app closes
  7. The test runs but fails with an exception. The app does not start.
  8. Repeat

Bug 2: App.Tap() cannot be performed when using ConnectToApp() on iOS

Description

I can reproduce the issue just by changing "StartApp()" to "ConnectToApp()" in UITestDemo, making sure the test app is launched on the target simulator; and then running a test that includes tapping.

There are some interesting additional observations when using the Repl() tool, specifically; if you set up this issue, I note that the tree command & app.Flash(x => x.Marked("Add")) command works normally as expected.

However, app.Tap(x => x.Marked("Add")) reproduces the issue, and in the Repl outputs a very large stacktrace. I've attached the output to this report as "raw stacktrace"

Because Tree & Flash work normally as expected; it seems like queries should be working, but I don't know much about how ConnectToApp() operates since I haven't used it many times in the past.

Assets 2
Loading