test: Add integration(e2e) test #424

hanwencheng · 2019-10-17T13:09:44Z

This is the configuration part of #326 which enable us to run CI locally.

For test it locally you need install detox-cli as global npm dependency. it has conflict with local detox dependency so that we could not add it locally and start with npx

npm install -g detox-cli

After testing with Cavy and Detox, I find Detox is a better choice for us, since we want to develop the test and the origin code base at the same time. And the only complex part of Detox is the configuration part, to use it we do not to use High Order Component, but just with an testID on the component. It is more powerful and It is in active development by Wix team.

There are some comparison articles like Detox vs Appium or Detox vs Cavy

c37c6e4 : fixes a recursive path problem caused by react-native-camera when build test on iOS, discussion here

Test Demo

iOS	Android

Further steps

After a lot of attempts trying to add the integration test on Travis CI, I keep get the failing test because the build process failed without variant configurations like Rust environment or Android NKD. This will need helps from other people who is experienced with setting Travis CI test environment, and I would like to suggest now we do the CI test locally, and work on Travis CI integration in another PR, to prevent blocking current development work.

hanwencheng · 2019-10-17T20:05:49Z

android/app/build.gradle

@@ -107,7 +108,7 @@ android {

    defaultConfig {
        applicationId "io.parity.signer"
-        minSdkVersion 16
+        minSdkVersion 18


minimal version for Detox

hanwencheng · 2019-10-17T20:07:09Z

android/build.gradle

+    afterEvaluate {project ->
+        if (project.hasProperty("android")) {


fix the dependency version problem, e.g. in react-native-secure-store the compiledSdkVersion is 27, which will leads a compiling error.

Tbaut

Thanks a lot for that. Any chance this will work with a real device? I get the following:

yarn test-e2e:android
yarn run v1.19.1
$ detox test -c android.emu.debug
detox[8550] INFO:  [test.js] configuration="android.emu.debug" artifactsLocation="artifacts/android.emu.debug.2019-10-21 09-57-31Z" recordLogs="none" takeScreenshots="manual" recordVideos="none" recordPerformance="none" reportSpecs=true readOnlyEmu=false node_modules/.bin/jest --config=e2e/config.json --maxWorkers=1 '--testNamePattern=^((?!:ios:).)*$' "e2e"
detox[8558] INFO:  [DetoxServer.js] server listening on localhost:41537...
detox[8558] ERROR: [DetoxExportWrapper.js/DETOX_INIT_ERROR] 
 Error: Can not boot Android Emulator with the name: 'Nexus_5_API_28',
      make sure you choose one of the available emulators: 
    at EmulatorDriver._validateAvd (/home/thib/github/paritytech/parity-signer/node_modules/detox/src/devices/drivers/EmulatorDriver.js:98:13)
Load test: should have account list screen
Load test: should have account list screen [FAIL]

 FAIL  e2e/firstTest.spec.js
  Load test
    ✕ should have account list screen (2ms)

  ● Load test › should have account list screen

    Can not boot Android Emulator with the name: 'Nexus_5_API_28',
          make sure you choose one of the available emulators:

      at EmulatorDriver._validateAvd (../node_modules/detox/src/devices/drivers/EmulatorDriver.js:98:13)

  ● Load test › should have account list screen

    ReferenceError: device is not defined

      3 | describe('Load test', () => {
      4 |   beforeEach(async () => {
    > 5 |     await device.reloadReactNative();
        |           ^
      6 |   });
      7 | 
      8 |   it('should have account list screen', async () => {

      at device (firstTest.spec.js:5:11)
      at tryCatch (../node_modules/regenerator-runtime/runtime.js:45:40)
      at Generator.invoke [as _invoke] (../node_modules/regenerator-runtime/runtime.js:271:22)
      at Generator.prototype.(anonymous function) [as next] (../node_modules/regenerator-runtime/runtime.js:97:21)
      at tryCatch (../node_modules/regenerator-runtime/runtime.js:45:40)
      at invoke (../node_modules/regenerator-runtime/runtime.js:135:20)
      at ../node_modules/regenerator-runtime/runtime.js:170:11
      at callInvokeWithMethodAndArg (../node_modules/regenerator-runtime/runtime.js:169:16)
      at AsyncIterator.enqueue (../node_modules/regenerator-runtime/runtime.js:192:13)
      at AsyncIterator.prototype.(anonymous function) [as next] (../node_modules/regenerator-runtime/runtime.js:97:21)
      at Object.<anonymous>.exports.async (../node_modules/regenerator-runtime/runtime.js:216:14)
      at Object._callee (firstTest.spec.js:4:14)

  ● Load test › should have account list screen

    ReferenceError: element is not defined

       6 |   });
       7 | 
    >  8 |   it('should have account list screen', async () => {
         |                                         ^
       9 |     await expect(element(by.id(testIDs.TacScreen.tacView))).toBeVisible();
      10 |     await element(by.id(testIDs.TacScreen.agreePrivacyButton)).tap();
      11 |     await element(by.id(testIDs.TacScreen.agreeTacButton)).tap();

      at _callee2$ (firstTest.spec.js:8:41)
      at tryCatch (../node_modules/regenerator-runtime/runtime.js:45:40)
      at Generator.invoke [as _invoke] (../node_modules/regenerator-runtime/runtime.js:271:22)
      at Generator.prototype.(anonymous function) [as next] (../node_modules/regenerator-runtime/runtime.js:97:21)
      at tryCatch (../node_modules/regenerator-runtime/runtime.js:45:40)
      at invoke (../node_modules/regenerator-runtime/runtime.js:135:20)
      at ../node_modules/regenerator-runtime/runtime.js:170:11
      at callInvokeWithMethodAndArg (../node_modules/regenerator-runtime/runtime.js:169:16)
      at AsyncIterator.enqueue (../node_modules/regenerator-runtime/runtime.js:192:13)
      at AsyncIterator.prototype.(anonymous function) [as next] (../node_modules/regenerator-runtime/runtime.js:97:21)
      at Object.<anonymous>.exports.async (../node_modules/regenerator-runtime/runtime.js:216:14)
      at Object._callee2 (firstTest.spec.js:8:41)

detox[8550] ERROR: [cli.js] Error: Command failed: node_modules/.bin/jest --config=e2e/config.json --maxWorkers=1 '--testNamePattern=^((?!:ios:).)*$' "e2e"

I had the issue both with yarn start running in another shell and without

README.md

.travis.yml

hanwencheng · 2019-10-21T10:54:42Z

I have tested it on Emulator, it is also possible to use real device, but need some configuration, basically we have to override device settings with device name, I will make an update on it soon.

The device should be listed in the output of adb devices command under provided name. Use this type to connect to Genymotion emulator.

hanwencheng · 2019-10-21T16:57:58Z

Tried on real device, but do not has the luck to run it success on device. It seems to be a problem of Detox with support of RN 0.60.x, will track the problem on this issue wix/Detox#1689

Tbaut

I can't review this PR further since I do not have any emulator installed.
@yjkimjunior or @sjeohp can you have a look?

Since this doesn't support physical devices, we should mention it and remove anything that tells what to do with a real device since it doesn't work.

pmespresso · 2019-10-23T13:23:54Z

@hanwencheng not sure if this is an Xcode version mismatch issue, but my binary path is at: ./ios/build/Build/Products/Debug-iphonesimulator/NativeSigner.app, as opposed to ios/build/NativeSigner/Build/Products/Debug-iphonesimulator/NativeSigner.app which led to the error I messaged you about earlier

The test itself works well otherwise

pmespresso

as discussed offline, looks like the xcode build and detox build artifacts are mismatched, so will need an extra script to check that as a workaround, potentially to be fixed in the detox repo directly.

everything else went smoothly for me. works well on android

pmespresso

latest commit fixes my grumble

Tbaut

My review is barely useful but my grumbles are addressed :)

sjeohp-zz · 2019-10-23T19:55:45Z

I can't review this PR further since I do not have any emulator installed.
@yjkimjunior or @sjeohp can you have a look?

Detox build gives me a linker error for ios: library not found for -lRNSecureStorage. Just that lib for some reason, the others are fine.

pmespresso · 2019-10-24T09:07:04Z

@sjeohp it looks like you need to link react-native-secure-storage again, maybe the link was lost while checking out the branch. react-native link react-native-secure-storage should do it.

Co-Authored-By: Thibaut Sardan <33178835+Tbaut@users.noreply.github.com>

Tbaut · 2019-10-28T13:46:42Z

android/app/build.gradle

@@ -146,6 +149,8 @@ android {
        release {
            minifyEnabled enableProguardInReleaseBuilds
            proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
+            // Detox-specific additions to pro-guard
+            // proguardFile "${rootProject.projectDir}/../node_modules/detox/android/detox/proguard-rules-app.pro"


Should we remove this stray comment?

These code is needed once ProGuard is Enabled, I will add one line comment above.

* init detox e2e test * Add first e2e test * add android configurations * fix detox rn-camera problem * add CI integration * update ci settings * update xcode version * use legacy build in CI * prettier xcode output * update with android * revert back to origin CI test * add E2E test in readme * Update .travis.yml * update detox guide * renaming * update readme * Update README.md Co-Authored-By: Thibaut Sardan <33178835+Tbaut@users.noreply.github.com> * Update .travis.yml * Update README.md * fix binary path problem * remove before each reloading * rephrase

hanwencheng self-assigned this Oct 17, 2019

hanwencheng commented Oct 17, 2019

View reviewed changes

Tbaut added the A3-inprogress label Oct 18, 2019

hanwencheng changed the title ~~WIP: test: Add integration(e2e) test~~ test: Add integration(e2e) test Oct 19, 2019

hanwencheng added A0-pleasereview and removed A3-inprogress labels Oct 19, 2019

hanwencheng mentioned this pull request Oct 20, 2019

WIP: Draft: fix CI integration on iOS #425

Closed

Tbaut reviewed Oct 21, 2019

View reviewed changes

README.md Outdated Show resolved Hide resolved

.travis.yml Outdated Show resolved Hide resolved

Tbaut added A5-grumble and removed A0-pleasereview labels Oct 21, 2019

hanwencheng added A0-pleasereview and removed A5-grumble labels Oct 21, 2019

hanwencheng requested a review from sjeohp-zz October 22, 2019 19:29

Tbaut requested review from Tbaut and pmespresso October 23, 2019 11:51

Tbaut suggested changes Oct 23, 2019

View reviewed changes

pmespresso suggested changes Oct 23, 2019

View reviewed changes

pmespresso approved these changes Oct 23, 2019

View reviewed changes

Tbaut self-requested a review October 23, 2019 15:02

Tbaut approved these changes Oct 23, 2019

View reviewed changes

hanwencheng added 4 commits October 28, 2019 13:07

init detox e2e test

7746328

Add first e2e test

1e013b7

add android configurations

e358a5f

fix detox rn-camera problem

48c4d68

hanwencheng and others added 19 commits October 28, 2019 13:07

add CI integration

fa59619

update ci settings

2c9e3ba

update xcode version

ba7b480

use legacy build in CI

c7ad39a

prettier xcode output

474d658

update with android

822b895

revert back to origin CI test

240f9c9

add E2E test in readme

1441a0b

Update .travis.yml

f3aa6a4

update detox guide

8790e86

renaming

b50e444

update readme

4c96920

Update README.md

829dff3

Co-Authored-By: Thibaut Sardan <33178835+Tbaut@users.noreply.github.com>

Update .travis.yml

14f34a1

Update README.md

e274ba9

Update README.md

d4f2166

Update README.md

2f76e6f

fix binary path problem

01a80f3

remove before each reloading

7f62dd6

hanwencheng force-pushed the hanwen-integration-test branch from cb08930 to 7f62dd6 Compare October 28, 2019 12:33

rephrase

a9cc4ef

hanwencheng merged commit 0ba9966 into master Oct 28, 2019

hanwencheng deleted the hanwen-integration-test branch October 28, 2019 12:47

Tbaut reviewed Oct 28, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: Add integration(e2e) test #424

test: Add integration(e2e) test #424

hanwencheng commented Oct 17, 2019 •

edited

Loading

hanwencheng Oct 17, 2019

hanwencheng Oct 17, 2019

Tbaut left a comment

hanwencheng commented Oct 21, 2019

hanwencheng commented Oct 21, 2019

Tbaut left a comment

pmespresso commented Oct 23, 2019 •

edited

Loading

pmespresso left a comment •

edited

Loading

pmespresso left a comment

Tbaut left a comment

sjeohp-zz commented Oct 23, 2019

pmespresso commented Oct 24, 2019

Tbaut Oct 28, 2019

hanwencheng Oct 29, 2019

		afterEvaluate {project ->
		if (project.hasProperty("android")) {

test: Add integration(e2e) test #424

test: Add integration(e2e) test #424

Conversation

hanwencheng commented Oct 17, 2019 • edited Loading

Test Demo

Further steps

hanwencheng Oct 17, 2019

Choose a reason for hiding this comment

hanwencheng Oct 17, 2019

Choose a reason for hiding this comment

Tbaut left a comment

Choose a reason for hiding this comment

hanwencheng commented Oct 21, 2019

hanwencheng commented Oct 21, 2019

Tbaut left a comment

Choose a reason for hiding this comment

pmespresso commented Oct 23, 2019 • edited Loading

pmespresso left a comment • edited Loading

Choose a reason for hiding this comment

pmespresso left a comment

Choose a reason for hiding this comment

Tbaut left a comment

Choose a reason for hiding this comment

sjeohp-zz commented Oct 23, 2019

pmespresso commented Oct 24, 2019

Tbaut Oct 28, 2019

Choose a reason for hiding this comment

hanwencheng Oct 29, 2019

Choose a reason for hiding this comment

hanwencheng commented Oct 17, 2019 •

edited

Loading

pmespresso commented Oct 23, 2019 •

edited

Loading

pmespresso left a comment •

edited

Loading