Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(tests): New match-strict? cljs.test directive #20825

Merged
merged 2 commits into from
Jul 25, 2024

Conversation

ilmotta
Copy link
Contributor

@ilmotta ilmotta commented Jul 19, 2024

Summary

This PR's issue explains in detail the problem being solved. In essence, equality checks in tests using = give a bad experience by default on test failures containing nested data structures. We use the cljs.test directive match? from matcher-combinators library to help compare nested structures. The problem with match? is that its default matcher for maps (embeds) can be too permissive, and this causes surprises.

This PR upgrades matcher-combinators to latest, where a new matcher called nested-equals is available. This matcher won't allow extra keys in maps. This matcher eliminates the need for manually adding nested equals matchers as we have to do currently.

  • Upgrades matcher-combinators from 3.8.8 to 3.9.1 (latest as of 2024-07-19)

What changes?

When asserting in tests, we now have the option to use match-strict? or match?. Both directives are available by integrating with cljs.test. The code implementing the new match-strict? directive was 100% copied from the library matcher-combinators because we need to wrap the expected value ourselves with matcher-combinators.matchers/nested-equals. It's ugly code, but it's how we can integrate with cljs.test/assert-expr. Hopefully we write once and profit forever.

Guidelines?

I think it's too early to worry about a guideline. I would personally be inclined to use match-strict? the majority of the time, but match? has its place too.

Areas that may be impacted

All safe 🦺

status: ready

@status-im-auto
Copy link
Member

status-im-auto commented Jul 19, 2024

Jenkins Builds

Click to see older builds (4)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ ebb9293 #1 2024-07-19 15:00:05 ~6 min tests 📄log
✔️ ebb9293 #1 2024-07-19 15:01:42 ~8 min android 🤖apk 📲
✔️ ebb9293 #1 2024-07-19 15:02:33 ~8 min android-e2e 🤖apk 📲
✔️ ebb9293 #1 2024-07-19 15:04:21 ~10 min ios 📱ipa 📲
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ aaa815a #2 2024-07-24 02:14:52 ~4 min tests 📄log
✔️ aaa815a #2 2024-07-24 02:16:30 ~6 min android 🤖apk 📲
✔️ aaa815a #2 2024-07-24 02:17:07 ~7 min android-e2e 🤖apk 📲
✔️ aaa815a #2 2024-07-24 02:19:44 ~9 min ios 📱ipa 📲
✔️ e02331c #3 2024-07-25 01:06:30 ~5 min tests 📄log
✔️ e02331c #3 2024-07-25 01:07:57 ~6 min android-e2e 🤖apk 📲
✔️ e02331c #3 2024-07-25 01:08:59 ~7 min android 🤖apk 📲
✔️ e02331c #3 2024-07-25 01:11:38 ~10 min ios 📱ipa 📲

@ilmotta ilmotta force-pushed the ilmotta-20590/new-matcher-for-nested-equality branch from ebb9293 to aaa815a Compare July 24, 2024 02:09
@status-im-auto
Copy link
Member

86% of end-end tests have passed

Total executed tests: 7
Failed tests: 0
Expected to fail tests: 1
Passed tests: 6
IDs of expected to fail tests: 727232 

Expected to fail tests (1)

Click to expand

Class TestWalletOneDevice:

1. test_wallet_add_remove_watch_only_account, id: 727232

Device 1: Find `EditBox` by `accessibility id`: `add-address-to-watch`
Device 1: Type `0x8d2413447ff297d30bdc475f6d5cb00254685aae` to `EditBox`

critical/test_wallet.py:249: in test_wallet_add_remove_watch_only_account
    self.wallet_view.add_watch_only_account(address=address_to_watch, account_name=new_account_name)
../views/wallet_view.py:163: in add_watch_only_account
    self.account_has_activity_label.wait_for_visibility_of_element()
../views/base_element.py:147: in wait_for_visibility_of_element
    raise TimeoutException(
 Device 1: Text by accessibility id:`account-has-activity` is not found on the screen after wait_for_visibility_of_element 

[[Missing networks in account address, https://github.com//issues/20166]]

Device sessions

Passed tests (6)

Click to expand

Class TestWalletMultipleDevice:

1. test_wallet_send_asset_from_drawer, id: 727230
2. test_wallet_send_eth, id: 727229

Class TestCommunityOneDeviceMerged:

1. test_community_copy_and_paste_message_in_chat_input, id: 702742
Device sessions

2. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133
Device sessions

Class TestOneToOneChatMultipleSharedDevicesNewUi:

1. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
Device sessions

Class TestWalletOneDevice:

1. test_wallet_add_remove_regular_account, id: 727231
Device sessions

@ilmotta
Copy link
Contributor Author

ilmotta commented Jul 24, 2024

@jakubgs, can I have your blessing in this PR? It upgrades one of the Clojure libraries. Thanks ;)

Copy link
Contributor

@mohsen-ghafouri mohsen-ghafouri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏

@ilmotta ilmotta force-pushed the ilmotta-20590/new-matcher-for-nested-equality branch from aaa815a to e02331c Compare July 25, 2024 01:01
@ilmotta ilmotta merged commit 60ad7c8 into develop Jul 25, 2024
5 checks passed
@ilmotta ilmotta deleted the ilmotta-20590/new-matcher-for-nested-equality branch July 25, 2024 02:06
ilmotta added a commit that referenced this pull request Jul 28, 2024
Equality checks in tests using = give a bad experience by default on test
failures containing nested data structures. We use the cljs.test directive
match? from matcher-combinators library to help compare nested structures. The
problem with match? is that its default matcher for maps (embeds) can be too
permissive, and this causes surprises.

Here we upgrade matcher-combinators to latest, where a new matcher called
nested-equals is available. This matcher won't allow extra keys in maps. This
matcher eliminates the need for manually adding nested equals matchers as we
have to do currently.

- Upgrades matcher-combinators from 3.8.8 to 3.9.1 (latest as of 2024-07-19)

What changes?

When asserting in tests, we now have the option to use match-strict? or match?.
Both directives are available by integrating with cljs.test. The code
implementing the new match-strict? directive was 100% copied from the library
matcher-combinators because we need to wrap the expected value ourselves with
matcher-combinators.matchers/nested-equals. It's ugly code, but it's how we can
integrate with cljs.test/assert-expr.
ilmotta added a commit that referenced this pull request Jul 30, 2024
Revisions from develop:

- 59ceddb develop origin/develop fix(wallet): fix bridge transactions (#20902)
- 99ccbc3 Cover wallet send events with tests Part 2 #20411 #20533 (#20721)
- 8c2d539 Enabling WalletConnect feature flag (#20906)
- 67c83b1 fix(wallet): remove edit routes button in bridging (#20874)
- 11a84ba feat(wallet): disable complex routing (#20901)
- 1f5bb57 chore(wallet): disable bridging on unsupported tokens (#20846)
- 4586f80 Add toggle in advanced settings for mobile data
- 55c620e fix: create password for small screen (#20645)
- 525609f Wallet Activity: transactions are not sorted by time #20808 (#20862)
- 9065395 chore(settings): Disable telemetry option (#20881)
- d27ab75 fix_:display group message using the new ui (#20787)
- c6a1db6 ci: enable split apks & build only for arm64-v8a (#20683)
- 73777e0 Ensure keycard account can send transaction after upgrading from v1 to v2 #20552 (#20845)
- a6d3fc3 [#20524] fix: the missed keypairs are shown in the key pair list screen (#20888)
- a671c70 fix broken screen and navigation when syncing fails (#20887)
- a45991b 🥅 Filter connected dapps based on testnet mode, reject proposals and requests gracefully (#20799)
- 2e9fa22 feat: wallet router v2 (#20631)
- 737d8c4 rename sub to fix error when requesting to join community (#20868)
- 3aa7e10 Sync process is blocked on Enabled notifications screen (#20883)
- c1d2d44 perf: Fix app freeze after login (#20729)
- 0fed811 e2e: updated testnet switching and added one test into smoke
- 53c35cb fix(wallet): Linear gradient exception on invalid colors for watched account cards (#20854)
- be82365 chore(settings)_: Remove testnet toggle from legacy advanced settings (#20875)
- eae8a65 feat(wallet)_: Add beta info box in activity tab (#20873)
- fe54a25 fix: not clearing network & web3-wallet on logout (#20886)
- 15a4219 Reject wallet-connect request by dragging the modal down (#20763) (#20836)
- 2ffbdac WalletConnect show expired toast (#20857)
- 402eb83 fix Issue with scrolling WalletConnect transaction on Android (#20867)
- ff88049 Fix WalletConnect header alignment on Android (#20860)
- cee2124 WalletConnect no internet edge-cases (#20826)
- 60ad7c8 chore(tests): New match-strict? cljs.test directive (#20825)
- 4989c92 fix_: Adding own address as saved addresses (#20839)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: DONE
Development

Successfully merging this pull request may close these issues.

Default map matcher used by the match? directive can be confusing
7 participants