chore(analytics): Simplify analytics testing

[ilmotta]

Fixes #21680

Summary

status-go PR: status-im/status-go#6385

This PR provides an alternative implementation to the one suggested in #21680 which is much simpler. The problem we are trying to resolve is that when QAs or even developers need to analyze events sent to MixPanel they cannot pinpoint exactly which events originated from their testing Status installation. Even if events end up in the test project (which is not as noisy as prod MixPanel), it's still cumbersome to test if there are other people concurrently generating events (which can be the case while we are testing releases).

The solution is to allow us to filter events based on the so called MixPanel User ID, which in the case of Status, is a UUID generated once and which varies per installation. This PR adds a new option in the Advanced Settings behind a toggle, therefore the feature is only available in PR builds because feature flags are only accessible if ENABLE_QUO_PREVIEW is 1.

Additional notes

The original solution suggested in the issue is more flexible but more involved, where clients would be able to specify their own UserID, which in the future would be useful to support tracking certain events more anonymously by not tying them to the global installation ID (User ID).

Areas that may be impacted

I don't anticipate any impact in the app.

Steps to test

1. Create at least one profile or log in to an existing one.
2. Enable usage tracking in Settings > Privacy and security.
3. Go to Settings > Feature flags and enable analytics > copy-user-id.
4. Log out so that the app initialization kicks in.
5. Log in and go to Settings > Advanced settings and press on the option Copy analytics User ID to copy the User ID to the clipboard.
6. Verify events are being sent to MixPanel (see screenshot below as an example on how to filter).

status: ready

[status-im-auto]

Jenkins Builds

❔	Commit	#️⃣	Finished (UTC)	Duration	Platform	Result
✔️	a811d16	#1	2025-03-04 20:01:05	~6 min	tests	📄log
✔️	a811d16	#1	2025-03-04 20:02:57	~8 min	android	🤖apk 📲
✔️	a811d16	#1	2025-03-04 20:03:55	~9 min	android-e2e	🤖apk 📲
✔️	a811d16	#1	2025-03-04 20:07:36	~13 min	ios	📱ipa 📲
✔️	3877269	#2	2025-03-07 08:39:35	~5 min	tests	📄log
✔️	3877269	#2	2025-03-07 08:41:38	~7 min	android-e2e	🤖apk 📲
✔️	3877269	#2	2025-03-07 08:42:41	~8 min	android	🤖apk 📲
✔️	3877269	#2	2025-03-07 08:45:37	~11 min	ios	📱ipa 📲

[status-im-auto]

86% of end-end tests have passed

Total executed tests: 14
Failed tests: 2
Expected to fail tests: 0
Passed tests: 12

IDs of failed tests: 741840,741841 

Failed tests (2)

Click to expand

Rerun failed tests

Class TestWalletCollectibles:

1. test_wallet_send_collectible, id: 741840

Device 1: Tap on found: Button Device 1: Find `CollectibleItemElement` by `xpath`: `//*[@content-desc='collectible-list-item']//*[contains(@text,'BVL')]/../..` critical/wallet/test_collectibles.py:104: in test_wallet_send_collectible self.wallet_view.get_collectible_element('BVL').click() ../views/base_element.py:89: in click element = self.find_element() ../views/base_element.py:78: in find_element raise NoSuchElementException( Device 1: CollectibleItemElement by xpath: `//*[@content-desc='collectible-list-item']//*[contains(@text,'BVL')]/../..` is not found on the screen; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception

2. test_wallet_collectible_send_from_expanded_info_view, id: 741841

Device 1: Find Button by accessibility id: collectibles-tab Device 1: Tap on found: Button critical/wallet/test_collectibles.py:150: in test_wallet_collectible_send_from_expanded_info_view self.wallet_view.get_collectible_element('Glitch Punks').wait_for_element().click() ../views/base_element.py:120: in wait_for_element raise TimeoutException( Device `1`: `CollectibleItemElement` by` xpath`: `//*[@content-desc='collectible-list-item']//*[contains(@text,'Glitch Punks')]/../..` is not found on the screen after wait_for_element

Passed tests (12)

Click to expand

Class TestWalletCollectibles:

1. test_wallet_collectibles_balance, id: 741839

Class TestWalletOneDevice:

1. test_wallet_swap_flow_mainnet, id: 741555

2. test_wallet_add_remove_regular_account, id: 727231

3. test_wallet_balance_mainnet, id: 740490

4. test_wallet_bridge_flow_mainnet, id: 741612

5. test_wallet_send_flow_mainnet, id: 741554

Class TestWalletMultipleDevice:

1. test_wallet_send_asset_from_drawer, id: 727230

2. test_wallet_send_eth, id: 727229

Class TestOneToOneChatMultipleSharedDevicesNewUi:

1. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745

Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs

Class TestCommunityOneDeviceMerged:

1. test_community_copy_and_paste_message_in_chat_input, id: 702742

Device sessions Device 1: Steps, video, logs

2. test_restore_multiaccount_with_waku_backup_remove_profile_switch, id: 703133

Device sessions Device 1: Steps, video, logs

Class TestCommunityMultipleDeviceMerged:

1. test_community_message_edit, id: 702843

Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs

[mariia-skrypnyk]

Hi @ilmotta !

I've checked User ID events on iOS/Android and unfortunately I do not see them.
I did search by User ID and also added User ID in my column.

I also have a filling that events are not tracked in PR builds at all.
I did a rebase and tested new builds but also checked at a previous PR build from 4/03.

One more thing: I've created 2 Profiles and:

1. they both have the same User ID
   OR
2. it doesn't work for the second Profile as I copying the same User ID for Profile 2.

Can you please take a look?

UserID_events.zip

[ilmotta]

I've checked User ID events on iOS/Android and unfortunately I do not see them. I did search by User ID and also added User ID in my column.

I also have a filling that events are not tracked in PR builds at all. I did a rebase and tested new builds but also checked at a previous PR build from 4/03.

Hey @mariia-skrypnyk, sorry for the late reply. I see from the logs that the metrics are being correctly sent to MixPanel's test project and the token is correct. I also see numerous events (600+) in the test project since last week and there are events as recent as March 10. This means that PR builds are generating test events.

I used the User ID found in the zip log you shared, and I got 100 events in the last few days.

---

Events may take some time to arrive in MixPanel, although usually for me they appear in less than a minute and most certainly in less than 5 min. It looks like this is what happened in your testing.

One more thing: I've created 2 Profiles and: they both have the same User ID OR it doesn't work for the second Profile as I copying the same User ID for Profile 2.

Every app installation generates a unique random ID that is shared by all profiles, therefore, metrics are collected per installation, not per profile.

[mariia-skrypnyk]

Thanks @ilmotta !
Will try now and let you know.

[mariia-skrypnyk]

Hey @ilmotta !

So the issue was in my access to the test project.
I found events by User ID.
Looks really great now.

PR finally 👏 can be merged!