Instrument metrics for web for the IA project

[kittykat]

We want to know if the IA changes are successful. For this we need to learn more about how users currently use element and whether the IA changes are improving or not making worse their workflows.

Details of what needs to be instrumented are in this internal spreadsheet.

Search

• Funnel for open search > visit room > open search again (within time constraint)
• Time and frequency to go from open search to click result
• Frequency of open search and close without clicking result
• Frequency of no search results returned (this needs more investigation before we know how we want to track this)

Room header menu

We should be able to use screen/previous screen features in PostHog to know where users are accessing things from, if there are multiple ways to access them.

• How are favourites being added (which path leads to a room being favourited)
• How Notifications settings for a room are being accessed
• Path to room member list is being accessed
• Path to invite to room (which menu is the link accessed from? are users using the "invite" button in member list?)
• Path to users are leaving rooms
• Path to room settings are being accessed

Rooms

• Path for users to join a room
• Point of entry to create room dialog
• How many favourited rooms does a user have (split between new and experienced users)

Metaspaces and quick settings

We can use user properties rather for users preferences, and then should be able to visualise from there.

• Number of (experienced) users with metaspaces changed from default
• Which metaspaces are being used
• How many users have Home set to "show all rooms"
• Path to enabling/disabling metaspace settings
• Path to theme change (available in User Menu, Quick Settings, User Settings)

Space header

Should add a user property for how many spaces the user is in

• Add a user property for how many spaces the user is in - to split users into cohorts
• Source of invite to space
• Number of times explore space is accessed and how
• Number of times a space is clicked in the left panel and "explore for that space is accessed within 3 seconds

Breadcrumbs

• Source of room open

Descoped

• Length of session for room accessed via breadcrumb (we want to compare old breadcrumbs against new ones)

[germain-gg]

@kittykat what kind of support do you believe is required here? From what I gather this should be ready to work on?
Unless there were some technical limitations to tracking the remaining unchecked list items

[kittykat]

@gsouquet this was probably requiring support from ops as we needed a posthog upgrade to be able to do time-based tracking. This has now happened.

I've descoped the breadcrumb tracking item, but space invites haven't been marked as implemented.

[kittykat]

I believe this issue is now done