Instrument metrics for Android for the IA project

[kittykat]

We want to know if the IA changes are successful. For this we need to learn more about how users currently use element and whether the IA changes are improving or not making worse their workflows.

Details of what needs to be instrumented are in this internal spreadsheet.

Search

Global search

• Bounce rate from funnel: open search > visit room > open search (within time constraint)
  could be solved with userPath:
  - for group chat: RoomDirectory -> RoomPreview -> RoomDirectory
  - for DMs StartChat -> Room -> People/Home -> Start Chat

• Time and frequency to go from open search to tap result
  userPath: RoomDirectory/StartChat -> Room/RoomPreview

• Frequency of open search and go back without tapping result
  userPath: RoomDirectory/StartChat -> not Room/RoomPreview

Needs steps in Funnel (screen->event->screen)

• Funnel for no progress from search results - guesstimate based on users that fell off search and how long did they stick around

User paths from Home, People, Rooms, etc:

• From which tabs do users open global search
  separate events for searching people and rooms globally: StartChat and RoomDirectory

Room header menu

User paths from RoomDetails:

• When users tap on the room header, where do they go next?
  userPath: Room -> RoomDetails-> any of 'Room*' events
• Add Home(create shortcut)
  Interaction(name: MobileRoomAddHome)
• Leave
  Interaction(name: MobileRoomLeave)

Left panel

User paths from SideBar:

• Which menu items are users using in the left panel and how often?
  userPath: SideBar -> SpaceMenu, Settings,and other related screens
• Do users switch spaces?
  Interaction event SideMenuSwitchSpace
• How often do users tap on the selected space?
  Interaction event SideMenuSelectedSpace

Settings

• How often are people accessing their settings and what are they doing? (track screens for the moment)
  As discussed: userPath from Settings

Spaces

• Add a user property for how many spaces the user is in - to split users into cohorts
  user property numSpace still to be discussed which event we will use
• Path to joining to space (were they invited, room directory, link, etc)
  event JoinedRoom. Taken out to Track user path to joining to room or space #5356
• Number of times explore space is accessed and how (there is a screen for this - note that navigation from space to subscace is a second event)
  screen SpaceExploreRooms
• Funnel from left panel to explore space
  userPath SideBar -> SpaceExploreRooms

Rooms

• Source of room open (did they click a link, did they search for the room, did they open it from one of the tabs? note that spaces look like rooms?)
  event ViewRoom. Taken out to Track source of room open #5355
• Path to joining to room (were they invited, room directory, link, etc)
  event JoinedRoom. Taken out to Track user path to joining to room or space #5356
• Point of entry to create room dialog
  screen events StartChat and CreateRoom

[fedrunov]

Global Search

Problem:

• No screen name for RoomPreview

Spaces

Problem:

• SideBarEvent is not tracked before Explore Rooms

  Question:
  -number of spaces - how accurate should be? A: ones per session is enough, web tracks on sync, if number differs from previously sent

Rooms

Problem:

• CreateRoom is not tracked from Global Search and Space Manage screens

  Question:
  -When do we track ViewRoom? A: View room is tracked when user opens room. It's a separate event from Room since more data required and we don't want to mix this data to Screen events
  -Joined Room has many triggers

[fedrunov]

JoinedRoom map of use case to trigger

Rooms

Screen	Use case	Trigger
timeline	accept invite (DM?) + notification ? + upgrade retry	-
message composer	slash command	-
room list	accept invite	-
invite accepter	auto accept invites	-
notification	notification's accept button	Notification
room preview screen	from global search	-
room list item	by button in room item on global search	RoomDirectory
space explore item	by button in room item on space explore	-

Spaces

Screen	Use case	Trigger
message composer	slash command	-
space directory	subspace	-
space invite item	invite item in sidebar	-
space preview screen	seems to be unused for now	-

Preview Dialog issue:

for both Space and Room we have dialog with some information about it and "join button". This dialog is called from different places:

Screen	Use case	Trigger
HomeActivity	from parsed link	-
SpaceExploreActivity	click on room in Space Explore	-
UserCodeActivity	on user code scanned	-
HomeActivity	from notification	-
Timeline	click on not joined room’s link	-
RoomList	on suggested room click	-

[fedrunov]

ViewRoom map of use case to trigger

Screen	Use case	Trigger
CreateDirectRoomActivity	after user creates a DM from users list	MessageUser
IncomingVerificationRequestHandler	verification request	Verification request
SearchFragment	navigate to event/message from search	MessageSearch
TimeLineFragmnet	handling a link in timeline	Timeline
PublicRoomsFragment	click on item in rooms global search	RoomDirectory
CreateRoomFragment	after room created	Created
RoomPreviewNoPreviewFragment	room preview	-
RoomJoinRuleActivity	switch to replacement room when user change join rules	-
IncomingShareFragment	share from outside the app	-
SpaceExploreActivity	click on “open” button on room list item in a space explore	-
SpacePeopleActivity	click on a space member in space members list (from fab on space’s dm tab)	-
RoomListFragment	click on a room on Home screen	RoomList

Also there is a bottom sheet with room preview from where user also can be navigated to Room, but this sheet is called in multiple places:

Screen	Use case	Trigger
HomeActivity	from parsed link	-
SpaceExploreActivity	click on room in Space Explore	-
UserCodeActivity	on user code scanned	-
HomeActivity	from notification	-
Timeline	click on not joined room’s link	-
RoomList	on suggested room click	-

[fedrunov]

map of room details items to analytics events

[fedrunov]

map of settings items to analytics events

[fedrunov]

• Room details screen tracked as RoomSettings instead of RoomDetails

[fedrunov]

Related PR - #5257

changes tracking screen from tracking when user leaves the screen to tracking when user enters the screen

[niquewoodhouse]

@nadonomy last week you suggested not having the subspace functionality on mobile. I totally understand the reasons but I just want to check please what you're thinking of stopping:

• people making subspaces on mobile
• people viewing a space by subspace on mobile

I'd like to us to add something to this issue to include a way to check if people on android are using the relevant feature(s) depending on what we intend to do.

[novocaine]

@niquewoodhouse added element-hq/element-meta#208

[nadonomy]

@nadonomy last week you suggested not having the subspace functionality on mobile. I totally understand the reasons but I just want to check please what you're thinking of stopping:

My core motivation is to avoid us spending time on things which are either not very impactful or just not needed in the first place.

• people making subspaces on mobile

We don't have any evidence users need to create or manage subspaces on mobile, so I don't think we should build this until we do, as it would require some effort to get right. Until we get more confidence I think we should consider pointing people to Web/Desktop to manage this.

• people viewing a space by subspace on mobile

I don't think we should assume we need to display nested hierarchies when browsing Spaces on mobile.

Need to learn more (analytics!) but I would advocate for us to (1) ensure the rooms are always discoverable (2) display the subspace as secondary info until we gain more confidence in investing our time further.

@niquewoodhouse Does that help? I'm replying to your comment rather than internalising the scope of this entire issue, but can discuss further if helpful.

[niquewoodhouse]

@niquewoodhouse Does that help? I'm replying to your comment rather than internalising the scope of this entire issue, but can discuss further if helpful.

Yes, thanks, that helps