zebra v1.0.0-alpha.0 Usability study

[mpguerra]

Usability Testing Plan

Purpose of test and research questions

We would like to understand how easy it is for a range of users (zcashd node operators, experienced "technical" users, developers) to build, install and run a zebrad node.

We would also like to understand any pain points and (both good and bad) surprises or otherwise unexpected behaviour when running zebrad and syncing to tip.

Finally we are interested in whether users understand what the node is doing from looking at the logs produced and if there is anything that they would expect to see.

Participants

We don't have a minimum number of participants and 5 should be enough.

Recruitment criteria

The following are some of the characteristics that our participants may have:

• have run a zcashd node
• rust developer
• technical ZFND "friend"

Recruitment strategy

We will recruit participants through:

• zfnd discord
• twitter
• direct invite

Protocol

Method

Tests will be remote, attended by zebra engineers with one moderator and one note taker.

Test environment, equipment and logistics

Users will use their own choice of hardware to build, install and run a zebrad node.

Their hardware will need to meet the following requirements:

• 4+ CPU cores
• 16+ GB RAM
• 50GB+ available disk space for finalized state
• 100+ Mbps network connections

Task(s)

Description

Scenario

Imagine you've heard about this neat new Zcash client called Zebra and you want to find out more about what it is capable of and possibly take it for a spin. How would you go about doing this?

Success criteria

• Participant is able to find Zebra
• Participant is able to download Zebra
• Participant is able to build Zebra
• Participant is able to run zebrad
• Participant understands what zebrad is doing as it executes
• Participant understands when there is a problem during zebrad execution
• Participant is able to recover from problems during zebrad execution

Session Outline and timing

1.Introduction to the session (5')

• Welcome participant to the session
• Explain the scenario to the participant
• Explain that they are not the ones being tested, the software is,
• Get their consent to go ahead with the user study

2. Introductory Interview (10')

• Ask participant why they agreed to participate in the user study,
• Ask participant if they may be interested in running a zebrad node in future
• Ask participant about any existing experience that they may have with Zcash.

3. Tasks (25')

1. Find zebra
2. Build zebra
3. Run zebra
4. Examine zebra execution

4. Post-test debriefing (5')

Thank participants for their time and gather any final impressions and general feedback.

Data to be collected and findings

Hardware specs of participants
Notes on impressions from participants
Task success rates

Required documentation

N/A

[mpguerra]

The description is still in progress but I figured I'd post it to get feedback early

[teor2345]

Currently our build instructions include downloading Zebra.

Do we need to change "Obtain Zebra" to "Find Zebra"?

[mpguerra]

We've undertaken 2 separate usability studies based on this plan.

We will discuss findings and create issues based on those items we wish to develop further.

[mpguerra]

We have identified a number of improvements to implement as a result of this initial user study.