Stats Refresh Master Issue

[ScoutHarris]

---

NOTE: All open issues have been added to the v2 Master Issue, and can be tracked there.

---

Milestones

• 12.6: Release v1

Time Away

• May 24, 27 : Stephenie AFK
• May 27 : Steve, Daniele AFK
• May 27 - 31 : Stephenie - groundskeeping
• May 31 : Steve's last sprint on the project

---

Required for v1 release (12.6):

Pre-release preparation

• Design review (TEAM) - marked as complete as @SylvesterWilmott has used the 12.5 release from HockeyApp to do the review.

Shared

• Add tracks & validate ([Stats Refresh] Implement & validate analytics events #11730) (@stevebaranski)

Issues

• Date bar now only shows the current day (PR [Stats Refresh] Fix period displayed on date bar #11756) (@ScoutHarris )
• "simple" stat cards are empty when no data (PR [Stats Refresh] empty "simple" stat cards now show No data yet label #11748) (@ScoutHarris )
• Posting Activity view is blank when no data (PR [Stats Refresh] always display Posting Activity months #11766) (@ScoutHarris )
• Posting Activity data is no longer correct (PR [Stats Refresh] correct Posting Activity data #11765) (@ScoutHarris )
• Period queries being cancelled ([Stats Refresh] Period queries being cancelled #11742) (@danielebogo )
• Browsing back from View More shows the wrong set of data ([Stats Refresh] browsing back from View More shows the wrong set of data #11815) (@danielebogo )
• Data discrepancies for Period Published (Stats Refresh: data discrepancies for "Published Posts" #11371) (@jklausa)
• Investigate Period summary fetch time ([Stats Refresh] investigate Period summary fetch time #11657) (@jklausa)
• Details view take a long time to load ([Stats Refresh] Investigate Stats parsing performance #11360) (@ScoutHarris)
  • Period Referrers (PR [Stats Refresh] Period Referrers: fix displaying details view #11741)

Release preparation

• Activate feature flag for 12.6 release (PR [Stats Refresh] Enable Stats Refresh for all. #11826) (@ScoutHarris)

---

Optional for v1 release / future releases:

Shared

• Hide navigation bar on scroll ([Stats Refresh] Hide navigation bar on scroll #10851)
• Add additional Tracks events ([Stats Refresh] Add additional Tracks events #11775)

DWMY

• Countries: Add map ([Stats Refresh] Add country map to Period Countries card #11083) (@danielebogo)

Enhancements:

• While loading icons a grey square should be used. (PR [Stats Refresh] placeholder while loading icon #11745) (@ScoutHarris )
• Referrers: ([Stats Refresh] Referrers card: add placeholder icon #11080)
  • When no icon is available then show gridicons-globe. (PR [Stats Refresh] Period Referrers: fix displaying details view #11741) (@ScoutHarris )
  • Replace the Search Engines magnifying glass icon with gridicons-search. (PR [Stats Refresh] StatsViewModels: Replace default search engine icon #11674) (@frosty )

Issues

• Stats page showing error when user doesn't have permissions to view stats on jetpack sites (Stats page showing error when user doesn't have permissions to view stats on jetpack sites #11823)
• Insights Comments details doesn't show all ([Stats Refresh] Insights Comments details doesn't show all  #11713) (@ScoutHarris / @jklausa)
• Retrieve country flags from the server (Unassigned)
• Retain cycle with StatsTotalRows ([Stats Refresh] Insights Comments > Posts & Pages: show in web view  #10743 (review))
• Emojis in post titles here get messed up ([Stats Refresh] Add Insights Tags and Categories card #10831 (review))
• New Stats section needs Dynamic Type review. (New Stats section needs Dynamic Type review. #10845)
• Verify that locale/timezone account for this Groundskeeping issue re: missing data
• As I user, if Posts and Pages is selected when navigating from Insights to Comments, I expect that it will be selected when I navigate back to the Insights screen. ([Stats Refresh] Insights Comments: add detail list view #11181 (comment))
• This issue is a remnant from legacy stats, but a reminder that we should be conduct a VoiceOver validating pass of Stats.
• Expanding/hiding rows makes the other rows "jump". ([Stats Refresh] Insights Tags and Categories: add drop-down rows for groups #10944 (comment))
• Toggling expandable rows updates all chevrons ([Stats Refresh] Stats details: toggling expandable rows updates all chevrons #11743) (@ScoutHarris )
• Date bar: In Hebrew (RTL), arrows in wrong direction. (Stats: Date bar and list arrows appear in a wrong direction on any RTL language #11767)
• Card titles not aligned correctly in Hebrew (RTL) (Stats: Card titles not aligned correctly in Hebrew (RTL) #11768)
• Post Stats: table extended in landscape ([Stats Refresh] Post Stats table extended in landscape #11602)
• Post Stats: extra spacing in post title label ([Stats Refresh] Post Stats: extra spacing in post title label #11637)
• Several cards missing lines (PR [Stats Refresh] Add missing separator lines #11754) (@ScoutHarris )
• Insights Most Popular Time icons are the incorrect color (PR [Stats Refresh] Insights Most Popular Time: change icon color to grey #11794) (@ScoutHarris )
• Insights All Time Stats: Best Views Ever subtitle should be 15pt (PR [Stats Refresh] Insights All Time Stats: correct Best Views Ever subtitle font size #11795) (@ScoutHarris )
• Insights Posting Activity detail: incorrect layout after rotation (PR [Stats Refresh] Insights Posting Activity detail: correct layout on rotation #11801) (@ScoutHarris )
• Extra space when 'No data for this period' shown (PR [Stats Refresh] remove extra space when 'No data' shown #11813) (@ScoutHarris )
• Extra separator line when row expanded ([Stats Refresh] extra separator line when row expanded #11799)
• Period Authors: expanded rows should be 44pt (PR [Stats Refresh] Period Authors: fix expanded row height #11825) (@ScoutHarris )
• Period Authors details: expanded rows should be 44pt ([Stats Refresh] Period Authors details: expanded rows should be 44pt #11824)

---

Post release:

• Remove feature flag & obsolesced code ([Stats Refresh] Remove feature flag & legacy code #10975)

[ScoutHarris]

UI estimates as of Jan 16:

• Shared:
  • Empty states (8)
  • List view (to display the View mores) (8)
  • Implement charting library (13)
  • Add tracks (5)
  • Hide nav bar on scroll (5)
  • Post Stats (32)
• Insights:
  • Insights: Add Annual Site Stats card (8)
  • Tags and Categories: update to have drop-down rows (8)
  • Annual Site Stats: update to use real data once backend is ready (8)
  • Loading view (8)
  • Latest Post Summary: Chart (8)
  • Tags and Categories: View more list view (8)
  • Comments: View more list view (8)
  • Followers: View more list (8)
• DWMY:
  • Create structure (8)
  • Videos (8)
  • Published (8)
  • Search Terms (8)
  • Countries (8)
  • Authors (8)
  • Clicks (8)
  • Referrers (8)
  • Posts and Pages (8)
  • Chart (8)

[jklausa]

My estimates for data work are as follows:

• Insights / Shared

  • Latest Post — 13 (this includes work on general structure of the data)
  • All Times — 5
  • Followers — 5
  • Most Popular — 8
  • Tags and Categories — 3
  • Annual — 13
  • Comments — Authors — 2
  • Comments — Pages — 3
  • Followers — WPCom — 2
  • Followers — Email — 2
  • Today's Stats — 5
  • Posting Activity — 5
  • Publicise — 3
  • Post Stats — months, and years — 5
  • Average Views Per day — 3
  • Recent Weeks — 3

• DWMY

  • Referers — 2
  • Posts and Pages — 2
  • Clicks — 2
  • Authors — 3
  • Countries — 5
  • Search Terms — 2
  • Published — 2

• Vaguely "other"

  • Persistence — 13
  • Insights management (reordering, enabling) — 8 (punted to V2)

Not included:

• Distinguishing between pages/posts for datapoints returning stats for those (basically the reverse of [Stats Refresh] Period Posts and Pages card: update icon based on type #10886)
• Keep around old logic of grouping tags/categories
• Decide on approach to timezones (https://a8c.slack.com/archives/CDHKS7X4M/p1544572700003300)
• WPStatsServiceRemote:operationForSearchTermsForDate (https://github.com/wordpress-mobile/WordPressKit-iOS/blob/260d5420b9e07c90ef2421f372de63b55175604a/WordPressKit/WPStatsServiceRemote.m#L1230) manually adds an item for "Unknown Search Terms" to the end of the array. Let's figure out a way to not deal with localising strings in the network layer (it now needs to be "Unknown search terms" (note capitalisiation)). It also now needs to be the first item.

[ScoutHarris]

UI estimates as of Jan 28:

• Shared:
  • Empty states (8)
  • Table footers (5)
  • List view (to display the View mores) (8)
  • Implement charting library (13)
  • Add tracks (5)
  • When scrolling hide both appbar/navigation bar and tabs, date chooser remains fixed to the top. (8)
  • Post Stats (32)
  • Table cell for stats with expanding rows (8)
• Insights:
  • Update card order to match Android. (1)
  • Loading view (8)
  • Tags and Categories: update to have drop-down rows (8)
  • Tags and Categories: View more list view (8)
  • Annual Site Stats: update to use real data once backend is ready (8)
  • Latest Post Summary: Chart (8)
  • Comments: Authors, Pages & Posts View more list views (8)
  • Followers: View more list (8)
• DWMY:
  • Loading view (8)
  • Countries (8)
  • Authors (8)
  • Clicks (8)
  • Referrers (8)
  • Chart (8)
  • Posts and Pages: update icon depending on type. (5)
  • Posts and Pages: View more list view (8)
  • Videos: View more list view (8)
  • Search Terms: View more list view (8)
  • Published: View more list view (8)

Enhancements stemming from feedback post:

• Insights:
  • Latest Post Summary: (8)
    • Move “views” to top and use full number
    • Move “likes” and “comments” to bottom as table rows
  • New design for “Most popular day and hour” (8)
  • New design for “annual site stats” (8)
  • Remove data bars from: (1)
    • Comments - Authors and Pages & Posts
    • Publicize
• DWMY:
  • Add date bar (8)
• Shared:
  • Start shortening numbers after 9,999 (1)

[ScoutHarris]

UI estimates as of Feb 12:

• Shared:
  • Empty states (8)
  • List view (to display the View mores) (8)
  • Implement charting library (13)
  • Add tracks (5)
  • Hide appbar/navigation bar and tabs on scroll, date chooser remains fixed to the top. (8)
  • Post Stats (32)
• Insights:
  • Loading view (8)
  • Tags and Categories: View more list view (8)
  • Latest Post Summary: Chart (8)
  • Comments: Authors, Pages & Posts View more list views (8)
  • Followers: View more list view (8)
  • Latest Post Summary: (8)
    • Move “views” to top and use full number
    • Move “likes” and “comments” to bottom as table rows
• DWMY:
  • Loading view (8)
  • Countries (8)
  • Referrers (8)
  • Chart (8)
  • Posts and Pages: update icon depending on type. (1)
  • Posts and Pages: View more list view (8)
  • Videos: View more list view (8)
  • Search Terms: View more list view (8)
  • Published: View more list view (8)
  • Authors: View more list view (8)
  • Clicks: View more list view (8)
  • Add date bar (8)

[jklausa]

Feb 25: status/estimates

• Insights / Shared

  • Latest Post — 13 (this includes work on general structure of the data) done
  • All Times — 5 done
  • Followers — 5 done
  • Most Popular — 8 done
  • Tags and Categories — 3 done
  • Annual — 13 done
  • Comments — Authors — 2 done
  • Comments — Pages — 3 done
  • Followers — WPCom — 2 done
  • Followers — Email — 2 done
  • Today's Stats — 5 done
  • Publicise — 3 done
  • Posting Activity — 5 done

• DWMY

  • Referers — 2 done
  • Posts and Pages — 2 done
  • Clicks — 2 done
  • Authors — 3 done
  • Countries — 5 done
  • Search Terms — 2 done
  • Published — 2 done
  • Post Stats — months, and years — 5 done
  • Average Views Per day — 3 done
  • Recent Weeks — 3

• Vaguely "other"

  • Persistence — 21
  • Insights management (reordering, enabling) — 8 (punted to V2)

[ghost]

I have created 9 issues that fulfill the Charting functionality as currently conceived. Each is listed below, along with a relative estimate associated:

1. Insights : Construct Chart for Latest Post Summary (LPS) (13)
2. Selected Post : Construct Chart for Selected Post Performance (8)
3. Days / Weeks / Months / Years : Construct Chart of Views for a Given Period (8)
4. Enhance DWMY Chart to Support More Dimensions (8)
5. Ensure proper accessibility support in Charts (5)
6. Implement Analytics (placeholder) (5)
7. Adapt Latest Post Chart to leverage persisted data (8)
8. Adapt Selected Post Chart to leverage persisted data (5)
9. Adapt DWMY Charts to leverage persisted data (8)

68 points; <= 3 person-sprints

[ghost]

Charts Estimate, updated Feb 25

Work Accomplished since last estimate

1. Insights : Construct Chart for Latest Post Summary (LPS) (13)

Previously Scoped Work - Still Outstanding

1. Selected Post : Construct Chart for Selected Post Performance (8)
2. Days / Weeks / Months / Years : Construct Chart of Views for a Given Period (8)
3. Enhance DWMY Chart to Support More Dimensions (8)
4. Ensure proper accessibility support in Charts (5)
5. Implement Analytics (placeholder) (5)
6. Adapt Latest Post Chart to leverage persisted data (8)
7. Adapt Selected Post Chart to leverage persisted data (5)
8. Adapt DWMY Charts to leverage persisted data (8)

Newly Identified Work

1. Correct number formatting on Chart axes (8)
2. Round the bars in Charts (13* => 0)

63 points; <2.5 person-sprints

[jklausa]

Bumped my estimates above. I'm (apart from any inevitable bugs/small issue follow-ups) now done with the networking layer and what's left is, roughly:

• mapping the objects we get from Network into Core Data entities
• making the Stores aware of the Core Data entites
• making the Stores handle data for multiple sites at once

in my head it's more-or-less all part of "Supporting Core Data", so I bumped that item up to 21 points in the estimates (and will break it down before actually digging into it).

I think it's realistically a one full sprint of work, but given that I'm away for most of this one (AFK Wed-Fri and then meetup the whole next one), it's gonna be two sprints of time.

[ScoutHarris]

UI estimates as of Feb 25:

• Shared:
  • Empty states (8)
  • Hide appbar/navigation bar and tabs on scroll, date chooser remains fixed to the top. (8)
  • Post Stats (32)
• Insights:
  • Loading view (8)
  • Tags and Categories: View more list view (8)
  • Comments: Authors, Pages & Posts View more list views (8)
  • Followers: View more list view (8)
  • Latest Post Summary: (8)
    • Move “views” to top and use full number
    • Move “likes” and “comments” to bottom as table rows
• DWMY:
  • Loading view (8)
  • Countries:
    • map (13)
    • View more list view (8)
  • Posts and Pages: update icon depending on type. (1)
  • Posts and Pages: View more list view (8)
  • Videos: View more list view (8)
  • Search Terms: View more list view (8)
  • Published: View more list view (8)
  • Authors: View more list view (8)
  • Clicks: View more list view (8)
  • Referrers: View more list view (8)
  • Add date bar (8)
• New Enhancements:
  • Icons: (5)
  • While loading icons a grey square should be used.
  • While loading avatars, a grey circle with gridicons-user should be used.
  • Referrers: (5)
    • When no icon is available then show gridicons-globe.
    • Replace the Search Engines magnifying glass icon with gridicons-search.
  • Tabs: Update filter tab bar to be blue. (5)

[ghost]

Charts Estimate, updated March 11

Work Accomplished since last estimate

1. Insights : Construct Chart for Latest Post Summary (LPS) (13)
2. Selected Post : Construct Chart for Selected Post Performance (8)
3. Days / Weeks / Months / Years : Construct Chart of Views for a Given Period (8)

Previously Scoped Work - Still Outstanding

1. Enhance DWMY Chart to Support More Dimensions (8)
2. Ensure proper accessibility support in Charts (5)
3. Implement Analytics (placeholder) (5)
4. Adapt Latest Post Chart to leverage persisted data (8)
5. Adapt Selected Post Chart to leverage persisted data (5)
6. Adapt DWMY Charts to leverage persisted data (8)
7. Correct number formatting on Chart axes (8)
8. Round the bars in Charts (13* => 0)

Newly Added / Identified Work

1. Improve styling of Period Chart when Views/Visitors are overlaid (3)

50 points; ~2 person-sprints

[ScoutHarris]

UI estimates as of March 12:

• Shared:
  • Empty states (8)
  • Hide appbar/navigation bar and tabs on scroll, date chooser remains fixed to the top. (8)
  • Post Stats (24)
  • Stats Detail lists: show loading view (8)
• Insights:
  • Loading view (8)
  • Latest Post Summary: (8)
    • Move “views” to top and use full number
    • Move “likes” and “comments” to bottom as table rows
• DWMY:
  • Loading view (8)
  • Countries:
    • map (13)
    • View more list view (8)
  • Posts and Pages: update icon depending on type. (1)
  • Posts and Pages: View more list view (8)
  • Videos: View more list view (8)
  • Search Terms: View more list view (8)
  • Published: View more list view (8)
  • Authors: View more list view (8)
  • Clicks: View more list view (8)
  • Referrers: View more list view (8)
  • Add date bar (8)
• Enhancements:
  • Icons: (5)
    • While loading icons a grey square should be used.
    • While loading avatars, a grey circle with gridicons-user should be used.
  • Referrers: (5)
    • When no icon is available then show gridicons-globe.
    • Replace the Search Engines magnifying glass icon with gridicons-search.

[ghost]

Charts Estimate, updated March 25

Work Accomplished since last estimate

1. Improve styling of Period Chart when Views/Visitors are overlaid (3)
2. Chart highlight bar overlaps the legend (1)

Previously Scoped Work - Still Outstanding

1. Enhance DWMY Chart to Support More Dimensions (8)
2. Ensure proper accessibility support in Charts (5)
3. Implement Analytics (placeholder) (5)
4. Adapt Latest Post Chart to leverage persisted data (8)
5. Adapt Selected Post Chart to leverage persisted data (5)
6. Adapt DWMY Charts to leverage persisted data (8)
7. Correct number formatting on Chart axes (8)
8. Round the bars in Charts (13* => 0)

Newly Added / Identified Work

(None)

Projection

47 points remaining (i.e., <2 person-sprints)

[ScoutHarris]

UI estimates as of March 26:

• Shared:
  • Empty states (8)
  • Hide appbar/navigation bar and tabs on scroll, date chooser remains fixed to the top. (8)
  • Post Stats (24)
  • Stats Detail lists: show loading view (8)
• Insights:
  • Loading view (8)
• DWMY:
  • Loading view (8)
  • Countries: add map (13)
  • Posts and Pages: update icon depending on type. (1)
  • Add date bar (13)
• Enhancements:
  • Icons: (5)
  • While loading icons a grey square should be used.
  • While loading avatars, a grey circle with gridicons-user should be used.
  • Referrers: (5)
    • When no icon is available then show gridicons-globe.
    • Replace the Search Engines magnifying glass icon with gridicons-search.
• Issues
  • Issues listed in description and any additional that may arise. (13)

[jklausa]

I'm hoping to wrap-up my part of the work by the end of the week!

[ghost]

Charts Estimate, updated April 9

Work Accomplished since last estimate

1. Enhance DWMY Chart to Support More Dimensions (8)
2. Implement Analytics (placeholder) (5)

Previously Scoped Work - Still Outstanding

1. Ensure proper accessibility support in Charts (5)
2. Adapt Latest Post Chart to leverage persisted data (8)
3. Adapt Selected Post Chart to leverage persisted data (5)
4. Adapt DWMY Charts to leverage persisted data (8)
5. Improve number formatting on Chart axes (8)

Newly Added / Identified Work

(None)

Projection

34 points remaining (i.e., <2 person-sprints)

[ScoutHarris]

UI estimates as of April 15:

• Shared:
  • Add tracks (5)
  • Hide appbar/navigation bar and tabs on scroll, date chooser remains fixed to the top. (8)
  • Post Stats (24)
  • Stats Detail lists: show loading view (8)
  • Post Stats: tap post title opens post in webview (5)
• Insights:
  • Loading view (8)
• DWMY:
  • Loading view (8)
  • Restart queries when period or date changes (5)
  • Overview card: populate with actual data (5)
  • Countries: add map (13)
  • Date bar: Add to Post Stats view (8)
• Enhancements:
  • Icons: (5)
  • While loading icons a grey square should be used.
  • While loading avatars, a grey circle with gridicons-user should be used.
  • Referrers: (5)
    • When no icon is available then show gridicons-globe.
    • Replace the Search Engines magnifying glass icon with gridicons-search.
• Issues
  • Issues listed in description and any additional that may arise. (13)

[ghost]

Charts Estimate, updated April 23

Work Accomplished since last estimate

1. Ensure proper accessibility support in Charts (5)

Previously Scoped Work - Still Outstanding (or In Progress)

1. Adapt Latest Post Chart to leverage persisted data (5)
2. Adapt Selected Post Chart to leverage persisted data (5)
3. Adapt DWMY Charts to leverage persisted data (5)
4. Improve number formatting on Chart axes (8)

Newly Added / Identified Work

(None)

Projection

23 points remaining (i.e., ~1 person-sprint)

[ghost]

Charts Estimate, updated May 6

Work Accomplished since last estimate

1. Adapt Latest Post Chart to leverage persisted data (5)
2. Adapt Selected Post Chart to leverage persisted data (5)
3. Improve number formatting on Chart axes (8)

Previously Scoped Work - Still Outstanding (or In Progress)

1. Adapt DWMY Charts to leverage persisted data (5)

Newly Added / Identified Work

1. Overview chart: when a bar is selected, update overview data accordingly; when the view first appears, auto select the most recent day; when the date changes via the date chooser, auto select corresponding bar. (8)

Projection

13 points remaining

[ScoutHarris]

UI estimates as of May 7:

• Shared:
  • Add tracks (5)
  • Hide appbar/navigation bar and tabs on scroll, date chooser remains fixed to the top. (8)
  • Post Stats: show loading view (8)
  • Stats Detail lists: show loading view (8)
• Insights:
  • Properly display cached data with no network connection (5)
• DWMY:
  • Loading view (8)
  • Countries: add map (13)
• Enhancements:
  • Icons: (5)
    • While loading icons a grey square should be used.
  • Referrers: (5)
    • When no icon is available then show gridicons-globe.
    • Replace the Search Engines magnifying glass icon with gridicons-search.
• Issues
  • Period summary query takes a long time to return data. (8)
  • Period data not refreshing correctly when queries finish. (5)
  • Details view take a long time to load (13)
  • Other issues listed in description and any additional that may arise. (13)

[danielebogo]

Estimation - May 8

Insights

• Properly display cached data with no network connection (5)

DWMY

• Loading view and error message (8)
• Countries: add map (13)

Remaining: 26 points

[ghost]

Charts Estimate, updated May 20

Work Accomplished since last estimate

1. Adapt DWMY Charts to leverage persisted data (5)
2. Overview chart: when a bar is selected, update overview data accordingly; when the view first appears, auto select the most recent day; when the date changes via the date chooser, auto select corresponding bar. (8)

Previously Scoped Work - Still Outstanding (or In Progress)

Bug fixes & polishing as needed.

Newly Added / Identified Work

N/A

Projection

0 points remaining

[danielebogo]

Estimation - May 21

Insights

• Properly display cached data with no network connection (5)
• Loading view on Stats detail

DWMY

• Loading view and error message (8)
• Loading view on Stats detail
• Countries: add map (13)

Remaining: 13 points

[ScoutHarris]

v2 Master Issue created, which includes all outstanding issues from v1. Closing this.