Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Style stats modal #91

Closed
3 tasks done
nathanielrindlaub opened this issue Jun 17, 2022 · 3 comments
Closed
3 tasks done

Style stats modal #91

nathanielrindlaub opened this issue Jun 17, 2022 · 3 comments
Assignees
Labels
admin tools enhancement New feature or request
Milestone

Comments

@nathanielrindlaub
Copy link
Member

nathanielrindlaub commented Jun 17, 2022

I think it would be pretty straight forward and useful to have an option to pull stats about a particular set of filtered images. Maybe we put a button next to the image count that triggers a modal to open with some analytics and simple data visualizations. Useful ones include:

  • total number of reviewed images / not-reviewed images
  • leader board of all labelers and their label count
  • break down of labels (number and percentage fox, skunk, etc)

We would need to create query handlers for all of these on the API side.

@nathanielrindlaub
Copy link
Member Author

I built out a crude version of this (currently just displaying the returned JSON "stats" payload, with the following:

  • imageCount - total # of images included in filters & in calculations
  • reviewedCount - breakdown of # of images that have been reviewed / still need review
  • reviewerList - a list of all the users who have edited images, and the number of images they have edited
  • labelList - breakdown of all the validated objects (# skunks, foxes, empties, etc)
  • multipleReviewerCount - number of images that have been edited by more than one user. Not that useful I don't think - I was just curious.

Next steps are to display these data in a more polished dashboar format (table, bar charts, etc.), and consider whether there are other things that might be interesting to show. I feel like there's opportunity to make the labelList more interesting...

We could possibly show total number of ML predictions for each label so you can compare roughly how well it performs against the validated label counts (i.e., there were 5k ML predictions for "skunks", but only 3k validated ones). Of course, that's a very imperfect gauge of model accuracy, because an invalidated false positive will be erased by a corrected false negative somewhere else.

For all the validated top labels, we could show the % that were ML predicted & simply validated vs. manually created? That might be more interesting.

Could also maybe show counts for the labels that have yet to be validated (objects still need review).

@nathanielrindlaub nathanielrindlaub added this to the v2.0 milestone Aug 10, 2023
@nathanielrindlaub nathanielrindlaub changed the title Create stats modal Style stats modal Aug 10, 2023
@nathanielrindlaub nathanielrindlaub modified the milestones: v2.0, v2.1 Sep 14, 2023
@nathanielrindlaub
Copy link
Member Author

nathanielrindlaub commented Sep 25, 2023

  • fix loading spinner placement

@nathanielrindlaub
Copy link
Member Author

Closing out with this PR! #215

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
admin tools enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant