As a programme specialist / data scientist, I want to know that I can trust this data

[binkymilk]

What I want:

As a programme specialist / data scientist, I want to know that I can trust this data

I will know that this works when

• It's clear who created this dataset
• It's clear when this dataset was created
• It's clear how this dataset was created
• It's clear how accurate this data is (When compared against another dataset, or the validation / holdout dataset)
• It's clear where the training data / underlying data came from
• It's clear who else has used this data
• There's some supporting material (A peer reviewed paper, an endorsement from another reputable organisation)

[binkymilk]

Questions:

• For supporting materials, are these links to papers or press releases?
• Are there any other info we want to add both to the model info and the dataset info?

Notes:

• Include structure / schema of the datasets here

[butchtm]

Fields to include in the catalog card:

• who created this item
• when this item was created
• how this item was created - description? link for more documentation?
• accuracy/metrics - applicable for models only?
• training data source - description? links?
• who else used the data - requires tracking of usage? alternatively, self-reported usage. e.g. citations
• Supporting material - RRL (Related Research Literature), peer reviewed paper, endorsements from other reputable organizations)

[butchtm]

Notes:

• Included in the catalog card page
• Included in the catalog json contents

[ghost]

Dependencies:

• Set up model/dataset page
• Organization
• Description
• Accuracy/metrics
• Training datasets
• Citations or who used this data
• Structure or schema of data
• Note on benefits/risks of retraining data
• Data preview (nice-to-have)
• Supporting material
• Links

[butchtm]

• Possible way to track who used this data: Post click survey or popup - don't hijack the on click. - TBD on how to operationalize - estimates pending definition of the approach. - leave it for now, maybe a task for a BIA to conceptualize, and design

• Grab the structure if its a csv - Related to Data Preview - automate fetching the sample data. - mostly BE (except for Data Preview Design - FE)

• Stretch goal : try it geojson

[butchtm]

• Data preview - show hxl tags
• suggested hxl tags: location, adm1,2,3,4, name,

[AnthonyMockler]

Quick User Test from Kath:

I will know this works when:

• It is clear who created this dataset
o Is the organization the “creator” of the dataset?
• It is clear when this dataset was created
o Is it the Year/Period or the Date Created?
• It is clear how this dataset was created
o Not sure if I just missed it but couldn’t find the specific information on how this dataset was created
• It is clear how accurate this data is
o There’s an evaluation metric and there’s a link to a paper
• It is clear where the training data/underlying data came from
o For this specific model, there are details for the training dataset – however, I am unsure if the requirement is to specify the data source of the training dataset?
• It is clear who else has used this data
o Not sure if I just missed it but couldn’t find the information
• There’s some supporting material
o I believe this should be under “Related Links” but I only see links to resources for the model, not really supporting materials
 
What I want:
As a data scientist / data analyst / programme specialist, I want to see a list of all models in the catalogue

I will know this works when:

• There’s a web interface to provide me with a list of all the models / datasets in the catalogue
o I think the search catalogue was easy to find and gave me exactly what I was looking for
• The list is structured in some useful way
o Might be helpful to have some option to sort by Year? Alphabetically?
• There’s sufficient metadata about the models for me to filter / choose the one I’m interested in
o The current metadata and filters seem to be enough

[ghost]

We've removed Citations or who used this data as a feature on June 22