Create benchmark data sets

[ejsegall]

Go through the data we have
Select a diverse range of possible formulations based on the actual data, but designed to be diverse so e.g. a small number of positives in a lot of data, a lot of positives, smaller number of genes, larger number of genes, different gene expression distributions, etc. @dhimmel: please review description

[rdvelazquez]

What's the status of this? I've looked for a benchmark data set before but haven't found one.

@dhimmel or @ejsegall: Let me know if this is something I should take a crack at... if so, we could discuss some specifics (which data sets/queries to include; whether to implement as a notebook in the explore repo, as a feature for cognoml (which may help with cognoma/cognoml#4 (comment)) or just save file(s) in the format outlined for the MVP #31 (comment)).

[dhimmel]

@rdvelazquez let's focus on #94 as a priority. Then we can reassess our needs regarding this issue, especially after #93 is merged.

[athril]

You may consider using PMLB as a benchmark instead of creating a new one: https://github.com/EpistasisLab/penn-ml-benchmarks

[rdvelazquez]

Thanks for the heads up about PMLB @athril. This is quite a long list of benchmark datasets; the benchmark data-set for testing cognoma is fairly specific and I didn't see anything that would meet our needs on PMLB.

[rdvelazquez]

@brankaj provided a very nice list of applicable genes in #52. I also expanded this by subsetting queries by disease in #113. I think we can close this issue for now.