Explore the eLife article data

[lwinfree]

eLife articles: a dump of all eLife articles in XML and JSON (up to Aug, 2020; ~ 60GB), https://github.com/elifesciences/elife-article-xml and https://github.com/elifesciences/elife-article-json

[chris48s]

• How can we identify and extract tabular data from these articles?
• What tools exist to help us parse these? Anything higher-level/smarter than plain old lxml?
• Is there a particular subset of tables/tabular data that we are interested in?
• Once we've identified tabular data in articles, how do we work with them?

[proccaserra]

@chris48s @lwinfree just asked Emmy about retrieving the table.

example of table of interest: table 1 in:
https://elifesciences.org/articles/57525/figures#tables
From Emmy, I understand we could get JATS xml format for the article, parse that and pull tables, get headers and look for entities/patterns (e.g. repeating headers as seen in table 1).

• The so-called key resource table can be filtered out. I have produced a "Frictionless Template" for it.
• attempt to detect tables containing statistical results (e.g. field header ~[pval|AdjPval|p-value|q-value]) etc...
• create bins for other motifs
• batch convert to Frictionless data package.
• attempt assigning rdfType to headers.