Create paper projections for paper similarity graph #63

georgepar · 2020-06-12T14:13:54Z

The 3 steps of #54

Changelog:

Add initial script scripts/create_papers_csv.py to create a first version of papers.csv. This was needed to proceed with this. @GuyAglionby will take over the next version of papers.csv
Modify scripts/reduce.py to include UMAP projection method
Modify templates/poster.html and main.py to add code for viewing similar papers
Add docs for the process

Also see the PR from sitedata repo:
https://github.com/acl-org/acl-2020-virtual-conference-sitedata/pull/3

georgepar · 2020-06-12T14:18:33Z

@hao-fang I've also created a branch in the sitedata private repo with the updated files.
Not sure what to do so we can see this on some CI-deployed site.

Currently one needs to grab the updated papers.csv and papers_projection.json and paste them into sitedata_acl2020 and run the site locally.

georgepar · 2020-06-12T17:01:26Z

Also I see in the CI make format-check fails.
I've run make format and make format-check locally and it passes. Any ideas?

hao-fang · 2020-06-12T17:21:57Z

templates/poster.html

+             {{a}},
+             {% endfor %}
+          </h6>
+          <center><img class="cards_img" src="https://iclr.github.io/iclr-images/{{openreview.content.iclr_id}}.png" width="80%"/></center>


Can we make https://iclr.github.io/iclr-images a config variable and use {{ config.xxxx_url }} here?
Then we can edit the sitedata_acl2020/config.yml file to add this field.

Updated with latest commit

hao-fang · 2020-06-12T17:24:45Z

@hao-fang I've also created a branch in the sitedata private repo with the updated files.
Not sure what to do so we can see this on some CI-deployed site.

Currently one needs to grab the updated papers.csv and papers_projection.json and paste them into sitedata_acl2020 and run the site locally.

Can you make a PR in the sitedata private repo and I'll approve?

After we merge that PR, you can go to this repo locally, run

cd sitedata_acl2020
git pull
cd ../
git add sitedata_acl2020
git commit -am "bump sitedata_acl2020"
git push

This way it should point to the latest private sitedata.
3. Once this PR is merged into master, it will publish to this website https://d7znt0uidgb1r.cloudfront.net/ (We have a GitHub action configured to automate this.)

hao-fang · 2020-06-12T17:26:53Z

Also I see in the CI make format-check fails.
I've run make format and make format-check locally and it passes. Any ideas?

It says

ERROR: /home/runner/work/acl-2020-virtual-conference/acl-2020-virtual-conference/scripts/create_papers_csv.py Imports are incorrectly sorted.

It seems that your local python venv is not synced with the CI venv.
Did you follow the instruction here to create your local virtualenv?

georgepar · 2020-06-12T17:39:02Z

Did you follow the instruction here to create your local virtualenv?

I will try it. Thanks :)

georgepar · 2020-06-15T12:14:35Z

Ok I did some refactoring to simplify code, since papers.csv was updated in sitedata repo.

Now the CI fails in make freeze because paper_recs.json is missing so I think this PR should be merged first.

https://github.com/acl-org/acl-2020-virtual-conference-sitedata/pull/3/files

georgepar · 2020-06-15T18:13:36Z

@hao-fang after bumping sitedata checks pass, so this can be merged after review

hao-fang

Thanks! I bumped the sitedata_acl2020 version and updated the master branch.
This PR is now good to merge 🎉

Co-authored-by: Sasha Rush <sasha.rush@gmail.com>

* cal with some magic numbers. * projection to site_data * cal v1 * timzones added * cal finshed. * cookie solution * local storage solution * add python format checks * add makefile * add CONTRIBUTING.md * update * Update README.md * Update README.md * Update README.md * Fix poster link * README * readme * Format python code * Prettier on CSS files * Move to make file * bugfix cal ? * speaker CSV format changed. * fixed lazy-load in schedule * css fixes * remove deploy * Update README.md * Run prettier and make formatter * Prettier format code * added authors * links added * minor changes on readme * drive-by * Update README.md * added ci * added new test * added newline * added make test * edit script * added pylint * removed pylint * used make format * style check on file * Create lint_code.yml (#39) * Create lint_code.yml * ok * test * ok * pylint * ok * ok... * remove node versin * Update README.md * Add files via upload * Delete miniconf.gif * Add files via upload * Update README.md * Customizable logo size, site_title, and background images (#40) * Merge mini-conf upstream (#7) * cal with some magic numbers. * projection to site_data * cal v1 * timzones added * cal finshed. * cookie solution * local storage solution * add python format checks * add makefile * add CONTRIBUTING.md * update * Update README.md * Update README.md * Update README.md * Fix poster link * README * readme * Format python code * Prettier on CSS files * Move to make file * bugfix cal ? * speaker CSV format changed. * fixed lazy-load in schedule * css fixes * remove deploy * Update README.md * Run prettier and make formatter * Prettier format code * added authors * links added * minor changes on readme * drive-by * Update README.md Co-authored-by: Strobelt Hendrik <hendrik@strobelt.com> Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * adjust base and index * make height width adjustable * make height width adjustable * readmeg * ok * use default value * quote Co-authored-by: Strobelt Hendrik <hendrik@strobelt.com> Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * typo fixed (#43) Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * Add Highwire Press meta tags for citation info (#42) * Add Highwire Press meta tags for citation info * Fix typos * Update citation meta tag generation * Add titles to pages (fixes #23) (#46) * Validates CSS files during CI (#44) * ok * eslint * backup stylelint * package * lint_code.yml * makefile * bug * use npx to execute node packages * use npm ci * Delete package-lock.json * Create package-lock.json Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * Add eslint and prettier for JS files (#45) * wip * ok * eslint * backup stylelint * package * lint_code.yml * makefile * bug * use npx to execute node packages * ok * use npm ci * prettier and eslint * ok * do not change typehead * do not change ical * add prettier ignore * ok * ok * revert * ok * ok * [EASY] Update CONTRIBUTING.md & use safe for config.date (#48) * Update CONTRIBUTING.md * use safe for date * Fix workshop organiser formatting + yaml/json paper data (#47) * Fix workshop author rendering * Fix handling of papers when stored as yml or json * linting * linting againe * pull list field handling to method * Use nice concatenator method Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * Cleaned up CDNs (#25) (#49) * Standardized CDNs * Update tui-code-snippet version * Update popper cjs -> umd * cjs -> umd cdn Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * pdf js for poster rendering (#50) Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * . (#51) * Setup make deploy (#31) (#53) * Standardized CDNs * Update tui-code-snippet version * Update popper cjs -> umd * cjs -> umd cdn * Update makefile * Update makefile to checkout previous branch * Updated Makefile comment Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * Integrate chat system (#52) * Readme * Chat updates * update * Format * Format * format chat * format * isort nonsense * Update chat.html * update poster page Co-authored-by: Hao Fang <haofang1990@gmail.com> * added hour start and end for a more compact calendar, if possible (#56) * support for reading papers in diff formats in make_posters (#57) * Format chat folder (#58) * check chat folder as well * format * drive-by * ok * remove unused imports * lint * format * Integrate SSO for chat and login (#62) * chat * auth * update * turn off auth by default * miniconf * . Co-authored-by: Sasha <arush@g.harvard.edu> * don't use const for variable variables (#63) Co-authored-by: Sasha Rush <sasha.rush@gmail.com> * Update README.md * Update README.md * Update README.md * Cache Google fonts locally. (#64) * chat * auth * update * turn off auth by default * miniconf * . * fonts * cuprum * prettier Co-authored-by: Sasha <arush@g.harvard.edu> * Fix login and uri (#65) * chat * auth * update * turn off auth by default * miniconf * . * fonts * cuprum * prettier * redirect * fix redirect Co-authored-by: Sasha <arush@g.harvard.edu> * Use different config field for citation date (#66) To allow HTML in the date field that is used in the index * bugfix: using correct tippy bundle * . * Updated README to show the use of reduce.py (#68) Mistake with embeddings.py instead of reduce.py * add lato 900 ttf (#70) * ok * ok Co-authored-by: Strobelt Hendrik <hendrik@strobelt.com> Co-authored-by: Sasha Rush <sasha.rush@gmail.com> Co-authored-by: animesh-007 <animeshagarwal7@gmail.com> Co-authored-by: Sasha Rush <srush.research@gmail.com> Co-authored-by: Guy Aglionby <GuyAglionby@users.noreply.github.com> Co-authored-by: Arpan Srivastava <arpansri98@gmail.com> Co-authored-by: Hendrik Strobelt <HendrikStrobelt@users.noreply.github.com> Co-authored-by: Sameer Singh <sameeersingh@gmail.com> Co-authored-by: Sasha <arush@g.harvard.edu> Co-authored-by: Vukosi <vimarambo@gmail.com> Co-authored-by: Tom Pollard <tpollard@mit.edu>

Add papers.csv creator and UMAP projection in scripts/reduce.py

d766d3e

georgepar requested a review from hao-fang June 12, 2020 14:14

georgepar added 3 commits June 12, 2020 17:24

Reformat code

d97a876

Add similar papers in poster page

2fd1435

Add guide to produce similar paper recommendations

31c19f7

georgepar mentioned this pull request Jun 12, 2020

Basic "Similar Papers" function #54

Closed

3 tasks

Reformat main.py

e556362

hao-fang reviewed Jun 12, 2020

View reviewed changes

hao-fang linked an issue Jun 12, 2020 that may be closed by this pull request

Basic "Similar Papers" function #54

Closed

3 tasks

make image_path configurable

5af073f

georgepar added 4 commits June 15, 2020 13:51

format create_papers_csv

857fc8c

Update with latest papers.csv and simplify code

07b2f8b

Remove unused imports

b213406

Ignore typecheck for openreview and umap-learn

1913635

georgepar and others added 6 commits June 15, 2020 15:37

Merge branch 'master' into 54_add_similar_papers_graph

766146e

Modify poster.html to get correct id field

2a52660

refactor templates/poster.html

7571602

Update README.md

9776a15

Update README.recommendations.md

b5a94a8

Merge branch 'master' into 54_add_similar_papers_graph

f9775d2

hao-fang approved these changes Jun 15, 2020

View reviewed changes

hao-fang merged commit e5a817e into master Jun 15, 2020

hao-fang deleted the 54_add_similar_papers_graph branch June 15, 2020 18:47

hao-fang pushed a commit that referenced this pull request Jun 19, 2020

don't use const for variable variables (#63)

38ad7bd

Co-authored-by: Sasha Rush <sasha.rush@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create paper projections for paper similarity graph #63

Create paper projections for paper similarity graph #63

georgepar commented Jun 12, 2020 •

edited

Loading

georgepar commented Jun 12, 2020 •

edited

Loading

georgepar commented Jun 12, 2020

hao-fang Jun 12, 2020

georgepar Jun 12, 2020

hao-fang commented Jun 12, 2020 •

edited

Loading

hao-fang commented Jun 12, 2020

georgepar commented Jun 12, 2020

georgepar commented Jun 15, 2020

georgepar commented Jun 15, 2020

hao-fang left a comment

Create paper projections for paper similarity graph #63

Create paper projections for paper similarity graph #63

Conversation

georgepar commented Jun 12, 2020 • edited Loading

georgepar commented Jun 12, 2020 • edited Loading

georgepar commented Jun 12, 2020

hao-fang Jun 12, 2020

Choose a reason for hiding this comment

georgepar Jun 12, 2020

Choose a reason for hiding this comment

hao-fang commented Jun 12, 2020 • edited Loading

hao-fang commented Jun 12, 2020

georgepar commented Jun 12, 2020

georgepar commented Jun 15, 2020

georgepar commented Jun 15, 2020

hao-fang left a comment

Choose a reason for hiding this comment

georgepar commented Jun 12, 2020 •

edited

Loading

georgepar commented Jun 12, 2020 •

edited

Loading

hao-fang commented Jun 12, 2020 •

edited

Loading