Skip to content

Commit

Permalink
[#3] Make the exported data available directly via the repo
Browse files Browse the repository at this point in the history
  • Loading branch information
bbatsov committed Dec 31, 2020
1 parent a2e09d7 commit eb2c880
Show file tree
Hide file tree
Showing 8 changed files with 26,375 additions and 125 deletions.
63 changes: 0 additions & 63 deletions .circleci/config.yml

This file was deleted.

48 changes: 20 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,35 @@
# clojuredocs-export-edn
[![CircleCI](https://img.shields.io/circleci/project/github/clojure-emacs/clojuredocs-export-edn/master.svg)](https://circleci.com/gh/clojure-emacs/clojuredocs-export-edn)

[![Dependencies Status](https://versions.deps.co/clojure-emacs/clojuredocs-export-edn/status.svg)](https://versions.deps.co/clojure-emacs/clojuredocs-export-edn)

Convert [ClojureDocs](https://clojuredocs.org)'s [clojuredocs-export.json](https://clojuredocs.org/clojuredocs-export.json) to EDN format, and publish them to this repository daily.

## Why?

Because ClojureDocs doens't have an official API and provides only a JSON export of its data currently. This makes
it harder for tools to leverage ClojureDocs and the EDN exports aim to bridge that gap.

This data is used by libraries like [Orchard](https://github.com/clojure-emacs/orchard) to fuse ClojureDocs and the standard Clojure documentation.

## Exported Data

Convert [ClojureDocs](https://clojuredocs.org)'s [clojuredocs-export.json](https://clojuredocs.org/clojuredocs-export.json) to EDN format, and deploy to [Netlify](https://www.netlify.com) daily.
The EDN data is made available in several variants:

* Deployed EDN file
* https://clojuredocs-edn.netlify.com/export.edn (all the data in the JSON export, retaining its format)
* https://clojuredocs-edn.netlify.com/export.compact.edn (drops all data about edits made from the JSON export, contains only current state of data for each symbol in an optimized format)
* Deploy task
* https://github.com/liquidz/clojuredocs-export-edn/blob/master/.circleci/config.yml
* The EDN file containing all the data from the JSON export is [here](https://github.com/clojure-emacs/clojuredocs-export-edn/blob/master/exports/export.edn).
* The compact EDN file (it doesn't include data about edits that were made to the data, only its current state) is [here](https://github.com/clojure-emacs/clojuredocs-export-edn/blob/master/exports/export.compact.edn).
* The compact minified EDN file (it strips all the optional whitespace) is [here](https://github.com/clojure-emacs/clojuredocs-export-edn/blob/master/exports/export.compact.min.edn).

## How to deploy to netlify
## Running the Exporter

* EDN files are deployed by CircleCI automatically.
* All steps are in [.circleci/config.yml](.circleci/config.yml).
* The job is triggered by cron [every day](https://github.com/liquidz/clojuredocs-export-edn/blob/b9edb899e0a5402fc82f43afa72cb61e047545bf/.circleci/config.yml#L48-L55).
* All you need is to add these environment variables to CircleCI.
* `NETLIFY_PERSONAL_ACCESS_TOKEN` which can be created [here](https://app.netlify.com/user/applications#personal-access-tokens)
* `NETLIFY_SITE_ID` which is your site's API ID
* Steps to deploy manually is below.
```console
# EDN files are generated in ./target/out/
bash generate.sh
You can also run the export procedure manually like this:

# Install netlify-cli
sudo npm install -g --silent netlify-cli
$ lein run

# Deploy EDN files via netlify-cli
netlify deploy \
--auth=${NETLIFY_PERSONAL_ACCESS_TOKEN} \
--site=${NETLIFY_SITE_ID} \
--dir=target/out \
--prod
```
The fresh exports will be placed under `exports/`.

## License

Copyright © 2019 [Masashi Iizuka](https://twitter.com/uochan)
Copyright © 2019-2020 [Masashi Iizuka](https://twitter.com/uochan), Bozhidar Batsov

This program and the accompanying materials are made available under the
terms of the Eclipse Public License 2.0 which is available at
Expand Down
Loading

0 comments on commit eb2c880

Please sign in to comment.