Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INFRA] Fix CircleCI workflows #764

Merged
merged 3 commits into from
Mar 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 53 additions & 39 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ jobs:
docker:
- image: circleci/python:3.8
steps:
# checkout code to default ~/project
- checkout
- run:
name: install dependencies
Expand All @@ -14,53 +15,56 @@ jobs:
name: generate docs
command: mkdocs build --clean --strict --verbose
- persist_to_workspace:
root: .
# the mkdocs build outputs are in ~/project/site
root: ~/project
paths: site

linkchecker:
docker:
- image: yarikoptic/linkchecker:9.4.0.anchorfix1-1
steps:
- attach_workspace:
at: ~/build
# mkdocs build outputs will be in ~/project/site
at: ~/project
- run:
name: check links
command: |
if (! git log -1 --pretty=%b | grep REL:) ; then
chmod a+rX -R ~
linkchecker -t 1 ~/build/site/
# check external separately by pointing to all *html so no
# failures for local file:/// -- yoh found no better way,
linkchecker -t 1 --check-extern \
--ignore-url 'file:///.*' \
--ignore-url https://fonts.gstatic.com \
--ignore-url "https://github.com/bids-standard/bids-specification/(pull|tree)/.*" \
--ignore-url "https://github.com/[^/]*" \
~/build/site/*html ~/build/site/*/*.html
chmod a+rX -R ~
linkchecker -t 1 ~/project/site/
# check external separately by pointing to all *html so no
# failures for local file:/// -- yoh found no better way,
linkchecker -t 1 --check-extern \
--ignore-url 'file:///.*' \
--ignore-url https://fonts.gstatic.com \
--ignore-url "https://github.com/bids-standard/bids-specification/(pull|tree)/.*" \
--ignore-url "https://github.com/[^/]*" \
~/project/site/*html ~/project/site/*/*.html
else
echo "Release PR - do nothing"
echo "Release PR - do nothing"
fi
build_docs_pdf:
working_directory: ~/bids-specification/pdf_build_src
docker:
- image: danteev/texlive:latest
steps:
- checkout:
path: ~/bids-specification
# checkout code to default ~/project
- checkout
- run:
name: install dependencies
command: |
python -m pip install --upgrade pip
pip install -r ../requirements.txt
pip install -r ~/project/requirements.txt
- run:
name: install font that works with unicode emojis
command: apt-get update && apt-get install -y fonts-symbola
- run:
name: generate pdf version docs
command: bash build_pdf.sh
command: |
cd ~/project/pdf_build_src
bash build_pdf.sh
- store_artifacts:
path: bids-spec.pdf
path: ~/project/pdf_build_src/bids-spec.pdf

# Auto changelog collector
github-changelog-generator:
Expand All @@ -69,84 +73,94 @@ jobs:
steps:
- setup_remote_docker:
version: 18.06.0-ce
# checkout code to default ~/project
- checkout
- run:
name: Build changelog
working_directory: ~/build
# $CHANGE_TOKEN is generated via the GitHub web UI, and then securely stored within CircleCI web UI
command: |
mkdir ~/changelog_build
git status
if (git log -1 --pretty=%s | grep Merge*) && (! git log -1 --pretty=%b | grep REL:) ; then
github_changelog_generator \
--user bids-standard \
--project bids-specification \
--token ${CHANGE_TOKEN} \
--output ~/build/CHANGES.md \
--base ~/build/src/pregh-changes.md \
--output ~/changelog_build/CHANGES.md \
--base ~/project/src/pregh-changes.md \
--header-label "# Changelog" \
--no-issues \
--no-issues-wo-labels \
--no-filter-by-milestone \
--no-compare-link \
--pr-label "" \
--release-branch master
cat ~/build/CHANGES.md
mv ~/build/CHANGES.md ~/build/src/CHANGES.md
cat ~/changelog_build/CHANGES.md
else
echo "Commit or Release, do nothing"
fi
- persist_to_workspace:
root: .
paths: src
# raw generated changelog in ~/changelog_build/CHANGES.md
root: ~/.
paths: changelog_build

# Run remark on the auto generated changes.md file
remark:
docker:
- image: node:latest
steps:
# checkout code to default ~/project
- checkout
- attach_workspace:
at: ~/build
# the freshly built CHANGES.md will be in ~/changelog_build/CHANGES.md
at: ~/.
- run:
name: install remark and extensions
command: npm install `cat npm-requirements.txt`
- run:
name: remark on autogenerated CHANGES.md
# format changelog, then use sed to change * to -, then lint changelog
command: |
mkdir ~/project/src/tmp
git status
if (git log -1 --pretty=%s | grep Merge*) && (! git log -1 --pretty=%b | grep REL:) ; then
cat ~/build/src/CHANGES.md
cp ~/build/src/CHANGES.md ~/project/src/tmp/CHANGES.md
npx remark ~/project/src/tmp/CHANGES.md --frail --rc-path .remarkrc
head -n 100 ~/changelog_build/CHANGES.md
npx remark-cli ~/changelog_build/CHANGES.md --rc-path ~/project/.remarkrc --output ~/changelog_build/CHANGES.md
head -n 100 ~/changelog_build/CHANGES.md
sed -i 's/* /- /' ~/changelog_build/CHANGES.md
head -n 100 ~/changelog_build/CHANGES.md
npx remark-cli ~/changelog_build/CHANGES.md --frail --rc-path ~/project/.remarkrc
else
echo "Commit or Release, do nothing"
touch ~/project/src/tmp/empty.txt
fi
- persist_to_workspace:
root: ~/project/src
paths: tmp
# fixed+linted changelog in ~/changelog_build/CHANGES.md
root: ~/.
paths: changelog_build

# Push built changelog to repo
Changelog-bot:
working_directory: ~/build
docker:
- image: circleci/openjdk:8-jdk
steps:
- setup_remote_docker:
version: 17.11.0-ce
# checkout code to default ~/project
- checkout
- attach_workspace:
at: ~/build
# fixed+linted changelog in ~/changelog_build/CHANGES.md
at: ~/.
- deploy:
name: Changelog deployment
working_directory: ~/build
# $CHANGE_TOKEN is generated via the GitHub web UI, and then securely stored within CircleCI web UI
command: |
if (git log -1 --pretty=%s | grep Merge*) && (! git log -1 --pretty=%b | grep REL:) ; then
mv ~/build/tmp/CHANGES.md ~/build/src/CHANGES.md
mv ~/changelog_build/CHANGES.md ~/project/src/CHANGES.md
merge_message=$(git log -1 | grep Merge | grep "pull")
PR_number=$(echo $merge_message | cut -d ' ' -f 4)
git config credential.helper 'cache --timeout=120'
git config user.email "bids.maintenance@gmail.com"
git config user.name "bids-maintenance"
git add ~/build/src/CHANGES.md
git add ~/project/src/CHANGES.md
git commit -m "[DOC] Auto-generate changelog entry for PR ${PR_number}"
git push https://${CHANGE_TOKEN}@github.com/bids-standard/bids-specification.git master
else
Expand Down
4 changes: 2 additions & 2 deletions DECISION-MAKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ and heavily depends on [GitHub Pull Request Review system](https://help.github.c
Request (PR) to the Repository.
1. Anyone can open a PR (this action is not limited to Contributors).
1. PRs adding new Contributors must also add their GitHub names to the
[CODEOWNERS](CODEOWNERS) file.
[CODEOWNERS](./CODEOWNERS) file.
1. A PR is eligible to be merged if and only if these conditions are met:
1. The last commit is at least 5 working days old to allow the community to
evaluate it.
Expand Down Expand Up @@ -138,7 +138,7 @@ and heavily depends on [GitHub Pull Request Review system](https://help.github.c
1. To facilitate triage of incoming PR you can subscribe to
notifications for new PRs proposing changes to specific files. To do this
add your GitHub name next to the file you want to subscribe to in the
[CODEOWNERS](CODEOWNERS). This way you will be ask to review each relevant
[CODEOWNERS](./CODEOWNERS). This way you will be ask to review each relevant
PR. Please mind that lack of your review will not prevent the PR from being
merged so if you think the PR needs your attention, please review it
promptly or request more time via Request changes.
Expand Down
2 changes: 1 addition & 1 deletion src/99-appendices/03-hed.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Each level of a hierarchical tag is delimited with a forward slash (`/`).
A HED string contains one or more HED tags separated by commas (`,`).
Parentheses (brackets, `()`) group tags and enable specification of multiple items
and their attributes in a single **HED string** (see section 2.4 in
[HED Tagging Strategy Guide](https://www.hedtags.org/downloads/HED%20Tagging%20Strategy%20Guide.pdf)).
[HED Tagging Strategy Guide](https://www.hedtags.org/docs/HEDTaggingStrategyGuide.pdf)).
For more information about HED and tools available to validate and match HED
strings, please visit [www.hedtags.org](https://www.hedtags.org).
Since dedicated fields already exist for the overall task classification in the
Expand Down