Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a workflow to update dependencies #490

Merged
merged 7 commits into from
Dec 21, 2020
Merged

Add a workflow to update dependencies #490

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
336db95
Add a workflow to update dependencies
mayeut Dec 20, 2020
3b465d8
Fix a few things in update workflow
mayeut Dec 20, 2020
17798c5
Use '--no-docker' to run dependency update using local pythons
mayeut Dec 20, 2020
5ffc7e4
Only run the dependency update workflow manually
mayeut Dec 20, 2020
55048c7
Update .github/workflows/update-dependencies.yml
mayeut Dec 20, 2020
13b7ed3
Create constraints for all python versions
mayeut Dec 20, 2020
d0b61cd
Use manylinux2010_x86_64:latest to update dependencies
mayeut Dec 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/update-dependencies.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Update dependencies

on:
workflow_dispatch:

jobs:
update-dependencies:
name: Update dependencies
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
name: Install Python 3.9
with:
python-version: 3.9
architecture: x64
- name: Install dependencies
run: python -m pip install requests pip-tools
- name: Run update
run: python3 ./bin/update-dependencies.py
- name: Create Pull Request
#if: github.ref == 'refs/heads/master' && always()
uses: peter-evans/create-pull-request@v3
with:
commit-message: Update dependencies
title: '[Bot] Update dependencies'
body: |
Update the versions of our dependencies.

PR generated by "Update dependencies" [workflow](https://github.com/joerick/cibuildwheel/actions/runs/${{github.run_id}}).
branch: update-dependencies-pr
delete-branch: true
draft: true
36 changes: 27 additions & 9 deletions bin/update_dependencies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,39 @@

# CUSTOM_COMPILE_COMMAND is a pip-compile option that tells users how to
# regenerate the constraints files
os.environ['CUSTOM_COMPILE_COMMAND'] = "bin/update_constraints.py"
os.environ['CUSTOM_COMPILE_COMMAND'] = "bin/update_dependencies.py"
subprocess.check_call([
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realised... this command is implicitly run in python3.8, because that's what my venv is in on my mac >.< (maybe a little sloppy, sorry!). Could it be made explicit? Perhaps 38 and 39 should be added to the list of versions and the default constraints.txt uses the final entry of this list? Then it should also be generated inside the docker container, so we're sure which version of Python is running it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we use a default constraints.txt at all ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great point. Maybe we don't need it at all, in fact!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, well actually our code does assert that it exists, currently

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also maybe useful as a starting point if somebody wanted to customise their constraints. But just copying the latest Python we support should do

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I had to modify a test so I updated all dependencies. Anyway, I think I'll have to rebase on #489 anyway.

'pip-compile',
'--allow-unsafe',
'--upgrade',
'cibuildwheel/resources/constraints.in',
])
for python_version in ['27', '35', '36', '37']:
subprocess.check_call([
f'./env{python_version}/bin/pip-compile',
'--allow-unsafe',
'--upgrade',
'cibuildwheel/resources/constraints.in',
'--output-file', f'cibuildwheel/resources/constraints-python{python_version}.txt'
])

if 'GITHUB_ACTIONS' in os.environ:
mayeut marked this conversation as resolved.
Show resolved Hide resolved
image = 'quay.io/pypa/manylinux2010_x86_64:2020-12-19-8df9e2d'
YannickJadoul marked this conversation as resolved.
Show resolved Hide resolved
subprocess.check_call(['docker', 'pull', image])
for python_version in ['27', '35', '36', '37']:
python_path = f'/opt/python/cp{python_version}-cp{python_version}m/bin/'
subprocess.check_call([
'docker', 'run', '--rm',
'-e', 'CUSTOM_COMPILE_COMMAND',
'-v', f'{os.getcwd()}:/volume',
'--workdir', '/volume', image,
'bash', '-c',
f'{python_path}pip install pip-tools &&'
f'{python_path}pip-compile --allow-unsafe --upgrade '
'cibuildwheel/resources/constraints.in '
f'--output-file cibuildwheel/resources/constraints-python{python_version}.txt'
])
else:
for python_version in ['27', '35', '36', '37']:
subprocess.check_call([
f'./env{python_version}/bin/pip-compile',
'--allow-unsafe',
'--upgrade',
'cibuildwheel/resources/constraints.in',
'--output-file', f'cibuildwheel/resources/constraints-python{python_version}.txt'
])

Image = namedtuple('Image', [
'manylinux_version',
Expand Down