Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Github Actions x86 and mac jobs to build python wheels #3024

Merged
merged 5 commits into from
Jan 15, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions .github/workflows/build-wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: Build wheels

on:
push:
branches: [ develop ]
pull_request:
branches: [ develop ]
schedule:
- cron: '0 0 * * 0,3' # 2/weekly
mpenkov marked this conversation as resolved.
Show resolved Hide resolved

jobs:
build:
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
python-version: [3.6, 3.7, 3.8]
os: [ubuntu-latest, macos-latest]
platform: [x64]
include:
- os: ubuntu-latest
python-version: 3.7
skip-nw-test: 1
mpenkov marked this conversation as resolved.
Show resolved Hide resolved
- os: ubuntu-latest
python-version: 3.8
skip-nw-test: 1
mpenkov marked this conversation as resolved.
Show resolved Hide resolved
- os: macos-latest
os-name: osx
mpenkov marked this conversation as resolved.
Show resolved Hide resolved
skip-nw-test: 1
mpenkov marked this conversation as resolved.
Show resolved Hide resolved
env:
PKG_NAME: gensim
REPO_DIR: gensim
BUILD_COMMIT: HEAD
PLAT: x86_64
UNICODE_WIDTH: 32
MB_PYTHON_VERSION: ${{ matrix.python-version }} # MB_PYTHON_VERSION is needed by Multibuild
TEST_DEPENDS: Morfessor==2.0.2a4 python-levenshtein==0.12.0 visdom==0.1.8.9 pytest mock cython nmslib pyemd testfixtures scikit-learn pyemd
DOCKER_TEST_IMAGE: multibuild/xenial_x86_64
TRAVIS_OS_NAME: ${{ matrix.os-name }}
mpenkov marked this conversation as resolved.
Show resolved Hide resolved
SKIP_NETWORK_TESTS: ${{ matrix.skip-nw-test }}
mpenkov marked this conversation as resolved.
Show resolved Hide resolved

steps:
- uses: actions/checkout@v2
with:
submodules: recursive
fetch-depth: 0
- name: Print some Environment variable
mpenkov marked this conversation as resolved.
Show resolved Hide resolved
run: |
echo "PLAT: ${PLAT}"
echo "DOCKER_TEST_IMAGE: ${DOCKER_TEST_IMAGE}"
echo "TEST_DEPENDS: ${TEST_DEPENDS}"
echo "TRAVIS_OS_NAME: ${TRAVIS_OS_NAME}"
echo "SKIP_NETWORK_TESTS: ${SKIP_NETWORK_TESTS}"
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install virtualenv
- name: Build and Install Wheels
run: |
echo ::group::Set up Multibuild
source multibuild/common_utils.sh
source multibuild/travis_steps.sh
source config.sh
echo ::endgroup::
echo ::group::Before install
before_install
echo ::endgroup::
echo ::group::Build wheel
build_wheel $REPO_DIR ${{ matrix.PLAT }}
echo ::endgroup::
echo ::group::Install run
install_run ${{ matrix.PLAT }}
echo ::endgroup::
- name: Upload wheels
mpenkov marked this conversation as resolved.
Show resolved Hide resolved
if: always()
run: |
pip install wheelhouse-uploader
ls wheelhouse/*.whl
python -m wheelhouse_uploader upload --local-folder wheelhouse/ --no-ssl-check gensim-wheels --provider S3 --no-enable-cdn
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "multibuild"]
path = multibuild
url = https://github.com/matthew-brett/multibuild.git
27 changes: 27 additions & 0 deletions config.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Define custom utilities

function pre_build {
# Any stuff that you need to do before you start building the wheels
# Runs in the root directory of this repository.
:
}

function build_wheel_cmd {
local cmd=${1:-pip_wheel_cmd}
local wheelhouse=$(abspath ${WHEEL_SDIR:-wheelhouse})
start_spinner
if [ -n "$(is_function "pre_build")" ]; then pre_build; fi
stop_spinner
if [ -n "$BUILD_DEPENDS" ]; then
pip install $(pip_opts) $BUILD_DEPENDS
fi
$cmd $wheelhouse
repair_wheelhouse $wheelhouse
}

function run_tests {
# Runs tests on installed distribution from an empty directory
python --version
pip freeze
pytest -rfxEXs --durations=20 --disable-warnings --showlocals --pyargs gensim
}
1 change: 1 addition & 0 deletions multibuild
Submodule multibuild added at a01ddf