[ci] Move pip dependencies to docker images, add ninja / shellcheck #10257
Conversation
driazati
requested review from
comaniac,
junrushao,
tqchen,
jroesch,
areusch,
yzhliu,
merrymercy,
icemelon and
leandron
as code owners
driazati
force-pushed
the
pip_deps
branch
2 times, most recently
from
062fb5e to
0f4858c
Compare
driazati
changed the title
Following on from apache#10246, this moves the `pip install`-at-runtime deps to the docker image install so they are baked in. commit-id:5fe5152b
|
@driazati what's the plan for removing task_ci_setup.sh tho? |
once this lands and we update the docker images we can remove the
the version pins are the same so the installs in |
driazati
changed the title
|
i think maybe we should change task_ci_setup.sh to not install those packages if they already exist. i can't remember exactly what happened here, but there was a CI problem before where a disagreement between the pinned version of the package in task_ci_setup.sh and the pinned version in the docker/install scripts caused CI to fail. i think maybe this was due to not specifying -U in pip install? anyway, it would be great to ensure that either we use task_ci_setup.sh or we use new docker image. |
There was a problem hiding this comment.
ok, discussed offline, since we are migrating all of these to the ci image, the main thing we need to avoid the state where one version of synr is installed in ci- images and another in $TVM_HOME/.local. since we are moving all of them to ci installation, we can just rm -rf $TVM_HOME/.local in task_ci_setup.sh when we bump the docker image version. we can revert the docker image with [skip ci] to avoid breakage.
|
@Mousius can you take another look here and merge if you are good with it? |
There was a problem hiding this comment.
LGTM! Apologies for the delay!
Agree dependencies which are used as Python dependencies should come via pip. If they're not used as Python dependencies and there's a stable OS package available then we should favour the more robust OS way of delivering packages.
…pache#10257) Following on from apache#10246, this moves the `pip install`-at-runtime deps to the docker image install so they are baked in.
Following on from #10246, this moves the
pip install-at-runtime deps to the docker image install so they are baked in. This adds some extra dependencies to support #10425 and make it so we can easily run things in parallel in CI scripts via GNU parallel and lint our bash scripts.