GitHub actions

[bernt-matthias]

Hi @peterjc it's about time to switch this repo to github actions :) .. fixes #140

I tested the PR workflow in a mock PR to my fork: bernt-matthias#2

The CI workflow has been tested using the slash command bernt-matthias#1 .. you can see the result listed here https://github.com/bernt-matthias/galaxy_blast/actions

[peterjc]

Per-tool test data has become the norm then, fair enough. We might be able to use separate folders, I forget now how much reuse there was between tools. The symlinks used here is pragmatic.

[peterjc]

Is there any reason to use setup.cfg over .flake8 here, other than following the IUC example repository?
https://flake8.pycqa.org/en/latest/user/configuration.html

[peterjc]

I'll try to look at this properly during the week, Tuesday at the earliest. Thank you!

I had very quick playing with the flake8 changes locally, there are a bunch of things flake8-import-order does not like - we'd best disable that initially or fix them first...

Also it looks like it would be easy to extend the initial actions to include all the flake8 plugins I was using on Travis,

pip install flake8 flake8-blind-except flake8-docstrings flake8-rst-docstrings

And while I'm sure it should be simple to add black too (you may know how), but just installing flake8-black would be even simpler in terms of configuration.

[bernt-matthias]

Is there any reason to use setup.cfg over .flake8 here, other than following the IUC example repository?

No :)

I had very quick playing with the flake8 changes locally, there are a bunch of things flake8-import-order does not like - we'd best disable that initially or fix them first...

The good thing is that python linting (like all linting jobs) only run for PRs. So this should nt create any noise and may be fixed step by step with the PRs that come in .. but might be an annoyance for potential contributors.

Also it looks like it would be easy to extend the initial actions to include all the flake8 plugins I was using on Travis,
pip install flake8 flake8-blind-except flake8-docstrings flake8-rst-docstrings
And while I'm sure it should be simple to add black too (you may know how), but just installing flake8-black would be even simpler in terms of configuration.

Should be easy. I guess this might be also an addition that we can play back to the IUC.

[bernt-matthias]

Should be easy. I guess this might be also an addition that we can play back to the IUC.

How about having something like a .flake_requirements file that allows to specify flake extensions. Should then be easy to add this here

galaxy_blast/.github/workflows/pr.yaml

Line 150 in 856062d

run: pip install flake8 flake8-import-order

[peterjc]

The .flake_requirements file idea would work (or flake8_requirements?), using the requirements.txt style it can include version qualifiers which can be very important (e.g. linters often add new rules).

I was initially thinking Galaxy tool repositories currently just copy the https://github.com/galaxyproject/galaxy-tool-repository-template template github action configuration, and then extend bits like adding to the pip install flake8 flake8-import-order line in their copy. But that breaks keeping the local CI setup up to date by copying the GitHub Actions YAML files from https://github.com/galaxyproject/galaxy-tool-repository-template periodically.

Would it be a big job to create a dedicated GitHub action for linting/testing Galaxy tools? I have no idea how much work that would be...

[peterjc]

Merged, thank you!

Watching the first run now... https://github.com/peterjc/galaxy_blast/actions/runs/3629223291/jobs/6121165190

[bernt-matthias]

Did you define the secrets TS_API_KEY and TTS_API_KEY?

Would it be a big job to create a dedicated GitHub action for linting/testing Galaxy tools? I have no idea how much work that would be...

This is (kind of) what galaxyproject/tools-iuc#4830 will do. It will make the IUC workflows callable. Then other repos can reuse the IUC workflows.

[bernt-matthias]

To use the slash command you need to setup a personal access token. A bit of docs is here: https://github.com/bernt-matthias/galaxy-tool-repository-template#setup

[peterjc]

No keys set yet, but it isn't getting that far anyway:

+Linting tool /home/runner/work/galaxy_blast/galaxy_blast/tools/ncbi_blast_plus/tools/ncbi_blast_plus/ncbi_blastp_wrapper.xml
Failed linting
Applying linter tests... FAIL
.. ERROR: Test 6: Test param database value= not found in the inputs
.. CHECK: 6 test(s) found.

And:

Unsuccessful tests found, inspect the 'All tool test results' artifact for details.

Update: That says:

ncbi_deltablast_wrapper, exit_code: 2, stderr: 'num_threads' is currently ignored when 'subject' is specified.

i.e. A real issue to address in the wrapper, perhaps an NCBI change, perhaps something we missed with the loss of TravisCI.

[bernt-matthias]

No keys set yet

You need to do that before merging the first PR. Otherwise the updated/fixed tools won't be deployed .. and you probably do not want to deploy them manually.

I would also suggest not to push to master directly.

[peterjc]

Cross reference galaxyproject/galaxy-tool-repository-template#12 - automated deployment to the main tool shed certainly does argue against pushing to master.

I assume you'd expect pull requests to include tool version bumps?

[bernt-matthias]

I assume you'd expect pull requests to include tool version bumps?

Indeed. Currently there is unfortunately no automatic check for this. At the IUC we sometimes make exceptions to bumping basically only if only the <tests> are changed.

[peterjc]

Fixed the warning from deltablast, need to understand the actual error still:

ncbi_deltablast_wrapper, exit_code: 2, stderr: BLAST Database error: No alias or index file found for protein database [cdd_delta] in search path [/tmp/tmpbn2s4qv9/job_working_directory/000/40/working::]
.

I can reproduce this locally:

$ deltablast  -query four_human_proteins.fasta  -subject rhodopsin_proteins.fasta  -evalue 1e-08 -out /tmp/out.txt -outfmt 5   -matrix 'BLOSUM62'  -seg no    -parse_deflines
BLAST Database error: No alias or index file found for protein database [cdd_delta] in search path [/Users/xxx/repositories/galaxy_blast/test-data::]

$ deltablast -version
deltablast: 2.10.1+
 Package: blast 2.10.1, build Dec  7 2020 04:08:37

[peterjc]

Got it, this was previously being excluded via .tt_skip:

tools/ncbi_blast_plus/ncbi_psiblast_wrapper.xml
tools/ncbi_blast_plus/ncbi_deltablast_wrapper.xml
tools/reciprocal_best_hits/reciprocal_best_hits.xml
tools/blast2go/blast2go.xml