Target citests in Makefile?

[frankluebeck]

In recent pull requests I found it difficult to understand what the CI tests bound to this repository are doing and to understand the problem that caused a test to fail.

Could someone who knows please add a target citests (or some other name) to the Makefile such that it becomes easy to locally run these tests before submitting a pull request. That would simplify the change-and-test cycle a lot.

[fingolfin]

Something in this direction definitely would be good to have. But there are some fundamental limitations and questions on that

E.g. it couldn't possibly install 32bit versions of compilers and libraries for you...

Also, what scope do you have in mind? Do you want something that really emulates all the tests, including setting up 32/64 builds, HPC-GAP builds, builds with and without --enable-debug etc? That would be quite complex and likely require lots of debugging on its own.

More reasonable seems to me to allow running specific tests against whatever builds of GAP you happen to have around, including combinations that we don't test on Travis.

For that, I think the original idea was that one should be able to just call TESTSUITE=foobar etc/ci.sh to get exactly that. But this is hampered by various issues; e.g. that script insists on cloning and building the io and profiling packages. This code should probably be moved to etc/ci-prepare.sh

Also, the make targets like testinstall do additional work (testing with and without packages), which is why we don't use them on Travis. It sure would be good to unify them (e.g. I don't like that make testinstall does two tests, so I also bypass it and do gap tst/testinstall.g).

In the meantime, you can get a reasonable approximation via

gap tst/testtravis.g
gap tst/testbugfix.g
( cd tst/test-error && ./run_error_tests.sh )
make testmanuals

[olexandr-konovalov]

@fingolfin testing things locally, I also just do gap tst/testinstall.g often. But doing it in the way like make testinstall does is important for checking that no packages break it. I am fine if people are doing gap tst/testinstall.g more often, and the full make testinstall less often.