Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add calls to TestPackageAvailability in bugfix tests that load packages #2810

Merged
merged 1 commit into from
Sep 19, 2018

Conversation

wilfwilson
Copy link
Member

Background: the purpose of this PR is to resolve the initial problem that I encountered in #2809. Basically: I want to run tst/testbugfix.g in the Semigroups CI setup, as an extra check that Semigroups doesn't break things in GAP. But I can't be bothered installing the necessary packages that some of the tests use.

However, if I don't install these packages, then I get diffs because of the way that LoadPackage produces Info statements when it returns fail. This causes the tests to fail. See #2809 for more discussion about this topic. I think that the behaviour of LoadPackage with respect to Info statements is actually a bug, as I discuss on issue #2809. This PR doesn't resolve that 'bug', but I will attempt to fix that bug in a separate PR if someone agrees with me that there is a bug.

This is an example of a diff that arises when you don't have a package installed:

testing: /home/travis/gap/tst/testbugfix/2012-06-18-t00328.tst
########> Diff in /home/travis/gap/tst/testbugfix/2012-06-18-t00328.tst:2
# Input is:
if LoadPackage("anupq",false) <> fail then
for i in [1..192] do Q:=Pq( FreeGroup(2) : Prime:=3, ClassBound:=1 ); od; fi;
# Expected output:
# But found:
#I  anupq package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
########

This PR avoids these problems by replacing all occurrences if LoadPackage("blah") <> fail then in testbugfix test files with if TestPackageAvailability("blah") <> fail and LoadPackage("blah") <> fail then, because TestPackageAvailability prints no unexpected output when it returns fail. Therefore the test files 'pass' cleanly when the relevant package is not able to be loaded.

There is one exception: tst/testbugfix/2005-12-08-t00323.tst. Please look at the relevant commit. This is not formulated with an if statement, and just blindly loads the package. The test involves printing a table to the screen. Therefore, for the test file to pass without the package loaded, I add the if statement as before, and when the condition is not met, I manually print the expected output to screen. It feels weird to do so, but it seems to be the only way to ensure that testbugfix.g still passes when you don't have additional packages installed.

The obvious problem with my changes is that, if one of these packages stops loading for some reason, then this won't be captured by the testbugfix test files. If this happens and we don't notice that packages are failing to load, then we won't notice a potential regression of the relevant tests.

I'd like people's opinions on whether it is okay to do what I have done. I won't be offended by your comments: I made this PR at the suggestion of @alex-konovalov so it's not my idea anyway; I don't mind finding a different solution to my problem if others think that's best. Perhaps it would be better to move the bugfix files that require a package into a new collection? Perhaps we should ensure that such tests are also present in the relevant packages? Perhaps I should just get over it and install the relevant packages in my Travis setup...?

@wilfwilson wilfwilson added kind: request for comments topic: tests issues or PRs related to tests release notes: not needed PRs introducing changes that are wholly irrelevant to the release notes labels Sep 13, 2018
@fingolfin
Copy link
Member

See also PR #1633 and the #@skipif extension to the .tst format it implements -- that could be applied to these tests, too, I think (but that doesn't mean we shouldn't merge this PR here as-is, just thought I'd bring it up for completeness.)

@@ -1,4 +1,5 @@
## bug 2 for fix 6
gap> if LoadPackage("tomlib", false) <> fail then
gap> if TestPackageAvailability("tomlib") <> fail and
> LoadPackage("tomlib", false) <> fail then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are using this a lot, perhaps it'd be better to add an option to LoadPackage to suppress warnings, and/or have a LoadPackageIfAvailable function? (That's not a change request for this PR, just thinking about how we could possibly refactor things in the future to make it easier to write this kind of logic)

@@ -1,3 +1,4 @@
# 2012/06/18 (MH)
gap> if LoadPackage("anupq",false) <> fail then
gap> if TestPackageAvailability("anupq") <> fail and
> LoadPackage("anupq",false) <> fail then
> for i in [1..192] do Q:=Pq( FreeGroup(2) : Prime:=3, ClassBound:=1 ); od; fi;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is about verifying that launch a couple hundred subprocesses via InputOutputLocalProcess works.

> LoadPackage("cvec",false) <> fail then
> mat := [[Z(2)]];
> ConvertToMatrixRep(mat,2); cmat := CMat(mat); cmat := cmat^1000;
> fi;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This goes back to this commit (only available in my "secret" git history):

commit 614132d823d43e6f036e6352a54015b0889e77a7
Author: Frank Lübeck <frank.luebeck@math.rwth-aachen.de>
Date:   2012-06-18 12:43:23 +0000

    Avoid ConvertToMatrixRep and use Matrix. Now POW_MAT_INT also works
    for matrices in IsCMatRep.
    FL

diff --git a/lib/matrix.gi b/lib/matrix.gi
index 7ebde84e4..3db395796 100644
--- a/lib/matrix.gi
+++ b/lib/matrix.gi
@@ -4168,7 +4168,7 @@ BindGlobal("POW_MAT_INT", function(mat, n)
         until r <> true;
       fi;
     od;
-    ConvertToMatrixRep(t);
+    t := Matrix(t, m);
     return t;
   end;
   # compared to standard method, we avoid some zero or identity matrices

Copy link
Member

@fingolfin fingolfin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I just noticed that this PR is targeting stable-4.10. That means if merged, it would not go into master. Is that really what you want? But then the problem would persist in master resp. 4.11, would it not

So I think this should be rebased on master, and the backport-to-4.10 be added.

And of course it'd be good if other GAP team members had a look at this, too...

@wilfwilson
Copy link
Member Author

Well-spotted @fingolfin. I will sort this out tomorrow at GAP Days

@wilfwilson wilfwilson changed the base branch from stable-4.10 to master September 17, 2018 09:26
@wilfwilson
Copy link
Member Author

@fingolfin How does the backporting work? I've rebased on master and added the label backport-to-4.10; if/when this PR is merged into master, will this then be cherry-picked into stable-4.10? Or do I then open another PR against stable-4.10?

@olexandr-konovalov
Copy link
Member

olexandr-konovalov commented Sep 17, 2018

@wilfwilson

when this PR is merged into master, will this then be cherry-picked into stable-4.10

yes, exactly - by one of the people who have appropriate access and who will then change the label from backport-to-4.10 to backport-to-4.10-DONE. No need for you to open a second PR.

@wilfwilson
Copy link
Member Author

Thanks @alex-konovalov

@fingolfin
Copy link
Member

Backported to stable-4.10 in da66cbc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-to-4.10-DONE release notes: not needed PRs introducing changes that are wholly irrelevant to the release notes topic: tests issues or PRs related to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants