Skip to content

Commit

Permalink
pperm: fix bug in PreImagePPermInt
Browse files Browse the repository at this point in the history
This occasionally caused incorrect values to be returned when i == deg
and by coincidence ptf2[i] == cpt. When ptf2[deg] is beyond the end of
the bag containing f, and so is not valid. For reference, this bug
caused errors in:

semigroups/Semigroups#296
semigroups/Semigroups#472
  • Loading branch information
James Mitchell authored and fingolfin committed Apr 6, 2018
1 parent 1b6e030 commit dd60f28
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions src/pperm.c
Original file line number Diff line number Diff line change
Expand Up @@ -316,17 +316,17 @@ static Obj PreImagePPermInt(Obj pt, Obj f)
if (TNUM_OBJ(f) == T_PPERM2) {
ptf2 = ADDR_PPERM2(f);
deg = DEG_PPERM2(f);
while (ptf2[i] != cpt && i < deg)
while (i < deg && ptf2[i] != cpt)
i++;
if (ptf2[i] != cpt)
if (i == deg || ptf2[i] != cpt)
return Fail;
}
else {
ptf4 = ADDR_PPERM4(f);
deg = DEG_PPERM4(f);
while (ptf4[i] != cpt && i < deg)
while (i < deg && ptf4[i] != cpt)
i++;
if (ptf4[i] != cpt)
if (i == deg || ptf4[i] != cpt)
return Fail;
}
return INTOBJ_INT(i + 1);
Expand Down

0 comments on commit dd60f28

Please sign in to comment.