Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reject GAP function definitions containing huge list or record expressions (millions of entries) instead of silently discarding part of the data #5509

Merged
merged 1 commit into from
Oct 11, 2023

Conversation

ChrisJefferson
Copy link
Contributor

Fixes #5508 (GAP produces incorrect code when there are very large arrays inside functions). GAP will silently fail whenever there is a function with 2097152 or more elements given explicitly, inside a function (note a list must be this long, a list of lists whose sublists contain this many elements would be fine).

We could also look at increasing how much allowance we give, but I'm tempted to put this commit in as is (unless there are any small fixes / suggestions), because then if anyone has code that has hit this bug in the past, they will get a clear error message that an error has occurred (in fact, perhaps I should even extend the error to say "this code may have given incorrect answers in previous versions of GAP"?)

src/code.c Outdated Show resolved Hide resolved
@@ -0,0 +1,3 @@
strlist := String(List([1..2097152], x -> x));;
func := Concatenation("lf := function() return ", strlist, "; end;");;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this in testspecial and not part of the .tst file above?

gap> strlist := String(List([1..2097152], x -> x));;
gap> func := Concatenation("lf := function() return ", strlist, "; end;");;
gap> Read(InputTextString(func));;
Error, function too large for parser

should work fine there, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason this was here is previously when this PRed "panic"ed, I couldn't do that in a normal test. Now it's an error, I can (I also added a record test as well. I don't know if anyone would ever create such a huge record in code, but it doesn't hurt to catch it!)

Copy link
Member

@fingolfin fingolfin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to say, despite my nitpicking, this is of course a good PR fixing a real issue in a sensible way -- thank you :-)

Error, function too large for parser

# Test functions with very large records
gap> r := rec();; for x in [1..2097150/2] do r.(x) := x; od;;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running the above code changes the behavior of GAP in a way that breaks another test later in the test suite. In a nutshell:

gap> r:=rec();
rec(  )
gap> \.(r, 1000000);
Error, Record Element: <rnam> must be a valid rnam (not the integer 1000000)
not in any function at *stdin*:2
type 'quit;' to quit to outer loop
brk>
gap> r2 := rec();; for x in [1..2097150/2] do r2.(x) := x; od;;
gap> \.(r, 1000000);
Error, Record Element: '<rec>.992517' must have an assigned value
not in any function at *stdin*:5
type 'quit;' to quit to outer loop

Possible fix: modify tst/testinstall/recordname.tst to use, say MAX_SIZE_LIST_INTERNAL instead of 1000000, should fix this...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've adjusted this test now, let's see if that is enough.

@fingolfin fingolfin force-pushed the HugeFunction branch 2 times, most recently from bdb4f57 to 697f58f Compare September 29, 2023 17:49
src/code.c Show resolved Hide resolved
@ChrisJefferson ChrisJefferson force-pushed the HugeFunction branch 3 times, most recently from 13f6621 to 3a68bbe Compare October 9, 2023 11:13
@ChrisJefferson
Copy link
Contributor Author

Just wanted to point out this is now updated (just fixed tests for 32-bit GAP).

@fingolfin fingolfin merged commit a8bd423 into gap-system:master Oct 11, 2023
22 checks passed
@fingolfin fingolfin added kind: bug Issues describing general bugs, and PRs fixing them topic: kernel labels Oct 11, 2023
@fingolfin fingolfin changed the title Catch overflow in header size Error out when trying to parse GAP functions with hug list or record expressions (millions of entries) instead of silently discarding part of the data Oct 11, 2023
@fingolfin fingolfin added the kind: bug: wrong result Issues describing bugs that result in mathematically or otherwise wrong results, and PRs fixing them label Oct 11, 2023
@fingolfin fingolfin changed the title Error out when trying to parse GAP functions with hug list or record expressions (millions of entries) instead of silently discarding part of the data Reject GAP function definitions containing huge list or record expressions (millions of entries) instead of silently discarding part of the data Oct 11, 2023
@ChrisJefferson ChrisJefferson deleted the HugeFunction branch December 18, 2023 07:10
@fingolfin fingolfin added the release notes: use title For PRs: the title of this PR is suitable for direct use in the release notes label Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: bug: wrong result Issues describing bugs that result in mathematically or otherwise wrong results, and PRs fixing them kind: bug Issues describing general bugs, and PRs fixing them release notes: use title For PRs: the title of this PR is suitable for direct use in the release notes topic: kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Very large arrays in functions are parsed incorrectly
2 participants