backend error in cypari2 when factoring: "RuntimeError: cannot resize PARI stack here"

[katestange]

This seems to occur on some setups and not others. It occurs inside the call to create a pari instance, during the setup of the stack. The relevant line in cypari2 is line 150 here https://github.com/sagemath/cypari2/blob/d1fd6ca09b9f6fafbf8b084613f88950a11e745e/cypari2/stack.pyx#L150

This is intermittent, but seems more likely to occur if you try lots of sequences quickly, particularly in the frontscope instead of just using the API in the URL bar (but that may be a misperception). It is not consistently related to any one sequence number, but occurs when factoring a new sequence for the database. Discussed in #67 .

Example:

127.0.0.1 - - [11/Apr/2023 16:41:09] "GET /api/get_oeis_factors/A000143/1000 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/.venv/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/flaskr/nscope/views.py", line 302, in get_oeis_factors
    seq = fetch_factors(valid_oeis_id, wants)
  File "/home/katestange/data/projects/numberscope/frontscope-backscope/backscope/flaskr/nscope/views.py", line 171, in fetch_factors
    pari = cypari2.Pari()
  File "cypari2/pari_instance.pyx", line 609, in cypari2.pari_instance.Pari.__init__
    
  File "cypari2/stack.pyx", line 157, in cypari2.stack.set_pari_stack_size
    
  File "cypari2/stack.pyx", line 150, in cypari2.stack.before_resize
    
RuntimeError: cannot resize PARI stack here

[katestange]

Here's some conversation with Luca de Feo that is very helpful:

This may be a "newbie" question, but when I make a cypari2 pari instance ( pari = cypari2.Pari() ), do I need to destroy it later for garbage collection reasons?

No. The Pari library has a bunch of global variables, the most important of which is the (in)famous stack. They are all allocated and initialized when you call cypari2.Pari() the first time. The Python object returned by the constructor is just a thin wrapper that manipulates the global PARI data. If you create two instances of cypari2.Pari() you'll have two Python objects that manipulate the same global PARI data.

When the Python variable falls out of scope, the Python object eventually gets destroyed by Python's garbage collector, but the global PARI data will stay there until the program is terminated. So, in a sense, you may say cypari2 leaks memory by design, although the PARI designers would disagree :) At least, PARI reclaims the unused memory from the stack.

I'm running into a lot of "RuntimeError: cannot resize PARI stack here" and I'm trying to debug, and I wonder if I'm creating a mess of pari instances.

Because of what I said above, PARI is not designed to be thread safe. libpari has its own thread API, but that is currently not exposed in cypari2, and it wouldn't be automatically compatible with Python's threads anyway.

Since you're calling PARI from from a web app, I suppose you're running into thread-safety issues. A quick fix would be to force Flask to run single-threaded, although that may cost you too much in terms of performance. See this issue: sagemath/sage#35051

Alternatively, you can try with sympy. Its integer factorization is probably slower than PARI's, but it may be enough for your purposes.

[katestange]

Some efficiency comparison:
https://dzone.com/articles/integer-factorization-software-parigp-mathematica