Custom scripts with job_timeout more than RQ_DEFAULT_TIMEOUT failed when rescheduled

[miaow2]

Deployment Type

Self-hosted

Triage priority

I volunteer to perform this work (if approved)

NetBox Version

v4.1.3

Python Version

3.12

Steps to Reproduce

1. Create script with job_timeout more than RQ_DEFAULT_TIMEOUT, for example, the default timeout is 300 secs, setting job_timeout to 500 and inside script only sleep for 400 secs

from time import sleep

from extras.scripts import Script


class TestScript(Script):
    class Meta:
        job_timeout = 500

    def run(self, data, commit):
        sleep(400)

2. Run script with interval, for example 3 minutes

Expected Behavior

Script runs successfully and all scheduled after scripts are successful too

Observed Behavior

First running will be successful, but after that all scheduled jobs will fail with JobTimeoutException

[DJANGO] ERROR 2024-10-15 11:45:28,255 scripts 160778 140373397688448 An exception occurred: `JobTimeoutException: Task exceeded maximum timeout value (300 seconds)`

Traceback (most recent call last):
  File "/home/miaow/work/netbox/netbox/extras/jobs.py", line 45, in run_script
    script.output = script.run(data, commit)
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miaow/work/netbox/netbox/scripts/test_script.py", line 11, in run
    sleep(400)
  File "/home/miaow/work/netbox/venv/lib/python3.12/site-packages/rq/timeouts.py", line 63, in handle_death_penalty
    raise self._exception('Task exceeded maximum timeout value ({0} seconds)'.format(self._timeout))
rq.timeouts.JobTimeoutException: Task exceeded maximum timeout value (300 seconds)

This happens because ScriptView passes job_timeout into ScriptJob,

but rescheduling mechanism does not pass this argument into enqueue method

[arthanson]

@miaow2 it looks like you know where this is, do you want to work on a fix for it?

[miaow2]

@arthanson yeah, I want to work on this fix

[jeremystretch]

Thanks @miaow2, I've assigned this to you.