MaxRetriesFailureHandler does not work with OnFailureReschedule

[beilCrxmarkets]

Configuration

@Configuration
public class DbSchedulerConfiguration {

    @Bean
    RecurringTask<Void> staticRecurringTask() {
        CronSchedule schedule = cron("*/1 * * * * *"); // High frequency for testing purposes. Normally once a day. 

        return Tasks.recurring("Trigger job", schedule)
                .onFailure(new FailureHandler.MaxRetriesFailureHandler<>(5,
                        new FailureHandler.ExponentialBackoffFailureHandler<>(Duration.ofSeconds(3), 1),
                        new FailureHandler.OnFailureReschedule<Void>(schedule)::onFailure))
                .execute((instance, ctx) -> {
                    System.out.println("Running staticRecurringTask");
                    throw new RuntimeException("Error");
                });
    }
}

Expected Behavior

"Trigger job" is rescheduled again after the 5 retry attempts have failed.

Current Behavior

"Trigger job" has been removed from the DB by calling executionOperations.stop(); See: https://github.com/kagkarlsson/db-scheduler/blob/master/db-scheduler/src/main/java/com/github/kagkarlsson/scheduler/task/FailureHandler.java#L96

Therefore the attempt to reschedule the task will fail.

Proposed solution

If a maxRetriesExceededHandler is defined, let it decide whether the task is stopped/removed.

Workaround

Define your own MaxRetriesFailureHandler which does not stop/remove the task by implementing the provided interface.

---

Context

• DB-Scheduler Version : 14.0.3
• Java Version : 21
• Spring Boot (check for Yes) : [X]
• Database and Version : PostgreSQL

Logs

Running staticRecurringTask
{"@timestamp":"2024-09-18T13:55:59.162+02:00","@Version":"1","message":"Unhandled exception RuntimeException: 'Error' during execution of task with name 'Trigger job'. Treating as failure.","logger_name":"com.github.kagkarlsson.scheduler.Scheduler","thread_name":"db-scheduler-pool-2-thread-1","level":"WARN","level_value":30000,"stack_trace":"java.lang.RuntimeException: Error\n\tat lu.crx.currencyservice.app.config.DbSchedulerConfiguration.lambda$staticRecurringTask$0(DbSchedulerConfiguration.java:26)\n\tat com.github.kagkarlsson.scheduler.task.helper.Tasks$RecurringTaskBuilder$2.executeRecurringly(Tasks.java:141)\n\tat com.github.kagkarlsson.scheduler.task.helper.RecurringTask.execute(RecurringTask.java:96)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.executePickedExecution(ExecutePicked.java:102)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.run(ExecutePicked.java:81)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$1(FetchCandidates.java:114)\n\tat java.base/java.util.Optional.ifPresent(Optional.java:178)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$2(FetchCandidates.java:100)\n\tat com.github.kagkarlsson.scheduler.Executor.lambda$addToQueue$0(Executor.java:53)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n","arg0":"Unhandled exception","arg1":"RuntimeException: 'Error'","arg2":"Trigger job"}

Running staticRecurringTask
{"@timestamp":"2024-09-18T13:56:09.155+02:00","@Version":"1","message":"Unhandled exception RuntimeException: 'Error' during execution of task with name 'Trigger job'. Treating as failure.","logger_name":"com.github.kagkarlsson.scheduler.Scheduler","thread_name":"db-scheduler-pool-2-thread-2","level":"WARN","level_value":30000,"stack_trace":"java.lang.RuntimeException: Error\n\tat lu.crx.currencyservice.app.config.DbSchedulerConfiguration.lambda$staticRecurringTask$0(DbSchedulerConfiguration.java:26)\n\tat com.github.kagkarlsson.scheduler.task.helper.Tasks$RecurringTaskBuilder$2.executeRecurringly(Tasks.java:141)\n\tat com.github.kagkarlsson.scheduler.task.helper.RecurringTask.execute(RecurringTask.java:96)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.executePickedExecution(ExecutePicked.java:102)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.run(ExecutePicked.java:81)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$1(FetchCandidates.java:114)\n\tat java.base/java.util.Optional.ifPresent(Optional.java:178)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$2(FetchCandidates.java:100)\n\tat com.github.kagkarlsson.scheduler.Executor.lambda$addToQueue$0(Executor.java:53)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n","arg0":"Unhandled exception","arg1":"RuntimeException: 'Error'","arg2":"Trigger job"}

Running staticRecurringTask
{"@timestamp":"2024-09-18T13:56:19.167+02:00","@Version":"1","message":"Unhandled exception RuntimeException: 'Error' during execution of task with name 'Trigger job'. Treating as failure.","logger_name":"com.github.kagkarlsson.scheduler.Scheduler","thread_name":"db-scheduler-pool-2-thread-3","level":"WARN","level_value":30000,"stack_trace":"java.lang.RuntimeException: Error\n\tat lu.crx.currencyservice.app.config.DbSchedulerConfiguration.lambda$staticRecurringTask$0(DbSchedulerConfiguration.java:26)\n\tat com.github.kagkarlsson.scheduler.task.helper.Tasks$RecurringTaskBuilder$2.executeRecurringly(Tasks.java:141)\n\tat com.github.kagkarlsson.scheduler.task.helper.RecurringTask.execute(RecurringTask.java:96)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.executePickedExecution(ExecutePicked.java:102)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.run(ExecutePicked.java:81)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$1(FetchCandidates.java:114)\n\tat java.base/java.util.Optional.ifPresent(Optional.java:178)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$2(FetchCandidates.java:100)\n\tat com.github.kagkarlsson.scheduler.Executor.lambda$addToQueue$0(Executor.java:53)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n","arg0":"Unhandled exception","arg1":"RuntimeException: 'Error'","arg2":"Trigger job"}

Running staticRecurringTask
{"@timestamp":"2024-09-18T13:56:29.175+02:00","@Version":"1","message":"Unhandled exception RuntimeException: 'Error' during execution of task with name 'Trigger job'. Treating as failure.","logger_name":"com.github.kagkarlsson.scheduler.Scheduler","thread_name":"db-scheduler-pool-2-thread-4","level":"WARN","level_value":30000,"stack_trace":"java.lang.RuntimeException: Error\n\tat lu.crx.currencyservice.app.config.DbSchedulerConfiguration.lambda$staticRecurringTask$0(DbSchedulerConfiguration.java:26)\n\tat com.github.kagkarlsson.scheduler.task.helper.Tasks$RecurringTaskBuilder$2.executeRecurringly(Tasks.java:141)\n\tat com.github.kagkarlsson.scheduler.task.helper.RecurringTask.execute(RecurringTask.java:96)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.executePickedExecution(ExecutePicked.java:102)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.run(ExecutePicked.java:81)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$1(FetchCandidates.java:114)\n\tat java.base/java.util.Optional.ifPresent(Optional.java:178)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$2(FetchCandidates.java:100)\n\tat com.github.kagkarlsson.scheduler.Executor.lambda$addToQueue$0(Executor.java:53)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n","arg0":"Unhandled exception","arg1":"RuntimeException: 'Error'","arg2":"Trigger job"}

Running staticRecurringTask
{"@timestamp":"2024-09-18T13:56:39.183+02:00","@Version":"1","message":"Unhandled exception RuntimeException: 'Error' during execution of task with name 'Trigger job'. Treating as failure.","logger_name":"com.github.kagkarlsson.scheduler.Scheduler","thread_name":"db-scheduler-pool-2-thread-5","level":"WARN","level_value":30000,"stack_trace":"java.lang.RuntimeException: Error\n\tat lu.crx.currencyservice.app.config.DbSchedulerConfiguration.lambda$staticRecurringTask$0(DbSchedulerConfiguration.java:26)\n\tat com.github.kagkarlsson.scheduler.task.helper.Tasks$RecurringTaskBuilder$2.executeRecurringly(Tasks.java:141)\n\tat com.github.kagkarlsson.scheduler.task.helper.RecurringTask.execute(RecurringTask.java:96)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.executePickedExecution(ExecutePicked.java:102)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.run(ExecutePicked.java:81)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$1(FetchCandidates.java:114)\n\tat java.base/java.util.Optional.ifPresent(Optional.java:178)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$2(FetchCandidates.java:100)\n\tat com.github.kagkarlsson.scheduler.Executor.lambda$addToQueue$0(Executor.java:53)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n","arg0":"Unhandled exception","arg1":"RuntimeException: 'Error'","arg2":"Trigger job"}

Running staticRecurringTask
{"@timestamp":"2024-09-18T13:56:49.184+02:00","@Version":"1","message":"Unhandled exception RuntimeException: 'Error' during execution of task with name 'Trigger job'. Treating as failure.","logger_name":"com.github.kagkarlsson.scheduler.Scheduler","thread_name":"db-scheduler-pool-2-thread-6","level":"WARN","level_value":30000,"stack_trace":"java.lang.RuntimeException: Error\n\tat lu.crx.currencyservice.app.config.DbSchedulerConfiguration.lambda$staticRecurringTask$0(DbSchedulerConfiguration.java:26)\n\tat com.github.kagkarlsson.scheduler.task.helper.Tasks$RecurringTaskBuilder$2.executeRecurringly(Tasks.java:141)\n\tat com.github.kagkarlsson.scheduler.task.helper.RecurringTask.execute(RecurringTask.java:96)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.executePickedExecution(ExecutePicked.java:102)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.run(ExecutePicked.java:81)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$1(FetchCandidates.java:114)\n\tat java.base/java.util.Optional.ifPresent(Optional.java:178)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$2(FetchCandidates.java:100)\n\tat com.github.kagkarlsson.scheduler.Executor.lambda$addToQueue$0(Executor.java:53)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n","arg0":"Unhandled exception","arg1":"RuntimeException: 'Error'","arg2":"Trigger job"}

{"@timestamp":"2024-09-18T13:56:49.184+02:00","@Version":"1","message":"Execution has failed 6 times for task instance TaskInstance: task=Trigger job, id=recurring. Cancelling execution.","logger_name":"com.github.kagkarlsson.scheduler.task.FailureHandler$MaxRetriesFailureHandler","thread_name":"db-scheduler-pool-2-thread-6","level":"ERROR","level_value":40000,"arg0":"6","arg1":"TaskInstance: task=Trigger job, id=recurring"}

{"@timestamp":"2024-09-18T13:56:49.188+02:00","@Version":"1","message":"Failed while completing execution Execution: task=Trigger job, id=recurring, executionTime=2024-09-18T11:56:42.184005Z, picked=true, pickedBy=Benedikts-MacBook-Pro.local, lastHeartbeat=2024-09-18T11:56:49.181421Z, version=12, because RuntimeException: 'Error'. Execution will likely remain scheduled and locked/picked. The execution should be detected as dead after a while, and handled according to the tasks DeadExecutionHandler.","logger_name":"com.github.kagkarlsson.scheduler.ExecutePicked","thread_name":"db-scheduler-pool-2-thread-6","level":"ERROR","level_value":40000,"stack_trace":"com.github.kagkarlsson.scheduler.exceptions.ExecutionException: Expected one execution to be updated, but updated 0. Indicates a bug. (task name: Trigger job, instance id: recurring)\n\tat com.github.kagkarlsson.scheduler.jdbc.JdbcTaskRepository.rescheduleInternal(JdbcTaskRepository.java:497)\n\tat com.github.kagkarlsson.scheduler.jdbc.JdbcTaskRepository.reschedule(JdbcTaskRepository.java:432)\n\tat com.github.kagkarlsson.scheduler.task.ExecutionOperations.reschedule(ExecutionOperations.java:56)\n\tat com.github.kagkarlsson.scheduler.task.FailureHandler$OnFailureReschedule.onFailure(FailureHandler.java:142)\n\tat com.github.kagkarlsson.scheduler.task.FailureHandler$MaxRetriesFailureHandler.onFailure(FailureHandler.java:97)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.failure(ExecutePicked.java:155)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.executePickedExecution(ExecutePicked.java:112)\n\tat com.github.kagkarlsson.scheduler.ExecutePicked.run(ExecutePicked.java:81)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$1(FetchCandidates.java:114)\n\tat java.base/java.util.Optional.ifPresent(Optional.java:178)\n\tat com.github.kagkarlsson.scheduler.FetchCandidates.lambda$run$2(FetchCandidates.java:100)\n\tat com.github.kagkarlsson.scheduler.Executor.lambda$addToQueue$0(Executor.java:53)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n","arg0":"Execution: task=Trigger job, id=recurring, executionTime=2024-09-18T11:56:42.184005Z, picked=true, pickedBy=Benedikts-MacBook-Pro.local, lastHeartbeat=2024-09-18T11:56:49.181421Z, version=12","arg1":"RuntimeException: 'Error'"}