fix(controller): delay deletion of failed job now works

[jialeicui]

Description

Modules

• UI
• Controller
• Agent
• Client
• Python-SDK
• Others

Checklist

• run code format and lint check
• add unit test
• add necessary doc

[codecov]

Codecov Report

Merging #1285 (1baf084) into main (10a9204) will decrease coverage by 12.99%.
The diff coverage is 33.33%.

@@              Coverage Diff              @@
##               main    #1285       +/-   ##
=============================================
- Coverage     73.87%   60.87%   -13.00%     
- Complexity     1212     1213        +1     
=============================================
  Files           287      225       -62     
  Lines         13392     6364     -7028     
  Branches        651      651               
=============================================
- Hits           9893     3874     -6019     
+ Misses         3277     2268     -1009     
  Partials        222      222               

Flag	Coverage Δ
controller	60.87% <33.33%> (-0.01%)	⬇️
standalone	?
unittests	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...n/task/status/watchers/TaskWatcherForSchedule.java	40.00% <0.00%> (-2.11%)	⬇️
...rwhale/mlops/domain/task/status/WatchableTask.java	85.36% <100.00%> (+2.43%)	⬆️
client/starwhale/core/eval/model.py
client/starwhale/utils/log.py
client/starwhale/core/runtime/view.py
client/starwhale/utils/process.py
client/starwhale/core/model/model.py
client/starwhale/api/_impl/dataset/builder.py
client/starwhale/base/cloud.py
client/starwhale/core/dataset/model.py
... and 55 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[anda-ren]

lgtm