DAG hanging #3739
Comments
alexec
added
the
type/regression
|
@simster7 created a fix and released in v2.9.5. Can you please run the controlled with the environment variable |
|
I'll investigate this today |
|
Confirmed this as a bug, working on a fix |
|
Thank you @simster7 |
|
Note: this bug is a very unusual edge case. For this bug to happen all of these conditions must be satisfied:
|
|
just happened to run into the same bug, where my generator script is returning an empty json payload, here's the yaml I used for reproducing it: apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: reproduce-bug-
spec:
entrypoint: start
templates:
- name: start
dag:
tasks:
- name: first-step
template: first-step
- name: second-step
dependencies: [first-step]
template: second-step
arguments:
parameters:
- name: chunk
value: "{{item}}"
withParam: "{{tasks.first-step.outputs.result}}"
- name: third-step
dependencies: [second-step]
template: third-step
- name: first-step
script:
image: python:3
command: [bash]
source: |
set -e
python3 -c 'import os, json; print(json.dumps([]))'
resources:
requests:
cpu: 250m
- name: second-step
inputs:
parameters:
- name: chunk
script:
image: python:3
command: [bash]
source: |
set -e
echo hi {{$inputs.parameters.chunk}}
resources:
requests:
cpu: 100m
- name: third-step
script:
image: busybox
command: [sh]
source: |
echo fin
resources:
requests:
cpu: 100mI return empty json on purpose, as I have a finish maker detection step in my real workflow, I wanted to skip the whole DAG if the finish marker file is already there in place in cloud storage. And JSON result is required even if I guess maybe it makes sense to have a feature to skip the whole DAG without even providing JSON result. Anyway, just added a possible case people may want to return empty JSON on purpose on top of it. |
|
I'm using V2.11.7 and running into the same issue again. |
|
Can you please try |
|
@tarunpantqb I just ran the sample workflow you posted with no issues. I also see that it's one of our examples, which means that if there was a bug that prevented it from running successfully, our testing infrastructure would have caught it already. Lastly, as you can see in #3739 (comment), this issue is only relevant for DAGs that contain a specific edge case when using Can you please ensure that you don't have an environment or similar issue that's preventing you from running workflows? If you are sure that it's not an issue on your end, feel free to open a new issue with a detailed bug report. |
Summary
What happened/what you expected to happen?
Workflow DAG step hangs when the DAG has a task for scheduled jobs to run, but there are no jobs scheduled, and a subsequent task depends on that task.
Please note that this is different than issue: Dag workflow will stay running forever in some long run situation. which was previously closed. I tested the examples in that issue and they still work as expected in version 2.9.5.
This issue is reproducible in v2.9.5, but works as expected in v2.8.2.
example_dag_workflow.yaml
Diagnostics
What version of Argo Workflows are you running? v2.9.5
Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.
The text was updated successfully, but these errors were encountered: