Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Orquesta workflow with a sub workflow with "with items" task gets stuck in the running state when empty array is passed for that task #4954

Closed
igcherkaev opened this issue May 19, 2020 · 2 comments · Fixed by #4959
Closed

Orquesta workflow with a sub workflow with "with items" task gets stuck in the running state when empty array is passed for that task #4954

igcherkaev opened this issue May 19, 2020 · 2 comments · Fixed by #4959
Labels
bug workflows: orquesta

Comments

@igcherkaev
Copy link

igcherkaev commented May 19, 2020 •
edited
Loading

SUMMARY

Orquesta workflow with a sub workflow with "with items" task gets stuck in the running state when empty array is passed for that task

STACKSTORM VERSION

Paste the output of st2 --version:

$ st2 --version
st2 3.2.0, on Python 2.7.5
OS, environment, install method

CentOS 7 install following the docs at: https://docs.stackstorm.com/install/rhel7.html

Steps to reproduce the problem

I have the following two workflows:

wf_orquesta_stuck1.meta.yaml:

---
pack: "playground"
name: "wf_orquesta_stuck1"
description: "Orquesta workflow gets stuck in running bug, st2 v3.2.0"
runner_type: orquesta
enabled: true
entry_point: "workflows/wf_orquesta_stuck1.yaml"
parameters:
  some_array:
    required: false
    default: []
    type: "array"
    description: "Some array."

workflows/wf_orquesta_stuck1.yaml:

---
version: '1.0'
input:
  - some_array
tasks:
  init_task:
    action: core.noop
    next:
      - when: <% succeeded() %>
        do:
          - subflow
  subflow:
    action: playground.wf_orquesta_stuck_sub
    input:
      some_array: <% ctx().some_array %>
    next:
      - when: <% succeeded() %>
        do:
          - finish

  finish:
    action: core.noop

And the sub workflow:

wf_orquesta_stuck_sub.meta.yaml:

---
pack: "playground"
name: "wf_orquesta_stuck_sub"
description: "Orquesta sub workflow that gets main workflow stuck in running, st2 v3.2.0"
runner_type: orquesta
enabled: true
entry_point: "workflows/wf_orquesta_stuck_sub.yaml"
parameters:
  some_array:
    required: false
    default: []
    type: "array"
    description: "Some array."

workflows/wf_orquesta_stuck_sub.yaml:

---
version: '1.0'
input:
  - some_array

tasks:
  process_some_array:
    with: <% ctx(some_array) %>
    action: core.local
    input:
      cmd: "echo <% item() %>"

Expected Results

The parent workflow to finish when empty array is passed.

Actual Results

The parent workflow gets stuck in the RUNNING state and never finishes.

image

workflow engine logs:

2020-05-19 12:52:23,605 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Processing request for workflow execution.
2020-05-19 12:52:23,616 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Requesting conductor to start running workflow execution.
2020-05-19 12:52:23,617 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Identifying next set (0) of tasks for workflow execution in status "running".
2020-05-19 12:52:23,619 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Identified the following set of tasks to execute next: init_task (route 0)
2020-05-19 12:52:23,619 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Mark task "init_task", route "0", in conductor as running.
2020-05-19 12:52:23,619 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Updating workflow execution from status "requested" to "running".
2020-05-19 12:52:23,654 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Requesting execution for task "init_task", route "0".
2020-05-19 12:52:23,655 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Processing task execution request for task "init_task", route "0".
2020-05-19 12:52:23,661 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Task execution "5ec438f729178af3686e28df" created for task "init_task", route "0".
2020-05-19 12:52:23,773 140150526377328 AUDIT action [-] Action execution requested. LiveAction.id=5ec438f729178af3686e28e0, ActionExecution.id=5ec438f729178af3686e28e1
2020-05-19 12:52:23,773 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Action execution "5ec438f729178af3686e28e1" requested for task "init_task", route "0".
2020-05-19 12:52:23,789 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Identifying next set (1) of tasks for workflow execution in status "running".
2020-05-19 12:52:23,790 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] No tasks identified to execute next.
2020-05-19 12:52:23,852 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Action execution "5ec438f729178af3686e28e1" for task "init_task" is updated and in "scheduled" state.
2020-05-19 12:52:23,909 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Action execution "5ec438f729178af3686e28e1" for task "init_task" is updated and in "running" state.
2020-05-19 12:52:23,967 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Action execution "5ec438f729178af3686e28e1" for task "init_task" is updated and in "succeeded" state.
2020-05-19 12:52:23,980 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Handling completion of action execution "5ec438f729178af3686e28e1" in status "succeeded" for task "init_task", route "0".
2020-05-19 12:52:24,013 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Publish task "init_task", route "0", with status "succeeded" to conductor.
2020-05-19 12:52:24,042 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Identifying next set (0) of tasks after completion of task "init_task", route "0".
2020-05-19 12:52:24,045 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Identified the following set of tasks to execute next: subflow (route 0)
2020-05-19 12:52:24,046 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Mark task "subflow", route "0", in conductor as running.
2020-05-19 12:52:24,085 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Requesting execution for task "subflow", route "0".
2020-05-19 12:52:24,086 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Processing task execution request for task "subflow", route "0".
2020-05-19 12:52:24,095 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Task execution "5ec438f829178af3686e28e2" created for task "subflow", route "0".
2020-05-19 12:52:24,213 140150526377328 AUDIT action [-] Action execution requested. LiveAction.id=5ec438f829178af3686e28e3, ActionExecution.id=5ec438f829178af3686e28e4
2020-05-19 12:52:24,213 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Action execution "5ec438f829178af3686e28e4" requested for task "subflow", route "0".
2020-05-19 12:52:24,230 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Identifying next set (1) of tasks for workflow execution in status "running".
2020-05-19 12:52:24,230 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] No tasks identified to execute next.
2020-05-19 12:52:24,386 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Action execution "5ec438f829178af3686e28e4" for task "subflow" is updated and in "scheduled" state.
2020-05-19 12:52:24,451 140150526377328 INFO workflows [-] [5ec438f76dcfffe45e24342b] Action execution "5ec438f829178af3686e28e4" for task "subflow" is updated and in "running" state.
2020-05-19 12:52:24,522 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Processing request for workflow execution.
2020-05-19 12:52:24,530 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Requesting conductor to start running workflow execution.
2020-05-19 12:52:24,530 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Identifying next set (0) of tasks for workflow execution in status "running".
2020-05-19 12:52:24,533 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Identified the following set of tasks to execute next: process_some_array (route 0)
2020-05-19 12:52:24,533 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Mark task "process_some_array", route "0", in conductor as running.
2020-05-19 12:52:24,534 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Updating workflow execution from status "requested" to "running".
2020-05-19 12:52:24,573 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Requesting execution for task "process_some_array", route "0".
2020-05-19 12:52:24,574 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Processing task execution request for task "process_some_array", route "0".
2020-05-19 12:52:24,581 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Task execution "5ec438f829178af3686e28e5" created for task "process_some_array", route "0".
2020-05-19 12:52:24,582 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Task "process_some_array", route "0", has no items and succeed by default.
2020-05-19 12:52:24,626 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Publish task "process_some_array", route "0", with status "succeeded" to conductor.
2020-05-19 12:52:24,627 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Updating workflow execution from status "running" to "succeeded".
2020-05-19 12:52:24,663 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Workflow action execution is completed and invoking post run.
2020-05-19 12:52:24,664 140150526377328 INFO utils [-] Invoking post run for action execution 5ec438f829178af3686e28e3.
2020-05-19 12:52:24,666 140150526377328 INFO utils [-] Action execution 5ec438f829178af3686e28e3 runs wf_orquesta_stuck_sub of runner type orquesta.
2020-05-19 12:52:24,684 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] Identifying next set (1) of tasks for workflow execution in status "succeeded".
2020-05-19 12:52:24,684 140150526377328 INFO workflows [-] [5ec438f829178af3686e28e4] No tasks identified to execute next.
@guzzijones
Copy link
Contributor

I can confirm I see this on some of my workflows too.
As a workaround I check the .length() of the array in a prior task with core.noop action. If the length is 0 then I skip the with_items task.

@emptywee
Copy link
Contributor

Yeah, as a temp workaround it's fine to add empty array tests, but it'd be nice for the engine to handle it properly (mistral did :P)

@m4dcoder m4dcoder mentioned this issue May 23, 2020
Merged
@igcherkaev igcherkaev mentioned this issue Jun 15, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug workflows: orquesta
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix empty with items list resulting in parent workflow stuck in running StackStorm/st2
4 participants
@arm4b @emptywee @guzzijones @igcherkaev