Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [master/task] work process with task 'sub_process' can't be stop when running #13940

Closed
2 of 3 tasks
albertzzas opened this issue Apr 17, 2023 · 5 comments
Closed
2 of 3 tasks
Labels
bug Something isn't working

Comments

@albertzzas
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

We use work process with 'sub_process' task to control the entry of every layer, such ods,ads.
If trigger the button 'STOP' when parent process is running, then work process is supposed to change to 'STOP'.
But it doesn't work in 3.1.5 , still hang with status 'PREPARE TO STOP' and child process is RUNNING.

Worker LOG is fine
Master LOG:

  • worker worker in work group default have not received the heartbeat
  • The task execution status code: 5 is invalidated

[ERROR] 2023-04-17 01:27:33.890 -0500 TaskLogLogger-class org.apache.dolphinscheduler.server.master.runner.task.SubTaskProcessor:[91] - [WorkflowInstance-22595][TaskInstance-246877] - work flow 22595 sub task 246877 exceptions
java.lang.IllegalArgumentException: The task execution status code: 5 is invalidated
at org.apache.dolphinscheduler.plugin.task.api.enums.TaskExecutionStatus.of(TaskExecutionStatus.java:60)
at org.apache.dolphinscheduler.server.master.runner.task.SubTaskProcessor.updateTaskState(SubTaskProcessor.java:133)
at org.apache.dolphinscheduler.server.master.runner.task.SubTaskProcessor.runTask(SubTaskProcessor.java:88)
at org.apache.dolphinscheduler.server.master.runner.task.BaseTaskProcessor.run(BaseTaskProcessor.java:237)
at org.apache.dolphinscheduler.server.master.runner.task.BaseTaskProcessor.action(BaseTaskProcessor.java:209)
at org.apache.dolphinscheduler.server.master.event.TaskStateEventHandler.handleStateEvent(TaskStateEventHandler.java:88)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnable.handleEvents(WorkflowExecuteRunnable.java:288)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[WARN] 2023-04-17 01:27:34.258 -0500 org.apache.dolphinscheduler.server.master.dispatch.host.LowerWeightHostManager:[132] - [WorkflowInstance-0][TaskInstance-0] - worker worker in work group default have not received the heartbeat
[WARN] 2023-04-17 01:27:34.258 -0500 org.apache.dolphinscheduler.server.master.dispatch.host.LowerWeightHostManager:[132] - [WorkflowInstance-0][TaskInstance-0] - worker worker in work group default have not received the heartbeat

What you expected to happen

if trigger the button 'STOP' when process is running, then work process is supposed to change to 'STOP'.
We use this function all the time via 1.3.6

How to reproduce

Test1:OK
Process Action: 'STOP'
Work process: doesn't use sub_process task
Task status:'KILL'
Result:'STOP'


Test2: Failed
Process Action: 'STOP'
Work process: use 'sub_process' task
Task status:'KILL'
Parent process result: 'PREPARE TO STOP' ,state code: 4
Child process result:'RUNNING' ,state code :1

Anything else

ENV and others
Version: git clone -b 3.1.5 https://github.com/apache/dolphinscheduler.git
Meta DB: Upgraded Mysql db via DS tools from 1.3.6->2.0.6->3.1.5
ZK version:<zookeeper.version>3.4.6</zookeeper.version>
Workgroup exists already , infor can be found in t_ds_task_instance.work_group

Version

3.1.x

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@albertzzas albertzzas added bug Something isn't working Waiting for reply Waiting for reply labels Apr 17, 2023
@github-actions
Copy link

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

  • In order for us to understand your request as soon as possible, please provide detailed information, version or pictures.
  • If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

@wqxs
Copy link
Contributor

wqxs commented Apr 24, 2023

image

@fuchanghai
Copy link
Member

fuchanghai commented Apr 24, 2023

@wqxs
Because the task instance of the subprocess task type does not have a stopped state
image

image

@fuchanghai
Copy link
Member

@ruanwenjun i will fix it

fuchanghai pushed a commit to fuchanghai/dolphinscheduler that referenced this issue Apr 26, 2023
fuchanghai pushed a commit to fuchanghai/dolphinscheduler that referenced this issue Apr 26, 2023
@zhongjiajie
Copy link
Member

close #14016

@zhongjiajie zhongjiajie removed the Waiting for reply Waiting for reply label May 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants