Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[yb-ctl] make remove_node destroy node data #3771

Open
jaki opened this issue Feb 28, 2020 · 2 comments
Open

[yb-ctl] make remove_node destroy node data #3771

jaki opened this issue Feb 28, 2020 · 2 comments
Assignees
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@jaki
Copy link
Contributor

jaki commented Feb 28, 2020

Jira Link: DB-10819
Currently, remove_node and stop_node commands do the same thing: stop the node. There is no way to kill a node. Add that ability, preferably by changing the behavior of the redundant remove_node command.

@jaki jaki added kind/enhancement This is an enhancement of an existing feature area/tools labels Feb 28, 2020
@jaki
Copy link
Contributor Author

jaki commented Feb 28, 2020

Actually, it seems remove_node does kill the process (along with --master flag), but status shows it still.

----------------------------------------------------------------------------------------------------
| Node 2: yb-tserver (Stopped), yb-master (Stopped)                                                |
----------------------------------------------------------------------------------------------------

@jaki jaki added the invalid label Feb 28, 2020
@jaki jaki closed this as completed Feb 28, 2020
@jaki
Copy link
Contributor Author

jaki commented Feb 28, 2020

Actually, I think that this is somewhat a somewhat reasonable issue:

  • yb-ctl stop stops all nodes without removing data
  • yb-ctl destroy stops all nodes and removes data

Similarly, we should have

  • yb-ctl stop_node stop the node without removing data
  • yb-ctl remove_node (or destroy_node) stop the node (both master and tserver) and remove the data

status should not show destroyed nodes (skip over them). UI pages should also not show these nodes as DEAD because they are gone for good, irrelevant. add_node should keep counting upwards, never going back to destroyed node numbers.

@jaki jaki reopened this Feb 28, 2020
@jaki jaki removed the invalid label Feb 28, 2020
@jaki jaki changed the title [yb-ctl] make remove_node kill the process [yb-ctl] make remove_node destroy node data Feb 28, 2020
nspiegelberg added a commit that referenced this issue Mar 16, 2020
Summary:
We had a couple data races in the CDC Poller.
1. Although the poller is called in a serial fashion, critical call stacks are placed upon
a ThreadPool.  This is not guaranteed to always enqueue on the same thread and can create a
situation where the ThreadPool is processing the call before a cache flush of the local
data has occured on the originating thread. Added a mutex to guarantee ordering.
2. A race existed within TwoDCOutputClient where failing call OpenTable would not properly reset the
OpID or ErrorStatus.  Modified code to call HandleError, which has standardized handling here.

Test Plan:
ybd tsan --cxx-test twodc-test --gtest_filter BatchSize/TwoDCTest.ApplyOperationsRandomFailures/1 -n
1000

Reviewers: rahuldesirazu, neha, hector

Reviewed By: neha, hector

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D8075
@rthallamko3 rthallamko3 assigned yusong-yan and unassigned bmatican and WesleyW Apr 11, 2024
@rthallamko3 rthallamko3 added the area/docdb YugabyteDB core features label Apr 11, 2024
@yugabyte-ci yugabyte-ci added priority/medium Medium priority issue and removed area/tools labels Apr 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Development

No branches or pull requests

6 participants