Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[2DC] Setup replication between two universes #1977

Closed
ndeodhar opened this issue Aug 6, 2019 · 0 comments
Closed

[2DC] Setup replication between two universes #1977

ndeodhar opened this issue Aug 6, 2019 · 0 comments
Assignees
@ndeodhar
Labels
area/cdc Change Data Capture
Milestone
v2.0

Comments

@ndeodhar
Copy link
Contributor

ndeodhar commented Aug 6, 2019

Add API to setup and delete replication between two YB universes.
This should:

  1. Validate that table schemas match across the two universes.
  2. Setup CDC streams on producer tables.
  3. Register subscribers on consumer universe for each table.
@ndeodhar ndeodhar self-assigned this Aug 6, 2019
@ndeodhar ndeodhar added the area/cdc Change Data Capture label Aug 6, 2019
@ndeodhar ndeodhar added this to the v2.0 milestone Aug 6, 2019
ndeodhar added a commit that referenced this issue Aug 27, 2019
@ndeodhar
[#1977][2DC] Producer and consumer master handshake to setup 2DC
beb0b3d 
Summary:
This diff adds two APIs:
1. SetupUniverseReplication which will be run on consumer DC to setup the DC to consumer data from another YB universe.
2. DeleteUniverseReplication which will be run on consumer DC to delete replication.

Setting up YB replication involves the following steps:
1. Consumer universe master leader gets the table schemas from producer universe master leader and validates that schemas match.
2. Consumer master then sets up CDC streams on producer universe for all tables that are to be replicated.
3. Consumer master sets up CDC subscribers to consume data from producers.

Information about universe replication is maintained in master sys catalog.

Sample command to setup 2DC:
```
./build/latest/bin/yb-admin -master_addresses 127.0.0.11:7100,127.0.0.12:7100,127.0.0.13:7100 setup_universe_replication e260b8b6-e89f-4505-bb8e-b31f74aa29f3 127.0.0.1:7100,127.0.0.2:7100,127.0.0.3:7100 000030a5000030008000000000004000,000030a5000030008000000000004005
```

Test Plan:
```
ctest -R sys_catalog
ctest -R master_master-test
ctest -R twodc
ctest -R client_client-test
```

Reviewers: rahuldesirazu, sergei, bogdan

Reviewed By: bogdan

Subscribers: mikhail, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D7046
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ndeodhar ndeodhar

Labels
area/cdc Change Data Capture
Projects
None yet
Milestone
v2.0
Development

No branches or pull requests
1 participant
@ndeodhar