Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incubating Project: TiDB over RoCE #630

Open
raygift opened this issue Jan 24, 2022 · 5 comments
Open

Incubating Project: TiDB over RoCE #630

raygift opened this issue Jan 24, 2022 · 5 comments

Comments

@raygift
Copy link

raygift commented Jan 24, 2022

Project Incubating Request

TiDB-over-RoCE

Describe the project you want to incubate:

Currently, message transmissions inside TiDB such as raft heartbeat RPCs are based on TCP. Specifically, TiDB server and PD using grpc-go , while TiKV using gRPC Core wrapped by gRPC-rs, both of these gRPC projects using TCP as the underlying communication protocol.

RDMA(Remote Direct Memory Access) allows one machine to read or write to the memory of another machine without requiring any processor cycles on the target. It has been paid more attention since its especial advantages such as Kernel-Bypass and Zero-Copy. As a RDMA protocol based on Ethernet, RoCEv2 (RDMA over Converged Ethernet v2) achieves a very well balance between efficiency of network communication and compatibility in existing data center infrastructure.

This proposal proposes running TiDB over RoCE protocol, which means all communications between tidb server, pd and tikv can benefit from RDMA technology, and higher performance and lower CPU workload are expected.

Estimated Time

10 months

Initial Team Members

张培承(raygift)
李毅伦(typedefstruct32)

Your RFC/Proposal?

@winkyao
Copy link
Contributor

winkyao commented Feb 7, 2022

LGTM

2 similar comments
@niubell
Copy link

niubell commented Feb 22, 2022

LGTM

@sunxiaoguang
Copy link
Contributor

LGTM

@sunxiaoguang
Copy link
Contributor

The project is very likely to be tightly coupled with gRPC implementation for both Rust and Golang. Therefore you may have to spend lots of efforts to make significant improvements to convenience people to accept such changes. Another thing I personally would like to see in this project is RDMA in virtualized environment. Good luck:)

@winkyao
Copy link
Contributor

winkyao commented Feb 28, 2022

@raygift https://github.com/tidb-incubator/TiDBOverRoCE have been created, already invited you to be the admin of this project. Please accept the invitation https://github.com/tidb-incubator/TiDBOverRoCE/invitations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants