Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage] Add target to virtio-scsi, clean up virtio-blk #207

Merged
merged 1 commit into from
Nov 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
167 changes: 160 additions & 7 deletions storage/v1alpha1/autogen.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,22 +110,31 @@
- [frontend_virtio_scsi.proto](#frontend_virtio_scsi-proto)
- [CreateVirtioScsiControllerRequest](#opi_api-storage-v1-CreateVirtioScsiControllerRequest)
- [CreateVirtioScsiLunRequest](#opi_api-storage-v1-CreateVirtioScsiLunRequest)
- [CreateVirtioScsiTargetRequest](#opi_api-storage-v1-CreateVirtioScsiTargetRequest)
- [DeleteVirtioScsiControllerRequest](#opi_api-storage-v1-DeleteVirtioScsiControllerRequest)
- [DeleteVirtioScsiLunRequest](#opi_api-storage-v1-DeleteVirtioScsiLunRequest)
- [DeleteVirtioScsiTargetRequest](#opi_api-storage-v1-DeleteVirtioScsiTargetRequest)
- [GetVirtioScsiControllerRequest](#opi_api-storage-v1-GetVirtioScsiControllerRequest)
- [GetVirtioScsiLunRequest](#opi_api-storage-v1-GetVirtioScsiLunRequest)
- [GetVirtioScsiTargetRequest](#opi_api-storage-v1-GetVirtioScsiTargetRequest)
- [ListVirtioScsiControllerRequest](#opi_api-storage-v1-ListVirtioScsiControllerRequest)
- [ListVirtioScsiControllerResponse](#opi_api-storage-v1-ListVirtioScsiControllerResponse)
- [ListVirtioScsiLunRequest](#opi_api-storage-v1-ListVirtioScsiLunRequest)
- [ListVirtioScsiLunResponse](#opi_api-storage-v1-ListVirtioScsiLunResponse)
- [ListVirtioScsiTargetRequest](#opi_api-storage-v1-ListVirtioScsiTargetRequest)
- [ListVirtioScsiTargetResponse](#opi_api-storage-v1-ListVirtioScsiTargetResponse)
- [UpdateVirtioScsiControllerRequest](#opi_api-storage-v1-UpdateVirtioScsiControllerRequest)
- [UpdateVirtioScsiLunRequest](#opi_api-storage-v1-UpdateVirtioScsiLunRequest)
- [UpdateVirtioScsiTargetRequest](#opi_api-storage-v1-UpdateVirtioScsiTargetRequest)
- [VirtioScsiController](#opi_api-storage-v1-VirtioScsiController)
- [VirtioScsiControllerStatsRequest](#opi_api-storage-v1-VirtioScsiControllerStatsRequest)
- [VirtioScsiControllerStatsResponse](#opi_api-storage-v1-VirtioScsiControllerStatsResponse)
- [VirtioScsiLun](#opi_api-storage-v1-VirtioScsiLun)
- [VirtioScsiLunStatsRequest](#opi_api-storage-v1-VirtioScsiLunStatsRequest)
- [VirtioScsiLunStatsResponse](#opi_api-storage-v1-VirtioScsiLunStatsResponse)
- [VirtioScsiTarget](#opi_api-storage-v1-VirtioScsiTarget)
- [VirtioScsiTargetStatsRequest](#opi_api-storage-v1-VirtioScsiTargetStatsRequest)
- [VirtioScsiTargetStatsResponse](#opi_api-storage-v1-VirtioScsiTargetStatsResponse)

- [FrontendVirtioScsiService](#opi_api-storage-v1-FrontendVirtioScsiService)

Expand Down Expand Up @@ -1501,10 +1510,9 @@ Front End (host-facing) APIs. Mostly used for NVMe/PCIe emulation and host prese
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |
| pcie_id | [PciEndpoint](#opi_api-storage-v1-PciEndpoint) | | |
| bdev | [string](#string) | | |
| pcie_id | [PciEndpoint](#opi_api-storage-v1-PciEndpoint) | | The PCI endpoint where this device should appear |
| volume_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | The back/middle-end volume to back this controller |
| max_io_qps | [int64](#int64) | | |
| serial_number | [string](#string) | | |



Expand Down Expand Up @@ -1619,6 +1627,21 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and



<a name="opi_api-storage-v1-CreateVirtioScsiTargetRequest"></a>

### CreateVirtioScsiTargetRequest



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| target | [VirtioScsiTarget](#opi_api-storage-v1-VirtioScsiTarget) | | |






<a name="opi_api-storage-v1-DeleteVirtioScsiControllerRequest"></a>

### DeleteVirtioScsiControllerRequest
Expand Down Expand Up @@ -1650,6 +1673,21 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and



<a name="opi_api-storage-v1-DeleteVirtioScsiTargetRequest"></a>

### DeleteVirtioScsiTargetRequest



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| target_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |






<a name="opi_api-storage-v1-GetVirtioScsiControllerRequest"></a>

### GetVirtioScsiControllerRequest
Expand Down Expand Up @@ -1681,6 +1719,21 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and



<a name="opi_api-storage-v1-GetVirtioScsiTargetRequest"></a>

### GetVirtioScsiTargetRequest



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| target_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |






<a name="opi_api-storage-v1-ListVirtioScsiControllerRequest"></a>

### ListVirtioScsiControllerRequest
Expand Down Expand Up @@ -1746,6 +1799,38 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and



<a name="opi_api-storage-v1-ListVirtioScsiTargetRequest"></a>

### ListVirtioScsiTargetRequest



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| page_size | [int32](#int32) | | |
| page_token | [string](#string) | | |






<a name="opi_api-storage-v1-ListVirtioScsiTargetResponse"></a>

### ListVirtioScsiTargetResponse



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| targets | [VirtioScsiTarget](#opi_api-storage-v1-VirtioScsiTarget) | repeated | |
| next_page_token | [string](#string) | | |






<a name="opi_api-storage-v1-UpdateVirtioScsiControllerRequest"></a>

### UpdateVirtioScsiControllerRequest
Expand Down Expand Up @@ -1776,6 +1861,21 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and



<a name="opi_api-storage-v1-UpdateVirtioScsiTargetRequest"></a>

### UpdateVirtioScsiTargetRequest



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| target | [VirtioScsiTarget](#opi_api-storage-v1-VirtioScsiTarget) | | |






<a name="opi_api-storage-v1-VirtioScsiController"></a>

### VirtioScsiController
Expand All @@ -1785,7 +1885,7 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |
| pcie_id | [PciEndpoint](#opi_api-storage-v1-PciEndpoint) | | |
| pcie_id | [PciEndpoint](#opi_api-storage-v1-PciEndpoint) | | xPU&#39;s PCI ID for the controller |



Expand Down Expand Up @@ -1832,8 +1932,8 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |
| controller_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |
| bdev | [string](#string) | | |
| target_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | The target that this LUN is in |
| volume_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | The middle/back-end volume for this LLUN |



Expand Down Expand Up @@ -1871,6 +1971,53 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and




<a name="opi_api-storage-v1-VirtioScsiTarget"></a>

### VirtioScsiTarget



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |
| max_luns | [int32](#int32) | | maximum LUNs within a target |






<a name="opi_api-storage-v1-VirtioScsiTargetStatsRequest"></a>

### VirtioScsiTargetStatsRequest



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| target_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |






<a name="opi_api-storage-v1-VirtioScsiTargetStatsResponse"></a>

### VirtioScsiTargetStatsResponse



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | |
| stats | [string](#string) | | |








Expand All @@ -1881,10 +2028,16 @@ Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and
<a name="opi_api-storage-v1-FrontendVirtioScsiService"></a>

### FrontendVirtioScsiService
Front End (host-facing) APIs. Mostly used for Virtio-scsi emulation emulation and host presentation as alternative to Virtio-blk.
Front End (host-facing) APIs. Mostly used for Virtio-scsi emulation and host presentation as alternative to Virtio-blk.

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
| CreateVirtioScsiTarget | [CreateVirtioScsiTargetRequest](#opi_api-storage-v1-CreateVirtioScsiTargetRequest) | [VirtioScsiTarget](#opi_api-storage-v1-VirtioScsiTarget) | |
| DeleteVirtioScsiTarget | [DeleteVirtioScsiTargetRequest](#opi_api-storage-v1-DeleteVirtioScsiTargetRequest) | [.google.protobuf.Empty](#google-protobuf-Empty) | |
| UpdateVirtioScsiTarget | [UpdateVirtioScsiTargetRequest](#opi_api-storage-v1-UpdateVirtioScsiTargetRequest) | [VirtioScsiTarget](#opi_api-storage-v1-VirtioScsiTarget) | |
| ListVirtioScsiTarget | [ListVirtioScsiTargetRequest](#opi_api-storage-v1-ListVirtioScsiTargetRequest) | [ListVirtioScsiTargetResponse](#opi_api-storage-v1-ListVirtioScsiTargetResponse) | |
| GetVirtioScsiTarget | [GetVirtioScsiTargetRequest](#opi_api-storage-v1-GetVirtioScsiTargetRequest) | [VirtioScsiTarget](#opi_api-storage-v1-VirtioScsiTarget) | |
| VirtioScsiTargetStats | [VirtioScsiTargetStatsRequest](#opi_api-storage-v1-VirtioScsiTargetStatsRequest) | [VirtioScsiTargetStatsResponse](#opi_api-storage-v1-VirtioScsiTargetStatsResponse) | |
| CreateVirtioScsiController | [CreateVirtioScsiControllerRequest](#opi_api-storage-v1-CreateVirtioScsiControllerRequest) | [VirtioScsiController](#opi_api-storage-v1-VirtioScsiController) | |
| DeleteVirtioScsiController | [DeleteVirtioScsiControllerRequest](#opi_api-storage-v1-DeleteVirtioScsiControllerRequest) | [.google.protobuf.Empty](#google-protobuf-Empty) | |
| UpdateVirtioScsiController | [UpdateVirtioScsiControllerRequest](#opi_api-storage-v1-UpdateVirtioScsiControllerRequest) | [VirtioScsiController](#opi_api-storage-v1-VirtioScsiController) | |
Expand Down
8 changes: 6 additions & 2 deletions storage/v1alpha1/frontend_virtio_blk.proto
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,14 @@ service FrontendVirtioBlkService {

message VirtioBlk {
common.v1.ObjectKey id = 1;

// The PCI endpoint where this device should appear
PciEndpoint pcie_id = 2;
string bdev = 3;

// The back/middle-end volume to back this controller
common.v1.ObjectKey volume_id = 3;

int64 max_io_qps = 4;
string serial_number = 5;
}

message CreateVirtioBlkRequest {
Expand Down
83 changes: 80 additions & 3 deletions storage/v1alpha1/frontend_virtio_scsi.proto
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,37 @@ import "object_key.proto";
import "google/protobuf/empty.proto";
import "google/api/annotations.proto";

// Front End (host-facing) APIs. Mostly used for Virtio-scsi emulation emulation and host presentation as alternative to Virtio-blk.
// Front End (host-facing) APIs. Mostly used for Virtio-scsi emulation and host presentation as alternative to Virtio-blk.
service FrontendVirtioScsiService {
rpc CreateVirtioScsiTarget (CreateVirtioScsiTargetRequest) returns (VirtioScsiTarget) {
option (google.api.http) = {
post: "/v1/virtioscsitargets"
Copy link
Contributor

@jainvipin jainvipin Nov 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URLs needs hierarchy - hopefully we can do it after this PR; the flattened url names are including the hierarchy in the names explicitly. So the URLs in this and in NVMe protos can be done separately.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #188 to track the fix of all the URLs at once

body: "virtioscsitarget"
};
}
rpc DeleteVirtioScsiTarget (DeleteVirtioScsiTargetRequest) returns (google.protobuf.Empty) {
option (google.api.http) = {
delete: "/v1/virtioscsitargets/{virtioscsitarget}"
};
}
rpc UpdateVirtioScsiTarget (UpdateVirtioScsiTargetRequest) returns (VirtioScsiTarget) {
option (google.api.http) = {
patch: "/v1/virtioscsitargets"
body: "virtioscsitarget"
};
}
rpc ListVirtioScsiTarget (ListVirtioScsiTargetRequest) returns (ListVirtioScsiTargetResponse) {
option (google.api.http) = {
get: "/v1/virtioscsitargets"
};
}
rpc GetVirtioScsiTarget (GetVirtioScsiTargetRequest) returns (VirtioScsiTarget) {
option (google.api.http) = {
get: "/v1/virtioscsitargets/{virtioscsitarget}"
};
}
rpc VirtioScsiTargetStats (VirtioScsiTargetStatsRequest) returns (VirtioScsiTargetStatsResponse) {}

rpc CreateVirtioScsiController (CreateVirtioScsiControllerRequest) returns (VirtioScsiController) {
option (google.api.http) = {
post: "/v1/virtioscsictrls"
Expand Down Expand Up @@ -75,15 +104,63 @@ service FrontendVirtioScsiService {
rpc VirtioScsiLunStats (VirtioScsiLunStatsRequest) returns (VirtioScsiLunStatsResponse) {}
}

message VirtioScsiTarget {
common.v1.ObjectKey id = 1;

// maximum LUNs within a target
int32 max_luns = 2;
}

message VirtioScsiController {
common.v1.ObjectKey id = 1;

// xPU's PCI ID for the controller
PciEndpoint pcie_id = 2;
}

message VirtioScsiLun {
common.v1.ObjectKey id = 1;
common.v1.ObjectKey controller_id = 2;
string bdev = 3;

// The target that this LUN is in
common.v1.ObjectKey target_id = 2;

// The middle/back-end volume for this LLUN
common.v1.ObjectKey volume_id = 3;
}

message CreateVirtioScsiTargetRequest {
VirtioScsiTarget target = 1;
}

message DeleteVirtioScsiTargetRequest {
common.v1.ObjectKey target_id = 1;
}

message UpdateVirtioScsiTargetRequest {
VirtioScsiTarget target = 1;
}

message ListVirtioScsiTargetRequest {
int32 page_size = 1;
string page_token = 2;
}

message ListVirtioScsiTargetResponse {
repeated VirtioScsiTarget targets = 1;
string next_page_token = 2;
}

message GetVirtioScsiTargetRequest {
common.v1.ObjectKey target_id = 1;
}

message VirtioScsiTargetStatsRequest {
common.v1.ObjectKey target_id = 1;
}

message VirtioScsiTargetStatsResponse {
common.v1.ObjectKey id = 1;
string stats = 2;
Copy link
Contributor

@jainvipin jainvipin Nov 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this makes any sense, returning a big string for stats; if we are unsure about stats protos, we can remove it from here and introduce if with a bit more structure/thinking later. I'd suggest removing statis req/response all together for now.

Copy link
Member

@glimchb glimchb Nov 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have now string stats defined in all our objects, so I'm personally OK with this now to align with what we have already and then fix stats later on. opened #209 to track

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, that would work.

}

message CreateVirtioScsiControllerRequest {
Expand Down
Loading