implement discovery service using python

[CongMinYin]

The NVMe-over-Fabrics specification defines the concept of a Discovery Controller that an NVMe Host can query on a fabric network to discover NVMe subsystems contained in NVMe Targets which it can connect to on the network. The Discovery Controller will return Discovery Log Pages that provide the NVMe Host with specific information (such as network address and unique subsystem NQN) the NVMe Host can use to issue an NVMe connect command to connect itself to a storage resource contained in that NVMe subsystem on the NVMe Target.

We will implement a discovery controller using Python. In order to better communicate and share progress, I will update here.

The discovery controller code will be in:

root@cephs:~/git/ceph-nvmeof# ls control/discovery.py
control/discovery.py

Discovery controller IP and port for discovery service can be read from ceph-nvmeof.conf.

to start discovery service:
python3 -m control.discovery

discovery controller functions:

Network packets included in the discover command. This was captured using Wireshark.
command: nvme discover -t tcp -a 10.239.241.113 -s 4420

A complete process for basic function is:
1->2->3->4->5->6->7->8->9->9->…-> 9->4->5

appendix from wireshark.

[CongMinYin]

Now, I have completed step 9, the initiator can receive a response and display the log pages in the screenshot below. When completing the basic functions, I will optimize the code structure and submit a PR.

[CongMinYin]

cc @PepperJo @sdpeters

[PepperJo]

@CongMinYin This is great progress! Thank you for the update. Let us know if there are any blockers or you need any help with any of the items.

[CongMinYin]

Update status, basic functions have been completed, and the initiator can receive the log pages, I need to refactor the code, including the naming of constants, adjusting the code structure, updating comments, etc. It will take some time before submitting the PR.

Discovery controller:

root@cephs:~/git/ceph-nvmeof# python3 -m control.discovery
INFO:__main__:omap_name: nvmeof.state
INFO:__main__:discovery addr: 10.239.241.113 port: 4420
INFO:__main__:Waiting for connection...
INFO:__main__:Accept new connection from 10.239.241.67:41952...
INFO:__main__:can't handle asycn event now. 
INFO:__main__:Connection from ('10.239.241.67', 41952) closed.

Initiator:

root@ssp-ceph105:~# nvme discover -t tcp -a 10.239.241.113 -s 4420

Discovery Log Number of Records 11, Generation counter 1
=====Discovery Log Entry 0======
trtype:  tcp
adrfam:  ipv4
subtype: nvme subsystem
treq:    not required
portid:  0
trsvcid: 4420
subnqn:  nqn.2016-06.io.spdk:cnode10
traddr:  10.239.241.113
sectype: none
=====Discovery Log Entry 1======
trtype:  tcp
adrfam:  ipv4
subtype: nvme subsystem
treq:    not required
portid:  1
trsvcid: 50011
subnqn:  nqn.2016-06.io.spdk:cnode11
traddr:  10.239.241.113
sectype: none
...

[CongMinYin]

Another thing, before using the discovery controller, we need to address #87. The solution for my current commits is that even the default IP and port will be stored in the listener field of omap. After discussing this issue, we may need to add a new omap object to store the information of the discovery controller, which will also facilitate the implementation of the watch/notify function in the future. For more discussion, please refer to this issue.

[sdpeters]

As this is specifically about the Python implementation of #63, it's worth mentioning here that most of the future DS requirements described in #63 that aren't SPDK specific still apply to this.

[CongMinYin]

basic function PR: #127
require ip/port PR: #128
filter hosts with access permissions: CongMinYin@f98d312

In order not to increase the complexity of the previous PR, I will wait for the merge and then push the PR of the added functions one by one. Now, I am learning IO multiplexing and advanced IO multiplexing libraries in Python networks, including select and selectors. IO multiplexing of the network is the foundation of discovery async functionality.

[CongMinYin]

multi persistent connection by using selectors: CongMinYin@afd02b2
It is for async feature. Currently, only a small number of persistent connections can be supported. It is not yet possible to support thousands of concurrent requests for persistent connections and is still being resolved.

[CongMinYin]

Recently, I have implemented draft versions that support many persistent connections, as well as keep alive, async, timeout, and more for persistent connections(devel...CongMinYin:ceph-nvmeof:wip-discovery-multi-connection). Later on, I will focus on upstream and integrate the features of these draft versions.

[CongMinYin]

The preliminary work for most of the functions has been completed. see #127 and #127 (comment).