[v1.3][nfs] Everyone is able to operate objects under a bucket with NFS

[yosukehara]

Unauthorized access must be avoided to keep reliability.

[Current]

$ sudo mkdir /mnt/leofs
$ sudo mount -t nfs -o nolock <IP>:/<BUCKET> <PATH>

[After]

$ sudo mkdir /mnt/leofs
$ sudo mount -t nfs -o nolock <IP>:/<BUCKET>/<ACCESS_KEY_ID>/<SECRET_ACCESS_KEY_ID> <PATH>

• Reference: LeoFS w/NFS

[yosukehara]

I've recognized someone knows ACCESS_KEY_ID and SECRET_ACCESS_KEY_ID with the df-command.

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      20G  13G  6.0G  68% /
nfs              4M    0    4M  0% /mnt/nfs/accesskey/secretkey

So I've reconsider this issue, and reached one idea as below:

• A client IP address and a bucket's info is associated with a TOKEN
• A TOKEN is generated by a client IP address and a bucket's info
• A client need to use the TOKEN when mounting LeoFS w/NFS v3 client
• A Leo's gateway compares a requested TOKEN with a generated token at own node

Token_1 = leo_hex:binary_to_hex(crypto:hmac(sha, <<"192.168.1.10">>, <<"BUCKET/ACCESS_KEY_ID/SECRET_ACCESS_KEY_ID">>)).
"446f52cdce6bef80bf7f69e3a76b16e160cc8bbf"

Token_2 = leo_hex:binary_to_hex(crypto:hmac(sha, <<"192.168.1.11">>, <<"BUCKET/ACCESS_KEY_ID/SECRET_ACCESS_KEY_ID">>)).
"c29704d052d402267291e4725dfffbe3bf03f828"

And also, a TOKEN is managed at Leo's Manager nodes.

[yosukehara]

In conclusion, we'll deliver the way of mounting LeoFS with a NFS client with v1.4.0 as bellow:

[Current - v1.2]

$ sudo mkdir /mnt/leofs
$ sudo mount -t nfs -o nolock <IP>:/<BUCKET> /mnt/leofs

[After - v1.4]

$ leofs-adm mktoken <IP> <BUCKET> <ACCESS_KEY_ID> <SECRET_KEY_ID>
<TOKEN>
$ sudo mkdir /mnt/leofs
$ sudo mount -t nfs -o nolock <IP>:/<BUCKET>/<ACCESS_KEY_ID>/<TOKEN> /mnt/leofs

[mocchira]

Note for implement.

[After]

$ sudo mkdir /mnt/leofs
$ sudo mount -t nfs -o nolock <IP>:/<BUCKET>/<TOKEN> <PATH>

The below files to be modified.

• https://github.com/leo-project/leo_gateway/blob/develop/src/leo_nfs_mount3_server.erl#L69
  • retrieve access_key, secret_key, token according to a bucket retrieved from a path
  • generate a token based on the source ip, access_key, secret_key
  • compare a token generated and the other retrieved from a path and return OK|ERR
• https://github.com/leo-project/leo_manager/blob/develop/src/leo_manager_console.erl
• https://github.com/leo-project/leo_manager/blob/develop/src/leo_manager_formatter_json.erl
• https://github.com/leo-project/leo_manager/blob/develop/src/leo_manager_formatter_text.erl
• https://github.com/leo-project/leo_manager/blob/develop/src/leo_manager_api.erl
• https://github.com/leo-project/leo_s3_libs/blob/develop/include/leo_s3_bucket.hrl
• https://github.com/leo-project/leo_s3_libs/blob/develop/src/leo_s3_bucket.erl
  • provide CRUD functionalities for TOKENs
  • provide seamless schema upgrade (TOKENs to be added into a bucket table)

[yosukehara]

Today, I've implemented gen_nfs_mnt_key/3 in leo_s3_libs, and also LeoFS' manager is able to handle gen-nfs-mnt-key/3.

[yosukehara]

In addition, LeoFS does not manage tokens into the system because each token is automatically generated when receiving a request from a client, then LeoFS can compare a generated token with a requested token as below

Client:

$ leofs-adm mktoken <IP> <BUCKET> <ACCESS_KEY_ID> <SECRET_KEY_ID>
<TOKEN>
$ sudo mount -t nfs -o nolock <IP>:/<BUCKET>/<ACCESS_KEY_ID>/<TOKEN> /mnt/leofs

LeoFS' procedure:

• Retrieve $BUCKET, $ACCESS_KEY_ID, $TOKEN and IP address from a request
• Retrieve $SECRET_KEY_ID from $ACCESS_KEY_ID
• Generate $TOKEN, which is a same way of leofs-adm mktoken command
• Compare $TOKEN with Client's $TOKEN

[windkit]

Created PR
leo-project/leo_manager#16
leo-project/leo_gateway#43

[yosukehara]

I've reviewed the pull-requests, then we need to test the NFS support with one of the CI's scenarios.

[yosukehara]

I've tested this issue as below. I've closed it because I could not find a problem.

Preparing a NFS mount

$ ./leofs-adm add-bucket test 05236
OK

$ ./leofs-adm get-buckets
cluster id   | bucket   | owner       | permissions      | redundancy method            | created at
-------------+----------+-------------+------------------+------------------------------+---------------------------
leofs_1      | test     | _test_leofs | Me(full_control) | copy, {n:1, w:1, r:1, d:1}   | 2016-08-08 01:56:18 +0000

$ ./leofs-adm gen-nfs-mnt-key test 05236 127.0.0.1
bb5034f0c740148a346ed663ca0cf5157efb439f

$ sudo mkdir /mnt/leofs
$ sudo mount -t nfs -o nolock 127.0.0.1:/test/05236/bb5034f0c740148a346ed663ca0cf5157efb439f /mnt/leofs
$ ls -al /mnt/leofs
total 8
drwxrwxrwx 2 root root 4096 Aug  8 01:29 .
drwxr-xr-x 3 root root 4096 Nov  9  2015 ..

Creating a new file

$ touch /mnt/leofs/newfile
$ ls -al /mnt/leofs
total 8
drwxrwxrwx 2 root root 4096 Aug  8 01:29 .
drwxr-xr-x 3 root root 4096 Nov  9  2015 ..
-rw-rw-rw- 1 root root    0 Aug  8 01:29 newfile

Updating a file

$ echo "hello world" > /mnt/leofs/newfile
$ cat /mnt/leofs/newfile
hello world

$ ls -l /mnt/leofs/
total 9
drwxrwxrwx 2 root root 4096 Aug  8 01:32 .
drwxr-xr-x 3 root root 4096 Nov  9  2015 ..
-rw-rw-rw- 1 root root   12 Aug  8 01:31 newfile

Copying a file

$ cp /mnt/leofs/newfile /mnt/leofs/newfile.copy
$ ls -al /mnt/leofs
total 9
drwxrwxrwx 2 root root 4096 Aug  8 01:34 .
drwxr-xr-x 3 root root 4096 Nov  9  2015 ..
-rw-rw-rw- 1 root root   12 Aug  8 01:31 newfile
-rw-rw-rw- 1 root root   12 Aug  8 01:33 newfile.copy

Removing a file

$ rm /mnt/leofs/newfile
$ ls -al /mnt/leofs
total 9
drwxrwxrwx 2 root root 4096 Aug  8 01:35 .
drwxr-xr-x 3 root root 4096 Nov  9  2015 ..
-rw-rw-rw- 1 root root   12 Aug  8 01:33 newfile.copy

$ leofs-adm whereis test/newfile
-------+--------------------------+--------------------------------------+------------+--------------+----------------+----------------+----------------+----------------------------
 del?  |           node           |             ring address             |    size    |   checksum   |  has children  |  total chunks  |     clock      |             when
-------+--------------------------+--------------------------------------+------------+--------------+----------------+----------------+----------------+----------------------------
  *    | storage_0@127.0.0.1      | 4d40d881a8b59698134800283999336c     |         0B |   d41d8cd98f | false          |              0 | 539856c751109  | 2016-08-08 01:34:56 +0000


$ leofs-adm whereis test/newfile.copy
-------+--------------------------+--------------------------------------+------------+--------------+----------------+----------------+----------------+----------------------------
 del?  |           node           |             ring address             |    size    |   checksum   |  has children  |  total chunks  |     clock      |             when
-------+--------------------------+--------------------------------------+------------+--------------+----------------+----------------+----------------+----------------------------
       | storage_0@127.0.0.1      | 180963d63a661767ac4200763641e4ac     |        12B |   6f5902ac23 | false          |              0 | 53985683609e8  | 2016-08-08 01:33:45 +0000

Creating directories

$ mkdir -p /mnt/leofs/1/2/3
$ ls -alR /mnt/leofs/1
/mnt/leofs/1:
total 12
drwxrwxrwx 2 root root 4096 Aug  8 01:36 .
drwxrwxrwx 3 root root 4096 Aug  8 01:20 ..
drwxrwxrwx 2 root root 4096 Aug  8 01:36 2

/mnt/leofs/1/2:
total 12
drwxrwxrwx 2 root root 4096 Aug  8 01:36 .
drwxrwxrwx 2 root root 4096 Aug  8 01:36 ..
drwxrwxrwx 2 root root 4096 Aug  8 01:36 3

/mnt/leofs/1/2/3:
total 8
drwxrwxrwx 2 root root 4096 Aug  8 01:36 .
drwxrwxrwx 2 root root 4096 Aug  8 01:36 ..

Testing LeoFS' authentication

$ sudo mount -t nfs -o nolock 127.0.0.1:/test/05236/incorrecthash /mnt/leofs
mount.nfs: access denied by server while mounting 127.0.0.1:/test/05236/incorrecthash

$ sudo mount -t nfs -o nolock 127.0.0.1:/test /mnt/leofs
mount.nfs: access denied by server while mounting 127.0.0.1:/test