[leo_gateway][nfs] Implicit limitation|premise of partial updates

[mocchira]

When handling a partial update via NFS
The current implementation assumes that the chunk size of a file already stored in leofs is set to the same with the current config in leo_gateway.conf.

So the below scenario will result in a undesirable consequence.

1. upload a 15MB file via S3/REST with 5MB chunk ( 3 chunks ${filename_[1-3]} generated )
2. partial update 5MB block at offset 5MB(+1) with 1MB chunk cause

• actual
  • ${filename_[6-10]} created
  • don't succeed in committing (large_obj_partial_commit) as ${filename_[4-5]} don't exist
• expected
  • rewrite a whole file with 1MB chunk and newly 5MB block with the proper offset or
  • deny partial update requests if the chunk size of a file requested is different from the one in leo_gateway.conf

[mocchira]

Considering the cost to rewrite whole and also getting logic complex(error prone)
the latter(simply deny a request) seems promising solution to me.

[windkit]

As we should not mix different chunk size for a file, it may be a good idea to fix it when the file is created and subsequent operations should follow.

If a file is created with S3/REST of 5MB chunk size, then NFS operations should follow afterwards.

What we have to confirm then is all API implementation check the chunk size before read/write operations.

[mocchira]

Yes ideally should be what you suggested.
but we've already provided NFS as a stable feature on 1.3.x and that typically means we need to take care any kind of backward compatibility. ( I think that there are many other things we should document but haven't. #532 aim to cover the implicit specs as much as possible )
In this case, some user may already rely on the fact that the chunk size in leo_gateway.conf prior to the one a file already has.

The other thing we need to take special care about NFS is the disk space problem.
Setting a chunk size to be different from a NFS client buffer size cause significant disk space loss due to its intrinsic append only disk format leofs has adopted.

so it's kind of tough work more than it looks.

@yosukehara @windkit

let's discuss on #532 .
we need to make a decision based on various perspectives.
The first thing we need to to is listing all implicit specs we haven't document.