CountWrapper stream_seek(0) should reset $readCount

[kesselb]

Motivation

server/lib/private/Files/ObjectStore/S3ObjectTrait.php

Lines 89 to 93 in 94c74fb

	$uploader = new MultipartUploader($this->getConnection(), $countStream, [
	'bucket' => $this->bucket,
	'key' => $urn,
	'part_size' => $this->uploadPartSize,
	]);

Replace the MultipartUploader with ObjectUploader. The ObjectUploader uses a PutObject operation for files < 16 MB (default, can be changed) and MultipartUpload otherwise. Amazon S3 charges you for storage and requests hence less requests are better.

Example: New text document	MultipartUpload	PutObject
1. Create the object	CreateMultipartUpload, UploadPart, CompleteMultipartUpload	PutObject
2. Generate max preview using	CreateMultipartUpload, UploadPart, CompleteMultipartUpload	PutObject
3. Generate another preview	CreateMultipartUpload, UploadPart, CompleteMultipartUpload	PutObject

Blocker

It's not possible to upload files with #21952 anymore (actually uploading works but the process is aborted).

server/apps/dav/lib/Connector/Sabre/File.php

Lines 237 to 239 in 57403c7

	if ($count !== $expected) {
	throw new BadRequest('Expected filesize of ' . $expected . ' bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) ' . $count . ' bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.');
	}

Is thrown because the number of bytes read is not correct. If $stream->getSize() is not available (at least on my dev setup it's never available), ObjectUploader reads the first 5 MB of a stream and sets the pointer back to 0.

server/lib/private/Files/ObjectStore/ObjectStoreStorage.php

Lines 454 to 460 in 734c62b

	$countStream = CountWrapper::wrap($stream, function ($writtenSize) use ($fileId, &$size) {
	$this->getCache()->update($fileId, [
	'size' => $writtenSize
	]);
	$size = $writtenSize;
	});
	$this->objectStore->writeObject($urn, $countStream);

Does not care about seek(0). The count for files uploaded via ObjectUploader is always 5 MB to high.

	public function stream_seek($offset, $whence = SEEK_SET) {
		$this->readCount = 0;
		return parent::stream_seek($offset, $whence); // TODO: Change the autogenerated stub
	}

Uploading files via ObjectUploader works if I patch the CountWrapper like above.

Question

Is this something for https://github.com/icewind1991/streams? It depends on the use case if all bytes ever read or the bytes after the last seek(0) are wanted. It might be easier to overwrite the CountWrapper here because reset the read count on offset = 0 is a breaking change.

Also most of the storage code relies on the assumption that bytes read = bytes written to external storage. Looks like a risky change 👀

[szaimen]

I suppose this is still valid?

[ghost]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

[kesselb]

#27877