Releases: cs3org/reva
v1.12.0
Changelog for reva 1.12.0 (2021-08-24)
The following sections list the changes in reva 1.12.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1819: Disable notifications
- Fix #2000: Fix dependency on tests
- Fix #1957: Fix etag propagation on deletes
- Fix #1960: Return the updated share after updating
- Fix #1993: Fix owncloudsql GetMD
- Fix #1954: Fix response format of the sharees API
- Fix #1965: Fix the file target of user and group shares
- Fix #1956: Fix trashbin listing with depth 0
- Fix #1987: Fix windows build
- Fix #1990: Increase oc10 compatibility of owncloudsql
- Fix #1978: Owner type is optional
- Fix #1980: Propagate the etag after restoring a file version
- Fix #1985: Add quota stubs
- Fix #1992: Check if symlink exists instead of spamming the console
- Fix #1913: Logic to restore files to readonly nodes
- Chg #1982: Move user context methods into a separate
userctx
package - Enh #1946: Add share manager that connects to oc10 databases
- Enh #1983: Add Codacy unit test coverage
- Enh #1803: Introduce new webdav spaces endpoint
- Enh #1998: Initial version of the Nextcloud storage driver
- Enh #1984: Replace OpenCensus with OpenTelemetry
- Enh #1861: Add support for runtime plugins
- Enh #2008: Site account extensions
Details
-
Bugfix #1819: Disable notifications
The presence of the key
notifications
in the capabilities' response would cause clients to
attempt to poll the notifications endpoint, which is not yet supported. To prevent the
unnecessary bandwidth we are disabling this altogether. -
Bugfix #2000: Fix dependency on tests
The Nextcloud storage driver depended on a mock http client from the tests/ folder This broke
the Docker build The dependency was removed A check was added to test the Docker build on each PR -
Bugfix #1957: Fix etag propagation on deletes
When deleting a file the etag propagation would skip the parent of the deleted file.
-
Bugfix #1960: Return the updated share after updating
When updating the state of a share in the in-memory share manager the old share state was
returned instead of the updated state. -
Bugfix #1993: Fix owncloudsql GetMD
The GetMD call internally was not prefixing the path when looking up resources by id.
-
Bugfix #1954: Fix response format of the sharees API
The sharees API wasn't returning the users and groups arrays correctly.
-
Bugfix #1965: Fix the file target of user and group shares
In some cases the file target of user and group shares was not properly prefixed.
-
Bugfix #1956: Fix trashbin listing with depth 0
The trashbin API handled requests with depth 0 the same as request with a depth of 1.
-
Bugfix #1987: Fix windows build
Add the necessary
golang.org/x/sys/windows
package import toowncloud
and
owncloudsql
storage drivers. -
Bugfix #1990: Increase oc10 compatibility of owncloudsql
We added a few changes to the owncloudsql storage driver to behave more like oc10.
-
Bugfix #1978: Owner type is optional
When reading the user from the extended attributes the user type might not be set, in this case we
now return a user with an invalid type, which correctly reflects the state on disk. -
Bugfix #1980: Propagate the etag after restoring a file version
The decomposedfs didn't propagate after restoring a file version.
-
Bugfix #1985: Add quota stubs
The
owncloud
andowncloudsql
drivers now read the available quota from disk to no longer
always return 0, which causes the web UI to disable uploads. -
Bugfix #1992: Check if symlink exists instead of spamming the console
The logs have been spammed with messages like
could not create symlink for ...
when using the
decomposedfs, eg. with the oCIS storage. We now check if the link exists before trying to create
it. -
Bugfix #1913: Logic to restore files to readonly nodes
This impacts solely the DecomposedFS. Prior to these changes there was no validation when a
user tried to restore a file from the trashbin to a share location (i.e any folder under
/Shares
).With this patch if the user restoring the resource has write permissions on the share, restore
is possible. -
Change #1982: Move user context methods into a separate
userctx
package -
Enhancement #1946: Add share manager that connects to oc10 databases
-
Enhancement #1983: Add Codacy unit test coverage
This PR adds unit test coverage upload to Codacy.
-
Enhancement #1803: Introduce new webdav spaces endpoint
Clients can now use a new webdav endpoint
/dav/spaces/<storagespaceid>/relative/path/to/file
to directly access storage
spaces.The
<storagespaceid>
can be retrieved using the ListStorageSpaces CS3 api call. -
Enhancement #1998: Initial version of the Nextcloud storage driver
This is not usable yet in isolation, but it's a first component of
https://github.com/pondersource/sciencemesh-nextcloud -
Enhancement #1984: Replace OpenCensus with OpenTelemetry
OpenTelemetry](https://opentelemetry.io/docs/concepts/what-is-opentelemetry/) is
an open standard a
sandbox CNCF project and it was formed through a merger of the OpenTracing and OpenCensus.OpenCensus and OpenTracing have merged to form OpenTelemetry, which serves as the next major
version of OpenCensus and OpenTracing. OpenTelemetry will offer backwards compatibility
with existing OpenCensus integrations, and we will continue to make security patches to
existing OpenCensus libraries for two years.There is a lot of outdated documentation as a result of this merger, and we will be better off
adopting the latest standard and libraries. -
Enhancement #1861: Add support for runtime plugins
This PR introduces a new plugin package, that allows loading external plugins into Reva at
runtime. The hashicorp go-plugin framework was used to facilitate the plugin loading and
communication. -
Enhancement #2008: Site account extensions
This PR heavily extends the site accounts service: * Extended the accounts information (not
just email and name) * Accounts now have a password * Users can now "log in" to their accounts and
edit it * Ability to grant access to the GOCDBFurthermore, these accounts can now be used to authenticate for logging in to our customized
GOCDB. More use cases for these accounts are also planned.
v1.11.0
Changelog for reva 1.11.0 (2021-08-03)
The following sections list the changes in reva 1.11.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1899: Fix chunked uploads for new versions
- Fix #1906: Fix copy over existing resource
- Fix #1891: Delete Shared Resources as Receiver
- Fix #1907: Error when creating folder with existing name
- Fix #1937: Do not overwrite more specific matches when finding storage providers
- Fix #1939: Fix the share jail permissions in the decomposedfs
- Fix #1932: Numerous fixes to the owncloudsql storage driver
- Fix #1912: Fix response when listing versions of another user
- Fix #1910: Get user groups recursively in the cbox rest user driver
- Fix #1904: Set Content-Length to 0 when swallowing body in the datagateway
- Fix #1911: Fix version order in propfind responses
- Fix #1926: Trash Bin in oCIS Storage Operations
- Fix #1901: Fix response code when folder doesnt exist on upload
- Enh #1785: Extend app registry with AddProvider method and mimetype filters
- Enh #1938: Add methods to get and put context values
- Enh #1798: Add support for a deny-all permission on references
- Enh #1916: Generate updated protobuf bindings for EOS GRPC
- Enh #1887: Add "a" and "l" filter for grappa queries
- Enh #1919: Run gofmt before building
- Enh #1927: Implement RollbackToVersion for eosgrpc (needs a newer EOS MGM)
- Enh #1944: Implement listing supported mime types in app registry
- Enh #1870: Be defensive about wrongly quoted etags
- Enh #1940: Reduce memory usage when uploading with S3ng storage
- Enh #1888: Refactoring of the webdav code
- Enh #1900: Check for illegal names while uploading or moving files
- Enh #1925: Refactor listing and statting across providers for virtual views
Details
-
Bugfix #1899: Fix chunked uploads for new versions
Chunked uploads didn't create a new version, when the file to upload already existed.
-
Bugfix #1906: Fix copy over existing resource
When the target of a copy already exists, the existing resource will be moved to the trashbin
before executing the copy. -
Bugfix #1891: Delete Shared Resources as Receiver
It is now possible to delete a shared resource as a receiver and not having the data ending up in
the receiver's trash bin, causing a possible leak. -
Bugfix #1907: Error when creating folder with existing name
When a user tried to create a folder with the name of an existing file or folder the service didn't
return a response body containing the error. -
Bugfix #1937: Do not overwrite more specific matches when finding storage providers
Depending on the order of rules in the registry it could happend that more specific matches
(e.g. /home/Shares) were overwritten by more general ones (e.g. /home). This PR makes sure
that the registry always returns the most specific match. -
Bugfix #1939: Fix the share jail permissions in the decomposedfs
The share jail should be not writable
-
Bugfix #1932: Numerous fixes to the owncloudsql storage driver
The owncloudsql storage driver received numerous bugfixes and cleanups.
-
Bugfix #1912: Fix response when listing versions of another user
The OCS API returned the wrong response when a user tried to list the versions of another user's
file. -
Bugfix #1910: Get user groups recursively in the cbox rest user driver
-
Bugfix #1904: Set Content-Length to 0 when swallowing body in the datagateway
When swallowing the body the Content-Lenght needs to be set to 0 to prevent proxies from reading
the body. -
Bugfix #1911: Fix version order in propfind responses
The order of the file versions in propfind responses was incorrect.
-
Bugfix #1926: Trash Bin in oCIS Storage Operations
Support for restoring a target folder nested deep inside the trash bin in oCIS storage. The use
case is:MOVE -H 'Destination: https://localhost:9200/remote.php/dav/files/einstein/destination' ``` The previous command creates the `destination` folder and moves the contents of `/trash-bin/einstein/f1/f2` onto it. Retro-compatibility in the response code with ownCloud 10. Restoring a collection to a non-existent nested target is not supported and MUST return `409`. The use case is: ```console curl 'https://localhost:9200/remote.php/dav/trash-bin/einstein/f1/f2' -X MOVE -H 'Destination: https://localhost:9200/remote.php/dav/files/einstein/this/does/not/exist' ``` The previous command used to return `404` instead of the expected `409` by the clients. https://github.com/cs3org/reva/pull/1926
-
Bugfix #1901: Fix response code when folder doesnt exist on upload
When a new file was uploaded to a non existent folder the response code was incorrect.
-
Enhancement #1785: Extend app registry with AddProvider method and mimetype filters
-
Enhancement #1938: Add methods to get and put context values
Added
GetKeyValues
andPutKeyValues
methods to fetch/put values from/to context. -
Enhancement #1798: Add support for a deny-all permission on references
And implement it on the EOS storage
-
Enhancement #1916: Generate updated protobuf bindings for EOS GRPC
-
Enhancement #1887: Add "a" and "l" filter for grappa queries
This PR adds the namespace filters "a" and "l" for grappa queries. With no filter will look into
primary and e-groups, with "a" will look into primary/secondary/service/e-groups and with
"l" will look into lightweight accounts. -
Enhancement #1919: Run gofmt before building
-
Enhancement #1927: Implement RollbackToVersion for eosgrpc (needs a newer EOS MGM)
-
Enhancement #1944: Implement listing supported mime types in app registry
-
Enhancement #1870: Be defensive about wrongly quoted etags
When ocdav renders etags it will now try to correct them to the definition as quoted strings
which do not contain"
. This prevents double or triple quoted etags on the webdav api. -
Enhancement #1940: Reduce memory usage when uploading with S3ng storage
The memory usage could be high when uploading files using the S3ng storage. By providing the
actual file size when triggeringPutObject
, the overall memory usage is reduced. -
Enhancement #1888: Refactoring of the webdav code
Refactored the webdav code to make it reusable.
-
Enhancement #1900: Check for illegal names while uploading or moving files
The code was not checking for invalid file names during uploads and moves.
-
Enhancement #1925: Refactor listing and statting across providers for virtual views
v1.10.0
Changelog for reva 1.10.0 (2021-07-13)
The following sections list the changes in reva 1.10.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1883: Pass directories with trailing slashes to eosclient.GenerateToken
- Fix #1878: Improve the webdav error handling in the trashbin
- Fix #1884: Do not send body on failed range request
- Enh #1744: Add support for lightweight user types
Details
-
Bugfix #1883: Pass directories with trailing slashes to eosclient.GenerateToken
-
Bugfix #1878: Improve the webdav error handling in the trashbin
The trashbin handles errors better now on the webdav endpoint.
-
Bugfix #1884: Do not send body on failed range request
Instead of send the error in the body of a 416 response we log it. This prevents the go reverse
proxy from choking on it and turning it into a 502 Bad Gateway response. -
Enhancement #1744: Add support for lightweight user types
This PR adds support for assigning and consuming user type when setting/reading users. On top
of that, support for lightweight users is added. These users have to be restricted to accessing
only shares received by them, which is accomplished by expanding the existing RBAC scope.
v1.9.1
Changelog for reva 1.9.1 (2021-07-09)
The following sections list the changes in reva 1.9.1 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1843: Correct Dockerfile path for the reva CLI and alpine3.13 as builder
- Fix #1835: Cleanup owncloudsql driver
- Fix #1868: Minor fixes to the grpc/http plugin: checksum, url escaping
- Fix #1885: Fix template in eoshomewrapper to use context user rather than resource
- Fix #1833: Properly handle name collisions for deletes in the owncloud driver
- Fix #1874: Use the original file mtime during upload
- Fix #1854: Add the uid/gid to the url for eos
- Fix #1848: Fill in missing gid/uid number with nobody
- Fix #1831: Make the ocm-provider endpoint in the ocmd service unprotected
- Fix #1808: Use empty array in OCS Notifications endpoints
- Fix #1825: Raise max grpc message size
- Fix #1828: Send a proper XML header with error messages
- Chg #1828: Remove the oidc provider in order to upgrad mattn/go-sqlite3 to v1.14.7
- Enh #1834: Add API key to Mentix GOCDB connector
- Enh #1855: Minor optimization in parsing EOS ACLs
- Enh #1873: Update the EOS image tag to be for revad-eos image
- Enh #1802: Introduce list spaces
- Enh #1849: Add readonly interceptor
- Enh #1875: Simplify resource comparison
- Enh #1827: Support trashbin sub paths in the recycle API
Details
-
Bugfix #1843: Correct Dockerfile path for the reva CLI and alpine3.13 as builder
This was introduced on 117adad while porting the
configuration on .drone.yml to starlark.Force golang:alpine3.13 as base image to prevent errors from Make when running on Docker
<20.10 as it happens on Drone
ref.https://gitlab.alpinelinux.org/alpine/aports/-/issues/12396 -
Bugfix #1835: Cleanup owncloudsql driver
Use
owncloudsql
string when returning errors and removed copyMD as it does not need to copy
metadata from files. -
Bugfix #1868: Minor fixes to the grpc/http plugin: checksum, url escaping
-
Bugfix #1885: Fix template in eoshomewrapper to use context user rather than resource
-
Bugfix #1833: Properly handle name collisions for deletes in the owncloud driver
In the owncloud storage driver when we delete a file we append the deletion time to the file name.
If two fast consecutive deletes happened, the deletion time would be the same and if the two
files had the same name we ended up with only one file in the trashbin. -
Bugfix #1874: Use the original file mtime during upload
The decomposedfs was not using the original file mtime during uploads.
-
Bugfix #1854: Add the uid/gid to the url for eos
-
Bugfix #1848: Fill in missing gid/uid number with nobody
When an LDAP server does not provide numeric uid or gid properties for a user we now fall back to a
configurablenobody
id (default 99). -
Bugfix #1831: Make the ocm-provider endpoint in the ocmd service unprotected
-
Bugfix #1808: Use empty array in OCS Notifications endpoints
-
Bugfix #1825: Raise max grpc message size
As a workaround for listing larger folder we raised the
MaxCallRecvMsgSize
to 10MB. This
should be enough for ~15k files. The proper fix is implementing ListContainerStream in the
gateway, but we needed a way to test the web ui with larger collections. -
Bugfix #1828: Send a proper XML header with error messages
-
Change #1828: Remove the oidc provider in order to upgrad mattn/go-sqlite3 to v1.14.7
In order to upgrade mattn/go-sqlite3 to v1.14.7, the odic provider service is removed, which
is possible because it is not used anymore -
Enhancement #1834: Add API key to Mentix GOCDB connector
The PI (programmatic interface) of the GOCDB will soon require an API key; this PR adds the
ability to configure this key in Mentix. -
Enhancement #1855: Minor optimization in parsing EOS ACLs
-
Enhancement #1873: Update the EOS image tag to be for revad-eos image
-
Enhancement #1802: Introduce list spaces
The ListStorageSpaces call now allows listing all user homes and shared resources using a
storage space id. The gateway will forward requests to a specific storage provider when a
filter by id is given. Otherwise it will query all storage providers. Results will be
deduplicated. Currently, only the decomposed fs storage driver implements the necessary
logic to demonstrate the implmentation. A new/dav/spaces
WebDAV endpoint to directly
access a storage space is introduced in a separate PR. -
Enhancement #1849: Add readonly interceptor
The readonly interceptor could be used to configure a storageprovider in readonly mode. This
could be handy in some migration scenarios. -
Enhancement #1875: Simplify resource comparison
We replaced ResourceEqual with ResourceIDEqual where possible.
-
Enhancement #1827: Support trashbin sub paths in the recycle API
The recycle API could only act on the root items of the trashbin. Meaning if you delete a deep
tree, you couldn't restore just one file from that tree but you had to restore the whole tree. Now
listing, restoring and purging work also for sub paths in the trashbin.
v1.9.0
Changelog for reva 1.9.0 (2021-06-23)
The following sections list the changes in reva 1.9.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1815: Drone CI - patch the 'store-dev-release' job to fix malformed requests
- Fix #1765: 'golang:alpine' as base image & CGO_ENABLED just for the CLI
- Chg #1721: Absolute and relative references
- Enh #1810: Add arbitrary metadata support to EOS
- Enh #1774: Add user ID cache warmup to EOS storage driver
- Enh #1471: EOEGrpc progress. Logging discipline and error handling
- Enh #1811: Harden public shares signing
- Enh #1793: Remove the user id from the trashbin key
- Enh #1795: Increase trashbin restore API compatibility
- Enh #1516: Use UidNumber and GidNumber fields in User objects
- Enh #1820: Tag v1.9.0
Details
-
Bugfix #1815: Drone CI - patch the 'store-dev-release' job to fix malformed requests
Replace the backquotes that were used for the date component of the URL with the
POSIX-confirmant command substitution '$()'. -
Bugfix #1765: 'golang:alpine' as base image & CGO_ENABLED just for the CLI
Some of the dependencies used by revad need CGO to be enabled in order to work. We also need to
install the 'mime-types' in alpine to correctly detect them on the storage-providers.The CGO_ENABLED=0 flag was added to the docker build flags so that it will produce a static
build. This allows usage of the 'scratch' image for reduction of the docker image size (e.g. the
reva cli). -
Change #1721: Absolute and relative references
We unified the
Reference_Id
endReference_Path
types to a combinedReference
that
contains both: - aresource_id
property that can identify a node using astorage_id
and an
opaque_id
- apath
property that can be used to represent absolute paths as well as paths
relative to the id based properties. While this is a breaking change it allows passing both:
absolute as well as relative references. -
Enhancement #1810: Add arbitrary metadata support to EOS
-
Enhancement #1774: Add user ID cache warmup to EOS storage driver
-
Enhancement #1471: EOEGrpc progress. Logging discipline and error handling
-
Enhancement #1811: Harden public shares signing
Makes golangci-lint happy as well
-
Enhancement #1793: Remove the user id from the trashbin key
We don't want to use the users uuid outside of the backend so I removed the id from the trashbin
file key. -
Enhancement #1795: Increase trashbin restore API compatibility
- The precondition were not checked before doing a trashbin restore in the ownCloud dav API.
Without the checks the API would behave differently compared to the oC10 API. * The restore
response was missing HTTP headers likeETag
* Update the name when restoring the file from
trashbin to a new target name
- The precondition were not checked before doing a trashbin restore in the ownCloud dav API.
-
Enhancement #1516: Use UidNumber and GidNumber fields in User objects
Update instances where CS3API's
User
objects are created and used to useGidNumber
, and
UidNumber
fields instead of storing them inOpaque
map. -
Enhancement #1820: Tag v1.9.0
Bump release number to v1.9.0 as it contains breaking changes related to changing the
reference type.
v1.8.0
Changelog for reva 1.8.0 (2021-06-09)
The following sections list the changes in reva 1.8.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1779: Set Content-Type header correctly for ocs requests
- Fix #1650: Allow fetching shares as the grantee
- Fix #1693: Fix move in owncloud storage driver
- Fix #1666: Fix public file shares
- Fix #1541: Allow for restoring recycle items to different locations
- Fix #1718: Use the -static ldflag only for the 'build-ci' target
- Enh #1719: Application passwords CLI
- Enh #1719: Application passwords management
- Enh #1725: Create transfer type share
- Enh #1755: Return file checksum available from the metadata for the EOS driver
- Enh #1673: Deprecate using errors.New and fmt.Errorf
- Enh #1723: Open in app workflow using the new API
- Enh #1655: Improve json marshalling of share protobuf messages
- Enh #1694: User profile picture capability
- Enh #1649: Add reliability calculations support to Mentix
- Enh #1509: Named Service Registration
- Enh #1643: Cache resources from share getter methods in OCS
- Enh #1664: Add cache warmup strategy for OCS resource infos
- Enh #1710: Owncloudsql storage driver
- Enh #1705: Reduce the size of all the container images built on CI
- Enh #1669: Mint scope-based access tokens for RBAC
- Enh #1683: Filter created shares based on type in OCS
- Enh #1763: Sort share entries alphabetically
- Enh #1758: Warn user for not recommended go version
- Enh #1747: Add checksum headers to tus preflight responses
- Enh #1685: Add share to update response
Details
-
Bugfix #1779: Set Content-Type header correctly for ocs requests
Before this fix the
Content-Type
header was guessed byw.Write
becauseWriteHeader
was
called to early. Now theContent-Type
is set correctly and to the same values as in ownCloud 10 -
Bugfix #1650: Allow fetching shares as the grantee
The json backend now allows a grantee to fetch a share by id.
-
Bugfix #1693: Fix move in owncloud storage driver
When moving a file or folder (includes renaming) the filepath in the cache didn't get updated
which caused subsequent requests togetpath
to fail. -
Bugfix #1666: Fix public file shares
Fixed stat requests and propfind responses for publicly shared files.
-
Bugfix #1541: Allow for restoring recycle items to different locations
The CS3 APIs specify a way to restore a recycle item to a different location than the original by
setting therestore_path
field in theRestoreRecycleItemRequest
. This field had not
been considered until now. -
Bugfix #1718: Use the -static ldflag only for the 'build-ci' target
It is not intended to statically link the generated binaries for local development workflows.
This resulted on segmentation faults and compiller warnings. -
Enhancement #1719: Application passwords CLI
This PR adds the CLI commands
token-list
,token-create
andtoken-remove
to manage
tokens with limited scope on behalf of registered users. -
Enhancement #1719: Application passwords management
This PR adds the functionality to generate authentication tokens with limited scope on behalf
of registered users. These can be used in third party apps or in case primary user credentials
cannot be submitted to other parties. -
Enhancement #1725: Create transfer type share
transfer-create
creates a share of type transfer. -
Enhancement #1755: Return file checksum available from the metadata for the EOS driver
-
Enhancement #1673: Deprecate using errors.New and fmt.Errorf
Previously we were using errors.New and fmt.Errorf to create errors. Now we use the errors
defined in the errtypes package. -
Enhancement #1723: Open in app workflow using the new API
This provides a new
open-in-app
command for the CLI and the implementation on the
appprovider gateway service for the new API, including the option to specify the appplication
to use, thus overriding the preconfigured one. -
Enhancement #1655: Improve json marshalling of share protobuf messages
Protobuf oneof fields cannot be properly handled by the native json marshaller, and the
protojson package can only handle proto messages. Previously, we were using a workaround of
storing these oneof fields separately, which made the code inelegant. Now we marshal these
messages as strings before marshalling them via the native json package. -
Enhancement #1694: User profile picture capability
Based on feedback in the new ownCloud web frontend we want to omit trying to render user avatars
images / profile pictures based on the backend capabilities. Now the OCS communicates a
corresponding value. -
Enhancement #1649: Add reliability calculations support to Mentix
To make reliability calculations possible, a new exporter has been added to Mentix that reads
scheduled downtimes from the GOCDB and exposes it through Prometheus metrics. -
Enhancement #1509: Named Service Registration
Move away from hardcoding service IP addresses and rely upon name resolution instead. It
delegates the address lookup to a static in-memory service registry, which can be
re-implemented in multiple forms. -
Enhancement #1643: Cache resources from share getter methods in OCS
In OCS, once we retrieve the shares from the shareprovider service, we stat each of those
separately to obtain the required info, which introduces a lot of latency. This PR introduces a
resoource info cache in OCS, which would prevent this latency. -
Enhancement #1664: Add cache warmup strategy for OCS resource infos
Recently, a TTL cache was added to OCS to store statted resource infos. This PR adds an interface
to define warmup strategies and also adds a cbox specific strategy which starts a goroutine to
initialize the cache with all the valid shares present in the system. -
Enhancement #1710: Owncloudsql storage driver
This PR adds a storage driver which connects to a oc10 storage backend (storage + database).
This allows for running oc10 and ocis with the same backend in parallel. -
Enhancement #1705: Reduce the size of all the container images built on CI
Previously, all images were based on golang:1.16 which is built from Debian. Using 'scratch'
as base, reduces the size of the artifacts well as the attack surface for all the images, plus
copying the binary from the build step ensures that only the strictly required software is
present on the final image. For the revad images tagged '-eos', eos-slim is used instead. It is
still large but it updates the environment as well as the EOS version. -
Enhancement #1669: Mint scope-based access tokens for RBAC
Primarily, this PR is meant to introduce the concept of scopes into our tokens. At the moment, it
addresses those cases where we impersonate other users without allowing the full scope of what
the actual user has access to.A short explanation for how it works for public shares: - We get the public share using the token
provided by the client. - In the public share, we know the resource ID, so we can add this to the
allowed scope, but not the path. - However, later OCDav tries to access by path as well. Now this
is not allowed at the moment. However, from the allowed scope, we have the resource ID and we're
allowed to stat that. We stat the resource ID, get the path and if the path matches the one passed
by OCDav, we allow the request to go through. -
Enhancement #1683: Filter created shares based on type in OCS
-
Enhancement #1763: Sort share entries alphabetically
When showing the list of shares to the end-user, the list was not sorted alphabetically. This PR
sorts the list of users and groups. -
Enhancement #1758: Warn user for not recommended go version
This PR adds a warning while an user is building the source code, if he is using a go version not
recommended. -
Enhancement #1747: Add checksum headers to tus preflight responses
Added
checksum
to the headerTus-Extension
and added theTus-Checksum-Algorithm
header. -
Enhancement #1685: Add share to update response
After accepting or rejecting a share the API includes the updated share in the response.
v1.7.0
Changelog for reva 1.7.0 (2021-04-19)
The following sections list the changes in reva 1.7.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1619: Fixes for enabling file sharing in EOS
- Fix #1576: Fix etag changing only once a second
- Fix #1634: Mentix site authorization status changes
- Fix #1625: Make local file connector more error tolerant
- Fix #1526: Fix webdav file versions endpoint bugs
- Fix #1457: Cloning of internal mesh data lost some values
- Fix #1597: Check for ENOTDIR on readlink error
- Fix #1636: Skip file check for OCM data transfers
- Fix #1552: Fix a bunch of trashbin related issues
- Fix #1: Bump meshdirectory-web to 1.0.2
- Chg #1562: Modularize api token management in GRAPPA drivers
- Chg #1452: Separate blobs from metadata in the ocis storage driver
- Enh #1514: Add grpc test suite for the storage provider
- Enh #1466: Add integration tests for the s3ng driver
- Enh #1521: Clarify expected failures
- Enh #1624: Add wrappers for EOS and EOS Home storage drivers
- Enh #1563: Implement cs3.sharing.collaboration.v1beta1.Share.ShareType
- Enh #1411: Make InsecureSkipVerify configurable
- Enh #1106: Make command to run litmus tests
- Enh #1502: Bump meshdirectory-web to v1.0.4
- Enh #1502: New MeshDirectory HTTP service UI frontend with project branding
- Enh #1405: Quota querying and tree accounting
- Enh #1527: Add FindAcceptedUsers method to OCM Invite API
- Enh #1149: Add CLI Commands for OCM invitation workflow
- Enh #1629: Implement checksums in the owncloud storage
- Enh #1528: Port drone pipeline definition to starlark
- Enh #110: Add signature authentication for public links
- Enh #1495: SQL driver for the publicshare service
- Enh #1588: Make the additional info attribute for shares configurable
- Enh #1595: Add site account registration panel
- Enh #1506: Site Accounts service for API keys
- Enh #116: Enhance storage registry with virtual views and regular expressions
- Enh #1513: Add stubs for storage spaces manipulation
Details
-
Bugfix #1619: Fixes for enabling file sharing in EOS
-
Bugfix #1576: Fix etag changing only once a second
We fixed a problem with the owncloud storage driver only considering the mtime with a second
resolution for the etag calculation. -
Bugfix #1634: Mentix site authorization status changes
If a site changes its authorization status, Mentix did not update its internal data to reflect
this change. This PR fixes this issue. -
Bugfix #1625: Make local file connector more error tolerant
The local file connector caused Reva to throw an exception if the local file for storing site
data couldn't be loaded. This PR changes this behavior so that only a warning is logged. -
Bugfix #1526: Fix webdav file versions endpoint bugs
Etag and error code related bugs have been fixed for the webdav file versions endpoint and
removed from the expected failures file. -
Bugfix #1457: Cloning of internal mesh data lost some values
This update fixes a bug in Mentix that caused some (non-critical) values to be lost during data
cloning that happens internally. -
Bugfix #1597: Check for ENOTDIR on readlink error
The deconstructed storage driver now handles ENOTDIR errors when
node.Child()
is called
for a path containing a path segment that is actually a file. -
Bugfix #1636: Skip file check for OCM data transfers
-
Bugfix #1552: Fix a bunch of trashbin related issues
Fixed these issues:
- Complete: Deletion time in trash bin shows a wrong date - Complete: shared trash status code -
Partly: invalid webdav responses for unauthorized requests. - Partly: href in trashbin
PROPFIND response is wrong
Complete means there are no expected failures left. Partly means there are some scenarios
left. - Complete: Deletion time in trash bin shows a wrong date - Complete: shared trash status code -
-
Bugfix #1: Bump meshdirectory-web to 1.0.2
Updated meshdirectory-web mod to version 1.0.2 that contains fixes for OCM invite API links
generation. -
Change #1562: Modularize api token management in GRAPPA drivers
This PR moves the duplicated api token management methods into a seperate utils package
-
Change #1452: Separate blobs from metadata in the ocis storage driver
We changed the ocis storage driver to keep the file content separate from the metadata by
storing the blobs in a separate directory. This allows for using a different (potentially
faster) storage for the metadata.Note This change makes existing ocis storages incompatible with the new code.
We also streamlined the ocis and the s3ng drivers so that most of the code is shared between them.
-
Enhancement #1514: Add grpc test suite for the storage provider
A new test suite has been added which tests the grpc interface to the storage provider. It
currently runs against the ocis and the owncloud storage drivers. -
Enhancement #1466: Add integration tests for the s3ng driver
We extended the integration test suite to also run all tests against the s3ng driver.
-
Enhancement #1521: Clarify expected failures
Some features, while covered by the ownCloud 10 acceptance tests, will not be implmented for
now: - blacklisted / ignored files, because ocis/reva don't need to blacklist.htaccess
files -OC-LazyOps
support was removed from the
clients. We are thinking about a state
machine for uploads to properly solve that scenario and also list the state of files in progress
in the web ui. The expected failures
files now have a dedicated Won't fix section for these items. -
Enhancement #1624: Add wrappers for EOS and EOS Home storage drivers
For CERNBox, we need the mount ID to be configured according to the owner of a resource. Setting
this in the storageprovider means having different instances of this service to cater to
different users, which does not scale. This driver forms a wrapper around the EOS driver and
sets the mount ID according to a configurable mapping based on the owner of the resource. -
Enhancement #1563: Implement cs3.sharing.collaboration.v1beta1.Share.ShareType
Interface method Share() in pkg/ocm/share/share.go now has a share type parameter.
-
Enhancement #1411: Make InsecureSkipVerify configurable
Add
InsecureSkipVerify
field tometrics.Config
struct and update examples to include
it. -
Enhancement #1106: Make command to run litmus tests
This updates adds an extra make command to run litmus tests via make.
make litmus-test
executes the tests. -
Enhancement #1502: Bump meshdirectory-web to v1.0.4
Updated meshdirectory-web version to v.1.0.4 bringing multiple UX improvements in provider
list and map.#1502
sciencemesh/meshdirectory-web#2
sciencemesh/meshdirectory-web#3 -
Enhancement #1502: New MeshDirectory HTTP service UI frontend with project branding
We replaced the temporary version of web frontend of the mesh directory http service with a new
redesigned & branded one. Because the new version is a more complex Vue SPA that contains image,
css and other assets, it is now served from a binary package distribution that was generated
using the github.com/rakyll/statik package. The
http.services.meshdirectory.static
config option was obsoleted by this change. -
Enhancement #1405: Quota querying and tree accounting
The ocs api now returns the user quota for the users home storage. Furthermore, the ocis storage
driver now reads the quota from the extended attributes of the user home or root node and
implements tree size accounting. Finally, ocdav PROPFINDS now handle the
DAV:quota-used-bytes
andDAV:quote-available-bytes
properties. -
Enhancement #1527: Add FindAcceptedUsers method to OCM Invite API
-
Enhancement #1149: Add CLI Commands for OCM invitation workflow
This adds a couple of CLI commands,
ocm-invite-generate
andocm-invite-forward
to
generate and forward ocm invitation tokens respectively. -
Enhancement #1629: Implement checksums in the owncloud storage
Implemented checksums in the owncloud storage driver.
-
Enhancement #1528: Port drone pipeline definition to starlark
Having the pipeline definition as a starlark script instead of plain yaml greatly improves the
flexibility and allows for removing lots of duplicated definitions.
...
v1.6.0
Changelog for reva 1.6.0 (2021-02-16)
The following sections list the changes in reva 1.6.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1425: Align href URL encoding with oc10
- Fix #1461: Fix public link webdav permissions
- Fix #1457: Cloning of internal mesh data lost some values
- Fix #1429: Purge non-empty dirs from trash-bin
- Fix #1408: Get error status from trash-bin response
- Enh #1451: Render additional share with in ocs sharing api
- Enh #1424: We categorized the list of expected failures
- Enh #1434: CERNBox REST driver for groupprovider service
- Enh #1400: Checksum support
- Enh #1431: Update npm packages to fix vulnerabilities
- Enh #1415: Indicate in EOS containers that TUS is not supported
- Enh #1402: Parse EOS sys ACLs to generate CS3 resource permissions
- Enh #1477: Set quota when creating home directory in EOS
- Enh #1416: Use updated etag of home directory even if it is cached
- Enh #1478: Enhance error handling for grappa REST drivers
- Enh #1453: Add functionality to share resources with groups
- Enh #99: Add stubs and manager for groupprovider service
- Enh #1462: Hash public share passwords
- Enh #1464: LDAP driver for the groupprovider service
- Enh #1430: Capture non-deterministic behavior on storages
- Enh #1456: Fetch user groups in OIDC and LDAP backend
- Enh #1429: Add s3ng storage driver, storing blobs in a s3-compatible blobstore
- Enh #1467: Align default location for xrdcopy binary
Details
-
Bugfix #1425: Align href URL encoding with oc10
We now use the same percent encoding for URLs in WebDAV href properties as ownCloud 10.
owncloud/ocis#1120
owncloud/ocis#1296
owncloud/ocis#1307
#1425
#1472 -
Bugfix #1461: Fix public link webdav permissions
We now correctly render
oc:permissions
on the root collection of a publicly shared folder
when it has more than read permissions. -
Bugfix #1457: Cloning of internal mesh data lost some values
This update fixes a bug in Mentix that caused some (non-critical) values to be lost during data
cloning that happens internally. -
Bugfix #1429: Purge non-empty dirs from trash-bin
This wasn't possible before if the directory was not empty
-
Bugfix #1408: Get error status from trash-bin response
Previously the status code was gathered from the wrong response.
-
Enhancement #1451: Render additional share with in ocs sharing api
Recipients can now be distinguished by their email, which is rendered as additional info in the
ocs api for share and file owners as well as share recipients. -
Enhancement #1424: We categorized the list of expected failures
We categorized all expected failures into File (Basic file management like up and download,
move, copy, properties, trash, versions and chunking), Sync (Synchronization features
like etag propagation, setting mtime and locking files), Share (File and sync features in a
shared scenario), User management (User and group management features) and Other (API,
search, favorites, config, capabilities, not existing endpoints, CORS and others). The
Review and fix the tests that have sharing step to work with
ocis reference has been removed, as we
now have the sharing category -
Enhancement #1434: CERNBox REST driver for groupprovider service
-
Enhancement #1400: Checksum support
We now support checksums on file uploads and PROPFIND results. On uploads, the ocdav service
now forwards theOC-Checksum
(and the similar TUSUpload-Checksum
) header to the storage
provider. We added an internal http status code that allows storage drivers to return checksum
errors. On PROPFINDs, ocdav now renders the<oc:checksum>
header in a bug compatible way for
oc10 backward compatibility with existing clients. Finally, GET and HEAD requests now return
theOC-Checksum
header.owncloud/ocis#1291
owncloud/ocis#1316
#1400
owncloud/core#38304 -
Enhancement #1431: Update npm packages to fix vulnerabilities
-
Enhancement #1415: Indicate in EOS containers that TUS is not supported
The OCDAV propfind response previously hardcoded the TUS headers due to which clients such as
phoenix used the TUS protocol for uploads, which EOS doesn't support. Now we pass this property
as an opaque entry in the containers metadata. -
Enhancement #1402: Parse EOS sys ACLs to generate CS3 resource permissions
-
Enhancement #1477: Set quota when creating home directory in EOS
-
Enhancement #1416: Use updated etag of home directory even if it is cached
We cache the home directory and shares folder etags as calculating these is an expensive
process. But if these directories were updated after the previously calculated etag was
cached, we can ignore this calculation and directly return the new one. -
Enhancement #1478: Enhance error handling for grappa REST drivers
-
Enhancement #1453: Add functionality to share resources with groups
-
Enhancement #99: Add stubs and manager for groupprovider service
Recently, there was a separation of concerns with regard to users and groups in CS3APIs. This PR
adds the required stubs and drivers for the group manager. -
Enhancement #1462: Hash public share passwords
The share passwords were only base64 encoded. Added hashing using bcrypt with configurable
hash cost. -
Enhancement #1464: LDAP driver for the groupprovider service
-
Enhancement #1430: Capture non-deterministic behavior on storages
As a developer creating/maintaining a storage driver I want to be able to validate the
atomicity of all my storage driver operations. * Test for: Start 2 uploads, pause the first one,
let the second one finish first, resume the first one at some point in time. Both uploads should
finish. Needs to result in 2 versions, last finished is the most recent version. * Test for:
Start 2 MKCOL requests with the same path, one needs to fail. -
Enhancement #1456: Fetch user groups in OIDC and LDAP backend
-
Enhancement #1429: Add s3ng storage driver, storing blobs in a s3-compatible blobstore
We added a new storage driver (s3ng) which stores the file metadata on a local filesystem
(reusing the decomposed filesystem of the ocis driver) and the actual content as blobs in any
s3-compatible blobstore. -
Enhancement #1467: Align default location for xrdcopy binary
v1.5.1
Changelog for reva 1.5.1 (2021-01-19)
The following sections list the changes in reva 1.5.1 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1401: Use the user in request for deciding the layout for non-home DAV requests
- Fix #1413: Re-include the '.git' dir in the Docker images to pass the version tag
- Fix #1399: Fix ocis trash-bin purge
- Enh #1397: Bump the Copyright date to 2021
- Enh #1398: Support site authorization status in Mentix
- Enh #1393: Allow setting favorites, mtime and a temporary etag
- Enh #1403: Support remote cloud gathering metrics
Details
-
Bugfix #1401: Use the user in request for deciding the layout for non-home DAV requests
For the incoming /dav/files/userID requests, we have different namespaces depending on
whether the request is for the logged-in user's namespace or not. Since in the storage drivers,
we specify the layout depending only on the user whose resources are to be accessed, this fails
when a user wants to access another user's namespace when the storage provider depends on the
logged in user's namespace. This PR fixes that.For example, consider the following case. The owncloud fs uses a layout {{substr 0 1
.Id.OpaqueId}}/{{.Id.OpaqueId}}. The user einstein sends a request to access a resource
shared with him, say /dav/files/marie/abcd, which should be allowed. However, based on the
way we applied the layout, there's no way in which this can be translated to /m/marie/. -
Bugfix #1413: Re-include the '.git' dir in the Docker images to pass the version tag
And git SHA to the release tool.
-
Bugfix #1399: Fix ocis trash-bin purge
Fixes the empty trash-bin functionality for ocis-storage
-
Enhancement #1397: Bump the Copyright date to 2021
-
Enhancement #1398: Support site authorization status in Mentix
This enhancement adds support for a site authorization status to Mentix. This way, sites
registered via a web app can now be excluded until authorized manually by an administrator.Furthermore, Mentix now sets the scheme for Prometheus targets. This allows us to also support
monitoring of sites that do not support the default HTTPS scheme. -
Enhancement #1393: Allow setting favorites, mtime and a temporary etag
We now let the ocis driver persist favorites, set temporary etags and the mtime as arbitrary
metadata. -
Enhancement #1403: Support remote cloud gathering metrics
The current metrics package can only gather metrics either from json files. With this feature,
the metrics can be gathered polling the http endpoints exposed by the owncloud/nextcloud
sciencemesh apps.
v1.5.0
Changelog for reva 1.5.0 (2021-01-12)
The following sections list the changes in reva 1.5.0 relevant to
reva users. The changes are ordered by importance.
Summary
- Fix #1385: Run changelog check only if there are changes in the code
- Fix #1333: Delete sdk unit tests
- Fix #1342: Dav endpoint routing to home storage when request is remote.php/dav/files
- Fix #1338: Fix fd leaks
- Fix #1343: Fix ocis move
- Fix #551: Fix purging deleted files with the ocis storage
- Fix #863: Fix dav api for trashbin
- Fix #204: Fix the ocs share with me response
- Fix #1351: Fix xattr.Remove error check for macOS
- Fix #1320: Do not panic on remote.php/dav/files/
- Fix #1379: Make Jaeger agent usable
- Fix #1331: Fix capabilities response for multiple client versions
- Fix #1281: When sharing via ocs look up user by username
- Fix #1334: Handle removal of public shares by token or ID
- Chg #990: Replace the user uuid with the username in ocs share responses
- Enh #1350: Add auth protocol based on user agent
- Enh #1362: Mark 'store-dev-release' CI step as failed on 4XX/5XX errors
- Enh #1364: Remove expired Link on Get
- Enh #1340: Add cache to store UID to UserID mapping in EOS
- Enh #1154: Add support for the protobuf interface to eos metadata
- Enh #1154: Merge-rebase from master 10/11/2020
- Enh #1359: Add cache for calculated etags for home and shares directory
- Enh #1321: Add support for multiple data transfer protocols
- Enh #1324: Log expected errors with debug level
- Enh #1351: Map errtypes to status
- Enh #1347: Support property to enable health checking on a service
- Enh #1332: Add import support to Mentix
- Enh #1371: Use self-hosted Drone CI
- Enh #1354: Map bad request and unimplement to http status codes
- Enh #929: Include share types in ocs propfind responses
- Enh #1328: Add CLI commands for public shares
- Enh #1388: Support range header in GET requests
- Enh #1361: Remove expired Link on Access
- Enh #1386: Docker image for cs3org/revad:VERSION-eos
- Enh #1368: Calculate and expose actual file permission set
Details
-
Bugfix #1385: Run changelog check only if there are changes in the code
-
Bugfix #1333: Delete sdk unit tests
These depend on a remote server running reva and thus fail in case of version mismatches.
-
Bugfix #1342: Dav endpoint routing to home storage when request is remote.php/dav/files
There was a regression in which we were not routing correctly to the right storage depending on
the url. -
Bugfix #1338: Fix fd leaks
There were some left over open file descriptors on simple.go.
-
Bugfix #1343: Fix ocis move
Use the old node id to build the target path for xattr updates.
-
Bugfix #551: Fix purging deleted files with the ocis storage
The ocis storage could load the owner information of a deleted file. This caused the storage to
not be able to purge deleted files. -
Bugfix #863: Fix dav api for trashbin
The api was comparing the requested username to the userid.
-
Bugfix #204: Fix the ocs share with me response
The path of the files shared with me was incorrect.
-
Bugfix #1351: Fix xattr.Remove error check for macOS
Previously, we checked the xattr.Remove error only for linux systems. Now macOS is checked
also -
Bugfix #1320: Do not panic on remote.php/dav/files/
Currently requests to /remote.php/dav/files/ result in panics since we cannot longer strip
the user + destination from the url. This fixes the server response code and adds an error body to
the response. -
Bugfix #1379: Make Jaeger agent usable
Previously, you could not use tracing with jaeger agent because the tracing connector is
always used instead of the tracing endpoint.This PR removes the defaults for collector and tracing endpoint.
-
Bugfix #1331: Fix capabilities response for multiple client versions
-
Bugfix #1281: When sharing via ocs look up user by username
The ocs api returns usernames when listing share recipients, so the lookup when creating the
share needs to search the usernames and not the userid. -
Bugfix #1334: Handle removal of public shares by token or ID
Previously different drivers handled removing public shares using different means, either
the token or the ID. Now, both the drivers support both these methods. -
Change #990: Replace the user uuid with the username in ocs share responses
The ocs api should not send the users uuid. Replaced the uuid with the username.
-
Enhancement #1350: Add auth protocol based on user agent
Previously, all available credential challenges are given to the client, for example, basic
auth, bearer token, etc ... Different clients have different priorities to use one method or
another, and before it was not possible to force a client to use one method without having a side
effect on other clients.This PR adds the functionality to target a specific auth protocol based on the user agent HTTP
header. -
Enhancement #1362: Mark 'store-dev-release' CI step as failed on 4XX/5XX errors
Prevent the errors while storing new 'daily' releases from going unnoticed on the CI.
-
Enhancement #1364: Remove expired Link on Get
There is the scenario in which a public link has expired but ListPublicLink has not run,
accessing a technically expired public share is still possible. -
Enhancement #1340: Add cache to store UID to UserID mapping in EOS
Previously, we used to send an RPC to the user provider service for every lookup of user IDs from
the UID stored in EOS. This PR adds an in-memory lock-protected cache to store this mapping. -
Enhancement #1154: Add support for the protobuf interface to eos metadata
-
Enhancement #1154: Merge-rebase from master 10/11/2020
-
Enhancement #1359: Add cache for calculated etags for home and shares directory
Since we store the references in the shares directory instead of actual resources, we need to
calculate the etag on every list/stat call. This is rather expensive so adding a cache would
help to a great extent with regard to the performance. -
Enhancement #1321: Add support for multiple data transfer protocols
Previously, we had to configure which data transfer protocol to use in the dataprovider
service. A previous PR added the functionality to redirect requests to different handlers
based on the request method but that would lead to conflicts if multiple protocols don't
support mutually exclusive sets of requests. This PR adds the functionality to have multiple
such handlers simultaneously and the client can choose which protocol to use. -
Enhancement #1324: Log expected errors with debug level
While trying to download a non existing file and reading a non existing attribute are
technically an error they are to be expected and nothing an admin can or even should act upon. -
Enhancement #1351: Map errtypes to status
When mapping errtypes to grpc statuses we now also map bad request and not implemented /
unsupported cases in the gateway storageprovider. -
Enhancement #1347: Support property to enable health checking on a service
This update introduces a new service property called
ENABLE_HEALTH_CHECKS
that must be
explicitly set totrue
if a service should be checked for its health status. This allows us to
only enable these checks for partner sites only, skipping vendor sites. -
Enhancement #1332: Add import support to Mentix
This update adds import support to Mentix, transforming it into a Mesh Entity Exchanger.
To properly support vendor site management, a new connector that works on a local file has been
added as well. -
Enhancement #1371: Use self-hosted Drone CI
Previously, we used the drone cloud to run the CI for the project. Due to unexpected and sudden
stop of the service for the cs3org we decided to self-host it. -
Enhancement #1354: Map bad request and unimplement to http status codes
We now return a 400 bad request when a grpc call fails with an invalid argument status and a 501 not
implemented when it fails with an unimplemented status. This prevents 500 errors when a user
tries to add resources to the Share folder or a storage does not implement an action. -
Enhancement #929: Include share types in ocs propfind responses
Added the share types to the ocs propfind response when a resource has been shared.
htt...