Client-Side Applications, Restricted Access and Application-Specific Tokens

[pik]

One particular deficit in the current Matrix Spec / Synapse implementation is the capability for non-registered users to gain read-only access to a room. There is a pretty common use case for this in terms of a client-side application built with Matrix as a messaging log backend.

For example consider an application for commenting a sports cast - only the authoring user sends writes to the room, while all other client-applications should be able to access the room with read-only permissions. In the latter case and for other similar use-cases the necessity of forcing users to register (at least they must as guests given the current spec/implementation) would hurt the usability of the client-side application. The alternative of providing all users with a mock account, even if its power-level is restricted from writing in the room in question - may still open an unnecessary window for abuse in other rooms permissive of guest access.

Possible Solutions

1. One possibility for this kind of read-only access is allowing client-applications to request 'ephemeral access' which has no user entry associated with it. In this case the room would have to have perhaps an additional readability state (on top of 'world_readable') which would be checked when a client requests a token for reading room events without a user account (user_info retrieval would have to account for this, e.g. by providing a mock user_info object).

2. An option, which could follow from perhaps a very different requirement would be for a user to create an application specific token that would have caveats/restrictions that they could then distribute publicly. e.g. Supposing Homeservers/Synapse supported something of the application-specific-token permission model (I'm sure there is a correct term for this in the literature but I mean e.g. Github API tokens) then a user authoring the relevant content for a client-application could request a token with read-only permissions for a particular room-id -- and then distribute this token to application users.
   This would be slightly more complicated than something like the 'ephemeral' token suggested above (since the room administrator would need to actively craft and distribute the tokens) but on the other-hand would be a particular use-case of a solution with other applications, rather than a solution limited to one particular problem/access restriction type.

Ways of restricting Application Specific Tokens

The two obvious choices I guess would be explicit DB based restrictions and Macaroon caveats. Since Synapse is already Caveat based I spent a little more time exploring the later, although since Caveats are no longer explicitly spec'd application-specific tokens for user accounts could also have an open implementation (via. DB stored parameters or caveats supposing a different HS also supported granting restricted tokens).

Thoughts on caveats and Macaroons

In this way for example a read-only access type to room events for a specific room might be crafted by adding the following caveats to an existing token:

'access_method = GET'
'room_id = <room_id>'

One thing that I'm not quite sure about is it seems that it would be necessary to restrict endpoint access to limited to a particular subset endpoints as well - particularly this is because of some weaknesses with both the current method of handling caveats and the possibility of e.g. <room_id> params or other exact caveat specifications to occur both in path params (e.g. /room/<room_id>/) and in query params (e.g. events?room_id=<room_id>).

One thing to note is that the current method of applying caveats adds verifications for caveat propositions which might not be provided by a Macaroon at all. e.g. (https://github.com/matrix-org/synapse/blob/master/synapse/api/auth.py#L812)

satisfy_exact('guest = true')

It relies on the behavior that if the guest = proposition is not provided then caveat verification is ignored. The other Macaroon spec requirement that all caveats be understood is checked separately in _verify_recognizes_caveats; this is somewhat weak (more on this later). Therefore when implementing application-specific restrictions, since a Macaroon may not or may not contain an additional caveat caveat e.g. room_id = <room_id>, but should always be verified with the current system, the method would have to be something like:

if caveat_prop_name in request.args:
    param = request.args[caveat_prop_name][0]
# In my testing I added a method to set `_kwargs` on the request when the kwargs dict is created by the HTTP Server dispatch method
# to make this kind of comparison possible
elif caveat_prop_name in request.kwargs:
    param = requets._kwargs[caveat_prop_name]
else:
    param = 'ANY'

verifier.satisfy_exact('%s = %s' %(caveat_var_name, param))

With something like this 'room_id = <room_id>' would need to match the arguments of the request, and a room_id specific caveat would always fail on 'ANY'. But the general permissiveness for query params is a disadvantage here (if permitted endpoints aren't explicitly specified in the caveat) since one could do something like /user_settings?room_id=<room_id> which would satisfy the caveat -- but not the permission restrictions the user had in mind when they crafted an application-token with that caveat.

Some other concerns are the current unfinished transfer of all user_info to Macaroons (listed as TODO https://github.com/matrix-org/synapse/blob/master/synapse/api/auth.py#L751):

Since _look_up_user_by_access_token is called even after Macaroon deserialization and validation then application-specific Macaroons would fail here - so either the base token must be regenerated, by applying all of the 'base' caveats without additional ones to a newly minted token (which might have performance consequences) or a different method of user look-up / token-expiration checking should be chosen.

Another concern with application-specific tokens and caveats is refresh_token, since refreshing the base token should in theory render all other tokens generated from the previous one invalid. In this case it might make sense for refresh to update only expiry and invalidate other short-lived tokens but not the base token (or otherwise provide a separate base-token for crafting application-specific tokens with additional caveats).

One alternative to the pre-emptive caveat verification method currently used by Synapse, that is the process of adding satisfy_exact statements for caveats which might or might not exist, would be something of a regex matcher for '<variable> = <param>' type propositions (similar to the way URL's are currently routed), each caveat provided by a Macaroon would have to be matched by at-least one pattern, which would then provide a verifier class for determining the the caveats validity. e.g. for room_id the verifier class might be something like:

class RoomIdCaveatVerifier:
    PATTERN = "(?P<variable>room_id) = (?P<param>![\w]+:[\w\-]+)"

    def __init__(self, caveat, context):
        # initialize with caveat and context

    def verify(self):
        # extract request - room_id from context check against <param>

A structure like this would also supercede the what seems to me like a somewhat vulnerable separation of concerns created by _verify_recognizes_caveats being a separate check from Synapse's ability to actually recognize them. Also it would be more extensible where caveat propositions required to do somewhat broader logic at any point e.g. '<variable> IN {<start>...<end>}'

[richvdh]

pik: I edited your description slightly to correct some typing errors. I hope I didn't misunderstand you.

Thanks very much for all your thoughts on this. There is a /lot/ of stuff here though, covering a range of aspects; I'm not sure this is an actionable issue as it stands.

Some random thoughts:

I like the idea of extending the macaroons to apply further restrictions; however I think it's a mistake to try to do it in a general way so that clients can apply arbitrary restrictions based on HTTP method, URL, and query-parameters: this is always going to be leaky. Better to specify higher-level caveats (allowed_room_ids, read_only) which the server then enforces appropriately across all APIs.

I'm not sure how plausible it is to have clients able to add a room_id caveat to an existing access-token: think of something like /sync, which returns data from many rooms (as well as non-room-specific information) - how would a room_id caveat apply to that? You could restrict the results from /sync based on the caveat, but that would mix up the authorisation with the behaviour in a bad way. Better to have separate users who are restricted in which rooms they can join/peek in. Or maybe even users that are joined to specific rooms as soon as they are created, and cannot join or peek in any others.

I don't necessarily think that having to register with the server necessarily harms UX. Your client, or indeed a server-side application can automatically register with the matrix server; provided this can be done without user-interaction there is no need for it to affect UX.

[pik]

I don't necessarily think that having to register with the server necessarily harms UX. Your client, or indeed a server-side application can automatically register with the matrix server; provided this can be done without user-interaction there is no need for it to affect UX.

Yeah any automatic registration to a restricted permission set would also work (guest accounts are really too powerful I think to hand out automatically atm). So an ephemeral kind of access token (as I mentioned above) or being able to register a guest account with further restrictions would work for my particular use-cases.

My concern with any too specific implementation is that next time someone needs a different kind of restrictive access e.g. consider a user wants to give an application read/write but only to a particular room_id then there is another unsolved problem.

I like the idea of extending the macaroons to apply further restrictions; however I think it's a mistake to try to do it in a general way so that clients can apply arbitrary restrictions based on HTTP method, URL, and query-parameters: this is always going to be leaky. Better to specify higher-level caveats (allowed_room_ids, read_only) which the server then enforces appropriately across all APIs.

I'm not sure how plausible it is to have clients able to add a room_id caveat to an existing access-token: think of something like /sync, which returns data from many rooms (as well as non-room-specific information) - how would a room_id caveat apply to that? You could restrict the results from /sync based on the caveat, but that would mix up the authorisation with the behaviour in a bad way.

Yeah I was also worried about this as well. I agree that endpoint restrictions sounds very leaky and having to pass caveat conditions down to behavior (e.g. handlers.get_stream(caveat_restrictions, *args) doesn't sound very nice.

What about making restrictions based on creating a kind of sub-user? Than a token would fetch (whether from DB or by parsing the macaroon) user_info with any valid subset of existing user_info. So if the current user is a member of rooms A, B, C - a restricted token might retrieve a 'mock' version of that user who is only a member of room B. At least that would solve the /sync issue.
For request method, I guess just read, write, read/write might be high level enough or alternatively allow the mock user token to specify a restricted a power-level for joined rooms. e.g.

joined_rooms = { 
  room_id: 'any room the original user is a member of', 
  access_type: ['read', 'write', 'read/write'] //one of
  // Or
  power_level: 'any power level equal or lower to the original user's' 
}

[ptman]

oauth2 + scopes? matrix-org/matrix-spec#636