MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC

[hughns]

Rendered

Dependencies/sub-proposals:

• MSC2964: Usage of OAuth 2.0 authorization code grant and refresh token grant #2964
• MSC2965: OIDC Provider discovery #2965
• MSC2966: Usage of OAuth 2.0 Dynamic Client Registration in Matrix #2966
• MSC2967: OIDC API scopes #2967
• MSC3824: OIDC aware clients #3824
• MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code #4108
• MSC4190: Device management for application services #4190
• MSC4191: Account management deep-linking #4191

[MTRNord]

Please make sure to check if those are actually solved by this when doing it. There might be some around that OIDC does not fix but actually complicates, like the Concealed login MSC.

[sandhose]

That is a good point. We should spend some time identifying those MSCs sooner than later, and see for each MSC if either:

• it solves the problem solved by the MSC
• the tradeoffs are acceptable if it is conceptually orthogonal

[MTRNord]

Since now half a year went by: is there any update on which mscs are compatible and which aren't?

[hughns]

@MTRNord there is a start of a list at https://github.com/matrix-org/matrix-spec-proposals/blob/hughns/delegated-oidc-architecture/proposals/3861-delegated-oidc-architecture.md#alternatives

[turt2live]

just a mid-read review. Submitting before I forget about it

[MTRNord]

I am not sure if there is a better place, but:

How do we protect from ending up in the same situation as email, where big companies (like google/microsoft) end up using OIDC/OAuth2 purely to gatekeep the clients that are allowed to use? As far as I understand, matrix is supposed to be an open protocol where anyone is allowed to bring their own tools and software. So how do we make sure that this is still possible? I am aware of certain enterprise parts of matrix that want to restrict client usage, however imho this isn't something that should leak into spec.

The reasoning behind is that the Matrix Manifesto ( https://matrix.org/foundation/ ) clearly defines that we do want to be an open communication network and people shall have full control over their communication. If we end up however in a situation where we give HS admins a (too) easy way to restrict the clients that are usable we would break the goals of that manifesto at it's core, since then it's not open anymore for the users and Singular community managers can decide what tools to use with 0 protection for the users.

(edit: typos)

[MTRNord]

For additional source of my email reference, see the comment made by a thunderbird dev in https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat#OAuth2

Due to a defficiency in the OAuth2 spec, the client is usually required to send a client credential key, which in turn requires the client to be registered and approved by the email provider. Unfortunately, this not only allows email providers to block specific email clients (which is contrary to the idea of Open-Source), but also makes it impossible to support arbitrary OAuth2 servers. That's why Thunderbird is forced to hardcode the servers that it supports and the respecive client keys. That means that you cannot use OAuth2 for your own server. Only the servers listed on OAuth2Providers.jsm will work.

I am aware of the dynamic registration MSC for OIDC, but that only somewhat solves this, since there a provider still is able to refuse your client at any given time without explanation.

[sandhose]

I'm not sure how to properly address this to be honest. MSC2966 outlines what the client must advertise, and what the homeserver should accept. If the HS doesn't accept a valid registration, I would argue that this HS is no longer Matrix-compliant on the C-S API side?

This was always a possibility in the ecosystem: nothing prevented a homeserver to have an allow-list of valid redirect URLs for the m.login.sso flow.
What this proposal change, is that the homeserver has more metadata on the client than before to work with, which is a good thing for giving homeserver admins and users a better idea of what is happening with their account, who has access to it.

Note that the notes from Thunderbird are completely valid for email, but not for Matrix, as this 'many client, many homeservers' constraint means we have to have some sort of open client registration. Mastodon has something similar, Bluesky/ATProto as well… I'm not sure why it would not work for Matrix

[hughns]

I've added references to the following MSCs that this proposal assumes are generally accepted:

• MSC3967: Do not require UIA when first uploading cross signing keys #3967
• MSC3970: Scope transaction IDs to devices #3970

[clokep]

From reviewing the Synapse implementation it seems that this MSC is missing information on how application services would register users. matrix-org/synapse#15582 keeps the old /register endpoint active for the m.login.application_service login type only.

This MSC needs to include some information IMO about how this interacts with application services.

[MTRNord]

I might be unable to find the right line in this huge amount of MSCs, but how does this new system translate users to mxids for events There needs to be some mapping right?

[sandhose]

I'm not sure I understand what you mean here. This proposal doesn't touch on events, and MXIDs are not really a concern throughout the flow. Could you clarify what you're asking here?

[tranquillity-codes]

Do I understand right that should this be merged into the spec, every client would have to support viewing or opening arbitrary websites with possibly complex CSS and JavaScript, making implementing a Matrix client essentially impossible on lightweight operating systems that lack support by any major web browser?

[programmerjake]

Do I understand right that should this be merged into the spec, every client would have to support viewing or opening arbitrary websites with possibly complex CSS and JavaScript, making implementing a Matrix client essentially impossible on lightweight operating systems that lack support by any major web browser?

maybe there's a way to use QR codes to transfer the necessary data so you can authenticate on some other system that supports web browsers

[tranquillity-codes]

Do I understand right that should this be merged into the spec, every client would have to support viewing or opening arbitrary websites with possibly complex CSS and JavaScript, making implementing a Matrix client essentially impossible on lightweight operating systems that lack support by any major web browser?

maybe there's a way to use QR codes to transfer the necessary data so you can authenticate on some other system that supports web browsers

That would be quite a subpar user experience, assuming said users even have access to such a device.

[richvdh]

@tranquillity-codes, @programmerjake: please leave comments on the files tab rather than the main PR, so that there is some semblance of threading.

[XxSNiPxX]

Hello guys,

My current project requires me to authenticate a user on my synapse instance via a signed message from a keplr crypto wallet . I have a wallet signed message used as the password. How would i approach the authentication on matrix? Would i use another server that is openid compatible, whose license is often not gplv3 or would i use the architecture before by contacting the synapse instance directly? Can someone please share and end to end example of setting up a IAM server between the client and synapse?

Thanks

[Xiretza]

The first three of these points may be relevant for large, open-registration servers like matrix.org, but I'm gonna go out on a limb and say that most servers out there are not that. In general, there needs to be extremely solid justification for forcing this increase in complexity onto all the servers that have nothing or very little to gain from it.

[sandhose]

This section talks about one particular example, which is also the homeserver where a lot of users will first experience this flow.

I'm not sure I understand the second part of your comment, how is this section talking about increasing the complexity onto all servers?

[Xiretza]

I'm not sure I understand the second part of your comment, how is this section talking about increasing the complexity onto all servers?

It's not about that section, which is why I wrote "in general" - the rewritten MSC later mentions:

The long-term goal is to deprecate the existing UIA APIs and replace them with the new OAuth 2.0/OIDC-based APIs.

This means that in the long term, all homeserver operators have to set up an entire OAuth/OIDC infrastructure, which is significantly more complex than the built-in username+password auth that is entirely sufficient for what I claim to be the vast majority of homeservers out there. Or am I misunderstanding something?

[sandhose]

The change is relatively transparent for server operators, it is an API change, not an architecture change.

It happens that the current implementation with matrix-authentication-service and Synapse is a separate service, but the long term goal is to be integrated just within synapse, so the change will be fairly transparent for server operators

Does that answer your concern?

[Xiretza]

the long term goal is to be integrated just within synapse, so the change will be fairly transparent for server operators

Synapse operators. What about other homeservers?

Actually, putting on my homeserver developer hat, this really only shifts the problem from operators to developers. Right now, a homeserver implementation that can cover most usecases (i.e. except for the "large, open-registration" niche) using a very simple, almost stub-like UIA implementation. I admit I haven't actually looked into it properly yet, but I suspect that OIDC will make even a minimal implementation much more complex.

[sandhose]

I suggest reading the rest of the proposal, as this is not about imposing stuff like email verification, CAPTCHA support, etc. on homeservers.

A very lightweight implementation of this can be done only supporting simple login and password. At a very high level, the new API look like this:

• the client sends its metadata (name, redirect URL, etc.) to the homeserver, gets back a client identifier
• the client redirects the user to a web page hosted by the homeserver
• the homeserver does whatever it needs on that homepage to authenticate the user, this can be a very simple HTML form
• once completed, the homeserver redirect backs to the client with single-use code
• the client exchanges this code to get back an access token

How does that feel in terms of complexity? I understand that it is substantially higher than just sending 'username and password' to an API and get back an access token, but the bar doesn't seem too high for new implementation, especially considering there are plenty of OAuth/OIDC related libraries in many ecosystems to help implement some of this.

[Xiretza]

One use case I have somewhat frequently is setting up bots. Currently, you can just type in username+password, it does the authentication and stores the access token somewhere. With OIDC, I'd already be stuck at the first step: what is the "redirect URL" for a bot running on a headless server somewhere? Step 2, "redirecting" the user to a login page, is still possible by making them click on a link, but this is then opened on the user's machine, which is totally separate from the machine running the bot. There would need to be a third service somewhere that displays the single-use code, which then needs to be copy-pasted back into the bot. Not only is this more annoying to implement, it's also much more complicated to use.

So no, I'm not on board with removing username+password auth, but how about this as a long term plan:

• Strip down UIA to a bare minimum username+password (this removes most of the complexity there and still covers most use cases),
• Optionally let servers force the use of OIDC, which can then include all the captcha/email/whathaveyou bells and whistles for complex installations without having to cover these complexities in the Matrix spec.

[sandhose]

The long-awaited big rewrite is here!

I've been working on making this series of MSCs a lot better, and I think I addressed a lot of concerns made on it. I commented on most open threads how I believe they are now addressed and closed them. If you believe those concerns are still not addressed, please reopen them or open new threads on the new MSC text

Happy reading!

[sandhose]

I'm not sure how to properly address this to be honest. MSC2966 outlines what the client must advertise, and what the homeserver should accept. If the HS doesn't accept a valid registration, I would argue that this HS is no longer Matrix-compliant on the C-S API side?

This was always a possibility in the ecosystem: nothing prevented a homeserver to have an allow-list of valid redirect URLs for the m.login.sso flow.
What this proposal change, is that the homeserver has more metadata on the client than before to work with, which is a good thing for giving homeserver admins and users a better idea of what is happening with their account, who has access to it.

Note that the notes from Thunderbird are completely valid for email, but not for Matrix, as this 'many client, many homeservers' constraint means we have to have some sort of open client registration. Mastodon has something similar, Bluesky/ATProto as well… I'm not sure why it would not work for Matrix

[sandhose]

This section talks about one particular example, which is also the homeserver where a lot of users will first experience this flow.

I'm not sure I understand the second part of your comment, how is this section talking about increasing the complexity onto all servers?

[Tynach]

The long-awaited big rewrite is here!

Shouldn't a big rewrite be a separate pull request/proposal? That way it's easier to distinguish between discussion about the old one, and discussion about the new one?

Edit: Though, I suppose doing that would require updating links everywhere that link to this proposal, and it would have potentially been less discoverable. Wish I'd have thought of that a few minutes earlier than I did.