MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

hughns · 2022-08-04T16:50:31Z

Rendered

Core sub-proposals:

Additional proposals:

SCT stuff:

checklist

FCP tickyboxes

proposals/3861-delegated-oidc-architecture.md

…ndation

turt2live

just a mid-read review. Submitting before I forget about it

proposals/3861-delegated-oidc-architecture.md

Co-authored-by: greizgh <greizgh@ephax.org>

proposals/3861-delegated-oidc-architecture.md

hughns · 2023-02-28T16:54:37Z

I've added references to the following MSCs that this proposal assumes are generally accepted:

proposals/3861-delegated-oidc-architecture.md

Synapse 1.86.0 (2023-06-20) =========================== No significant changes since 1.86.0rc2. Synapse 1.86.0rc2 (2023-06-14) ============================== Bugfixes -------- - Fix an error when having workers of different versions running. ([\matrix-org#15774](matrix-org#15774)) Synapse 1.86.0rc1 (2023-06-13) ============================== This version was tagged but never released. Features -------- - Stable support for [MSC3882](matrix-org/matrix-spec-proposals#3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\matrix-org#15388](matrix-org#15388)) - Support resolving a room's [canonical alias](https://spec.matrix.org/v1.7/client-server-api/#mroomcanonical_alias) via the module API. ([\matrix-org#15450](matrix-org#15450)) - Enable support for [MSC3952](matrix-org/matrix-spec-proposals#3952): intentional mentions. ([\matrix-org#15520](matrix-org#15520)) - Experimental [MSC3861](matrix-org/matrix-spec-proposals#3861) support: delegate auth to an OIDC provider. ([\matrix-org#15582](matrix-org#15582)) - Add Synapse version deploy annotations to Grafana dashboard which enables easy correlation between behavior changes witnessed in a graph to a certain Synapse version and nail down regressions. ([\matrix-org#15674](matrix-org#15674)) - Add a catch-all * to the supported relation types when redacting an event and its related events. This is an update to [MSC3912](matrix-org/matrix-spec-proposals#3861) implementation. ([\matrix-org#15705](matrix-org#15705)) - Speed up `/messages` by backfilling in the background when there are no backward extremities where we are directly paginating. ([\matrix-org#15710](matrix-org#15710)) - Expose a metric reporting the database background update status. ([\matrix-org#15740](matrix-org#15740)) Bugfixes -------- - Correctly clear caches when we delete a room. ([\matrix-org#15609](matrix-org#15609)) - Check permissions for enabling encryption earlier during room creation to avoid creating broken rooms. ([\matrix-org#15695](matrix-org#15695)) Improved Documentation ---------------------- - Simplify query to find participating servers in a room. ([\matrix-org#15732](matrix-org#15732)) Internal Changes ---------------- - Log when events are (maybe unexpectedly) filtered out of responses in tests. ([\matrix-org#14213](matrix-org#14213)) - Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters`. ([\matrix-org#15649](matrix-org#15649)) - Add support for tracing functions which return `Awaitable`s. ([\matrix-org#15650](matrix-org#15650)) - Cache requests for user's devices over federation. ([\matrix-org#15675](matrix-org#15675)) - Add fully qualified docker image names to Dockerfiles. ([\matrix-org#15689](matrix-org#15689)) - Remove some unused code. ([\matrix-org#15690](matrix-org#15690)) - Improve type hints. ([\matrix-org#15694](matrix-org#15694), [\matrix-org#15697](matrix-org#15697)) - Update docstring and traces on `maybe_backfill()` functions. ([\matrix-org#15709](matrix-org#15709)) - Add context for when/why to use the `long_retries` option when sending Federation requests. ([\matrix-org#15721](matrix-org#15721)) - Removed some unused fields. ([\matrix-org#15723](matrix-org#15723)) - Update federation error to more plainly explain we can only authorize our own membership events. ([\matrix-org#15725](matrix-org#15725)) - Prevent the `latest_deps` and `twisted_trunk` daily GitHub Actions workflows from running on forks of the codebase. ([\matrix-org#15726](matrix-org#15726)) - Improve performance of user directory search. ([\matrix-org#15729](matrix-org#15729)) - Remove redundant table join with `room_memberships` when doing a `is_host_joined()`/`is_host_invited()` call (`membership` is already part of the `current_state_events`). ([\matrix-org#15731](matrix-org#15731)) - Remove superfluous `room_memberships` join from background update. ([\matrix-org#15733](matrix-org#15733)) - Speed up typechecking CI. ([\matrix-org#15752](matrix-org#15752)) - Bump minimum supported Rust version to 1.60.0. ([\matrix-org#15768](matrix-org#15768)) * Bump importlib-metadata from 6.1.0 to 6.6.0. ([\matrix-org#15711](matrix-org#15711)) * Bump library/redis from 6-bullseye to 7-bullseye in /docker. ([\matrix-org#15712](matrix-org#15712)) * Bump log from 0.4.18 to 0.4.19. ([\matrix-org#15761](matrix-org#15761)) * Bump phonenumbers from 8.13.11 to 8.13.13. ([\matrix-org#15763](matrix-org#15763)) * Bump pyasn1 from 0.4.8 to 0.5.0. ([\matrix-org#15713](matrix-org#15713)) * Bump pydantic from 1.10.8 to 1.10.9. ([\matrix-org#15762](matrix-org#15762)) * Bump pyo3-log from 0.8.1 to 0.8.2. ([\matrix-org#15759](matrix-org#15759)) * Bump pyopenssl from 23.1.1 to 23.2.0. ([\matrix-org#15765](matrix-org#15765)) * Bump regex from 1.7.3 to 1.8.4. ([\matrix-org#15769](matrix-org#15769)) * Bump sentry-sdk from 1.22.1 to 1.25.0. ([\matrix-org#15714](matrix-org#15714)) * Bump sentry-sdk from 1.25.0 to 1.25.1. ([\matrix-org#15764](matrix-org#15764)) * Bump serde from 1.0.163 to 1.0.164. ([\matrix-org#15760](matrix-org#15760)) * Bump types-jsonschema from 4.17.0.7 to 4.17.0.8. ([\matrix-org#15716](matrix-org#15716)) * Bump types-pyopenssl from 23.1.0.2 to 23.2.0.0. ([\matrix-org#15766](matrix-org#15766)) * Bump types-requests from 2.31.0.0 to 2.31.0.1. ([\matrix-org#15715](matrix-org#15715))

@Fizzadar

Synapse 1.86.0 (2023-06-20) =========================== No significant changes since 1.86.0rc2. Synapse 1.86.0rc2 (2023-06-14) ============================== Bugfixes -------- - Fix an error when having workers of different versions running. ([\#15774](matrix-org/synapse#15774)) Synapse 1.86.0rc1 (2023-06-13) ============================== This version was tagged but never released. Features -------- - Stable support for [MSC3882](matrix-org/matrix-spec-proposals#3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\#15388](matrix-org/synapse#15388)) - Support resolving a room's [canonical alias](https://spec.matrix.org/v1.7/client-server-api/#mroomcanonical_alias) via the module API. ([\#15450](matrix-org/synapse#15450)) - Enable support for [MSC3952](matrix-org/matrix-spec-proposals#3952): intentional mentions. ([\#15520](matrix-org/synapse#15520)) - Experimental [MSC3861](matrix-org/matrix-spec-proposals#3861) support: delegate auth to an OIDC provider. ([\#15582](matrix-org/synapse#15582)) - Add Synapse version deploy annotations to Grafana dashboard which enables easy correlation between behavior changes witnessed in a graph to a certain Synapse version and nail down regressions. ([\#15674](matrix-org/synapse#15674)) - Add a catch-all * to the supported relation types when redacting an event and its related events. This is an update to [MSC3912](matrix-org/matrix-spec-proposals#3861) implementation. ([\#15705](matrix-org/synapse#15705)) - Speed up `/messages` by backfilling in the background when there are no backward extremities where we are directly paginating. ([\#15710](matrix-org/synapse#15710)) - Expose a metric reporting the database background update status. ([\#15740](matrix-org/synapse#15740)) Bugfixes -------- - Correctly clear caches when we delete a room. ([\#15609](matrix-org/synapse#15609)) - Check permissions for enabling encryption earlier during room creation to avoid creating broken rooms. ([\#15695](matrix-org/synapse#15695)) Improved Documentation ---------------------- - Simplify query to find participating servers in a room. ([\#15732](matrix-org/synapse#15732)) Internal Changes ---------------- - Log when events are (maybe unexpectedly) filtered out of responses in tests. ([\#14213](matrix-org/synapse#14213)) - Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters`. ([\#15649](matrix-org/synapse#15649)) - Add support for tracing functions which return `Awaitable`s. ([\#15650](matrix-org/synapse#15650)) - Cache requests for user's devices over federation. ([\#15675](matrix-org/synapse#15675)) - Add fully qualified docker image names to Dockerfiles. ([\#15689](matrix-org/synapse#15689)) - Remove some unused code. ([\#15690](matrix-org/synapse#15690)) - Improve type hints. ([\#15694](matrix-org/synapse#15694), [\#15697](matrix-org/synapse#15697)) - Update docstring and traces on `maybe_backfill()` functions. ([\#15709](matrix-org/synapse#15709)) - Add context for when/why to use the `long_retries` option when sending Federation requests. ([\#15721](matrix-org/synapse#15721)) - Removed some unused fields. ([\#15723](matrix-org/synapse#15723)) - Update federation error to more plainly explain we can only authorize our own membership events. ([\#15725](matrix-org/synapse#15725)) - Prevent the `latest_deps` and `twisted_trunk` daily GitHub Actions workflows from running on forks of the codebase. ([\#15726](matrix-org/synapse#15726)) - Improve performance of user directory search. ([\#15729](matrix-org/synapse#15729)) - Remove redundant table join with `room_memberships` when doing a `is_host_joined()`/`is_host_invited()` call (`membership` is already part of the `current_state_events`). ([\#15731](matrix-org/synapse#15731)) - Remove superfluous `room_memberships` join from background update. ([\#15733](matrix-org/synapse#15733)) - Speed up typechecking CI. ([\#15752](matrix-org/synapse#15752)) - Bump minimum supported Rust version to 1.60.0. ([\#15768](matrix-org/synapse#15768)) ### Updates to locked dependencies * Bump importlib-metadata from 6.1.0 to 6.6.0. ([\#15711](matrix-org/synapse#15711)) * Bump library/redis from 6-bullseye to 7-bullseye in /docker. ([\#15712](matrix-org/synapse#15712)) * Bump log from 0.4.18 to 0.4.19. ([\#15761](matrix-org/synapse#15761)) * Bump phonenumbers from 8.13.11 to 8.13.13. ([\#15763](matrix-org/synapse#15763)) * Bump pyasn1 from 0.4.8 to 0.5.0. ([\#15713](matrix-org/synapse#15713)) * Bump pydantic from 1.10.8 to 1.10.9. ([\#15762](matrix-org/synapse#15762)) * Bump pyo3-log from 0.8.1 to 0.8.2. ([\#15759](matrix-org/synapse#15759)) * Bump pyopenssl from 23.1.1 to 23.2.0. ([\#15765](matrix-org/synapse#15765)) * Bump regex from 1.7.3 to 1.8.4. ([\#15769](matrix-org/synapse#15769)) * Bump sentry-sdk from 1.22.1 to 1.25.0. ([\#15714](matrix-org/synapse#15714)) * Bump sentry-sdk from 1.25.0 to 1.25.1. ([\#15764](matrix-org/synapse#15764)) * Bump serde from 1.0.163 to 1.0.164. ([\#15760](matrix-org/synapse#15760)) * Bump types-jsonschema from 4.17.0.7 to 4.17.0.8. ([\#15716](matrix-org/synapse#15716)) * Bump types-pyopenssl from 23.1.0.2 to 23.2.0.0. ([\#15766](matrix-org/synapse#15766)) * Bump types-requests from 2.31.0.0 to 2.31.0.1. ([\#15715](matrix-org/synapse#15715)) Synapse 1.85.2 (2023-06-08) =========================== Bugfixes -------- - Fix regression where using TLS for HTTP replication between workers did not work. Introduced in v1.85.0. ([\#15746](matrix-org/synapse#15746)) Synapse 1.85.1 (2023-06-07) =========================== Note: this release only fixes a bug that stopped some deployments from upgrading to v1.85.0. There is no need to upgrade to v1.85.1 if successfully running v1.85.0. Bugfixes -------- - Fix bug in schema delta that broke upgrades for some deployments. Introduced in v1.85.0. ([\#15738](matrix-org/synapse#15738), [\#15739](matrix-org/synapse#15739)) Synapse 1.85.0 (2023-06-06) =========================== No significant changes since 1.85.0rc2. ## Security advisory The following issues are fixed in 1.85.0 (and RCs). - [GHSA-26c5-ppr8-f33p](GHSA-26c5-ppr8-f33p) / [CVE-2023-32682](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32682) — Low Severity It may be possible for a deactivated user to login when using uncommon configurations. - [GHSA-98px-6486-j7qc](GHSA-98px-6486-j7qc) / [CVE-2023-32683](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32683) — Low Severity A discovered oEmbed or image URL can bypass the `url_preview_url_blacklist` setting potentially allowing server side request forgery or bypassing network policies. Impact is limited to IP addresses allowed by the `url_preview_ip_range_blacklist` setting (by default this only allows public IPs). See the advisories for more details. If you have any questions, email security@matrix.org. Synapse 1.85.0rc2 (2023-06-01) ============================== Bugfixes -------- - Fix a performance issue introduced in Synapse v1.83.0 which meant that purging rooms was very slow and database-intensive. ([\#15693](matrix-org/synapse#15693)) Deprecations and Removals ------------------------- - Deprecate calling the `/register` endpoint with an unspecced `user` property for application services. ([\#15703](matrix-org/synapse#15703)) Internal Changes ---------------- - Speed up background jobs `populate_full_user_id_user_filters` and `populate_full_user_id_profiles`. ([\#15700](matrix-org/synapse#15700)) Synapse 1.85.0rc1 (2023-05-30) ============================== Features -------- - Improve performance of backfill requests by performing backfill of previously failed requests in the background. ([\#15585](matrix-org/synapse#15585)) - Add a new [admin API](https://matrix-org.github.io/synapse/v1.85/usage/administration/admin_api/index.html) to [create a new device for a user](https://matrix-org.github.io/synapse/v1.85/admin_api/user_admin_api.html#create-a-device). ([\#15611](matrix-org/synapse#15611)) - Add Unix socket support for Redis connections. Contributed by Jason Little. ([\#15644](matrix-org/synapse#15644)) Bugfixes -------- - Fix a long-standing bug where setting the read marker could fail when using message retention. Contributed by Nick @ Beeper (@Fizzadar). ([\#15464](matrix-org/synapse#15464)) - Fix a long-standing bug where the `url_preview_url_blacklist` configuration setting was not applied to oEmbed or image URLs found while previewing a URL. ([\#15601](matrix-org/synapse#15601)) - Fix a long-standing bug where filters with multiple backslashes were rejected. ([\#15607](matrix-org/synapse#15607)) - Fix a bug introduced in Synapse 1.82.0 where the error message displayed when validation of the `app_service_config_files` config option fails would be incorrectly formatted. ([\#15614](matrix-org/synapse#15614)) - Fix a long-standing bug where deactivated users were still able to login using the custom `org.matrix.login.jwt` login type (if enabled). ([\#15624](matrix-org/synapse#15624)) - Fix a long-standing bug where deactivated users were able to login in uncommon situations. ([\#15634](matrix-org/synapse#15634)) Improved Documentation ---------------------- - Warn users that at least 3.75GB of space is needed for the nix Synapse development environment. ([\#15613](matrix-org/synapse#15613)) - Remove outdated comment from the generated and sample homeserver log configs. ([\#15648](matrix-org/synapse#15648)) - Improve contributor docs to make it more clear that Rust is a necessary prerequisite. Contributed by @grantm. ([\#15668](matrix-org/synapse#15668)) Deprecations and Removals ------------------------- - Remove the old version of the R30 (30-day retained users) phone-home metric. ([\#10428](matrix-org/synapse#10428)) Internal Changes ---------------- - Create dependabot changelogs at release time. ([\#15481](matrix-org/synapse#15481)) - Add not null constraint to column `full_user_id` of tables `profiles` and `user_filters`. ([\#15537](matrix-org/synapse#15537)) - Allow connecting to HTTP Replication Endpoints by using `worker_name` when constructing the request. ([\#15578](matrix-org/synapse#15578)) - Make the `thread_id` column on `event_push_actions`, `event_push_actions_staging`, and `event_push_summary` non-null. ([\#15597](matrix-org/synapse#15597)) - Run mypy type checking with the minimum supported Python version to catch new usage that isn't backwards-compatible. ([\#15602](matrix-org/synapse#15602)) - Fix subscriptable type usage in Python <3.9. ([\#15604](matrix-org/synapse#15604)) - Update internal terminology. ([\#15606](matrix-org/synapse#15606), [\#15620](matrix-org/synapse#15620)) - Instrument `state` and `state_group` storage-related operations to better picture what's happening when tracing. ([\#15610](matrix-org/synapse#15610), [\#15647](matrix-org/synapse#15647)) - Trace how many new events from the backfill response we need to process. ([\#15633](matrix-org/synapse#15633)) - Re-type config paths in `ConfigError`s to be `StrSequence`s instead of `Iterable[str]`s. ([\#15615](matrix-org/synapse#15615)) - Update Mutual Rooms ([MSC2666](matrix-org/matrix-spec-proposals#2666)) implementation to match new proposal text. ([\#15621](matrix-org/synapse#15621)) - Remove the unstable identifiers from faster joins ([MSC3706](matrix-org/matrix-spec-proposals#3706)). ([\#15625](matrix-org/synapse#15625)) - Fix the olddeps CI. ([\#15626](matrix-org/synapse#15626)) - Remove duplicate timestamp from test logs (`_trial_temp/test.log`). ([\#15636](matrix-org/synapse#15636)) - Fix two memory leaks in `trial` test runs. ([\#15630](matrix-org/synapse#15630)) - Limit the size of the `HomeServerConfig` cache in trial test runs. ([\#15646](matrix-org/synapse#15646)) - Improve type hints. ([\#15658](matrix-org/synapse#15658), [\#15659](matrix-org/synapse#15659)) - Add requesting user id parameter to key claim methods in `TransportLayerClient`. ([\#15663](matrix-org/synapse#15663)) - Speed up rebuilding of the user directory for local users. ([\#15665](matrix-org/synapse#15665)) - Implement "option 2" for [MSC3820](matrix-org/matrix-spec-proposals#3820): Room version 11. ([\#15666](matrix-org/synapse#15666), [\#15678](matrix-org/synapse#15678)) ### Updates to locked dependencies * Bump furo from 2023.3.27 to 2023.5.20. ([\#15642](matrix-org/synapse#15642)) * Bump log from 0.4.17 to 0.4.18. ([\#15681](matrix-org/synapse#15681)) * Bump prometheus-client from 0.16.0 to 0.17.0. ([\#15682](matrix-org/synapse#15682)) * Bump pydantic from 1.10.7 to 1.10.8. ([\#15685](matrix-org/synapse#15685)) * Bump pygithub from 1.58.1 to 1.58.2. ([\#15643](matrix-org/synapse#15643)) * Bump requests from 2.28.2 to 2.31.0. ([\#15651](matrix-org/synapse#15651)) * Bump sphinx from 6.1.3 to 6.2.1. ([\#15641](matrix-org/synapse#15641)) * Bump types-bleach from 6.0.0.1 to 6.0.0.3. ([\#15686](matrix-org/synapse#15686)) * Bump types-pillow from 9.5.0.2 to 9.5.0.4. ([\#15640](matrix-org/synapse#15640)) * Bump types-pyyaml from 6.0.12.9 to 6.0.12.10. ([\#15683](matrix-org/synapse#15683)) * Bump types-requests from 2.30.0.0 to 2.31.0.0. ([\#15684](matrix-org/synapse#15684)) * Bump types-setuptools from 67.7.0.2 to 67.8.0.0. ([\#15639](matrix-org/synapse#15639)) Synapse 1.84.1 (2023-05-26) =========================== This patch release fixes a major issue with homeservers that do not have an `instance_map` defined but which do use workers. If you have already upgraded to Synapse 1.84.0 and your homeserver is working normally, then there is no need to update to this patch release. Bugfixes -------- - Fix a bug introduced in Synapse v1.84.0 where workers do not start up when no `instance_map` was provided. ([\#15672](matrix-org/synapse#15672)) Internal Changes ---------------- - Add `dch` and `notify-send` to the development Nix flake so that the release script can be used. ([\#15673](matrix-org/synapse#15673)) Synapse 1.84.0 (2023-05-23) =========================== The `worker_replication_*` configuration settings have been deprecated in favour of configuring the main process consistently with other instances in the `instance_map`. The deprecated settings will be removed in Synapse v1.88.0, but changing your configuration in advance is recommended. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/release-v1.84/docs/upgrade.md#upgrading-to-v1840) for more information. Bugfixes -------- - Fix a bug introduced in Synapse 1.84.0rc1 where errors during startup were not reported correctly on Python < 3.10. ([\#15599](matrix-org/synapse#15599)) Synapse 1.84.0rc1 (2023-05-16) ============================== Features -------- - Add an option to prevent media downloads from configured domains. ([\#15197](matrix-org/synapse#15197)) - Add `forget_rooms_on_leave` config option to automatically forget rooms when users leave them or are removed from them. ([\#15224](matrix-org/synapse#15224)) - Add redis TLS configuration options. ([\#15312](matrix-org/synapse#15312)) - Add a config option to delay push notifications by a random amount, to discourage time-based profiling. ([\#15516](matrix-org/synapse#15516)) - Stabilize support for [MSC2659](matrix-org/matrix-spec-proposals#2659): application service ping endpoint. Contributed by Tulir @ Beeper. ([\#15528](matrix-org/synapse#15528)) - Implement [MSC4009](matrix-org/matrix-spec-proposals#4009) to expand the supported characters in Matrix IDs. ([\#15536](matrix-org/synapse#15536)) - Advertise support for Matrix 1.6 on `/_matrix/client/versions`. ([\#15559](matrix-org/synapse#15559)) - Print full error and stack-trace of any exception that occurs during startup/initialization. ([\#15569](matrix-org/synapse#15569)) Bugfixes -------- - Don't fail on federation over TOR where SRV queries are not supported. Contributed by Zdzichu. ([\#15523](matrix-org/synapse#15523)) - Experimental support for [MSC4010](matrix-org/matrix-spec-proposals#4010) which rejects setting the `"m.push_rules"` via account data. ([\#15554](matrix-org/synapse#15554), [\#15555](matrix-org/synapse#15555)) - Fix a long-standing bug where an invalid membership event could cause an internal server error. ([\#15564](matrix-org/synapse#15564)) - Require at least poetry-core v1.1.0. ([\#15566](matrix-org/synapse#15566), [\#15571](matrix-org/synapse#15571)) Deprecations and Removals ------------------------- - Remove need for `worker_replication_*` based settings in worker configuration yaml by placing this data directly on the `instance_map` instead. ([\#15491](matrix-org/synapse#15491)) Updates to the Docker image --------------------------- - Add pkg-config package to Stage 0 to be able to build Dockerfile on ppc64le architecture. ([\#15567](matrix-org/synapse#15567)) Improved Documentation ---------------------- - Clarify documentation of the "Create or modify account" Admin API. ([\#15544](matrix-org/synapse#15544)) - Fix path to the `statistics/database/rooms` admin API in documentation. ([\#15560](matrix-org/synapse#15560)) - Update and improve Mastodon Single Sign-On documentation. ([\#15587](matrix-org/synapse#15587)) Internal Changes ---------------- - Use oEmbed to generate URL previews for YouTube Shorts. ([\#15025](matrix-org/synapse#15025)) - Create new `Client` for use with HTTP Replication between workers. Contributed by Jason Little. ([\#15470](matrix-org/synapse#15470)) - Bump pyicu from 2.10.2 to 2.11. ([\#15509](matrix-org/synapse#15509)) - Remove references to supporting per-user flag for [MSC2654](matrix-org/matrix-spec-proposals#2654). ([\#15522](matrix-org/synapse#15522)) - Don't use a trusted key server when running the demo scripts. ([\#15527](matrix-org/synapse#15527)) - Speed up rebuilding of the user directory for local users. ([\#15529](matrix-org/synapse#15529)) - Speed up deleting of old rows in `event_push_actions`. ([\#15531](matrix-org/synapse#15531)) - Install the `xmlsec` and `mdbook` packages and switch back to the upstream [cachix/devenv](https://github.com/cachix/devenv) repo in the nix development environment. ([\#15532](matrix-org/synapse#15532), [\#15533](matrix-org/synapse#15533), [\#15545](matrix-org/synapse#15545)) - Implement [MSC3987](matrix-org/matrix-spec-proposals#3987) by removing `"dont_notify"` from the list of actions in default push rules. ([\#15534](matrix-org/synapse#15534)) - Move various module API callback registration methods to a dedicated class. ([\#15535](matrix-org/synapse#15535)) - Proxy `/user/devices` federation queries to application services for [MSC3984](matrix-org/matrix-spec-proposals#3984). ([\#15539](matrix-org/synapse#15539)) - Factor out an `is_mine_server_name` method. ([\#15542](matrix-org/synapse#15542)) - Allow running Complement tests using [podman](https://podman.io/) by adding a `PODMAN` environment variable to `scripts-dev/complement.sh`. ([\#15543](matrix-org/synapse#15543)) - Bump serde from 1.0.160 to 1.0.162. ([\#15548](matrix-org/synapse#15548)) - Bump types-setuptools from 67.6.0.5 to 67.7.0.1. ([\#15549](matrix-org/synapse#15549)) - Bump sentry-sdk from 1.19.1 to 1.22.1. ([\#15550](matrix-org/synapse#15550)) - Bump ruff from 0.0.259 to 0.0.265. ([\#15551](matrix-org/synapse#15551)) - Bump hiredis from 2.2.2 to 2.2.3. ([\#15552](matrix-org/synapse#15552)) - Bump types-requests from 2.29.0.0 to 2.30.0.0. ([\#15553](matrix-org/synapse#15553)) - Add `org.matrix.msc3981` info to `/_matrix/client/versions`. ([\#15558](matrix-org/synapse#15558)) - Declare unstable support for [MSC3391](matrix-org/matrix-spec-proposals#3391) under `/_matrix/client/versions` if the experimental implementation is enabled. ([\#15562](matrix-org/synapse#15562)) - Implement [MSC3821](matrix-org/matrix-spec-proposals#3821) to update the redaction rules. ([\#15563](matrix-org/synapse#15563)) - Implement updated redaction rules from [MSC3389](matrix-org/matrix-spec-proposals#3389). ([\#15565](matrix-org/synapse#15565)) - Allow `pip install` to use setuptools_rust 1.6.0 when building Synapse. ([\#15570](matrix-org/synapse#15570)) - Deal with upcoming Github Actions deprecations. ([\#15576](matrix-org/synapse#15576)) - Export `run_as_background_process` from the module API. ([\#15577](matrix-org/synapse#15577)) - Update build system requirements to allow building with poetry-core==1.6.0. ([\#15588](matrix-org/synapse#15588)) - Bump serde from 1.0.162 to 1.0.163. ([\#15589](matrix-org/synapse#15589)) - Bump phonenumbers from 8.13.7 to 8.13.11. ([\#15590](matrix-org/synapse#15590)) - Bump types-psycopg2 from 2.9.21.9 to 2.9.21.10. ([\#15591](matrix-org/synapse#15591)) - Bump types-commonmark from 0.9.2.2 to 0.9.2.3. ([\#15592](matrix-org/synapse#15592)) - Bump types-setuptools from 67.7.0.1 to 67.7.0.2. ([\#15594](matrix-org/synapse#15594)) Synapse 1.83.0 (2023-05-09) =========================== No significant changes since 1.83.0rc1. Synapse 1.83.0rc1 (2023-05-02) ============================== Features -------- - Experimental support to recursively provide relations per [MSC3981](matrix-org/matrix-spec-proposals#3981). ([\#15315](matrix-org/synapse#15315)) - Experimental support for [MSC3970](matrix-org/matrix-spec-proposals#3970): Scope transaction IDs to devices. ([\#15318](matrix-org/synapse#15318)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/experimental_features.html) to support per-user feature flags. ([\#15344](matrix-org/synapse#15344)) - Add a module API to send an HTTP push notification. ([\#15387](matrix-org/synapse#15387)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/statistics.html#get-largest-rooms-by-size-in-database) to query the largest rooms by disk space used in the database. ([\#15482](matrix-org/synapse#15482)) Bugfixes -------- - Disable push rule evaluation for rooms excluded from sync. ([\#15361](matrix-org/synapse#15361)) - Fix a long-standing bug where cached server key results which were directly fetched would not be properly re-used. ([\#15417](matrix-org/synapse#15417)) - Fix a bug introduced in Synapse 1.73.0 where some experimental push rules were returned by default. ([\#15494](matrix-org/synapse#15494)) Improved Documentation ---------------------- - Add Nginx loadbalancing example with sticky mxid for workers. ([\#15411](matrix-org/synapse#15411)) - Update outdated development docs that mention restrictions in versions of SQLite that we no longer support. ([\#15498](matrix-org/synapse#15498)) Internal Changes ---------------- - Speedup tests by caching HomeServerConfig instances. ([\#15284](matrix-org/synapse#15284)) - Add denormalised event stream ordering column to membership state tables for future use. Contributed by Nick @ Beeper (@Fizzadar). ([\#15356](matrix-org/synapse#15356)) - Always use multi-user device resync replication endpoints. ([\#15418](matrix-org/synapse#15418)) - Add column `full_user_id` to tables `profiles` and `user_filters`. ([\#15458](matrix-org/synapse#15458)) - Update support for [MSC3983](matrix-org/matrix-spec-proposals#3983) to allow always returning fallback-keys in a `/keys/claim` request. ([\#15462](matrix-org/synapse#15462)) - Improve type hints. ([\#15465](matrix-org/synapse#15465), [\#15496](matrix-org/synapse#15496), [\#15497](matrix-org/synapse#15497)) - Support claiming more than one OTK at a time. ([\#15468](matrix-org/synapse#15468)) - Bump types-pyyaml from 6.0.12.8 to 6.0.12.9. ([\#15471](matrix-org/synapse#15471)) - Bump pyasn1-modules from 0.2.8 to 0.3.0. ([\#15473](matrix-org/synapse#15473)) - Bump cryptography from 40.0.1 to 40.0.2. ([\#15474](matrix-org/synapse#15474)) - Bump types-netaddr from 0.8.0.7 to 0.8.0.8. ([\#15475](matrix-org/synapse#15475)) - Bump types-jsonschema from 4.17.0.6 to 4.17.0.7. ([\#15476](matrix-org/synapse#15476)) - Ask bug reporters to provide logs as text. ([\#15479](matrix-org/synapse#15479)) - Add a Nix flake for use as a development environment. ([\#15495](matrix-org/synapse#15495)) - Bump anyhow from 1.0.70 to 1.0.71. ([\#15507](matrix-org/synapse#15507)) - Bump types-pillow from 9.4.0.19 to 9.5.0.2. ([\#15508](matrix-org/synapse#15508)) - Bump packaging from 23.0 to 23.1. ([\#15510](matrix-org/synapse#15510)) - Bump types-requests from 2.28.11.16 to 2.29.0.0. ([\#15511](matrix-org/synapse#15511)) - Bump setuptools-rust from 1.5.2 to 1.6.0. ([\#15512](matrix-org/synapse#15512)) - Update the check_schema_delta script to account for when the schema version has been bumped locally. ([\#15466](matrix-org/synapse#15466)) Synapse 1.82.0 (2023-04-25) =========================== No significant changes since 1.82.0rc1. Synapse 1.82.0rc1 (2023-04-18) ============================== Features -------- - Allow loading the `/directory/room/{roomAlias}` endpoint on workers. ([\#15333](matrix-org/synapse#15333)) - Add some validation to `instance_map` configuration loading. ([\#15431](matrix-org/synapse#15431)) - Allow loading the `/capabilities` endpoint on workers. ([\#15436](matrix-org/synapse#15436)) Bugfixes -------- - Delete server-side backup keys when deactivating an account. ([\#15181](matrix-org/synapse#15181)) - Fix and document untold assumption that `on_logged_out` module hooks will be called before the deletion of pushers. ([\#15410](matrix-org/synapse#15410)) - Improve robustness when handling a perspective key response by deduplicating received server keys. ([\#15423](matrix-org/synapse#15423)) - Synapse now correctly fails to start if the config option `app_service_config_files` is not a list. ([\#15425](matrix-org/synapse#15425)) - Disable loading `RefreshTokenServlet` (`/_matrix/client/(r0|v3|unstable)/refresh`) on workers. ([\#15428](matrix-org/synapse#15428)) Improved Documentation ---------------------- - Note that the `delete_stale_devices_after` background job always runs on the main process. ([\#15452](matrix-org/synapse#15452)) Deprecations and Removals ------------------------- - Remove the broken, unspecced registration fallback. Note that the *login* fallback is unaffected by this change. ([\#15405](matrix-org/synapse#15405)) Internal Changes ---------------- - Bump black from 23.1.0 to 23.3.0. ([\#15372](matrix-org/synapse#15372)) - Bump pyopenssl from 23.1.0 to 23.1.1. ([\#15373](matrix-org/synapse#15373)) - Bump types-psycopg2 from 2.9.21.8 to 2.9.21.9. ([\#15374](matrix-org/synapse#15374)) - Bump types-netaddr from 0.8.0.6 to 0.8.0.7. ([\#15375](matrix-org/synapse#15375)) - Bump types-opentracing from 2.4.10.3 to 2.4.10.4. ([\#15376](matrix-org/synapse#15376)) - Bump dawidd6/action-download-artifact from 2.26.0 to 2.26.1. ([\#15404](matrix-org/synapse#15404)) - Bump parameterized from 0.8.1 to 0.9.0. ([\#15412](matrix-org/synapse#15412)) - Bump types-pillow from 9.4.0.17 to 9.4.0.19. ([\#15413](matrix-org/synapse#15413)) - Bump sentry-sdk from 1.17.0 to 1.19.1. ([\#15414](matrix-org/synapse#15414)) - Bump immutabledict from 2.2.3 to 2.2.4. ([\#15415](matrix-org/synapse#15415)) - Bump dawidd6/action-download-artifact from 2.26.1 to 2.27.0. ([\#15441](matrix-org/synapse#15441)) - Bump serde_json from 1.0.95 to 1.0.96. ([\#15442](matrix-org/synapse#15442)) - Bump serde from 1.0.159 to 1.0.160. ([\#15443](matrix-org/synapse#15443)) - Bump pillow from 9.4.0 to 9.5.0. ([\#15444](matrix-org/synapse#15444)) - Bump furo from 2023.3.23 to 2023.3.27. ([\#15445](matrix-org/synapse#15445)) - Bump types-pyopenssl from 23.1.0.0 to 23.1.0.2. ([\#15446](matrix-org/synapse#15446)) - Bump mypy from 1.0.0 to 1.0.1. ([\#15447](matrix-org/synapse#15447)) - Bump psycopg2 from 2.9.5 to 2.9.6. ([\#15448](matrix-org/synapse#15448)) - Improve DB performance of clearing out old data from `stream_ordering_to_exterm`. ([\#15382](matrix-org/synapse#15382), [\#15429](matrix-org/synapse#15429)) - Implement [MSC3989](matrix-org/matrix-spec-proposals#3989) redaction algorithm. ([\#15393](matrix-org/synapse#15393)) - Implement [MSC2175](matrix-org/matrix-spec-proposals#2175) to stop adding `creator` to create events. ([\#15394](matrix-org/synapse#15394)) - Implement [MSC2174](matrix-org/matrix-spec-proposals#2174) to move the `redacts` key to a `content` property. ([\#15395](matrix-org/synapse#15395)) - Trust dtonlay/rust-toolchain in CI. ([\#15406](matrix-org/synapse#15406)) - Explicitly install Synapse during typechecking in CI. ([\#15409](matrix-org/synapse#15409)) - Only load the SSO redirect servlet if SSO is enabled. ([\#15421](matrix-org/synapse#15421)) - Refactor `SimpleHttpClient` to pull out a base class. ([\#15427](matrix-org/synapse#15427)) - Improve type hints. ([\#15432](matrix-org/synapse#15432)) - Convert async to normal tests in `TestSSOHandler`. ([\#15433](matrix-org/synapse#15433)) - Speed up the user directory background update. ([\#15435](matrix-org/synapse#15435)) - Disable directory listing for static resources in `/_matrix/static/`. ([\#15438](matrix-org/synapse#15438)) - Move various module API callback registration methods to a dedicated class. ([\#15453](matrix-org/synapse#15453)) Synapse 1.81.0 (2023-04-11) =========================== Synapse now attempts the versioned appservice paths before falling back to the [legacy paths](https://spec.matrix.org/v1.6/application-service-api/#legacy-routes). Usage of the legacy routes should be considered deprecated. Additionally, Synapse has supported sending the application service access token via [the `Authorization` header](https://spec.matrix.org/v1.6/application-service-api/#authorization) since v1.70.0. For backwards compatibility it is *also* sent as the `access_token` query parameter. This is insecure and should be considered deprecated. A future version of Synapse (v1.88.0 or later) will remove support for legacy application service routes and query parameter authorization. No significant changes since 1.81.0rc2. Synapse 1.81.0rc2 (2023-04-06) ============================== Bugfixes -------- - Fix the `set_device_id_for_pushers_txn` background update crash. ([\#15391](matrix-org/synapse#15391)) Internal Changes ---------------- - Update CI to run complement under the latest stable go version. ([\#15403](matrix-org/synapse#15403)) Synapse 1.81.0rc1 (2023-04-04) ============================== Features -------- - Add the ability to enable/disable registrations when in the OIDC flow. ([\#14978](matrix-org/synapse#14978)) - Add a primitive helper script for listing worker endpoints. ([\#15243](matrix-org/synapse#15243)) - Experimental support for passing One Time Key and device key requests to application services ([MSC3983](matrix-org/matrix-spec-proposals#3983) and [MSC3984](matrix-org/matrix-spec-proposals#3984)). ([\#15314](matrix-org/synapse#15314), [\#15321](matrix-org/synapse#15321)) - Allow loading `/password_policy` endpoint on workers. ([\#15331](matrix-org/synapse#15331)) - Add experimental support for Unix sockets. Contributed by Jason Little. ([\#15353](matrix-org/synapse#15353)) - Build Debian packages for Ubuntu 23.04 (Lunar Lobster). ([\#15381](matrix-org/synapse#15381)) Bugfixes -------- - Fix a long-standing bug where edits of non-`m.room.message` events would not be correctly bundled. ([\#15295](matrix-org/synapse#15295)) - Fix a bug introduced in Synapse v1.55.0 which could delay remote homeservers being able to decrypt encrypted messages sent by local users. ([\#15297](matrix-org/synapse#15297)) - Add a check to [SQLite port_db script](https://matrix-org.github.io/synapse/latest/postgres.html#porting-from-sqlite) to ensure that the sqlite database passed to the script exists before trying to port from it. ([\#15306](matrix-org/synapse#15306)) - Fix a bug introduced in Synapse 1.76.0 where responses from worker deployments could include an internal `_INT_STREAM_POS` key. ([\#15309](matrix-org/synapse#15309)) - Fix a long-standing bug that Synpase only used the [legacy appservice routes](https://spec.matrix.org/v1.6/application-service-api/#legacy-routes). ([\#15317](matrix-org/synapse#15317)) - Fix a long-standing bug preventing users from rejoining rooms after being banned and unbanned over federation. Contributed by Nico. ([\#15323](matrix-org/synapse#15323)) - Fix bug in worker mode where on a rolling restart of workers the "typing" worker would consume 100% CPU until it got restarted. ([\#15332](matrix-org/synapse#15332)) - Fix a long-standing bug where some to_device messages could be dropped when using workers. ([\#15349](matrix-org/synapse#15349)) - Fix a bug introduced in Synapse 1.70.0 where the background sync from a faster join could spin for hours when one of the events involved had been marked for backoff. ([\#15351](matrix-org/synapse#15351)) - Fix missing app variable in mail subject for password resets. Contributed by Cyberes. ([\#15352](matrix-org/synapse#15352)) - Fix a rare bug introduced in Synapse 1.66.0 where initial syncs would fail when the user had been kicked from a faster joined room that had not finished syncing. ([\#15383](matrix-org/synapse#15383)) Improved Documentation ---------------------- - Fix a typo in login requests ratelimit defaults. ([\#15341](matrix-org/synapse#15341)) - Add some clarification to the doc/comments regarding TCP replication. ([\#15354](matrix-org/synapse#15354)) - Note that Synapse 1.74 queued a rebuild of the user directory tables. ([\#15386](matrix-org/synapse#15386)) Internal Changes ---------------- - Use `immutabledict` instead of `frozendict`. ([\#15113](matrix-org/synapse#15113)) - Add developer documentation for the Federation Sender and add a documentation mechanism using Sphinx. ([\#15265](matrix-org/synapse#15265), [\#15336](matrix-org/synapse#15336)) - Make the pushers rely on the `device_id` instead of the `access_token_id` for various operations. ([\#15280](matrix-org/synapse#15280)) - Bump sentry-sdk from 1.15.0 to 1.17.0. ([\#15285](matrix-org/synapse#15285)) - Allow running the Twisted trunk job against other branches. ([\#15302](matrix-org/synapse#15302)) - Remind the releaser to ask for changelog feedback in [#synapse-dev](https://matrix.to/#/#synapse-dev:matrix.org). ([\#15303](matrix-org/synapse#15303)) - Bump dtolnay/rust-toolchain from e12eda571dc9a5ee5d58eecf4738ec291c66f295 to fc3253060d0c959bea12a59f10f8391454a0b02d. ([\#15304](matrix-org/synapse#15304)) - Reject events with an invalid "mentions" property per [MSC3952](matrix-org/matrix-spec-proposals#3952). ([\#15311](matrix-org/synapse#15311)) - As an optimisation, use `TRUNCATE` on Postgres when clearing the user directory tables. ([\#15316](matrix-org/synapse#15316)) - Fix `.gitignore` rule for the Complement source tarball downloaded automatically by `complement.sh`. ([\#15319](matrix-org/synapse#15319)) - Bump serde from 1.0.157 to 1.0.158. ([\#15324](matrix-org/synapse#15324)) - Bump regex from 1.7.1 to 1.7.3. ([\#15325](matrix-org/synapse#15325)) - Bump types-pyopenssl from 23.0.0.4 to 23.1.0.0. ([\#15326](matrix-org/synapse#15326)) - Bump furo from 2022.12.7 to 2023.3.23. ([\#15327](matrix-org/synapse#15327)) - Bump ruff from 0.0.252 to 0.0.259. ([\#15328](matrix-org/synapse#15328)) - Bump cryptography from 40.0.0 to 40.0.1. ([\#15329](matrix-org/synapse#15329)) - Bump mypy-zope from 0.9.0 to 0.9.1. ([\#15330](matrix-org/synapse#15330)) - Speed up unit tests when using SQLite3. ([\#15334](matrix-org/synapse#15334)) - Speed up pydantic CI job. ([\#15339](matrix-org/synapse#15339)) - Speed up sample config CI job. ([\#15340](matrix-org/synapse#15340)) - Fix copyright year in SSO footer template. ([\#15358](matrix-org/synapse#15358)) - Bump peaceiris/actions-gh-pages from 3.9.2 to 3.9.3. ([\#15369](matrix-org/synapse#15369)) - Bump serde from 1.0.158 to 1.0.159. ([\#15370](matrix-org/synapse#15370)) - Bump serde_json from 1.0.94 to 1.0.95. ([\#15371](matrix-org/synapse#15371)) - Speed up membership queries for users with forgotten rooms. ([\#15385](matrix-org/synapse#15385))

turt2live

Overall, this looks great to me. Thanks for putting so much time and energy into this project.

My review here is long, but the vast majority are non-blocking concerns for stating FCP. They will ultimately block FCP from finishing, but aren't critical enough to warrant a total block on progress, in my opinion.

The theme for the blocking concerns is largely confusion for this MSC's purpose: it feels like this proposal is trying to confirm that we want to go down this road for authentication, but promptly puts up dependencies which would make it a net-zero change once they're all accepted. I've left some suggestions on how to change the wording slightly to flip this around: we would use this MSC to confirm the direction, and leave implementation/specification details to the underlying MSCs. The suggestions use "or similar" language to cover a process quirk where we know that we're most likely going to put the listed MSCs forward for FCP, but there's a non-zero chance that an alternative shows up, so we escape the pigeon hole early with "or similar".

If the intention is to have the listed MSCs accepted first, I'm not clear on this MSC's intended purpose. Clarification in the MSC body would be appreciated.

proposals/3861-next-generation-auth.md

turt2live · 2025-01-21T00:56:49Z

proposals/3861-next-generation-auth.md

+In practice, to provide a good user experience, homeservers have to implement web views for the authentication flows, which is complex to implement well.
+This means having proper accessibility, translations, and UX.
+Those concerns were previously only affecting client implementations, and will now also affect homeserver implementations.
+
+On the other hand, the previous registration flow was notoriously complex to implement both for clients and homeservers, and this proposal removes a lot of that complexity from the client side.


non-blocking concern: it may be good to mention that servers are supposed to have fallback UI for UIA anyways, which is likely subject to many of the same concerns in a full production build. Like UIA fallback though, servers can also do the bare minimum for their web UI if they want.

proposals/3861-next-generation-auth.md

turt2live · 2025-01-21T01:22:06Z

MSCs proposed for Final Comment Period (FCP) should meet the requirements outlined in the checklist prior to being accepted into the spec. This checklist is a bit long, but aims to reduce the number of follow-on MSCs after a feature lands.

SCT members: please check off things you check for, and raise a concern against FCP if the checklist is incomplete. If an item doesn't apply, prefer to check it rather than remove it. Unchecking items is encouraged where applicable.

Checklist:

Co-authored-by: Travis Ralston <travisr@matrix.org>

turt2live · 2025-01-25T21:51:59Z

proposals/3861-next-generation-auth.md

+
+The goal of this proposal is to keep the ecosystem open by clearly defining how clients register themselves dynamically with the homeserver.
+This won't prevent the homeserver from rejecting clients arbitrarily, but this was already the case with UIA.
+On the contrary, this proposal will provide homeserver operators greater visibility into which clients are being used, helping with decisions to approve third-party clients.


non-blocking: and helps homeserver operators make decisions about their homeservers, like when to roll out features which break in some clients.

turt2live

Thanks for continuing to put work into keeping this updated! There's a few open (non-blocking) things which would be good to include in the proposal, but overall the MSC appears ready to go to FCP.

I expect that the underlying MSCs will also go into FCP quickly, but it's important (to me) that we have a document somewhere which explains why next-gen auth over UIA in particular - an MSC is a great way to do that.

turt2live · 2025-01-25T21:58:16Z

@mscbot fcp merge

mscbot · 2025-01-25T21:58:19Z

erikjohnston

Looks good! I think the motivation section accidentally ends up focusing on the tangential benefits of the change rather than the core rationale of why we're doing this (see the comment)

erikjohnston · 2025-01-28T10:00:29Z

proposals/3861-next-generation-auth.md

+- The agreement to the terms of service can be completely ignored by the client, as it only requires the client to send a request with no specific data to validate it.
+  This means the server has to trust the goodwill of the client to make sure the user agrees to the terms of service.
+- The current design of the email verification flow has a confusing user experience.
+  The link sent to the user by email leads to a screen where the user is prompted to return to their client.


Isn't this a) how some MAS flows work, and b) is a relatively common paradigm?

The flow in MAS works by sending a code instead of a link for email verification. This has two advantages:

some platforms (like iOS/macOS) can autofill '2FA' codes got by email

this makes sure the user doesn't break the context of what they were doing. By making them enter a code, they will keep the browser window where they started the registration flow open while they check their email

I can see why adding that code avoids a potentially annoying UX problem, but it fundamentally feels more or less the same style of flow. And something that we could fix relatively easily in the existing system.

This isn't a concern with the actual substance, but by having these relatively minor concerns listed in bullet points the attention is drawn away from the actual reasons we want to move to oauth2. Can we not replace these more minor issues with the actual problems/benefits that are driving this? Things like security considerations, protecting client credentials, not re-inventing the wheel, 2FA, etc.

Things like security considerations, protecting client credentials, not re-inventing the wheel, 2FA, etc.

How are those first two achieved more when using OAuth, compared to the old way of doing things?

What if Google is forced by a government agency to keylog specific users' passwords entered into Matrix OAuth pages, as part of a secret operation against those users? Remember, Google Chrome is the most popular browser, and while parts of it are open source, the browser itself is closed source.

Relying on the security of a third party application means you have less ability to fix or even react to security holes that are found — or purposefully built into — those third party applications.

Besides, the arguably more important cryptographic keys that protect a user's messages and chat histories have to be known to the client anyway, and shouldn't be known by the server. Matrix's whole design is centered around trusting the client more than the server, and requiring OAuth breaks that paradigm.

erikjohnston · 2025-01-28T10:03:38Z

proposals/3861-next-generation-auth.md

+
+- Password managers can be used to store the password for the homeserver domain, and the user can use a different client on a different domain without having to remember the password.
+- WebAuthn credentials/Passkeys are bound to the domain, and would be impractical to introduce with the current authentication paradigm.
+- The user could benefit from sharing a single browser session with multiple clients, requiring them to enter their credentials only once per physical device.


Most of these feel like "nice to haves" rather than reasons to move to OAuth2, and as such I think actually hurt the argument. We wouldn't be making such a large change for these sorts of things. Honestly, I think we could either just get rid of these, or replace them with a paragraph at the end of the section, which brings the focus back on the the paragraphs that give the actual rationale for the move.

I think the same is true for the bullet points in the previous section.

Maybe something like:

We also get other, more minor, benefits from the change to a more common auth mechanism. For example, password managers work out the box, it'd be possible to use webauthm/passkeys, etc.

erikjohnston · 2025-01-28T10:21:38Z

proposals/3861-next-generation-auth.md

+
+**Note**: Many of these points could be improved with individual improvements to each of those stages, and multiple MSCs already exist to address some of them.
+
+### Benefits of authenticating end-users through the system browser


What are the benefits of using the system browser, versus any old browser? Do we even need to be emphasising the use of system browser?

I'm using 'system browser' here as opposed to 'an embedded web view'. Maybe I should make it clearer by calling it the 'default browser'?

Ah right, then: why not an embedded web view?

dbkr · 2025-01-28T12:44:17Z

proposals/3861-next-generation-auth.md

+
+### Discovery [MSC2965]
+
+First step is to discover the homeserver's authorization server metadata.


Struggling a little with the duplication between MSCs here but I see Erik has already asked what I was going to ask at https://github.com/matrix-org/matrix-spec-proposals/pull/2965/files#r1931976488

dbkr · 2025-01-28T12:50:29Z

proposals/3861-next-generation-auth.md

+[`m.login.application_service`]: https://spec.matrix.org/v1.13/client-server-api/#appservice-login
+[`m.login.sso`]: https://spec.matrix.org/v1.13/client-server-api/#single-sign-on
+[`/_matrix/client/v3/capabilities`]: https://spec.matrix.org/v1.13/client-server-api/#get_matrixclientv3capabilities
+[`/_matrix/client/v3/login`]: https://spec.matrix.org/v1.13/client-server-api/#post_matrixclientv3login


Having an overview like this is incredibly useful and thanks so much for putting it together, although I am struggling to pick out what this MSC is actually intending to change about the spec between the duplicated bits of other MSCs. Is it largely the glue of how they various different MSCs fit together?

richvdh · 2025-01-28T14:49:53Z

proposals/3861-next-generation-auth.md

+[MSC4198]: https://github.com/matrix-org/matrix-spec-proposals/pull/4198
+[MSC4254]: https://github.com/matrix-org/matrix-spec-proposals/pull/4254
+[`m.login.application_service`]: https://spec.matrix.org/v1.13/client-server-api/#appservice-login
+[`m.login.sso`]: https://spec.matrix.org/v1.13/client-server-api/#single-sign-on


this isn't really the right link for SSO: it's talking about SSO-during-UIA, which is particularly janky.

Suggested change

[`m.login.sso`]: https://spec.matrix.org/v1.13/client-server-api/#single-sign-on

[`m.login.sso`]: https://spec.matrix.org/v1.13/client-server-api/#sso-client-loginauthentication

richvdh · 2025-01-28T14:52:11Z

proposals/3861-next-generation-auth.md

+## Motivation
+
+The goal of this MSC is to propose a new set of authentication APIs for Matrix, based on the OAuth 2.0 and OpenID Connect (OIDC) specifications.
+To understand this proposal, it is important to lay out the reasoning which led to this conclusion.


nit: which conclusion? the conclusion that we should move to OAuth 2?

Suggested change

To understand this proposal, it is important to lay out the reasoning which led to this conclusion.

To understand this proposal, it is important to lay out the reasoning behind it.

richvdh · 2025-01-28T14:56:24Z

proposals/3861-next-generation-auth.md

+This has been a notorious problem with [OAuth 2.0 in email protocols][thunderbird-oauth2], where email clients are forced to register their applications with each email provider, giving the email provider the right to reject any application.
+
+The goal of this proposal is to keep the ecosystem open by clearly defining how clients register themselves dynamically with the homeserver.
+This won't prevent the homeserver from rejecting clients arbitrarily, but this was already the case with UIA.


but this was already the case with UIA.

was it? How so?

richvdh · 2025-01-28T14:59:37Z

proposals/3861-next-generation-auth.md

+
+The goal of this proposal is to keep the ecosystem open by clearly defining how clients register themselves dynamically with the homeserver.
+This won't prevent the homeserver from rejecting clients arbitrarily, but this was already the case with UIA.
+On the contrary, this proposal will provide homeserver operators greater visibility into which clients are being used, helping with decisions to approve third-party clients.


Slightly confused by what this line is trying to say. It seems to be saying that allowing the homeserver to reject clients arbitrarily is a good thing... but that conflicts with our assessment of OAuth 2.0 in email?

richvdh · 2025-01-28T15:03:47Z

proposals/3861-next-generation-auth.md

+As a first step, it introduces those APIs as alternatives to the existing User-Interactive Authentication (UIA) APIs, acknowledging the complexity of covering all the use cases of the existing APIs.
+The long-term goal is to deprecate the existing UIA APIs and replace them with the new OAuth 2.0/OIDC-based APIs.


as alternatives to the existing User-Interactive Authentication (UIA) APIs
deprecate the existing UIA APIs

and /login, right?

(and /logout, maybe?)

To be honest, the term "UIA APIs" isn't very clear... there are APIs which currently use UIA, but I'm not sure if you plan to remove them all, or just stop them using UIA. Would be good to be more explicit here.

And /register?

richvdh · 2025-01-28T15:47:25Z

proposals/3861-next-generation-auth.md

+### Account management
+
+This moves the user-interface for some account management tasks from the client to the homeserver.
+Existing APIs like `/_matrix/client/v3/capabilities` help clients understand which account-management API endpoints are unavailable, but they don't offer alternatives to a homeserver-provided user-interface.


still no link here.

clokep · 2025-01-28T18:12:40Z

proposals/3861-next-generation-auth.md

+- When setting up a CAPTCHA challenge, CAPTCHA services expect the challenge to be served from a specific domain. Because the client can be on any domain, [Synapse currently advises disabling host verification](https://element-hq.github.io/synapse/latest/CAPTCHA_SETUP.html#getting-api-keys).
+  When this option is disabled, the CAPTCHA service expects the server to verify the domain of the challenge, which is not possible.


Not sure if listing more is useful, but captcha design is specific to certain providers, support for 2FA would require modifications, etc.

clokep · 2025-01-28T18:20:53Z

proposals/3861-next-generation-auth.md

+Authenticating end-users through the system browser is a well-established approach for many applications and would help solve most of the UI quirks mentioned above.
+Though, some applications may wish to retain browser-less authentication, which this proposal supports thanks to the inherited authentication specifications.
+
+The general idea is simple: to authenticate a user, the client redirects the user to a URL on the homeserver, which completes the authentication flow, and then redirects the user back to the client.


Doesn't it really redirect to the IdP, not necessarily the homeserver?

clokep · 2025-01-28T18:22:24Z

proposals/3861-next-generation-auth.md

+This has two important implications:
+
+- The client can't store the user's credentials, and thus can't use them to gain access without the user's consent.
+- The user can use different clients without worrying about revealing their account credentials to unknown parties. Only their homeserver ever interacts with their credentials.


Again -- isn't this really the IdP? Doesn't this also protect the homeserver ever knowing the user's password?

clokep · 2025-01-28T18:23:14Z

proposals/3861-next-generation-auth.md

+This opens up the possibility of giving more restrictive access to the user's account.
+It would open up Matrix to a new class of clients with only partial access to the user's account.


It feels like this should be prominent in the proposal.

clokep · 2025-01-28T18:30:23Z

proposals/3861-next-generation-auth.md

+It is a great fit for connecting identity providers to other pieces of software, and this is already what homeservers do with the [`m.login.sso`] flow.
+
+Knowing that, it can feel like adopting OpenID Connect fully would help using off-the-shelf identity providers for Matrix homeservers.
+In practice, OpenID Connect does not cover continuous exchanges between the application and the identity providers: there is no well-supported standard to signal new sessions, new users, sessions ending, users deactivation, etc. from the identity provider to the application.


I can make some assumption, but why do we care about this in Matrix?

clokep · 2025-01-28T18:40:03Z

proposals/3861-next-generation-auth.md

+As a first step, it introduces those APIs as alternatives to the existing User-Interactive Authentication (UIA) APIs, acknowledging the complexity of covering all the use cases of the existing APIs.
+The long-term goal is to deprecate the existing UIA APIs and replace them with the new OAuth 2.0/OIDC-based APIs.


And /register?

Matrix architecture change to delegate authentication via OIDC

4521510

hughns changed the title ~~MSCxxxx: Matrix architecture change to delegate authentication via OIDC~~ MSC3861: Matrix architecture change to delegate authentication via OIDC Aug 4, 2022

MSC3861

4550bbc

turt2live added proposal A matrix spec change proposal client-server Client-Server API kind:core MSC which is critical to the protocol's success needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Aug 4, 2022

ara4n added 3 commits August 9, 2022 11:29

typoe

00cd76d

typoes

b684b84

typoes

dd60163

MTRNord reviewed Aug 9, 2022

View reviewed changes

Add proposal for Matrix.org Foundation to become member of OpenID Fou…

9e58990

…ndation

turt2live reviewed Aug 11, 2022

View reviewed changes

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

ShadowJonathan mentioned this pull request Aug 14, 2022

Accomodate OIDC authentication ruma/ruma#1258

Open

greizgh reviewed Jan 1, 2023

View reviewed changes

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

hughns and others added 4 commits January 4, 2023 13:55

Update proposals/3861-delegated-oidc-architecture.md

fb0dfe7

Co-authored-by: greizgh <greizgh@ephax.org>

Move images inline

45c0b1c

Use term OpenID Provider

ba0c3dc

Add note about extending UIA as alternative

ef76bfa

MTRNord reviewed Feb 24, 2023

View reviewed changes

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

MTRNord mentioned this pull request Feb 24, 2023

MSC2966: Usage of OAuth 2.0 Dynamic Client Registration in Matrix #2966

Open

Add reference to related MSCs

4bffdc9

hughns mentioned this pull request Feb 28, 2023

MSC3970: Scope transaction IDs to devices #3970

Merged

turt2live added the matrix-2.0 Required for Matrix 2.0 label Mar 3, 2023

clokep mentioned this pull request May 12, 2023

Experimental MSC3861 support: delegate auth to an OIDC provider matrix-org/synapse#15582

Merged

5 tasks

clokep reviewed May 26, 2023

View reviewed changes

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

omentic mentioned this pull request Jun 20, 2023

Remove arbitrary restrictions on passwords cinnyapp/cinny#1285

Closed

sandhose added 2 commits January 17, 2025 18:40

List a few potential issues

185e0a5

Mention areweoidcyet.com

90667ed

sandhose marked this pull request as ready for review January 17, 2025 17:56

turt2live added implementation-needs-checking The MSC has an implementation, but the SCT has not yet checked it. and removed needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Jan 18, 2025

turt2live requested changes Jan 21, 2025

View reviewed changes

sandhose and others added 9 commits January 22, 2025 10:36

Apply suggestions from code review

5ca8111

Co-authored-by: Travis Ralston <travisr@matrix.org>

§ about how we keep the ecosystem open

dbede1e

Update the alternatives table to stop mentioning 'OP'

56a6278

Reword how we mention MSC dependencies that are already in the spec

152cd98

Reformat with prettier

f5f02c2

Make it clearer what proposals are adjacente, write about ASes

d76b9a2

Add links about the current C-S API

9ff541c

Add links to the spec

0a86b69

Add links about OIDC and OAuth 2.0

2acf438

sandhose requested a review from turt2live January 22, 2025 16:19

turt2live reviewed Jan 25, 2025

View reviewed changes

turt2live approved these changes Jan 25, 2025

View reviewed changes

mscbot added proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. disposition-merge labels Jan 25, 2025

turt2live removed the implementation-needs-checking The MSC has an implementation, but the SCT has not yet checked it. label Jan 25, 2025

parisni mentioned this pull request Jan 26, 2025

[auth] support for oidc esl/MongooseIM#4472

Open

erikjohnston reviewed Jan 28, 2025

View reviewed changes

dbkr reviewed Jan 28, 2025

View reviewed changes

richvdh reviewed Jan 28, 2025

View reviewed changes

clokep reviewed Jan 28, 2025

View reviewed changes

TheOneWithTheBraid mentioned this pull request Feb 7, 2025

BREAKING: implement Native OIDC as per MSC 3861 famedly/matrix-dart-sdk#2024

Open

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

hughns commented Aug 4, 2022 •

edited by turt2live

Loading

turt2live left a comment

hughns commented Feb 28, 2023

turt2live left a comment

turt2live Jan 21, 2025

turt2live commented Jan 21, 2025 •

edited

Loading

turt2live Jan 25, 2025

turt2live left a comment

turt2live commented Jan 25, 2025

mscbot commented Jan 25, 2025 •

edited by ara4n

Loading

erikjohnston left a comment

erikjohnston Jan 28, 2025

sandhose Jan 28, 2025

erikjohnston Jan 28, 2025

Tynach Jan 31, 2025

erikjohnston Jan 28, 2025

erikjohnston Jan 28, 2025

sandhose Jan 28, 2025

erikjohnston Jan 28, 2025

dbkr Jan 28, 2025

dbkr Jan 28, 2025

richvdh Jan 28, 2025

richvdh Jan 28, 2025

richvdh Jan 28, 2025

richvdh Jan 28, 2025 •

edited

Loading

richvdh Jan 28, 2025

richvdh Jan 28, 2025

clokep Jan 28, 2025

richvdh Jan 28, 2025

clokep Jan 28, 2025

clokep Jan 28, 2025

clokep Jan 28, 2025

clokep Jan 28, 2025

clokep Jan 28, 2025

clokep Jan 28, 2025


		Note: Many of these points could be improved with individual improvements to each of those stages, and multiple MSCs already exist to address some of them.

		### Benefits of authenticating end-users through the system browser


		### Discovery [MSC2965]

		First step is to discover the homeserver's authorization server metadata.

	[`m.login.sso`]: https://spec.matrix.org/v1.13/client-server-api/#single-sign-on
	[`m.login.sso`]: https://spec.matrix.org/v1.13/client-server-api/#sso-client-loginauthentication

	To understand this proposal, it is important to lay out the reasoning which led to this conclusion.
	To understand this proposal, it is important to lay out the reasoning behind it.

		As a first step, it introduces those APIs as alternatives to the existing User-Interactive Authentication (UIA) APIs, acknowledging the complexity of covering all the use cases of the existing APIs.
		The long-term goal is to deprecate the existing UIA APIs and replace them with the new OAuth 2.0/OIDC-based APIs.

		- When setting up a CAPTCHA challenge, CAPTCHA services expect the challenge to be served from a specific domain. Because the client can be on any domain, [Synapse currently advises disabling host verification](https://element-hq.github.io/synapse/latest/CAPTCHA_SETUP.html#getting-api-keys).
		When this option is disabled, the CAPTCHA service expects the server to verify the domain of the challenge, which is not possible.

		This opens up the possibility of giving more restrictive access to the user's account.
		It would open up Matrix to a new class of clients with only partial access to the user's account.

MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

Are you sure you want to change the base?

MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

Conversation

hughns commented Aug 4, 2022 • edited by turt2live Loading

turt2live left a comment

Choose a reason for hiding this comment

hughns commented Feb 28, 2023

turt2live left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turt2live commented Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

turt2live left a comment

Choose a reason for hiding this comment

turt2live commented Jan 25, 2025

mscbot commented Jan 25, 2025 • edited by ara4n Loading

erikjohnston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richvdh Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hughns commented Aug 4, 2022 •

edited by turt2live

Loading

turt2live commented Jan 21, 2025 •

edited

Loading

mscbot commented Jan 25, 2025 •

edited by ara4n

Loading

richvdh Jan 28, 2025 •

edited

Loading