Screencast restore #638

GeorgesStavracas · 2021-09-29T01:47:29Z

This is my first working version of a working way to restore screencast streams. First, I'd like to remind what goals I had in mind when working on it:

Users must be able to control whether or not any application will be able to restore streams. Users must be able to revoke these permissions at any point in time. This is non-negotiable. ¹
Avoid a broad "allow all" permission.
The fallback is always to drop permissions and query the user again.

That being said, let's jump right into the implementation.

Implementation

There are 2 important workflows happening in parallel: acquiring a token, and using it.

1. Acquiring a restore token

Applications that want to be able to restore screencast streams pass the persist_mode option to the SelectSources() method. I'll explain persist modes below.
xdg-desktop-portal pass the persist_mode option to the portal implementation.
The portal implementation queries the user (usually via a dialog) which monitor or window to share, and whether they allow it to be restored. (It should default to not allow.)
If the screencast session should be restored, the portal implementation hands xdg-desktop-portal the implementation-specific data to be restored. This is passed via the restore_data option, at the response of the Start() call.
xdg-desktop-portal stores this data somewhere with an unique identifier that I called restore_token.
xdg-desktop-portal pass restore_token to the application.

The application then should save this restore token wherever it thinks it's appropriate.

2. Using the restore token

Now that the application has a restore token, it can restore a previous stream. This is how it happens:

Applications that want to restore a screencast stream pass the restore_token option to the SelectSources() method.
xdg-desktop-portal receives the token, and performs a lookup for the implementation-specific data associated with this token.
xdg-desktop-portal consumes this token and removes the permission.
xdg-desktop-portal sends this restore data to the portal implementation as the restore_data option to the SelectSources() method.
The portal implementation uses this restore data to match windows and monitors without presenting a dialog. If it fails to match (due to, for example, any monitors or windows not being available), it fallbacks to presenting the dialog.

Token lifecycle

Tokens can only be used once to restore streams. The permissions they hold are revoked after use. That's what xdg-desktop-portal does at step 2.3. Applications must not assume anything about tokens. Portal implementations never have access to the restore tokens.

Persist modes

In this first implementation, I've added two modes of persistence:

Transient: the restore token is valid as long as the application is alive. It's stored in memory and revoked when the application closes its D-Bus connection. The use-case I had in mind with this mode is browsers.
Persistent: the restore token is stored in disk, using the permissions store, and is valid until the user manually revokes it.

See https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/merge_requests/14 for a reference implementation.

GeorgesStavracas · 2021-09-29T02:31:39Z

@smcv here's the screencast restore thing I've mentioned. I'd love to hear your thoughts on this. I'm not confident that this is a good approach, much less a secure one.

smcv · 2021-09-29T09:27:44Z

The restore token can be stolen by any host system app via monitoring the session bus

Host system apps can also arbitrarily impersonate each other, e.g. if a host system app suspects that you might give Steam permission to record your screen (for Remote Play), it can tell xdg-desktop-portal "hello, I'm Steam". Or it can impersonate a web browser, or a standalone video-streaming client, or anything else, in a way that is indistinguishable from a Flatpak app. Anything Flatpak can do (not a Flatpak app, but Flatpak itself!), a host system app can do too.

In general, a host system app can also ptrace xdg-desktop-portal, or append export LD_PRELOAD="$HOME/.malicious-code-goes-here" to ~/.profile and use that to load its own arbitrary code into xdg-desktop-portal and host system apps. We are not going to be able to avoid any of this without making use of LSMs (SELinux/AppArmor/Smack/etc.), which are non-portable, but provide the only mechanism currently available for Unix sockets (and hence D-Bus) to distinguish between "applications" running as the same uid in the face of potentially malicious host-system applications.

The solution is to avoid running host system apps that you don't completely trust: they are in the trusted computing base (TCB), on the trusted side of the airtight hatchway.

Of course, this is far from ideal - if we were designing a modern app framework from scratch, without backward-compatibility concerns, we'd want literally every app to be identifiable as part of the app framework, as they are in Android or iOS, with a minimum of code in the TCB. However, users of Unix OSs are used to being able to add arbitrary programs to the TCB, with the tradition being that host system apps are the majority and sandboxed apps are a minority; and the price we pay for that freedom is that the programs added to the TCB are ... in the TCB.

smcv · 2021-09-29T09:40:44Z

A host system app could also do the equivalent of xdg-desktop-portal --replace and become the portal.

If we assume an attacker that somehow only has the ability to monitor D-Bus passively (Monitoring and/or legacy eavesdropping, the same mechanisms used in new and old dbus-monitor respectively), then we could craft a way to avoid the token being visible on the bus. The easiest way would be for the portal's client (browser, VNC server, Steam, etc.) to fd-pass the write end of a pipe to the portal, and the portal could write the token into the pipe, then close the pipe; the client could read into a buffer until EOF, and that's their token. An eavesdropper can't read from the write end of a pipe, so it would be useless to them.

However, that's 100% security theatre, because a host-system app that is only a passive attacker is not a realistic security model. If an attacker has the unfiltered access to the session bus that is necessary to carry out Monitoring or legacy eavesdropping, then they can also carry out active attacks. Perhaps the simplest active attack would be to ask the bus nicely for the org.freedesktop.portal.Desktop name, become the portal, and reimplement anything it can do.

grulja · 2021-10-06T14:02:30Z

From browser point of view, I think it only makes sense to use the restore_token option upon preview dialog confirmation, where we want to avoid the portal dialog again. Any other request to share a screen we want the user to again pick what he wants to share, therefore we should revoke the token after we successfuly share something on a web page, not after the browser is closed.

data/org.freedesktop.portal.ScreenCast.xml

GeorgesStavracas · 2021-10-07T18:45:31Z

@grulja this proposal contemplates this specific case in the form of the "transient" persist mode. It wouldn't touch the permissions store in this mode. I'm curious as to why it's not possible to simply pass the PipeWire stream between dialogs though.

GeorgesStavracas · 2021-11-09T13:38:59Z

This is ready for review now. The missing bits (removing transient permissions when a client disconnects, and apply review comments) are implemented.

data/org.freedesktop.impl.portal.ScreenCast.xml

src/screen-cast.c

data/org.freedesktop.portal.ScreenCast.xml

GeorgesStavracas · 2021-11-11T03:19:28Z

See https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/merge_requests/14 for a working, complete implementation of this feature.

GeorgesStavracas · 2021-11-11T04:13:23Z

While working on the corresponding API in libportal, a question arised: should "restoring a stream" also restore extra information like the cursor mode, output types, etc?

To expand a bit, libportal has this API:

void xdp_portal_create_screencast_session (XdpPortal           *portal,
                                           XdpOutputType        outputs,
                                           XdpScreencastFlags   flags,
                                           XdpCursorMode        cursor_mode,
                                           GCancellable        *cancellable,
                                           GAsyncReadyCallback  callback,
                                           gpointer             data);

Inspired by that, I thought about adding a sibling function that would look like this:

void xdp_portal_restore_screencast_session (XdpPortal                *portal,
                                            XdpScreencastPersistMode  persist_mode,
                                            const char               *restore_token,
                                            GCancellable             *cancellable,
                                            GAsyncReadyCallback       callback,
                                            gpointer                  data);

Which seems quite natural, almost obvious of an API. Which is great.

I'm still pondering the pros and cons of making stream restore also account for cursor mode, output types, etc.

Pros:

Easier to write an elegant corresponding libportal API
Portal implementations don't need to check if the to-be-restored stuff matches the output types (e.g. trying to restore a window stream, but the session is configured to monitor streams only)

Cons:

The more state we store, the harder it gets to add introduce new features or change existing ones
It might be annoying for apps to be "stuck" with their previous screencast state if they want to restore. Specially when some of these options can affect performance (e.g. embedded vs metadata cursors), it might be better to not restrict the API in this way.

I'm currently leaning towards the current pull request, which does not store any additional information. I'd love to hear your thoughts on this

GeorgesStavracas · 2021-11-11T19:46:37Z

I've been talking to various design-oriented folks about this feature, and what would an ideal interaction would look like, and apparently this proposal is flexible enough to allow portal implementations to do pretty sophisticated things (like, waiting for a window or monitor to be plugged when restoring a previous stream, etc). It doesn't seem necessary to add more properties to SelectSources() to be able to do that. So I guess this API design is playing out nicely in practice!

jadahl · 2021-11-11T20:11:25Z

...
I'm currently leaning towards the current pull request, which does not store any additional information. I'd love to hear your thoughts on this

Another reason to not attempt to restore cursor mode etc is that the session might not be available to be restored, meaning the application still needs to provide those parameters in case a new session will be created.

GeorgesStavracas · 2021-11-11T21:33:32Z

Another reason to not attempt to restore cursor mode etc is that the session might not be available to be restored, meaning the application still needs to provide those parameters in case a new session will be created.

Great point... scratch that idea then :)

data/org.freedesktop.impl.portal.ScreenCast.xml

First implementation of a mechanism to restore previously configured screencast sessions. There are 2 important workflows happening in parallel: acquiring a token, and using it. # Acquiring a restore token 1. Applications that want to be able to restore screencast streams pass the `persist_mode` option to the `SelectSources()` method. Persist modes are explained below. 2. `xdg-desktop-portal` pass the `persist_mode` option to the portal implementation. 3. The portal implementation queries the user (usually via a dialog) which monitor or window to share, and whether they allow it to be restored. 4. If the screencast session should be restored, the portal implementation hands `xdg-desktop-portal` the implementation-specific data to be restored. This is passed via the `restore_data` option, at the response of the `Start()` call. 5. `xdg-desktop-portal` stores this data somewhere with an unique identifier that I called `restore_token`. 6. `xdg-desktop-portal` pass `restore_token` to the application. The application then should save this restore token however it thinks it's appropriate. # Using the restore token Now that the application has a restore token, it can restore a previous stream. This is how it happens: 1. Applications that want to restore a screencast stream pass the `restore_token` option to the `SelectSources()` method. 2. `xdg-desktop-portal` receives the token, and performs a lookup for the implementation-specific data associated with this token. 3. `xdg-desktop-portal` consumes this token and removes the permission. 4. `xdg-desktop-portal` sends this restore data to the portal implementation as the `restore_data` option to the `SelectSources()` method. 5. The portal implementation uses this restore data to match windows and monitors without presenting a dialog. If it fails to match (due to, for example, any monitors or windows not being available), it fallbacks to presenting the dialog. Tokens can only be used once to restore streams. The permissions they hold are revoked after use. That's what xdg-desktop-portal does at step 2.3. Applications must not assume anything about tokens. Portal implementations never have access to the restore tokens. # Persistence modes There are two modes of persistence: 1. Transient: the restore token is valid as long as the application is alive. It's stored in memory and revoked when the application closes its D-Bus connection. 2. Persistent: the restore token is stored in disk, using the permissions store, and is valid until the user manually revokes it. Extra care is taken to ensure that restore tokens aren't predictable. For the multi-stream case, add a new 'id' property to the list of stream properties. This allows applications to uniquely identify each stream.

With the version 4 of the screencast portal, it is now possible to request and use restore tokens [1] so that apps can restore a previously configured screencast session without user interaction. Add the corresponding code to linux-capture's PipeWire source. Store the restore token in the source data, since each restore token corresponds to an OBS source, and use it as soon as we try to create a new session. Implement the obs_source_info.save vfunc, and save the restore token when it's received by the Start() response using obs_source_save(). [1] flatpak/xdg-desktop-portal#638

All3xJ · 2022-03-20T17:49:01Z

In which portal version will this be implemented? I'm looking forward this "remember the choice" feature to not have always these prompts.

aleixpol · 2022-03-21T01:51:28Z

In which portal version will this be implemented?

I'm pretty sure GNOME already has it, for Plasma it should be available for 5.25 https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/79/

All3xJ · 2022-03-23T06:36:53Z

In which portal version will this be implemented?

I'm pretty sure GNOME already has it, for Plasma it should be available for 5.25 https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/79/

which version of GNOME? I have 41.5 and don't have it

mwleeds · 2022-03-23T16:38:42Z

In which portal version will this be implemented?

I'm pretty sure GNOME already has it, for Plasma it should be available for 5.25 https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/79/

which version of GNOME? I have 41.5 and don't have it

42 according to https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/blob/main/NEWS

All3xJ · 2022-04-07T19:06:10Z

In which portal version will this be implemented?

I'm pretty sure GNOME already has it, for Plasma it should be available for 5.25 https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/79/

which version of GNOME? I have 41.5 and don't have it

42 according to https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/blob/main/NEWS

I have updated my archlinux to GNOME 42, but portal window still has not the option to remember the choice

jadahl · 2022-04-08T07:12:41Z

I have updated my archlinux to GNOME 42, but portal window still has not the option to remember the choice

Does the application make use of the new feature? Are you running it in a flatpak?

All3xJ · 2022-04-08T13:27:16Z

Are you running it in a flatpak?

no.

Does the application make use of the new feature?

I thought that this new update would automatically add the option "remember the choice". So the application devs have to manually implement it? I can make a feature request in flameshot git repo

jadahl · 2022-04-08T13:31:24Z

no.

If not via flatpak (or snap) then they need to be run in a cgroup compatible with #719

I thought that this new update would automatically add the option "remember the choice".

No, the application needs to keep track of a token that it can use to restore a previous session.

All3xJ · 2022-04-08T14:05:44Z

Ok thanks. But is this only for screencast, or can it be implemented on Flameshot (that uses Screenshot portal)?

jadahl · 2022-04-08T14:09:08Z

That's #649

All3xJ · 2022-04-10T18:25:22Z

Is there a way to readapt the "Screencast portal" restore code by @GeorgesStavracas to "Screenshot portal" in a fast and easy way so that a new volunteer who has to take time to do it is not needed? I mean, can the Screencast code be reused easily to apply it to Screenshot portal? If yes, I can try to do it, if it's not difficult, just suggest me how.

marler8997 · 2023-02-13T18:53:27Z

Is there any reason this couldn't also be added to remote desktop sessions?

P.S. the documentation in the PR description is great, but, would be very helpful to include some of those details in the documentation (i.e. details about the Token lifecycle, now all the "funny business" I saw in WebRTC's screencast usage makes sense).

jadahl · 2023-02-13T18:57:37Z

Is there any reason this couldn't also be added to remote desktop sessions?

See #850.

jadahl reviewed Oct 6, 2021

View reviewed changes

data/org.freedesktop.portal.ScreenCast.xml Outdated Show resolved Hide resolved

smcv mentioned this pull request Oct 16, 2021

Screenshot portal without prompt #649

Closed

GeorgesStavracas force-pushed the gbsneto/screencast-persist branch from b5b420f to 84fec1e Compare November 9, 2021 13:26

GeorgesStavracas marked this pull request as ready for review November 9, 2021 13:28

GeorgesStavracas changed the title ~~Draft: Screencast restore~~ Screencast restore Nov 9, 2021

GeorgesStavracas force-pushed the gbsneto/screencast-persist branch from 84fec1e to 70851b0 Compare November 9, 2021 13:34

GeorgesStavracas requested a review from smcv November 9, 2021 13:40

jadahl reviewed Nov 9, 2021

View reviewed changes

data/org.freedesktop.impl.portal.ScreenCast.xml Show resolved Hide resolved

matthiasclasen reviewed Nov 9, 2021

View reviewed changes

data/org.freedesktop.impl.portal.ScreenCast.xml Outdated Show resolved Hide resolved

GeorgesStavracas force-pushed the gbsneto/screencast-persist branch 2 times, most recently from ebf7007 to 67ed97d Compare November 9, 2021 18:16

jadahl reviewed Nov 10, 2021

View reviewed changes

GeorgesStavracas force-pushed the gbsneto/screencast-persist branch from 67ed97d to 1935bb4 Compare November 10, 2021 17:57

columbarius mentioned this pull request Nov 11, 2021

Create a more extendable chooser protocol emersion/xdg-desktop-portal-wlr#169

Open

GeorgesStavracas force-pushed the gbsneto/screencast-persist branch from 1935bb4 to f64fe30 Compare November 13, 2021 21:07

rohmishra mentioned this pull request Nov 14, 2021

Steam asks to share my screen on every startup ValveSoftware/steam-for-linux#8098

Closed

tinywrkb mentioned this pull request Nov 14, 2021

PoC Flatpak packaging H-M-H/Weylus#122

Open

columbarius reviewed Nov 14, 2021

View reviewed changes

data/org.freedesktop.impl.portal.ScreenCast.xml Show resolved Hide resolved

aleixpol approved these changes Nov 15, 2021

View reviewed changes

GeorgesStavracas and others added 2 commits November 19, 2021 19:44

data: Fix a couple typos

9aad2ff

mwleeds force-pushed the gbsneto/screencast-persist branch from e92b34b to 9aad2ff Compare November 20, 2021 03:45

mwleeds merged commit 610b3ad into flatpak:master Nov 20, 2021

GeorgesStavracas deleted the gbsneto/screencast-persist branch November 20, 2021 15:10

psqli mentioned this pull request Nov 21, 2021

PipeWire Virtual Camera obsproject/obs-studio#5377

Draft

6 tasks

GeorgesStavracas mentioned this pull request Nov 21, 2021

Screencast session restore for Wayland / PipeWire obsproject/obs-studio#5559

Merged

6 tasks

H-M-H mentioned this pull request Nov 22, 2021

Implement Screencast session restore for Wayland with pipewire H-M-H/Weylus#127

Open

X-m7 mentioned this pull request Mar 12, 2022

screencast: Add a way to remember (or programmatically assign) granted permissions #324

Closed

All3xJ mentioned this pull request Apr 8, 2022

A new version of xdg-portal has been released, and it's now possibile to "remember the choice" instead of clicking "Share" at every screenshot. flameshot-org/flameshot#2536

Closed

dc740 mentioned this pull request Sep 27, 2022

In-home Streaming + Wayland ValveSoftware/steam-for-linux#6148

Open

westonp42 mentioned this pull request Oct 14, 2023

[Feature Request] Implement Screencast Restore with -pipewire launch option ValveSoftware/steam-for-linux#10134

Open

bbb651 mentioned this pull request Jun 10, 2024

[Feature Request] Screencast: Allow some way to request windows by name or process #1064

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Screencast restore #638

Screencast restore #638

GeorgesStavracas commented Sep 29, 2021 •

edited

Loading

GeorgesStavracas commented Sep 29, 2021

smcv commented Sep 29, 2021

smcv commented Sep 29, 2021 •

edited

Loading

grulja commented Oct 6, 2021

GeorgesStavracas commented Oct 7, 2021

GeorgesStavracas commented Nov 9, 2021

GeorgesStavracas commented Nov 11, 2021

GeorgesStavracas commented Nov 11, 2021

GeorgesStavracas commented Nov 11, 2021

jadahl commented Nov 11, 2021

GeorgesStavracas commented Nov 11, 2021

All3xJ commented Mar 20, 2022

aleixpol commented Mar 21, 2022

All3xJ commented Mar 23, 2022

mwleeds commented Mar 23, 2022

All3xJ commented Apr 7, 2022

jadahl commented Apr 8, 2022

All3xJ commented Apr 8, 2022 •

edited

Loading

jadahl commented Apr 8, 2022

All3xJ commented Apr 8, 2022

jadahl commented Apr 8, 2022

All3xJ commented Apr 10, 2022 •

edited

Loading

marler8997 commented Feb 13, 2023 •

edited

Loading

jadahl commented Feb 13, 2023

Screencast restore #638

Screencast restore #638

Conversation

GeorgesStavracas commented Sep 29, 2021 • edited Loading

Implementation

1. Acquiring a restore token

2. Using the restore token

Token lifecycle

Persist modes

GeorgesStavracas commented Sep 29, 2021

smcv commented Sep 29, 2021

smcv commented Sep 29, 2021 • edited Loading

grulja commented Oct 6, 2021

GeorgesStavracas commented Oct 7, 2021

GeorgesStavracas commented Nov 9, 2021

GeorgesStavracas commented Nov 11, 2021

GeorgesStavracas commented Nov 11, 2021

GeorgesStavracas commented Nov 11, 2021

jadahl commented Nov 11, 2021

GeorgesStavracas commented Nov 11, 2021

All3xJ commented Mar 20, 2022

aleixpol commented Mar 21, 2022

All3xJ commented Mar 23, 2022

mwleeds commented Mar 23, 2022

All3xJ commented Apr 7, 2022

jadahl commented Apr 8, 2022

All3xJ commented Apr 8, 2022 • edited Loading

jadahl commented Apr 8, 2022

All3xJ commented Apr 8, 2022

jadahl commented Apr 8, 2022

All3xJ commented Apr 10, 2022 • edited Loading

marler8997 commented Feb 13, 2023 • edited Loading

jadahl commented Feb 13, 2023

GeorgesStavracas commented Sep 29, 2021 •

edited

Loading

smcv commented Sep 29, 2021 •

edited

Loading

All3xJ commented Apr 8, 2022 •

edited

Loading

All3xJ commented Apr 10, 2022 •

edited

Loading

marler8997 commented Feb 13, 2023 •

edited

Loading