High-dpi / high-resolution support, GUI scaling

[diizy]

This is again something that has been discussed in mailing lists which I'm posting here so that it exists in the issue tracker. This is all post-1.0.0 stuff and in no hurry to get implemented, though.

High-dpi screens are getting more common, and this presents a challenge to applications to function in diverse resolutions. Some desktop environments provide the option to automatically scale the GUI of applications, but it would be even better if the application does it themselves, or provides the option for the user to set the scaling factor.

I'm thinking the latter would be the best choice here, since it would also benefit people with impaired vision to be able to scale up the GUI.

So this would require two steps:

1. create a global "scaling factor" property, which varies from 1.0 (no scaling) to maybe 2.0, 2.5 or so - 2x size should be enough for all screens, although we can increase that size in the future if need arises.
2. modify all graphic and UI elements to take the scaling factor in account in all coordinates, sizes, pixmaps, etc.

This would also mean we should implement support for .svg in the pixmap embedding code. But that doesn't mean everything should be in .svg - we should use it where it makes sense and not compromise on visual style. I'm thinking a smart way to do this would be to modify embed.cpp so that when it's given a request to search for eg. "add_automation", instead of just searching for "add_automation.png" it will first search for .svg, if that isn't found, it searches for a png.

For the latter part, I'm thinking we can provide two versions of every png image: normal and 2x size. The way it would work is, in regular 1x scale, the regular "automation.png" would be used, but if the scaling factor is >1, then it would search for "automation.x2.png" and scale it down according to the scaling factor. This is because with bitmaps, scaling down is always cleaner than scaling up.

Most of the current UI elements already have either a .svg version or a 2x size png, which is good. But some don't. For the future, I'd like to request that all graphics contributions should either provide a .svg, or both 1x and 2x sized versions of a png. I've tried to ask this before but I haven't really enforced it as a rule - we should probably take a more strict approach to it in the future.

[eagles051387]

Vesa in regards to scaling I think you are making it really complex for
nothing here. There must be a way to get the screen resolution that the
system is running and adjust the UI according to the system set resolution
and adapt if the resolution is changed.

On Fri, Mar 14, 2014 at 8:50 AM, Vesa V notifications@github.com wrote:

This is again something that has been discussed in mailing lists which I'm
posting here so that it exists in the issue tracker. This is all post-1.0.0
stuff and in no hurry to get implemented, though.

High-dpi screens are getting more common, and this presents a challenge to
applications to function in diverse resolutions. Some desktop environments
provide the option to automatically scale the GUI of applications, but it
would be even better if the application does it themselves, or provides the
option for the user to set the scaling factor.

I'm thinking the latter would be the best choice here, since it would also
benefit people with impaired vision to be able to scale up the GUI.

So this would require two steps:

1. create a global "scaling factor" property, which varies from 1.0 (no
   scaling) to maybe 2.0, 2.5 or so - 2x size should be enough for all
   screens, although we can increase that size in the future if need arises.
2. modify all graphic and UI elements to take the scaling factor in
   account in all coordinates, sizes, pixmaps, etc.

This would also mean we should implement support for .svg in the pixmap
embedding code. But that doesn't mean everything should be in .svg - we
should use it where it makes sense and not compromise on visual style. I'm
thinking a smart way to do this would be to modify embed.cpp so that when
it's given a request to search for eg. "add_automation", instead of just
searching for "add_automation.png" it will first search for .svg, if that
isn't found, it searches for a png.

For the latter part, I'm thinking we can provide two versions of every png
image: normal and 2x size. The way it would work is, in regular 1x scale,
the regular "automation.png" would be used, but if the scaling factor is

1, then it would search for "automation.x2.png" and scale it down
according to the scaling factor. This is because with bitmaps, scaling down
is always cleaner than scaling up.

Most of the current UI elements already have either a .svg version or a 2x
size png, which is good. But some don't. For the future, I'd like to
request that all graphics contributions should either provide a .svg, or
both 1x and 2x sized versions of a png. I've tried to ask this before but I
haven't really enforced it as a rule - we should probably take a more
strict approach to it in the future.

Reply to this email directly or view it on GitHubhttps://github.com//issues/455
.

Jonathan Aquilina

[diizy]

On 03/14/2014 09:54 AM, eagles051387 wrote:

Vesa in regards to scaling I think you are making it really complex for
nothing here. There must be a way to get the screen resolution that the
system is running and adjust the UI according to the system set resolution
and adapt if the resolution is changed.

What exactly is really complex?

The problem with detecting screen resolution is that the DPI doesn't
tell us the actual size of the screen, or the distance from the viewer,
and when it comes to multi-monitor setups, it gets even more complicated...

No, it's better to let the user set the scaling factor. It's not a big
ordeal, it's just one more settings option which you only have to set
once (unless you change the monitor or something). And it also allows
the user to set the scaling to their preference.

[eagles051387]

Here is another thing to pick your brain. What if a user with a dual
monitor setup has to different screen sizes?

On Fri, Mar 14, 2014 at 9:21 AM, Vesa V notifications@github.com wrote:

On 03/14/2014 09:54 AM, eagles051387 wrote:

Vesa in regards to scaling I think you are making it really complex for
nothing here. There must be a way to get the screen resolution that the
system is running and adjust the UI according to the system set
resolution
and adapt if the resolution is changed.

What exactly is really complex?

The problem with detecting screen resolution is that the DPI doesn't
tell us the actual size of the screen, or the distance from the viewer,
and when it comes to multi-monitor setups, it gets even more complicated...

No, it's better to let the user set the scaling factor. It's not a big
ordeal, it's just one more settings option which you only have to set
once (unless you change the monitor or something). And it also allows
the user to set the scaling to their preference.

Reply to this email directly or view it on GitHubhttps://github.com//issues/455#issuecomment-37625217
.

Jonathan Aquilina

[diizy]

On 03/14/2014 10:42 AM, eagles051387 wrote:

Here is another thing to pick your brain. What if a user with a dual
monitor setup has to different screen sizes?

Well then that user is likely facing a lot of other problems as well.
Particularly, since LMMS doesn't provide any kind of multi-window mode.

That's a pretty narrow corner case though, most people with
multi-monitor setups specifically choose monitors with matching sizes
and resolutions.

In any case it's still better to let the user choose the scaling factor,
because the user knows best what size they like their GUI in.

[eagles051387]

I think a multi multi window mode should be setup in lmms. Then again isn't
that usually managed on a system basis as you can set it the resolution for
the 2nd screen separately

On Fri, Mar 14, 2014 at 9:53 AM, Vesa V notifications@github.com wrote:

On 03/14/2014 10:42 AM, eagles051387 wrote:

Here is another thing to pick your brain. What if a user with a dual
monitor setup has to different screen sizes?

Well then that user is likely facing a lot of other problems as well.
Particularly, since LMMS doesn't provide any kind of multi-window mode.

That's a pretty narrow corner case though, most people with
multi-monitor setups specifically choose monitors with matching sizes
and resolutions.

In any case it's still better to let the user choose the scaling factor,
because the user knows best what size they like their GUI in.

Reply to this email directly or view it on GitHubhttps://github.com//issues/455#issuecomment-37627186
.

Jonathan Aquilina

[diizy]

On 03/14/2014 10:58 AM, eagles051387 wrote:

I think a multi multi window mode should be setup in lmms. Then again
isn't
that usually managed on a system basis as you can set it the
resolution for
the 2nd screen separately

Ok sure but that's getting a bit off topic here. Maybe this should be
discussed in a separate topic or in the mailing list...

[Sti2nd]

It would be cool with high res icons and great if you could decide for yourself exactly how big you want them.

I am really wondering if it is necessary to manually edit all the graphics to take the scaling factor in account... Someone could probably make a program for that ❓

[diizy]

On 03/14/2014 04:08 PM, Stian Jørgensrud wrote:

It would be cool with high res icons and great if you could decide for
yourself exactly how big you want them.

I am really wondering if it is necessary to manually edit all the
graphics to take the scaling factor in account... Someone could
probably make a program for that ❓

Stian, if we were capable of making programs that are that good at
editing graphics, graphic designers would be out of jobs... :)

There are some pretty neat scaling algos, but they're not perfect and
each have their problems and they all produce some kinds of artifacts.
We already have either .svg or 2x versions of most our graphics, so it's
not that big of an issue to update the rest.

[Sti2nd]

I didn't meant auto scaling the pictures into higher res, which is impossible I think. I meant that adding editing the code manually could probably be done automatically. Isn't it just searching for specific words and then add the new lines of code?
I might have misunderstood you.

[diizy]

On 03/14/2014 10:40 PM, Stian Jørgensrud wrote:

I didn't meant auto scaling the pictures into higher res, which is
impossible I think.

Not impossible, just not necessarily very good quality. Although, to be
fair, it might not matter all that much on a high-dpi screen, where the
pixel details get too small to see...

I meant that adding editing the code manually could probably be done
automatically. Isn't it just searching for specific words and then add
the new lines of code?
I might have misunderstood you.

Well no, we can't edit the code automatically, it needs to be done by
hand. It's not just searching for specific words - there are so many
different cases, different data types, different ways of representing
size/location/length which all have to be handled slightly differently...

[boulder74]

regarding multi-mon it would be maybe an idea that you can manually type in the overal screen resolution.
so i.e. if you have 2 montitors with 1680x1050 (or whatever) then you simply enter
3360x1050 and thus the main window will extend also to the second mointor.
doing so it would be also necessary to have a kind of "reset to default settings" because if you move i.e a instrument window to the second monitor and its position is remembered on restart of LMMS - you would have a problem when the second monitor is not available anymore (for whatever reason).

JustMy2Cents

regards,
boulder

[eagles051387]

That is usually handled by the OS. On windows at the os level you tell it
to extend the desktop. What this would mean is that we should be able to
stretch the lmms GUI across multiple screens.

On Mon, Mar 17, 2014 at 12:24 PM, boulder74 notifications@github.comwrote:

regarding multi-mon it would be maybe an idea that you can manually type
in the overal screen resolution.
so i.e. if you have 2 montitors with 1680x1050 (or whatever) then you
simply enter
3360x1050 and thus the main window will extend also to the second mointor.
doing so it would be also necessary to have a kind of "reset to default
settings" because if you move i.e a instrument window to the second monitor
and its position is remembered on restart of LMMS - you would have a
problem when the second monitor is not available anymore (for whatever
reason).

JustMy2Cents

regards,
boulder

Reply to this email directly or view it on GitHubhttps://github.com//issues/455#issuecomment-37805531
.

Jonathan Aquilina

[boulder74]

on starting an application you can hand over the resolution to the GUI too...
i had once the case where the IE (Internet Explorer) should be opened over 2 screens (IE normally opens at primary monitor only). so i made a small program (.exe) which starts the IE spanning over 2 monitors.

that was the idea mentioned in my previous email.

[pgiblock]

Just for reference: Apple's work-around for their high DPI "retina" display is to provide assets at 1x and 2x resolutions. Regarding multiple monitors with "low" and "high" resolutions: a window only exists on one display at a time.

[diizy]

On 03/18/2014 03:49 PM, Paul Giblock wrote:

Just for reference: Apple's work-around for their high DPI "retina"
display is to provide assets at 1x and 2x resolutions. Regarding
multiple monitors with "low" and "high" resolutions: a window only
exists on one display at a time.

Well not necessarily, some people may want to stretch their LMMS window
across two or more displays, especially since LMMS doesn't provide any
kind of multi-window mode.

Although, in those cases people usually use monitors with similar
resolutions.

[diizy]

I'm kind of rethinking the necessity of this.

It seems that most modern desktop environments are already implementing HiDPI support on the OS level, which would make any application support somewhat redundant.

Of course, it could still be nice to have as it would mean better visual quality in HiDPI systems. But I think this is probably not going to be a priority at least in the near future.

[eagles051387]

I think one platform we can test this on is ubuntu to see how well its
supported at the OS level, but that is something that would need to be
researched further

On Sun, Mar 30, 2014 at 3:29 AM, Vesa V notifications@github.com wrote:

I'm kind of rethinking the necessity of this.

It seems that most modern desktop environments are already implementing
HiDPI support on the OS level, which would make any application support
somewhat redundant.

Of course, it could still be nice to have as it would mean better visual
quality in HiDPI systems. But I think this is probably not going to be a
priority at least in the near future.

Reply to this email directly or view it on GitHubhttps://github.com//issues/455#issuecomment-39014774
.

Jonathan Aquilina

[diizy]

I hear Qt 5 is getting HiDPI support on the toolkit level, so for now, I'm thinking we should just wait until the time comes to migrate to Qt5, and when that happens we'll get HiDPI support "for free". I don't think it makes much sense to implement application-wide HiDPI support only to have it replaced later on by Qt's solution.

Until that, people with HiDPI screens can just use the OS/DE scaling capabilities...

[tresf]

The font issues are mostly a duplicate of #455. Issue #455 has specific files that can be fixed right now to help support better font scaling and is a place people can help right away.

The Qt5 "HiDPI" stuff is as described a "free" benefit from switching from Qt4 to Qt5, so that is mostly addressed here #2108.

The rest (svg support, etc) will come with time through long-term work with theme updates such as #1839, #880, #1911.

Closing for now, but please feel free to reference or reopen at a later time.