Issues with window/frame sizes

[CrashBurnRepeat]

This is a continuation of issue #207 on GLVisualize.jl, but I want to focus on its manifestation in MakiE so as to use the newest interfaces. To provide a step-by-step of what I've done so far:

julia> using MakiE

julia> scene = Scene()

This creates the following blank scene:

Using the scene.data dictionary, we can see the current parameters for the window and the frame:

julia> scene.data[:window_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 840
 525

julia> scene.data[:window_area]
<Node: GeometryTypes.SimpleRectangle{Int64}(0, 0, 840, 525)>

julia> scene.data[:framebuffer_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 1680
 1050

If I change the window size at all, the window seems to adjust to the correct size:

Not the most exciting visualization, but running the same Julia commands:

julia> scene.data[:window_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 846
 530

julia> scene.data[:window_area]
<Node: GeometryTypes.SimpleRectangle{Int64}(0, 0, 1692, 1060)>

julia> scene.data[:framebuffer_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 1692
 1060

The main observation here is that in the first case, window_area matches the value of window_size, but after resizing the frame, window_area matches framebuffer_size. I've tried to hunt down the logic for initializing and later updating window_area, since it seems to be the parameter linked to this behavior, but admittedly I'm kind of running in circles. Hopefully this is enough evidence to point to a culprit, and if there's more I can provide I'd be happy to do so.

[CrashBurnRepeat]

Oh, another data point that might be useful. If we specify a resolution, for instance

julia> scene = Scene(resolution = (500,500))

The initial settings for the data entries above is:

julia> scene.data[:window_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 500
 500

julia> scene.data[:window_area]
<Node: GeometryTypes.SimpleRectangle{Int64}(0, 0, 500, 500)>

julia> scene.data[:framebuffer_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 1000
 1000

It would seem then that the frame is twice the dimensions it should be, if I'm understanding correctly. I've also seen a lot of adjustments in the code for macOS, which I'm using. Could it be something going wrong with that adjustment process, perhaps even something that's been changed recently in macOS?

[SimonDanisch]

yes! macos is special in this regard, and i was thinking i fixed it... i suspect there is a bug somewhere in the macos adjustment code.

[CrashBurnRepeat]

I will happily be the guinea pig if you need a macOS system to test things on.

[SimonDanisch]

Great! Can you try the same with JuliaGL/GLWindow.jl#51 ? I hope that fixes it.

[CrashBurnRepeat]

I just tried after checking out the sd/makie branch of GLWindow and the behavior is the same. Are there particular tests I can run on my end that could help?

[SimonDanisch]

Thanks! I've added some debug statements to the sd/makie branch, if you could try them out :)

[CrashBurnRepeat]

My new output:

julia> scene = Scene()
GLFW.GetFramebufferSize(window) = (1680, 1050)
fb_size = [1680, 1050]
Scene scene:
Dict{Symbol,Any} with 27 entries:
  :screen                => name: GLWindow…
  :mouse_button_released => <Node: 0>
  :mouse_buttons_pressed => <Node: Set{Int64}()>
  :time                  => <Node: 1.51035e9>
  :scroll                => <Node: [0.0, 0.0]>
  :buttons_pressed       => <Node: Set{Int64}()>
  :window_size           => <Node: [840, 525]>
  :window_area           => <Node: GeometryTypes.SimpleRectangle{Int64}(0, 0, 8…
  :theme                 => Scene: scene
  :cursor_position       => <Node: [0.0, 0.0]>
  :mouseinside           => <Node: true>
  :mouse_button_down     => <Node: 0>
  :mouseposition         => <Node: [0.0, 1050.0]>
  :arrow_navigation      => <Node: :nothing>
  :key_pressed           => <Node: false>
  :mouse2id              => <Node: [-1, -1]>
  :framebuffer_size      => <Node: [1680, 1050]>
  :button_down           => <Node: 0>
  :unicode_input         => <Node: Char[]>
  ⋮                      => ⋮

julia> scene.data[:window_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 840
 525

julia> scene.data[:window_area]
<Node: GeometryTypes.SimpleRectangle{Int64}(0, 0, 840, 525)>

julia> scene.data[:framebuffer_size]
2-element MakiE.ArrayNode{Int64,1,Base.#identity,GeometryTypes.Vec{2,Int64}}:
 1680
 1050

Should I be seeing other debug messages?

[SimonDanisch]

Ah I think I know what it is. Please tell me that the newest commit at sd/makie fixes it! :P
JuliaGL/GLWindow.jl@0fcefb3

[CrashBurnRepeat]

Looks the same, unfortunately. Again, I don't know if this helps to narrow things down, but I noticed that mouseposition is at (0.0, 1050.0), so at the full frame height. Is there some shared logic, or is mouseposition just reading the frame height after whatever code is causing the issue?

[CrashBurnRepeat]

Ok, I think I've narrowed it down to a single operation! In GLWindow in screen.jl, the instance of push! with the dispatch of:

function Base.resize!(x::Screen, w::Integer, h::Integer)

seems to be causing the issue, as when I eliminate this line, I get a properly sized window. Specifically, within this function, the line:

push!(x.area, SimpleRectangle(area.x, area.y, w, h))

seems to trigger some re-evaluation of the window that causes the issue. Oddly, resizing with the GUI still works properly, so I'm guessing that operation uses another dispatch. I'll try to look into some of the callbacks, but hopefully this helps to get closer to a root cause.

[SimonDanisch]

Ah, is it the resize! in MakiE?

[SimonDanisch]

Thanks for the detective work, I will have a look... But I think it makes sense, that this messes things up...

[mschauer]

• On my Debian machine the initial window size corresponds to the size of the white (displayed area), changing the windowsize does not affect the size of the painted/white area.
• On my OS X machine I have the same problem as OP, especially changing the windowsize does change (and fixes) the the size of the painted/white area.

[SimonDanisch]

Sorry, it's always annoying for me to work on osx issues, so I swiftly forgot about it altogether :D
Does JuliaGL/GLWindow.jl#53 fix it?
Btw, I also found another bug in Makie which I just fixed, that if you don't close the window, resize won't work anymore. Haven't pushed it yet, since It's a bit intermingled with other, not yet working changes ...

[CrashBurnRepeat]

I tried that branch, but I had to revert all the other graphics packages back to master to be able to do so, because it needs empty_shader_cache! from GLAbstraction. I didn't see a change, but is there some part of the fix that should be in the MakiE branches of these packages?

[SimonDanisch]

The list of packages has changed:

Pkg.clone("https://github.com/SimonDanisch/Makie.jl.git")
Pkg.checkout("GLAbstraction")
Pkg.checkout("GLVisualize")

# For UV examples, e.g. earth texture on sphere, or textured cat
Pkg.checkout("MeshIO")

# For image loading
Pkg.add("ImageMagick")

# For precompilation
Pkg.clone("https://github.com/SimonDanisch/PackageCompiler.jl.git")
Pkg.build("PackageCompiler")
Pkg.checkout("GLWindow")
```

[mschauer]

Unfortunately not. Makie started to print a lot of "auto" though. ;-)

[SimonDanisch]

oh yeah, thought printing auto would change it :-P

[CrashBurnRepeat]

I just submitted a pull request to the sd/osx branch with that I think is the solution. It looks like the final push! in the resize! function was still using the unscaled width and height, so I replaced that with the scaled values and everything seems to work properly. @mschauer, as a fellow macOS user, would you mind checking if that fix works for you too? JuliaGL/GLWindow.jl#54

[SimonDanisch]

so you checked out the GLwindow thing, and no behaviour change!? i'm multiplying the resize resolution by a value, so it'd be weird if nothing at all happens. .

[SimonDanisch]

nevermind :-D thanks @CrashBurnRepeat!!
JuliaGL/GLWindow.jl#54 (comment), so my change was well intended, but i had a typo^^

[mschauer]

Works!

[CrashBurnRepeat]

Awesome, I will be so happy to close this out/see it closed out once everything is merged!

[mschauer]

A minor thing: If I create the window with scene = Scene(resolution = (400, 400)) now I get

julia> scene[:window_area]
<Node: GeometryTypes.SimpleRectangle{Int64}(0, 0, 800, 800)>

[CrashBurnRepeat]

@mschauer, is this a breaking change for you? Overall though, I think it would help to go through the terminology in this package and straighten it out if needed. So far I've noticed window, frame, scene, screen, and I think I saw billboard mentioned on another issue. Perhaps these are all distinct, but I'm not sure I understand what that distinction is. I mention this all because I'm not sure what window is meant to represent conceptually, and if window_size and window_area should be different or not. Between these and framebuffer_size, it does seem like one is superfluous.
@SimonDanisch, is there anything holding up getting the osx branch and my changes to it merged into master?

[mschauer]

No problem caused by that at all, I just reported it because it surprised me.