CSS3DRenderer offset for Safari on High DPI displays with odd resolution

[JohannesDeml]

Description of the problem

When combining a webgl renderer with a cssrenderer (by punching holes through the webgl renderer to show the css renderer below, there is sometimes a missmatch between the renderers. It looks as if the cssRenderer has a slight offset compared to the webglrenderer. Interestingly enough, this happens on iPhoneX, but not iPhoneSE or iPad Mini4 and also not on android and desktop. Not sure about newer iPhones. The problem is especially present when looking at the css area at a very shallow angle.

The problem is not bound to the css object present (also tried it with a simple ).
There should be no white area around the youtube video:
Video

• codepen (latest release branch)

Three.js version

• Dev
• r118
• ...

Browser

• All of them
• Chrome
• Firefox
• Safari

OS

• All of them
• Windows
• macOS
• Linux
• Android
• iOS

Hardware Requirements (graphics card, VR Device, ...)

• iPhone SE
• iPhone 8
• iPhone X
• iPhone 11 Pro
• MacBook Pro with external High DPI Display
• iMac

[yomotsu]

Confirm and reproduced the problem on my iPhoneX too.
I also checked another example: https://threejs.org/examples/?q=css#css3d_sandbox but this seems to work with no gap on the same iPhoneX

[JohannesDeml]

Confirm and reproduced the problem on my iPhoneX too.
I also checked another example: https://threejs.org/examples/?q=css#css3d_sandbox but this seems to work with no gap on the same iPhoneX

That is very interesting, thanks a lot for pointing it out! I somehow missed that example.

Maybe there is a precision problem for handling matrix3d css stacks that is even more apparent for my example (since it is scaled by a factor of 100). I will try to expose the difference that creates the very visible problem in my example.

[JohannesDeml]

I was able to check the example @yomotsu posted with an iPhoneX and the offset also happens there.

So I guess it is safe to say that the problem does exist (since I built my example without knowing of the other example) and it is a device specific limitation for iPhone X (and maybe newer, sadly I don't have a newer iOS device at hand).

[JohannesDeml]

I was able to reproduce the problem on an iPhone 11 Pro as well. Could this have something to do with the notch / Rounded corners? I am already using the cover meta tag to avoid such problems, but it's really strange that the problem starts to appear on iOS devices with a notch...
<meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0, initial-scale=1.0, viewport-fit=cover" />

[mrdoob]

I've seen similar issues in the past and I think the issue was something like not being able to match CSS's perspective code with the current PerspectiveCamera updateProjectionMatrix code. Didn't end up digging deeper.

[WestLangley]

FWIW, on Safari on my iMac, showing the favorites bar causes a similar offset artifact (refresh required). It also occurs with page zoom not equal to 100%.

[JohannesDeml]

I've seen similar issues in the past and I think the issue was something like not being able to match CSS's perspective code with the current PerspectiveCamera updateProjectionMatrix code. Didn't end up digging deeper.

Hm, I think that might be a different issue, I built a repro with a orthographic camera, which shows the same problem.

FWIW, on Safari on my iMac, showing the favorites bar causes a similar offset artifact (refresh required). It also occurs with page zoom not equal to 100%.

Interesting, thanks for pointing it out. I dug a bit deeper, and I get the same issue when using a MacBook connected to a high DPI monitor with scaling set to the smallest value (might be needed just to get to a higher css resolution).

As for zooming, that might be connected to this issue (Repro), which was fixed in every browser except safari.

Anyways, I think I might have a lead on how to fix the problem:
As mentioned, the problem can be reproduced on MacOS. There I took a closer look at the css styling of the camera element. I noticed that changing the last translate value (which is half width, halfHeight) by 0.5-1.0 pixels seems to fix the problem. I haven't found out, when we have to change it in one other the other direction. I need to dig a bit deeper there.

If that is the case and the fix is to handle the transform values differently in the camera on safari, where would I need to put the user agent read to validate if the browser is safari? Should I put that in the Camera code, or somewhere else?

[WestLangley]

On your orhto example, with Safari on iMac, resizing the window vertically only or horizontally only is illustrative.

Maybe try

_widthHalf = Math.round( _width / 2 );
_heightHalf = Math.round( _height / 2 );

Just a guess.

[JohannesDeml]

Maybe try

_widthHalf = Math.round( _width / 2 );
_heightHalf = Math.round( _height / 2 );

Just a guess.

Yep, that does it for safari

So we will need to implement a check for safari (similar to what is done for ie) and use rounded values there.
I will also port my perspective example to a test branch to see, if the rounding fixes that problem as well.

Update: Nope, for the perspective camera, I will need to dig a bit deeper 😣

[JohannesDeml]

Okay, so I fixed the orthographic problem, sadly it seems that it does not have too much to do with the perspective problem other than having problems with odd resolutions.

I made a playground branch with a perspective and orthographic css scene which show the problem quite well:
Orthographic
Perspective

The problem that still exists now is:

CSS rendering for safari with odd width and or height. In that case the css looks like it is pushed away a bit from the camera:

When rotating the camera orthogonal to the plane, the plane offsets in interesting ways: https://www.youtube.com/watch?v=5YeipB0ZKGs

This is not a translation problem anymore. Has anybody any idea how to tackle that one? Maybe perspective or scaling at some point?

[JohannesDeml]

@mrdoob my PR sadly only fixed the problem for the orthographic camera, but not the perspective camera. Should I open another issue for that, or should we keep that one open?

[manthrax]

I've encountered similar problems on a past product. The "page zoom" controls also affect these transforms. We ended up setting preferred min-zoom/max-zoom in the CSS to mitigate problems..
I also recall that chromes handling of the CSS transforms changed somewhat arbitrarily a number of times over the course of 6 months of work on the feature, requiring it to be addressed many times, only to have chrome revert the breaking changes, or, alternatively, respond to our bug reports of the issue.

[Mugen87]

This needs to be fixed in Webkit: https://bugs.webkit.org/show_bug.cgi?id=215590

[zh-pan]

This issue beset me too, I debug a few hours. The result I've come to is this, you need set canvas width/height as even number at first, and set css3DObject's element width/height as even number too. The code maybe like this:

css3DRenderer.setSize(
  width % 2 === 0 ? width : width - 1,
  height % 2 === 0 ? height : height - 1
 );
----------------------------------
css3DObject.element.style.width = (width % 2 === 0 ? width : width - 1)+'px'
css3DObject.element.style.height= (height% 2 === 0 ? height: height- 1)+'px'