urxvt very slow after d7cff56cad779ee30769ec6b1a78ebb0c23281b8

[CyberShadow]

Hi,

The commit d7cff56 seems to have introduced a regression: urxvt font drawing becomes very slow. More specifically, urxvt with the font 7x13 and window size of about 1920x2160 takes about 300-500ms to draw its contents with mc running in it.

Platform

Arch Linux (x64)

GPU, drivers, and screen setup

RX Vega 64, AMDGPU drivers (Linux 4.18.7)

[yshui]

@CyberShadow I do suspect that change can add overhead, but didn't expect it to be this bad.

I will look into it.

[yshui]

@CyberShadow BTW, what is your compton configuration? I can't seem to directly reproduce the problem on my side.

[CyberShadow]

Hmm, I use a shader for transparency, but I did not think it was important. I will try to reduce the configuration to a minimal one in which the problem manifests.

[CyberShadow]

Hi,

I can reproduce the problem with the default (no arguments) configuration for Compton and urxvt.

If it helps, here's what urxvt's stack trace looks like while it's repainting:

Thread 1 (Thread 0x7f93a22ad780 (LWP 10360)):
#0  0x00007f93a3826b78 in poll () from /usr/lib/libc.so.6
#1  0x00007f93a2fea180 in ?? () from /usr/lib/libxcb.so.1
#2  0x00007f93a2fea895 in ?? () from /usr/lib/libxcb.so.1
#3  0x00007f93a2fea91e in xcb_writev () from /usr/lib/libxcb.so.1
#4  0x00007f93a407ed1f in _XSend () from /usr/lib/libX11.so.6
#5  0x00007f93a40672fc in XDrawImageString () from /usr/lib/libX11.so.6
#6  0x0000556973137b4f in rxvt_font_x11::draw(rxvt_drawable&, int, int, unsigned int const*, int, int, int) ()
#7  0x000055697313361b in rxvt_term::scr_refresh() ()
#8  0x0000556973134589 in rxvt_term::flush() ()
#9  0x000055697314f3ec in ev_invoke_pending() ()
#10 0x000055697314fc91 in ev_run ()
#11 0x000055697312fd69 in main ()
Detaching from program: /usr/bin/urxvt, process 10360

Let me know if I can do anything to help diagnose this.

[yshui]

@CyberShadow By default, do you mean you have an empty ~/.config/compton.conf? Is urxvt transparent? Is the background blurred?

I tried compton --config /dev/null and nothing in my xrdb, but I can't reproduce this problem.

[CyberShadow]

Yes, I just tested again with an empty compton.conf and the problem still happens.

I don't know if it matters, but I use i3 (i3-gaps).

Did you test with mc? The redraw speed seems to depend on what's in urxvt's window.

[yshui]

@CyberShadow yes, I tested mc.

I wonder if enabling vsync would help, is vsync previously enabled?

[CyberShadow]

Sorry, I don't know. I'm using completely default video/X11 configuration (I don't have an Xorg.conf).

[CyberShadow]

Sorry, if you mean the Compton setting, I just tried all 6 listed in the configuration and saw the same result.

[yshui]

@CyberShadow I tried to make urxvt transparent and blur the background, but I still can't reproduce this.

I will leave this open in case someone has something to add. If you want to sink some time into debugging this, that will be much appreciated. I will provide support if you want.

[CyberShadow]

Okay, I know a little bit about X11 programming so I could have a look.

Could you tell me a bit more for why d7cff56 is needed? And, is there a test case for it? Maybe I can find a way to fix it that doesn't result in slowness on my machine.

[yshui]

@CyberShadow Basically, compton uses the damage event to determine when to update the screen. To detect damage of a window (or rather, a drawable), you create a damage attached to the drawable. There are several event report mechanism. Before d7cff56, the one used is "NonEmpty", which generates event when damaged region goes from empty to non empty. After getting this event, compton will clear the damaged region (so more event can be generated), and repaint the screen.

The problem is, X server sometimes just doesn't send the damage event to the compton. The event seems to be generated, but is kept on the server and not sent, causing the whole screen to freeze. So d7cff56 change the reporting mechanism to "Raw", cause all damages to generate events. In theory, this change will increase the traffic between compton and X server, but it shouldn't increase the number of repaints. Since compton always tries to process as many events as possible before repainting.

And even if compton is doing more repaints than necessary, you shouldn't see your urxvt window freeze during the repaints.

[yshui]

@CyberShadow the freeze is a hard to reproduce problem, but several people have encountered this problem, see chjj/compton#494

[CyberShadow]

I will have a look.

In the worst case, we can add an option.

But, if it is a problem in the X server, it should be fixed there...

[yshui]

But, if it is a problem in the X server, it should be fixed there...

Indeed. But X is a much larger and scarier project to get into... I have made inquiries on xorg mailing list. I have yet to receive a reply.

[CyberShadow]

Thanks for the explanation. With it, and after reading the code and Damage protocol spec, I think I understand the situation.

Here is what I found so far:

• Before the patch, I only get a few DamageNotify events per repaint.

• After the patch, I get a LOT (about 3220 with mc). What about you?

• Looking at the CPU usage while urxvt is repainting, I see Xorg is busy using 100% CPU on one core.

• Here is a sample stack trace of Xorg:

  #0  0x00007fd1be809b02 in ?? () from /usr/lib/dri/radeonsi_dri.so
  #1  0x00007fd1be7e721b in ?? () from /usr/lib/dri/radeonsi_dri.so
  #2  0x00007fd1bee03242 in ?? () from /usr/lib/dri/radeonsi_dri.so
  #3  0x00007fd1bee005d0 in ?? () from /usr/lib/dri/radeonsi_dri.so
  #4  0x00007fd1bee01818 in ?? () from /usr/lib/dri/radeonsi_dri.so
  #5  0x00007fd1beb2c182 in ?? () from /usr/lib/dri/radeonsi_dri.so
  #6  0x00007fd1bf62b9b9 in ?? () from /usr/lib/xorg/modules/drivers/amdgpu_drv.so
  #7  0x000056306755649c in _CallCallbacks ()
  #8  0x000056306749048b in ?? ()
  #9  0x0000563067490a7b in WriteToClient ()
  #10 0x000056306754b702 in WriteEventsToClient ()
  #11 0x0000563067529264 in ?? ()
  #12 0x000056306750be5c in DamageReportDamage ()
  #13 0x000056306750c7b8 in ?? ()
  #14 0x0000563067510ebb in ?? ()
  #15 0x000056306755ac3e in ?? ()
  #16 0x0000563067416fd8 in ?? ()
  #17 0x00007fd1c3104223 in __libc_start_main () from /usr/lib/libc.so.6
  #18 0x00005630674172fe in _start ()
  

  Whatever the problem is, the GPU driver is definitely involved. (I could rebuild with symbols but I'm not sure they would give more insight.)

• I noticed that there are set_ignore_next calls before XDamageSubtract calls. Maybe the XDamageSubtract calls are sometimes failing for some reason, which causes the X server to not clear the damage and thus not send more events? It might be interesting to look at compton's output with --show-all-xerrors when the freeze bug happens.

• Whether or not there is drastic slowdown with urxvt/mc/etc., this still results in a lot more X11 traffic, so I think it needs to be an option. This is especially true if we're working around a bug in another program, because the bug might be fixed in the future, and then compton users might get stuck with a very inefficient workaround wasting cycles.

I think I had the freeze in chjj/compton#494 happen to me too. Too bad I didn't pay attention to it or how to reproduce it, because now I can't...

[yshui]

I noticed that there are set_ignore_next calls before XDamageSubtract calls

I tried removing those calls, but didn't get any error regarding DamageSubtract

Whatever the problem is, the GPU driver is definitely involved.

Interesting how sending packets to the client needs to involve the GPU driver...

this still results in a lot more X11 traffic, so I think it needs to be an option

I guess it probably has to be ..

[yshui]

@CyberShadow BTW, urxvt is somewhat an outlier nowadays. most program does their rendering and won't generate so many damage reports.

[CyberShadow]

Should I make a PR to add the option?

We could even go as far as make it a per-window rule (XDamageReportRawRectangles for mpv, XDamageReportNonEmpty for urxvt)... but that's probably too much.

[yshui]

@CyberShadow I'm looking for an alternative work around. It's not that nice to ask the user to choose between screen freeze and laggy windows.

[CyberShadow]

Thanks, let me know if you find something.

[yshui]

@CyberShadow Can you test this branch: https://github.com/yshui/compton/tree/lotta-damage ?

Probably won't work, but worth a try.

[CyberShadow]

That is MUCH better (I can no longer see mc repaint), but it's still more sluggish than before d7cff56.

Also, as I keep switching desktops in i3, it seems to randomly bug out and windows become invisible.

[fungible]

@CyberShadow Have you tried using the options --xrender-sync and --xrender-sync-fence. I had the same issue running urxvt on Arch Linux. Setting these options seemed to have solved it for me.

[CyberShadow]

Still slow with xrender-sync = true; xrender-sync-fence = true; in my compton.conf, though the redraws seem to be batched together a bit more.

[yshui]

I investigated the original problem a bit more, and I thinking I am getting close to the real cause.

[yshui]

@CyberShadow Can you try the next branch? I implemented the proper fix for screen freeze there.

(This branch also has some extra changes that might contain bug).

[CyberShadow]

Wow, nice work finding the root of the problem and porting to XCB!

If I suspected the problem was in the communication between the X server and Compton I could have lent a hand.

As expected, the next branch works perfectly for me. Thanks!

[yshui]

@CyberShadow Beware that that branch could crash quite often.

Which looks like an interoperability bug between xcb and xlib.

[CyberShadow]

No problem. No crashes so far. I'll run it in gdb and get some stack traces if I run into any.

[yshui]

Closing as this is fixed both in next and upstream Xlib.