libwx_gtk2 caret + tags file bug

[petergeoghegan]

The blinking block caret within the main hex control view sometimes does not appear initially, which seems like a UI bug. I'll click within the hex control view, and see values within the DataInterpreter view, indicating that the clicks are in some sense "registered". However, as I said, the clicks don't "fully register" since the blinking block caret will not appear. I can continually click around, and the caret will not appear at any point (though the contents of DataInterpreter will change as I click around). I'm working against git tip here. Hopefully, a fix for this issue will make it into the final v0.24.

If I open a file for editing that has a large tag file associated with it already, then this often happens. I've been working around the issue by quickly changing the focus to another window within my desktop environment. Once I do that, the blinking block caret appears as expected. It's been a long time since I wrote any wxWidgets code, but I'd guess that this refocusing creates some kind of invalidation of the view's contents. That's why my workaround fixes the issue. The fix seems to last the entire rest of the session.

This is not the expected/correct behavior, of course. I expect the block caret to appear immediately, as happens on my machine with the same file when there is no associated tag file (for whatever reason). My workaround shouldn't be necessary.

Note that this seems to consistently happen with particular files, and consistently not happen with other files (there is always an associated tag file when the bug hits). There is probably some aspect of the file itself (e.g. the overall size), and/or my large tag file that accounts for this. Some random thing about the input into wxHexEditor makes the difference between the bug hitting or not hitting. I have no idea what this may be, but I am willing to investigate with some help.

Details of my build:

pg@marmot:~/code/wxHexEditor$ uname --all
Linux marmot 4.10.0-42-generic #46~16.04.1-Ubuntu SMP Mon Dec 4 15:57:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
pg@marmot:~/code/wxHexEditor$ ldd wxHexEditor
	linux-vdso.so.1 =>  (0x00007ffe543ef000)
	libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f6d0adf3000)
	libwx_gtk2u_adv-3.0.so.0 => /usr/lib/x86_64-linux-gnu/libwx_gtk2u_adv-3.0.so.0 (0x00007f6d0aa0d000)
	libwx_gtk2u_core-3.0.so.0 => /usr/lib/x86_64-linux-gnu/libwx_gtk2u_core-3.0.so.0 (0x00007f6d0a193000)
	libwx_baseu_xml-3.0.so.0 => /usr/lib/x86_64-linux-gnu/libwx_baseu_xml-3.0.so.0 (0x00007f6d09f83000)
	libwx_baseu_net-3.0.so.0 => /usr/lib/x86_64-linux-gnu/libwx_baseu_net-3.0.so.0 (0x00007f6d09d37000)
	libwx_baseu-3.0.so.0 => /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0 (0x00007f6d098a4000)
	libwx_gtk2u_aui-3.0.so.0 => /usr/lib/x86_64-linux-gnu/libwx_gtk2u_aui-3.0.so.0 (0x00007f6d0960b000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6d09289000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6d08f7f000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6d08d69000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6d08b4c000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6d08781000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6d0857d000)
	libgtk-x11-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0 (0x00007f6d07f32000)
	libgdk-x11-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdk-x11-2.0.so.0 (0x00007f6d07c7c000)
	libgdk_pixbuf-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007f6d07a5a000)
	libpango-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007f6d0780e000)
	libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f6d075ba000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f6d072a9000)
	libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f6d06f6f000)
	libnotify.so.4 => /usr/lib/x86_64-linux-gnu/libnotify.so.4 (0x00007f6d06d66000)
	libpangocairo-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007f6d06b59000)
	libcairo.so.2 => /usr/lib/x86_64-linux-gnu/libcairo.so.2 (0x00007f6d06845000)
	libXxf86vm.so.1 => /usr/lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x00007f6d0663e000)
	libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6 (0x00007f6d06436000)
	libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007f6d06211000)
	libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f6d05fb7000)
	libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f6d05d43000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f6d05b1a000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6d058ff000)
	/lib64/ld-linux-x86-64.so.2 (0x000055dd6a02c000)
	libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f6d056fb000)
	libXfixes.so.3 => /usr/lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007f6d054f4000)
	libatk-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libatk-1.0.so.0 (0x00007f6d052cf000)
	libgio-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007f6d04f47000)
	libpangoft2-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007f6d04d30000)
	libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f6d04aed000)
	libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f6d048e2000)
	libXinerama.so.1 => /usr/lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007f6d046df000)
	libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007f6d044cf000)
	libXrandr.so.2 => /usr/lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007f6d042c4000)
	libXcursor.so.1 => /usr/lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007f6d040b9000)
	libXcomposite.so.1 => /usr/lib/x86_64-linux-gnu/libXcomposite.so.1 (0x00007f6d03eb6000)
	libXdamage.so.1 => /usr/lib/x86_64-linux-gnu/libXdamage.so.1 (0x00007f6d03cb3000)
	libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007f6d03aa0000)
	libthai.so.0 => /usr/lib/x86_64-linux-gnu/libthai.so.0 (0x00007f6d03897000)
	libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f6d0368e000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f6d0341e000)
	libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f6d031fc000)
	libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f6d02f51000)
	libpixman-1.so.0 => /usr/lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007f6d02ca9000)
	libxcb-shm.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007f6d02aa5000)
	libxcb-render.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007f6d0289a000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6d02692000)
	libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6 (0x00007f6d02477000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f6d02272000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f6d02050000)
	libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f6d01e42000)
	libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f6d01c1f000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f6d01a04000)
	libharfbuzz.so.0 => /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007f6d017a5000)
	libdatrie.so.1 => /usr/lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007f6d0159d000)
	libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f6d01398000)
	libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f6d01192000)
	libgraphite2.so.3 => /usr/lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007f6d00f6b000)

Thanks!

[petergeoghegan]

Additional information: As an alternative to my "change focus of window to invalidate hex control view", another workaround that I see working is to right click, showing the right click context menu. It seems to me that this reinforces the original theory: there is some issue with a failure to redraw/invalidate the view.

A similar bug can be observed when bytes are actually edited within the view. There appears to be some "ghosting" at times, where individual byte positions are sometimes not immediately redrawn to reflect the changes. Again, changing the window focus in my desktop environment makes the issue correct itself. Similarly, moving the caret around with the arrow keys redraws, once the actually-modified nibble/hex character is highlighted with the caret.

My uneducated guess is that the underlying cause is something like this: https://forums.wxwidgets.org/viewtopic.php?t=10114

Should I track this other issue by opening a separate Github issue?

[petergeoghegan]

Following a little debugging with a make OPTFLAGS="-O0 -g" CPPFLAGS="-D_DEBUG_CARET_" build, it now looks like the wxHexCtrl caret's state becomes confused. I added some wxCaret::IsVisible() traces, and they indicated that the caret was not visible when it should be (when the bug hits). It seems very clear that there are too many or too few wxCaret::Show() calls.

It looks like the issue is actually timing sensitive (this may be why some files don't seem to have the issue at all, as previously reported). I can see the bug or not see the bug by varying how quickly I make the first click. Then, the state of the caret is confused (it's incorrectly invisible) unless and until I change window in the desktop environment, and change back to wxHexEditor (i.e. apply my workaround).

So, my newly refined theory is that the event loop assumes a pairing of calls to wxCaret::Show() for the hex ctrl caret, but that this pairing doesn't always work out based on event backpressure or something like that (how many users have pushed the tags feature as far as I have before now?). Per the wx docs [1], if you call Show(false) n times, you must call Show(true) n times before the caret actually reappears. That's probably what fails to happen here. (Not sure about the second, "ghosting" issue I mentioned, though.)

[1] http://docs.wxwidgets.org/3.0/classwx_caret.html#a5112409bd3a83148f364c4b08086fe28

[EUA]

Caret shadow is something problematic on wxWidgets. I remember that it make big noise before but work good now even if it has some glitches that we need to work on it.
Indeed it acts different at different OSes. I remember that I even think about to build my custom caret...

If you figure out how to re-produce it, I will try to understand what happening.

[petergeoghegan]

Hi @EUA ,

I uploaded this video, which shows exactly what I'm talking about: https://youtu.be/CbN6rXSMUIw

Thanks!

[EUA]

The issue is related with Tags. More precisely:
Tag.cpp:52 - void TagElement::Show( const wxPoint& pos, wxWindow *parent )
If I return this function without doing anything, It just working.

So when Tag Popup is visible, caret doesn't want to work at first.
At least we identify the issue. :)

[EUA]

I dig deeper.
I put 2 breakpoints:

1. wxHexCtrl.cpp:1114 wxHexCtrl::OnFocus()
2. HexEditorCtrl.cpp::698 void HexEditorCtrl::OnMouseLeft()

If I click to TAG at first run, "hex_ctrl->SetFocus();" runs BUT it doesn't activate first breakpoint somehow. How could this be possible? I directly call it and it doesn't work!

Probably TAG Popup receive the event somehow and doesn't want to Skip it to the it's parents which is wxHexCtrl class...

At this point, I don't think if there are an easy solution.

[petergeoghegan]

I've stumbled upon a partial solution, or perhaps a clue:

@@ -682,17 +682,18 @@ inline uint8_t HexEditorCtrl::GetCharToHexSize( void ){
 void HexEditorCtrl::OnMouseLeft(wxMouseEvent& event){
     select->SetState( false );
 
-    if( event.GetEventObject() == hex_ctrl ){
-        hex_ctrl->SetFocus();
+    if( event.GetEventObject()->IsSameAs(*hex_ctrl)){
         focus=HEX_CTRL;
         SetLocalHexInsertionPoint( hex_ctrl->PixelCoordToInternalPosition( event.GetPosition() ) );
+        hex_ctrl->SetFocusIgnoringChildren();
         }
-    else if( event.GetEventObject() == text_ctrl ){
+    else if( event.GetEventObject()->IsSameAs(*text_ctrl)){
         text_ctrl->SetFocus();
         focus=TEXT_CTRL;
         SetLocalHexInsertionPoint( GetCharToHexSize() * text_ctrl->PixelCoordToInternalPosition( event.GetPosition() ) );
         }
-    else if( event.GetEventObject() == offset_ctrl ){
+    else if( event.GetEventObject()->IsSameAs(*offset_ctrl )){
         event.Skip(); //to lower level for copy offset to clipboard
         }

This almost fixes the problem. I find that I can click directly on a tag, and the caret appears. However, this only works after I scroll up and down a bit within the hex ctrl. I have to scroll up and down a bit when wxHexEditor initially starts, and then it's fine. I have to do the same procedure when I select the DataInterpreter dialog, and reselect the hex ctrl -- scroll a bit, and then clicking on a tag makes the caret appear.

BTW, I've also noticed that another workaround works (with or without this rough patch): If I click on an area in the hex ctrl that happens to not have a tag, the caret appears immediately, and continues to appear if I click on a tag.

In case it isn't obvious, I don't really have a clue what I'm doing here.

[EUA]

After all those years, the bug hides at here:

@@ -1175,13 +1175,14 @@ void wxHexCtrl::OnMouseMove( wxMouseEvent& event ){
                        if( (TagDetect >= TAX->start ) && (TagDetect < TAX->end ) ){    //end not included!
                                if( !(*TagMutex) && wxTheApp->IsActive() ) {
                                        *TagMutex=true;
-                                       TAX->Show( this->ClientToScreen(event.GetPosition() ) , this );
+                                       TAX->Show( this->ClientToScreen(event.GetPosition() ) , GetParent() );
                                        }
                                break;
                                }
                        }

Thank you.

[petergeoghegan]

@EUA Thanks!