IME Support

[lunabunn]

Update (April 2021):
Since this issue was initially opened, good progress has been made on IME support.
#258 Implemented proper candidate window positioning for egui_web
#278 Implemented IME composition events for egui as well as logic to fire them for egui_web
This means egui_web, as well as coreegui, now has IME support. (Thanks @n2 !)

However, eframe on native targets still DOES NOT have IME support. And with egui-glium (and many other winit-based egui backends), it is unlikely support will land any time in the near future. This is not an egui issue but rather an upstream issue. The biggest blocker is rust-windowing/winit#1497. winit currently does not have IME support, which means most of the Rust ecosystem does not have IME support. Glium/glutin are no exception.

egui-miniquad, despite not relying on winit, also has a similar issue because sokol-app, miniquad's windowing library of choice, doesn't have IME support either. The only egui backend listed in the README that would be able to support IMEs is egui_sdl2_gl. I opened an issue there.

---

Original issue:
Describe the bug

There are no events available for IME composition, nor do text inputs display IME composition strings. The candidate window is not shown in the appropriate location.

To Reproduce
Steps to reproduce the behavior:

1. See that https://docs.rs/egui/0.10.0/egui/enum.Event.html does not have any IME-related events.
2. Open up any egui program, regardless of the backend impl, and observe the lack of IME support.

Expected behavior
There are events available for when the IME composition string/state changes, and text inputs display said string/state correctly. The candidate window should be shown right below the cursor.

Screenshots
N/A

Desktop (please complete the following information):
N/A

Smartphone (please complete the following information):
N/A

Additional context
IME support is a(n often overlooked) must for many people around the world in any sort of production GUI.

[emilk]

It's being worked on in #247 !

[lunabunn]

It's being worked on in #247 !

As far as I can tell that PR is unrelated? On PC there needs to be a proper hook to pass composition events into egui. That PR seems like a "fix" at the egui_web level.

[emilk]

Ah, I read your issue to quickly! Yeah, #247 is IME for web, but you want IME for desktop.

[n2]

Please let me do it. I have a plan on it, and, it is important to me, too.

[lunabunn]

@n2 cool! I have other projects to work on so that'd be great. I think all you'd have to do in the egui space is to add Composition and CompositionEnd events and then add those to text inputs' caches state, displaying em at the cursor and clearing when the composition ends. Note that right now winit doesnt support any of this either so it will only work on web. I personally think having an invisible textbox that steals focus and fires egui events is better than implementing the whole thing in egui_web or whatever it is you're doing (for extensibility's sake)

[n2]

Thank you for your advise @lunabunn . Hidding text element(called agent) on web was what I did in #247 .
If I am right, there are tow things to do:

1. Positioning IME candidate window:
   Moving IME candidate window following text cursor, like this:
   
2. Showing composition text:
   Handing composition event correctly, like this:
   
   Let's discuss them separately.

• Positioning IME candidate window
  • On web, the winit team think it is diffifcult, but since we have agent, it may be done by moving agent following text cursor.
  • On native, winit has a API to do that(set_ime_position).
• Showing composition text:
  • On web, composition events was introduced to egui_web in IME enabled on web #247 , next things to do is to handing them correctly in egui.
  • On native, just as @lunabunn said, there is no support on the events.

So, below is my plan:

\	Positioning IME candidate	Showing composition text
web	Moving agent	Handling events
native	winit api	N/A

If you have any idea, please tell me, suggestions are welcome.

[lunabunn]

@n2 Ah, right. I know see what you're trying to do. Personally, I'd prefer to split this into the 2 parts (separate issues?):

1. Composition events and displaying composition text (only modifies egui crate)
2. Fire composition events and set candidate window position correctly for the egui_web backend crate (IME enabled on web #247)

egui_winit_platform is not an official backend crate, so doesn't belong in here. However, you might be interested in rust-windowing/winit#1497.

[emilk]

@lunabunn The official native backend is egui_glium, and glium uses winit (and exposes the set_ime_position function).

[lunabunn]

@lunabunn The official native backend is egui_glium, and glium uses winit (and exposes the set_ime_position function).

Ah, sorry, missed that. Regardless though, winit does not have IME support at the moment (beyond setting the IME position as you mentioned) so it's largely irrelevant. I guess we could add a third part?
3. Fire composition events and set candidate window position correctly for the egui_glium backend crate (blocked on rust-windowing/winit#1497)

[n2]

Yes, these three parts are exactly the same thing in the table above: @lunabunn

\	Positioning IME candidate	Showing composition text
web	Moving agent	Handling events
native	winit api

[lunabunn]

Yes, these three parts are exactly the same thing in the table above: @lunabunn

\ Positioning IME candidate Showing composition text
web Moving agent Handling events
native winit api

Mhm. I was mainly noting that those should be separate issues, as well as the actual impl within egui. This can be used as a tracking issue if needed.

[emilk]

How much work is left on this? Is it done?

[n2]

It's done, I think. Sorry for not informing you.

[emilk]

On the contrary - thanks for making it done :D

[SheldonNico]

winit add some IME support. Convert winit::event::Ime to egui::Event::Composition* and adjust the IME position seem to be enough for native. @emilk Do you think PR is needed?

[emilk]

@SheldonNico ooh, nice! Yeah, if you want to make a PR that would be awesome!

[SheldonNico]

Update:

IME now support Windows/MacOS/Linux.
winit still doesn't support IOS/Android. For those platforms, check rust-windowing/winit/issues/1497 for updates .