Adding GLES/EGL support to Termux apps

[twaik]

Hello there. As I mentioned here I found a way to share Android Native Window between processes. The library is here.
Some details of the implementation (the most important):

• Android Native Window is a queue of some ANativeWindowBuffers with a set of some methods manipulating them.
• ANativeWindowBuffer is a wrapper over gralloc buffer.
• gralloc buffer can be shared over Binder IPC or UNIX-socket.
• ANativeWindowBuffer can be binded to EGL/GLES texture and drawn on Surface usin shaders. Also the ANativeWindowBufffer can be attached to Surface and queued like a usual GraphicBuffer. That should improve perfomance but I didn't test this option.

About integration: for now the output program uses libgui to draw image to Surfaceflinger directly. It draws the shared window on top of other windows. That is a problem. The integration into Termux apk is necessary. I think the app should scan some folder for new UNIX-sockets and open windows the termux apps can bind to (like the button "New session" and bash).

[twaik]

Termux apps are apps built for Termux, like cmus or termux-wake-lock. Firefox is X11 app, it means it is hardware acceleration-enabled if running Xorg supports GLX. Termux have no Xorg support yet but I will write xorg-video-android driver when libcw(*1) will be tested and improved. I have already ported Xorg(*2) to android, but it is unusable without video output library. libgui supports video output but it is sdk-dependent and can not be used in non-Java userspace apps without hacks or AOSP-prebuilt binary blobs. So I will just improve libcw, then I will write xorg driver. And this Xorg will be usable inside chroot with DISPLAY=:0 environment variable. It will behave like XServer XSDL but it will be much faster. And only then I will add GLX support to that server. Something like this.
(*1) libcw is just the lib which shares drawing surface to another process which is able to draw it to the screen because Android does not support drawing without libgui. libgui is android-version dependent and it is can be built only with AOSP. Dalvik/ART are linked to libgui and that's why only Android apps (apks) can draw images to the screen. That's why I need Termux to support this lib.
(*2) Xorg built with my built script segfaults on some devices, that's why I want to port it to Termux first.

[twaik]

@fornwall I need you to implement window listener in Java side. I can create JNI-bindings to make this lib openable by Java or you can write something like JNI-dlopener for the app.

[twaik]

X11 server is Xorg. Xorg running in chroot or LXC uses glibc and it can not use Android libraries such as Android libEGL.so or Android libGLESv2.so.

an we just modify a component of X11 so X11 to communicate with a Android app ?

Look at sparkle driver. It renders image to the tmpfs and Android app draws it to the screen. It can not draw to the screen without Android app and can not use hardware acceleration. My solution will use the hardware acceleration without any bindings.

If X11 can just put a android app at work (that app being 3d capable) ,and X11 to receive the result back

It would be much slower than software acceleration.

We used long time ago a android app to play sound from Debian that runs inside lxc

And than I found a way to not use Android app...

[twaik]

It needs libGLESv2 and libEGL in chroot or LXC container too. The same problem as with Xorg. And XWayland + Wayland will be much slower in this case.

[twaik]

I have updated the lib git.
@fornwall I added GLSurfaceView based activity example. Maybe you can say something to improve it. Or you can just say what to do with lib to be integrated into Termux codebase.

[twaik]

Looks like this lib can solve this issue.

[twaik]

@fornwall I think it is ready to be integrated. I need only the activity/GLSurfaceView to be integrated into Termux code and I will be able to improve it. For now the only things needed to be adapted are JNI functio names to match JNI expectations (Java_com_example_SanAngeles_DemoRenderer_onSurfaceCreated -> Java_com_termux_libcwRenderer(?)_onSurfaceCreated)

[thebestnom]

any news on this?

[seisdr]

What about libhybris

[twaik]

No news. Libhybris is just android libc + library loader + some android library wrappers. We do not need to use external bionic and loader because termux uses device's bionic implementation. The only things we can use are wrappers but without universal API for hardware buffers or native windows they are completely useless. So we can not use GLES for rendering separate windows, only the whole screen. That is the reason we can not use EGL/GLES for X11/Wayland apps.

[RalfWerner]

Moin @twaik
I am satisfied with your X11-termux app together with Xwayland, except a few restrictions in "direct mode". It is definitely better than Xvnc. Most recently, I worked with proot/ubuntu to avoid the ELF restrictions of Q10-termux - you had read/detect. Unfortunately I could not yet use your app there. Have you checked this once and can you give me a tip what is missing?

[twaik]

@RalfWerner the app should be signed with the same key as Termux::App is.

[RalfWerner]

the app should be signed with the same key as Termux::App is

@twaik I think signing is not the problem, because otherwise I would not be able to install the app, but it worked in the real and emulated device in the termux environment. Only in ubuntu not!
The process PID3 (Xwayland - is different in termux/ubuntu) follows PID2 (x11 is the same for both OS). So PID3 must be stopped in termux before it is started again in ubuntu. Three shots from the Q10-emulator session to explain:

Shot 3 is made with nedit (PC-Xserver) from ubuntu and shows the PID3 logs (at the bottom the kill from shot2). I then added: apt-get install libegl1 in ubuntu, and tried again.:

X=Xwayland; px=`pgrep -o $X`; x11=com.termux.x11/.MainActivity
export XDG_RUNTIME_DIR=u/tmp; [ -z $px ]&& am start -n $x11; [ -z $px ]&& $X :0>TMP/TX 2>&1 &

Then am is not found in ubuntu (PID2 - service should still exist, but cannot be recognized with ps and ln -s /home/u/bin/am u/bin/am doesn't help) and when I start PID3 I get:
could not connect to wayland server (EE) Fatal server error: (EE) Couldn't add screen
Can you understand/repeat that and do you still have a solution?

[twaik]

@RalfWerner how did you install Ubuntu? Using proot? The server creates server socket only in Termux's environment, not Ubuntu's one.

[RalfWerner]

how did you install Ubuntu? Using proot?

yes! What would be an alternative? Are you interested in install script?
The 2nd choice Xserver: Xvnc works without problems with ubuntu but also only as an alternative

[twaik]

@RalfWerner even if it will be available to be used from Ubuntu you still wont be able to use Ubuntu's EGL/GLES implementation. Android has different graphical subsystem structure.

[twaik]

But if you start Xwayland in Termux you will be able to use it from Ubuntu. Using "export DISPLAY=..."

[RalfWerner]

That was my first unsuccessful attempt here again:

[~]$ u uu
a=arm, c= p=proot -b /storage -b /system -b /data -b /dev -b /proc -b /data/data/com.termux/files/home:/home --link2symlink -0 -r ../uu -w /root /usr/bin/env -i   HOME=/root PATH=.:/bin:/sbin:/system/bin RT=uu NL= TERM=xterm LANG=C.UTF-8
groups: cannot find name for group ID 3003
groups: cannot find name for group ID 9997
groups: cannot find name for group ID 20116
groups: cannot find name for group ID 50116
uu[~]# pp
  PID  PPID COMM                            ELAPSED
 2102     1 bash                              01:31
 2104  2102 aterm                             01:31
 2140 30128 proot                             00:04
 2144  2140 bash                              00:04
 2158  2144 toybox                            00:00
 8013  7979 bash                         6-04:01:40
 8045     1 sshd                         6-04:01:39
30126  8045 sshd                           03:22:36
30127  8045 sshd                           03:22:35
30128 30126 bash                           03:22:34
30608     1 bash                           03:01:38
30609 30608 Xwayland                       03:01:38
uu[~]# xterm -display :0
Warning: This program is an suid-root program or is being run by the root user.
The full text of the error or warning message cannot be safely formatted
in this environment. You may get a more descriptive message by running the
program as a non-root user or by removing the suid bit on the executable.
xterm: Xt error: Can't open display: %s
xterm: DISPLAY is not set

uu[~]# nedit uu.sh -display :0
NEdit: Can't open display
uu[~]# echo $DISPLAY
uu[~]# export DISPLAY=:0
uu[~]# nedit
NEdit: Can't open display

[ghost]

I don't see where /tmp was mounted in your command:

a=arm, c= p=proot -b /storage -b /system -b /data -b /dev -b /proc -b /data/data/com.termux/files/home:/home --link2symlink -0 -r ../uu -w /root /usr/bin/env -i   HOME=/root PATH=.:/bin:/sbin:/system/bin RT=uu NL= TERM=xterm LANG=C.UTF-8

Make -b $PREFIX/tmp:/tmp.

[RalfWerner]

aterm is the XClient from termux. It won't get any better if I finish it. env is in ubuntu the same as above. I try export XDG_RUNTIME_DIR=$PREFIX/tmp with the same effect!

uu[~]# export XDG_RUNTIME_DIR=$PREFIX/tmp
uu[~]# ll $PREFIX/tmp
total 11
drwx------.  3 root root 3488 Mar 11 13:12 .
drwx------. 20 root root 3488 Mar 11 16:15 ..
drwxrwxrwt.  2 root root 3488 Mar 11 13:12 .X11-unix
uu[~]# xterm -display :0
Warning: This program is an suid-root program or is being run by the root user.
The full text of the error or warning message cannot be safely formatted
in this environment. You may get a more descriptive message by running the
program as a non-root user or by removing the suid bit on the executable.
xterm: Xt error: Can't open display: %s
uu[~]#

But with your change suggestion in u uu it works! Hallelujah and many thanks.
I will test this 1st choice Xserver a little bit and describe it there. Could we possibly also solve the "direct mode crash problem"? I have little hope at SSM.

[twaik]

@RalfWerner you did not export DISPLAY

[RalfWerner]

XClients from termux and ubuntu can be seen here - that's more than I expected and Xvnc can.
everything is fine - thanks again! Can you understand/repeat the "direct mode crash problem" and possibly solve it?

[twaik]

For now I am spending all the time at work and don't have a time for development. In a month I am joining IDF, so I won't have a free time.

[Grimler91]

@seisdr let's be respectful

[RalfWerner]

In a month I am joining IDF, so I won't have a free time

@twaik Good for Israel but bad for termux - how sad! I read 3 years are common. I wish you success as well as God's support and protection for the time. You are apparently still young and very busy, while I'm old and have too much time - deal with A-Studio :)
For termux-x11: I successfully created *x11.apk last year with a build process in my debian V-Box emulation, but not edited it. With my signing I was able to use this plugin on all devices and emulators.
Studio uses a own signing so I tried to install termux-x11 in R11 where this is also for termux-app did and got:

Can you or @xeffyr give me a tip on how to fix this?

[ghost]

tip on how to fix this?

Uninstall all addons & previous Termux app before installing a new one. You are using a different signature key.

[RalfWerner]

How can it be? I've installed both APKs one after the other with Studio today and previously uninstalled all termux APKs from the R11 emulation - should I repeat?
The only thing that stayed before I started the installation with Studio is the data in /storage/* (google file app)

[ghost]

termux-x11 does not use our shared development key. As result, termux-x11 will have a key different from which is used by termux-app and other addons.

See https://github.com/termux/termux-app/blob/master/app/build.gradle#L34 for more information.

[RalfWerner]

Thanks @xeffyr for the tip. Both apps use targetSdkVersion 28 (see shot above) In addition to x11, I also use the plugins Widget - already contained the *keystore* and api - I copied them from app as with x11. This enabled me to install three *.apk (list) on both emulators - long wait and 65Gb now.

WSL[AS]$ ll `find x/wd/ -name "*keystore*"`
-rwxrwxrwx 1 th80wr th80wr 2097 Mar  6 16:10 x/wd/termux-api/app/dev_keystore.jks
-rwxrwxrwx 1 th80wr th80wr 2097 Mar  6 16:10 x/wd/termux-app/app/dev_keystore.jks
-rwxrwxrwx 1 th80wr th80wr 2097 Mar  4 17:23 x/wd/termux-widget/app/dev_keystore.jks
-rwxrwxrwx 1 th80wr th80wr 2097 Mar  6 16:10 x/wd/termux-x11/app/dev_keystore.jks
WSL[AS]$ ll `find x/wd/ -name "*.apk"`
-rwxrwxrwx 1 th80wr th80wr 17486369 Mar 14 11:36 x/wd/termux-app/app/build/outputs/apk/debug/app-debug.apk
-rwxrwxrwx 1 th80wr th80wr    53777 Mar 14 15:05 x/wd/termux-widget/app/build/outputs/apk/debug/app-debug.apk
-rwxrwxrwx 1 th80wr th80wr  2848568 Mar 14 16:52 x/wd/termux-x11/app/build/outputs/apk/debug/app-debug.apk

[~]$ t # start favorite check
Starting: Intent { cmp=com.termux.x11/.MainActivity } Warning: Activity not started, its current task has been brought to the front
IP:10.0.2.16, X:Xwayland/ p=*.xpm *.c bashrc :162034.1 make: /data/data/com.termux/files/home/TMP/check.exe is up to date.
E=> Display not opened! STOP # log: could not connect to wayland server (EE) Fatal server error: (EE) Couldn't add screen

[~]$ pp # process check
PID   PPID  COMMAND          ELAPSED
 8635  1740 com.termux.x11    7h38
 9195  1740 com.termux        7h30
 9274     1 sshd              7h30
10703     1 Xvnc              7h25
16731  9195 bash              1h32
16950 16731 proot             1h24
16960 16950 bash              1h23
17005 16960 nedit             1h19
19083  9195 bash              5:23

[~]$ env # check (partial) with "ln -s ../usr/ u" in $HOME
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so:/data/data/com.termux/files/usr/lib//libandroid-shmem.so
LD_LIBRARY_PATH=/data/data/com.termux/files/home/u/lib
XDG_RUNTIME_DIR=/data/data/com.termux/files/home/u/tmp

Compared to my previous checks with *t32.apk (also shot below shot 1-3).
I could not start PID3 (Xwayland) so far, but here (app-debug) PID1+2 (9195+8635) is visible. Processes and env are visible in the log above.
Shot 1,2 are from setup, startscreen of my gigaset device (with crash in x11-direct mode). Shot 3 startscreen of Q10 emulator. Shot 4 Studio with widget and api.

The fourth plugin api obviously lacks the Default Activity or is incorrect. Questions:

• What could be the error in PID3 (with *t32.apk it was still possible yesterday)?
• Where do I've to enter the API Default Activity to get this *.apk?
• Would one C:\Users\th80wr\.android\dev_keystore.jks suffice?

Addendum to the PID3 start: In the meantime I became successful (five shots below).
The order PID1-2 above is not arbitrary, although PID2 is a service. First I used kill 8635 to get a PID2-crash (shot1) and then switched to Emulated mouse (shot2) to avoid Direct mode-crash. The renewed OTS (alias t="bash .bashrc" in the api29 object) was then successful (shot3) and I started nedit=XCient from Ubuntu bash (shot4). This message (I chose Wait) often appears on emulated devices because they are very slow. In the last shot5 I started WM=fluxbox (PID4=XClient) from ubuntu.
Then I stopped this XClient and started it again from Termux - it looks a bit better/more readable. However, after a few minutes there was a PID2 crash. On real devices is X11 in this mode mainly stable!

[ghost]

What could be the error in PID3 (with *t32.apk it was still possible yesterday)?

Crash log is needed. adb logcat

Where do I've to enter the API Default Activity to get this *.apk?

Termux:API is not normal app and don't have default activity.

Would one C:\Users\th80wr.android\dev_keystore.jks suffice?

I have added keystore and config recently, so do git pull origin master for termux-x11 and do debug build again. It should be signed with correct key.

[RalfWerner]

Crash log is needed. adb logcat

As described in Addendum to the PID3 start X11 in api28 works satisfactorily for a Studio emulation. I would repeat that if targetSdkVersion 29 is possible - thanks so far!

Termux:API is not normal app and don't have default activity.

As described above, I test real devices (shot1: Termux:API 5.18 MB =api28) and emulations in Studio (>=api29). Because of Error in shot4 I can not do this here - what to do?

I have added keystore and config recently, ... for termux-x11 and do debug build again.

Your patch I've done in same way. My question was whether only one instead of four keystore files are necessary if all plugins use the same file is it?

[ghost]

You need to install Termux:API but not run it. Its a plug-in package and not app.

Regarding key - it is one for all APKs. Just added in per-repository mode.