[Request] Add support for Serial Comm protocols like Gemini PR or TX Bolt

[ar664]

I want to use "The Uni v3" on my android phone. Adding support for serial comms could probably support other steno machines that use it as well.

Example in Plover:
https://github.com/openstenoproject/plover/blob/3e7ba1afce0baf8992d43bc411100ca18e8d0d03/plover/machine/geminipr.py

A serial port android library I found googling:
https://github.com/lemoclone/SerialPortComm

[nimble0]

I started a branch several years ago with serial support. From what I remember it was working but I wasn't able to do all the testing I wanted to. It also needed a bit of clean up. I'll try and make time to clean it up and make the branch public at least.

[ar664]

Oh ok that's great to hear. I can at least test it with my steno board.

[nimble0]

Branch is up.

[ar664]

Sorry it took a while, to get back to this, but here is the crash log on permission accept.
Dotterel crash log.txt

Crash part:

--------- beginning of crash
04-10 23:18:38.041 E/AndroidRuntime(20418): FATAL EXCEPTION: main
04-10 23:18:38.041 E/AndroidRuntime(20418): Process: nimble.dotterel, PID: 20418
04-10 23:18:38.041 E/AndroidRuntime(20418): java.lang.RuntimeException: Error receiving broadcast Intent { act=nimble.dotterel.USB_PERMISSION flg=0x10 (has extras) } in nimble.dotterel.IntentForwarder$broadcastReceiver$1@28d8257
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.app.LoadedApk$ReceiverDispatcher$Args.lambda$getRunnable$0$LoadedApk$ReceiverDispatcher$Args(LoadedApk.java:1815)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.app.LoadedApk$ReceiverDispatcher$Args$$ExternalSyntheticLambda0.run(Unknown Source:2)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.os.Handler.handleCallback(Handler.java:938)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.os.Handler.dispatchMessage(Handler.java:99)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.os.Looper.loopOnce(Looper.java:226)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.os.Looper.loop(Looper.java:313)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.app.ActivityThread.main(ActivityThread.java:8641)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at java.lang.reflect.Method.invoke(Native Method)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:567)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1133)
04-10 23:18:38.041 E/AndroidRuntime(20418): Caused by: java.lang.Exception: Error probing serial USB device UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=36864,mProductId=1,mClass=239,mSubclass=2,mProtocol=1,mManufacturerName=stenokeyboards,mProductName=The Uni,mVersion=0.02,mSerialNumberReader=android.hardware.usb.IUsbSerialReader$Stub$Proxy@1290ac2, mHasAudioPlayback=false, mHasAudioCapture=false, mHasMidi=false, mHasVideoCapture=false, mHasVideoPlayback=false, mConfigurations=[
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbConfiguration[mId=1,mName=null,mAttributes=160,mMaxPower=250,mInterfaces=[
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=3,mSubclass=1,mProtocol=1,mEndpoints=[
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbEndpoint[mAddress=129,mAttributes=3,mMaxPacketSize=8,mInterval=10]]
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbInterface[mId=1,mAlternateSetting=0,mName=null,mClass=3,mSubclass=0,mProtocol=0,mEndpoints=[
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbEndpoint[mAddress=130,mAttributes=3,mMaxPacketSize=32,mInterval=10]]
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbInterface[mId=2,mAlternateSetting=0,mName=null,mClass=2,mSubclass=2,mProtocol=1,mEndpoints=[
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbEndpoint[mAddress=131,mAttributes=3,mMaxPacketSize=8,mInterval=255]]
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbInterface[mId=3,mAlternateSetting=0,mName=null,mClass=10,mSubclass=0,mProtocol=0,mEndpoints=[
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbEndpoint[mAddress=5,mAttributes=2,mMaxPacketSize=16,mInterval=5]
04-10 23:18:38.041 E/AndroidRuntime(20418): UsbEndpoint[mAddress=132,mAttributes=2,mMaxPacketSize=16,mInterval=5]]]]
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at nimble.dotterel.machines.UsbForAndroidSerialSocket.<init>(UsbForAndroidSerialSocket.kt:63)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at nimble.dotterel.machines.SerialStenoMachineKt$SERIAL_LIBRARIES$2.invoke(SerialStenoMachine.kt:56)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at nimble.dotterel.machines.SerialStenoMachineKt$SERIAL_LIBRARIES$2.invoke(Unknown Source:4)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at nimble.dotterel.machines.SerialStenoMachine.setConfig(SerialStenoMachine.kt:210)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at nimble.dotterel.Dotterel.configureMachine(Dotterel.kt:183)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at nimble.dotterel.machines.SerialStenoMachine.onIntent(SerialStenoMachine.kt:290)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at nimble.dotterel.IntentForwarder$broadcastReceiver$1.onReceive(IntentForwarder.kt:27)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	at android.app.LoadedApk$ReceiverDispatcher$Args.lambda$getRunnable$0$LoadedApk$ReceiverDispatcher$Args(LoadedApk.java:1805)
04-10 23:18:38.041 E/AndroidRuntime(20418): 	... 9 more

[nimble0]

If you try the library felHr (there should be a selection box where you configure the machine) it might work. Otherwise I need the details for the chip the machine uses. The issue is Dotterel doesn't know what driver to use so it just throws an exception (I should change it to show an error instead).

[ar664]

I don't see it in the settings. Is there anything special I need to do to build Dotterel? I'm using the latest android studio on Windows 10.

[nimble0]

Oh, it looks like I didn't put it in the settings. Change this line from "library": "usb-serial-for-android", to "library": "felHr",.

The serial machine settings only appear after the device has been plugged in as they're specific to that machine. Do they show up if you reject the permission?

[zenmetsu]

My apology for the issue necromancy...

I built the project with the library change called out above to include felHr library. The app launches and even detects the Uni v3 when it is attached.

Initially I had no luck getting a response out of the phone, but after manually setting the serial parameters (9600 8N2, no flow control, Gemini PR protocol), when i switch to the Dotterel keyboard and attach the Uni, it throws an error message which is partially displayed on screen. My apology for not knowing enough of android to provide better log output, I will gladly provide more data if instructed how to do so.

[nimble0]

I added a share log button in the top-right of the main settings page on the feature/serial-support branch. You can use that access the full log.

[leira]

Will this be merged to the mainline soon? What is the gap to merge it?

[nimble0]

Unlikely to be merged soon as I haven't been working on it. I think it's mainly bug testing that needs to be done.

[zenmetsu]

On my Samsung S22, Dotterel just goes into a crashloop whenever my Uni v3 is plugged in and the application is configured to speak to it via GeminiPR protocol. As soon as I am prompted to allow Dotterel access to the Uni, it goes into a crash loop.

I would like to provide logs to you, but I do not see a means to get to the logs.

When I plug in the keyboard, it shows up, and I am permitted to configure the Serial parameters. It isn't until I attempt to grant Dotterel permission to access the hardware that it crashes.

[zenmetsu]

Ah. I just realized that I am still running the previous build. I did not notice the build failures.

\StudioProjects\dotterel\app\src\main\java\nimble\dotterel\DotterelSettings.kt: (220, 27): Unresolved reference: root

\StudioProjects\dotterel\app\src\main\java\nimble\dotterel\DotterelSettings.kt: (249, 9): Unresolved reference: share_log

[zenmetsu]

found the crash log...

log1680051429444.txt

[zenmetsu]

Is this a simple permissions problem? I do not understand android well enough to develop for it, but the crash log gives me the impression that there is a permissions issue. I granted all permissions that were available to the app and it did not change the outcome.

[nimble0]

Thanks for the log. It's not a permissions issue. It doesn't have a serial driver for your machine, or at least doesn't know what driver to use. https://github.com/mik3y/usb-serial-for-android#compatible-devices Your machine maker might know if your machine is compatible based on this list.

[zenmetsu]

Well, that is unfortunate.

Is the library you linked the one used by dotterel? I can try to get support for this device if so.

This machine uses an ATMEGA32u4 which is listed as supported, but perhaps it is not using CDC/ACM protocol.

[zenmetsu]

Looking at the code, it seems that the Uni machine firmware is built using CDC. And since it is an ATMEGA32u4, I suspect they are reporting a different vendor/product ID pair. I will look into patching the code and report back if I have any success.

[nimble0]

I think you're right. I've been looking back at my code and there's a bit of code that tells it what driver to use for my test device's vendor and product ID.

I also noticed that in newer versions of the serial library it mentioned being able to recognise CDC devices without knowing the specific vendor and product ID in advance. https://play.google.com/store/apps/details?id=de.kai_morich.serial_usb_terminal You could try with this app which is based off the same serial library (but a newer version).

[zenmetsu]

This program works with my machine with the default firmware:

[zenmetsu]

i could not see where that library was being pulled in, but i don't know java either. is this a matter of updating your repository to use a newer version of the usb serial library?

[nimble0]

Yes, I'll update the branch when I get time. There's another fix that I need to push as well.

[nimble0]

I've updated the feature/serial-support branch with a newer version of the USB serial library and a few fixes.

[zenmetsu]

Thanks for the update. I am trying to fix my IDE since it is complaining about incompatible gradle versions. I'll check back in once i get those sorted and am able to test the branch.

[zenmetsu]

With the newest build, the app crashes or freezes when the keyboard is attached and the machine menu is pulled up. The app just becomes unresponsive, and I do not get any indication that a crash log has been created.

[nimble0]

You might be able to get a log if you use the share log button on the Dotterel settings page (top-right). If you can run the app from Android Studio you'll be able to see the log in real-time.

[zenmetsu]

Attaching logs. It looks like USB permissions issue.
dotterel.log

[nimble0]

Ah, I see, at some point they changed the UsbDevice API to require permission to access a device's serial number. I updated the branch to not use the serial number.

[zenmetsu]

OK, so the crashing problem goes away... but now I am left with another problem.

After configuring the serial settings, when i attempt to switch to the Dotterel input, I receive an "Invalid Type found while reading" error
dotterel-log.txt

[nimble0]

Hmm, strange, that error appears to happen while it's asking for permission to access the USB device. Do you get the prompt to grant permission to access the USB device (just before the invalid type error)?

[zenmetsu]

No. No prompt.

[nimble0]

I can't tell where the error is coming from. I suggest removing these lines https://github.com/nimble0/dotterel/blob/feature/serial-support/app/src/main/java/nimble/dotterel/machines/SerialStenoMachine.kt#L242-L249 so that you get a crash with the raw exception.

[Monniasza]

@nimble0 Do you still work on it?

[nimble0]

I'm not actively working on it, but I'll check any bug reports.