Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Android system TTS engine #508

Merged
merged 7 commits into from
Dec 31, 2023
Merged

Conversation

csukuangfj
Copy link
Collaborator

You can find two demo videos below:

The following two repositories are used as a reference during implementation:

Note that you can use all supported models with this PR, e.g., models from piper and coqui-ai/TTS, etc.

Fixes #488

See also
rhasspy/piper#257 (comment)

@Pantyhose-X
@christiankaye
@ekorn
@LuekWasHere
@beqabeqa473
@paolo-caroni

@LuekWasHere
Copy link

I can't seem to figure how to build this pull request.
It seems (from my unqualified perspective), that there is an error to do with a file that is being declared though doesnt exist.

It gives this error specifically:
Expecting a top level declaration

image

Is this an issue with me or perhaps an issue with the Package directive?

@csukuangfj
Copy link
Collaborator Author

Looks like you are using windows.

That file is a symlink. Please delete that file and use the real file to replace it.

Windows does not support symlinks.

@csukuangfj
Copy link
Collaborator Author

By the way, we don't hard-code a model in the code since we try to support as many models as possible.

You need to change the code to select a model. The following is an example:

diff --git a/android/SherpaOnnxTtsEngine/app/src/main/java/com/k2fsa/sherpa/onnx/tts/engine/TtsEngine.kt b/android/SherpaOnnxTtsEngine/app/src/main/java/com/k2fsa/sherpa/onnx/tts/engine/TtsEngine.kt
index 4fc935b..52626e1 100644
--- a/android/SherpaOnnxTtsEngine/app/src/main/java/com/k2fsa/sherpa/onnx/tts/engine/TtsEngine.kt
+++ b/android/SherpaOnnxTtsEngine/app/src/main/java/com/k2fsa/sherpa/onnx/tts/engine/TtsEngine.kt
@@ -60,12 +60,12 @@ object TtsEngine {
         // The purpose of such a design is to make the CI test easier
         // Please see
         // https://github.com/k2-fsa/sherpa-onnx/blob/master/scripts/apk/generate-tts-apk-script.py
-        modelDir = null
-        modelName = null
+        modelDir = "vits-piper-en_GB-southern_english_female-low"
+        modelName = "en_GB-southern_english_female-low.onnx"
         ruleFsts = null
         lexicon = null
-        dataDir = null
-        lang = null
+        dataDir = "vits-piper-en_GB-southern_english_female-low/espeak-ng-data"
+        lang = "eng"

We give several detailed examples in the code. Please do read them before you build an APK.

@LuekWasHere
Copy link

LuekWasHere commented Jan 1, 2024

Oh I see I did see those, though it seems I assumed that it would automatically swap out the details when built. I'll change that now, thank you.
Do i just swap in the details where it says 'null'?

@LuekWasHere
Copy link

LuekWasHere commented Jan 1, 2024

It seems so so close, to functioning. I am getting this error which if i'm interpretting correctly means that it doesnt function with my two phones chipset?

logcat:

---------------------------- PROCESS STARTED (29519) for package com.k2fsa.sherpa.onnx.tts.engine ----------------------------
2024-01-01 14:03:27.346 29519-29519 Compatibil...geReporter com.k2fsa.sherpa.onnx.tts.engine     D  Compat change id reported: 171979766; UID 10246; state: ENABLED
2024-01-01 14:03:27.351 29519-29519 ziparchive              com.k2fsa.sherpa.onnx.tts.engine     W  Unable to open '/data/app/~~7NVuguQ1TPkSsdQHxd54QQ==/com.k2fsa.sherpa.onnx.tts.engine-gl5i0272h3A4d7WKloPwuw==/base.dm': No such file or directory
2024-01-01 14:03:27.351 29519-29519 ziparchive              com.k2fsa.sherpa.onnx.tts.engine     W  Unable to open '/data/app/~~7NVuguQ1TPkSsdQHxd54QQ==/com.k2fsa.sherpa.onnx.tts.engine-gl5i0272h3A4d7WKloPwuw==/base.dm': No such file or directory
2024-01-01 14:03:27.523 29519-29519 nativeloader            com.k2fsa.sherpa.onnx.tts.engine     D  Configuring clns-4 for other apk /data/app/~~7NVuguQ1TPkSsdQHxd54QQ==/com.k2fsa.sherpa.onnx.tts.engine-gl5i0272h3A4d7WKloPwuw==/base.apk. target_sdk_version=34, uses_libraries=, library_path=/data/app/~~7NVuguQ1TPkSsdQHxd54QQ==/com.k2fsa.sherpa.onnx.tts.engine-gl5i0272h3A4d7WKloPwuw==/lib/arm64:/data/app/~~7NVuguQ1TPkSsdQHxd54QQ==/com.k2fsa.sherpa.onnx.tts.engine-gl5i0272h3A4d7WKloPwuw==/base.apk!/lib/arm64-v8a, permitted_path=/data:/mnt/expand:/data/user/0/com.k2fsa.sherpa.onnx.tts.engine
2024-01-01 14:03:27.530 29519-29519 Perf                    com.k2fsa.sherpa.onnx.tts.engine     I  Connecting to perf service.
2024-01-01 14:03:27.533 29519-29519 GraphicsEnvironment     com.k2fsa.sherpa.onnx.tts.engine     V  ANGLE Developer option for 'com.k2fsa.sherpa.onnx.tts.engine' set to: 'default'
2024-01-01 14:03:27.533 29519-29519 GraphicsEnvironment     com.k2fsa.sherpa.onnx.tts.engine     V  ANGLE GameManagerService for com.k2fsa.sherpa.onnx.tts.engine: false
2024-01-01 14:03:27.533 29519-29519 GraphicsEnvironment     com.k2fsa.sherpa.onnx.tts.engine     V  Neither updatable production driver nor prerelease driver is supported.
2024-01-01 14:03:27.534 29519-29519 NetworkSecurityConfig   com.k2fsa.sherpa.onnx.tts.engine     D  No Network Security Config specified, using platform default
2024-01-01 14:03:27.535 29519-29519 NetworkSecurityConfig   com.k2fsa.sherpa.onnx.tts.engine     D  No Network Security Config specified, using platform default
2024-01-01 14:03:27.545 29519-29519 Monotype                com.k2fsa.sherpa.onnx.tts.engine     V  SetAppTypeFace- try to flip, app = com.k2fsa.sherpa.onnx.tts.engine
2024-01-01 14:03:27.547 29519-29519 Monotype                com.k2fsa.sherpa.onnx.tts.engine     V      Typeface getFontPathFlipFont - systemFont = default#default
2024-01-01 14:03:27.757 29519-29519 Monotype                com.k2fsa.sherpa.onnx.tts.engine     V  SetAppTypeFace- try to flip, app = com.k2fsa.sherpa.onnx.tts.engine
2024-01-01 14:03:27.757 29519-29519 Monotype                com.k2fsa.sherpa.onnx.tts.engine     V      Typeface getFontPathFlipFont - systemFont = default#default
2024-01-01 14:03:27.761 29519-29519 sherpa-onnx-tts-engine  com.k2fsa.sherpa.onnx.tts.engine     I  Init Next-gen Kaldi TTS
2024-01-01 14:03:27.761 29519-29519 System.out              com.k2fsa.sherpa.onnx.tts.engine     I  data dir is vits-piper-en_US-glados
2024-01-01 14:03:31.321 29519-29519 System.out              com.k2fsa.sherpa.onnx.tts.engine     I  newDataDir: /storage/emulated/0/Android/data/com.k2fsa.sherpa.onnx.tts.engine/files
2024-01-01 14:03:31.341 29519-29519 sherpa-onnx             com.k2fsa.sherpa.onnx.tts.engine     W  config:
                                                                                                    OfflineTtsConfig(model=OfflineTtsModelConfig(vits=OfflineTtsVitsModelConfig(model="/storage/emulated/0/Android/data/com.k2fsa.sherpa.onnx.tts.engine/files/vits-piper-en_US-glados/en_US-glados.onnx", lexicon="/storage/emulated/0/Android/data/com.k2fsa.sherpa.onnx.tts.engine/files/vits-piper-en_US-glados/", tokens="/storage/emulated/0/Android/data/com.k2fsa.sherpa.onnx.tts.engine/files/vits-piper-en_US-glados/tokens.txt", data_dir="/storage/emulated/0/Android/data/com.k2fsa.sherpa.onnx.tts.engine/files/vits-piper-en_US-glados/espeak-ng-data", noise_scale=0.667, noise_scale_w=0.8, length_scale=1), num_threads=2, debug=True, provider="cpu"), rule_fsts="", max_num_sentences=1)
2024-01-01 14:03:32.633 29519-29519 libc                    com.k2fsa.sherpa.onnx.tts.engine     W  Access denied finding property "ro.hardware.chipname"
2024-01-01 14:03:38.592 29519-29519 sherpa-onnx             com.k2fsa.sherpa.onnx.tts.engine     W  ---vits model---
                                                                                                    sample_rate=22050
                                                                                                    has_espeak=1
                                                                                                    n_speakers=1
                                                                                                    voice=en-us
                                                                                                    model_type=vits
                                                                                                    language=English
                                                                                                    comment=piper
                                                                                                    ----------input names----------
                                                                                                    0 input
                                                                                                    1 input_lengths
                                                                                                    2 scales
                                                                                                    ----------output names----------
                                                                                                    0 output
---------------------------- PROCESS ENDED (29519) for package com.k2fsa.sherpa.onnx.tts.engine ----------------------------

It does this in a loop. Im going to try rebuilt with a different app build.

@csukuangfj
Copy link
Collaborator Author

https://k2-fsa.github.io/sherpa/onnx/tts/apk-engine.html

The above page contains TTS engine APKs for different languages.

@LuekWasHere
Copy link

https://k2-fsa.github.io/sherpa/onnx/tts/apk-engine.html

The above page contains TTS engine APKs for different languages.

You have done incredible mate. This is simply exactly what I have been wanting. I have lots of difficulty reading and the other tts options seem to be challenging to read along side. This though is awesome. I cannot express how greatful I am. I hope my persistent communication hasnt been difficult. You're a legend.

@paolo-caroni
Copy link
Contributor

https://k2-fsa.github.io/sherpa/onnx/tts/apk-engine.html

The above page contains TTS engine APKs for different languages.

I LOVE it, would you please to consider to mantain a repo for F-Droid?
You work at Xiaomi, you are being payed to do it?

@csukuangfj
Copy link
Collaborator Author

https://k2-fsa.github.io/sherpa/onnx/tts/apk-engine.html

The above page contains TTS engine APKs for different languages.

I LOVE it, would you please to consider to mantain a repo for F-Droid?
You work at Xiaomi, you are being payed to do it?

Please see #520

Users from the community are supporting it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Android system: Google Text-to-speech Engine App Alternative
3 participants