Issues with RealTimeSTT?

[holdengand]

Are you still providing any kind of support for this project?

I am trying to put this together and get it running in VS Code. I spent a day getting DeepSpeed to install before realizing I was using the wrong python version for my environment for the deepspeed 0.15.0 version.

But now when trying to run the main.py I am getting an issue with CoquiEngine hanging up because of issues with RealTimeSTT.

I'm not really a programmer or developer so I'm struggling to piece together what needs to be done or what I have overlooked. But I am always struggling getting the dependencies for RealTimeSTT in order. It always needs a torch version higher or lower then what everything else needs.

I'm not sure its worth even listing what all I have tired as I'm not really even sure I am taking the right approach. I will say I started with installing the requierments and pipfreeze; but got hung up on deepspeed as the version this project used was being a real hassle to get installed on windows; but the 0.15.0 version has sort of resolved the extra things that used to be needed, like doing a WSL install. Swapping the virtual environment to python3.11 made it very easy to get DeepSpeed 0.15.0 installed. But now I can't get past this STT/Torch compatibility thing.

I think I have hit my current limit on what I can do to diagnose this without a smarter mind helping me out. This project seems like the best of its type to me at the moment with the base features I'd like without a bunch of excessive stuff just bloating it up. So its a shame its not getting a bit more work currently. I hope someone can help me figure this out, or someone more capable then me can put together an update.

Here is my current pip freeze
pipfreeze.txt

[kimjammer]

Hey there! Thanks for your interest.
The most likely problem is that you're not using the right deepspeed wheel.

DeepSpeed (For TTS) will need to be installed separately. I was using instructions from AllTalkTTS, and using their provided wheels.

Click on the "provided wheels" link, and it'll bring you to a place to download the DeepSpeed 0.14 version I was using. If you have further problems, lmk! I don't have as much free time as I did in the summer, but I'll try my best to help.

[kimjammer]

I haven't done a clean install in a bit, I should probably have transformers in the requirements, haha.
You might want to try using Python 3.11.9, which is the version I am using. Minor versions shouldn't have breaking changes, but you never know with building wheels and stuff - just might be worth trying. You also might want to see if you can install realtimeTTS by itself in a different venv.
The opentelemetry was because one of my dependencies needed something like version 1.0-1.9(made up numbers for example) and another needed 1.0-2.x and by default pip installed the newest version and broke the library that needed the older version. So it's just explicitly saying to use the older version that's compatible with both libraries.

[holdengand]

Python 3.11.9
Fresh venv
pip install realtimetts > Installs just fine, no issues.
pip install requirements.txt > still errors out on the numba thing.

pip install realtimetts
(removing RealTimeTTS[coqui]==0.4.1 from the requirements file)
pip install requirements.txt > completes fine
pip install transformers
pip install deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
python main.py

Starting Project...
Traceback (most recent call last):
  File "D:\_AI_\main.py", line 115, in <module>
    asyncio.run(main())
  File "Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Python\Python311\Lib\asyncio\base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "D:\_AI_\main.py", line 50, in main
    tts = TTS(signals)
          ^^^^^^^^^^^^
  File "D:\_AI_\tts.py", line 13, in __init__
    engine = CoquiEngine(
             ^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

And then I realized I forgot to install torch, torchvision and torchaudio; as directed on the readme. Torch 2.5.1 and torchaudio 2.5.1 had been installed by something in the process. Used your pip command in the readme, tried the main.py again and... same error. TypeError: 'NoneType' object is not callable.

I THINK I am past the instillation of dependencies, but I'm not sure what to do about TypeError: 'NoneType' object is not
callable in relation to the CoquiEngine. It's the same error I had before. I attached a file of my current pip list for good measure.

But if its working for you then I am sure there must be something on my end I just am not understanding and simply messing up. This level of hands-on installation is about the most involved I have done in a while; and when it comes to my luck nothing like this 'just works' for me for some reason. I appreciate your efforts and I'm sorry all I have to offer is errors. Don't feel you need to rush to solve my minor problems!

pipcurrent.txt

[kimjammer]

Ah that's because if you install "realtimetts" by itself it doesn't install any of the engines (ie coqui engine is not installed) so it's NoneType when you try to use it. You have to specify realtimetts[coqui] to install the engine. Make sure you can install "realtimetts[coqui]==0.4.1" in an empty venv.

Don't worry about bothering me, I had to fight through my fair share of python library installation errors so I know how you feel.

[kimjammer]

If you can install realtimetts with coqui in another venv, try pulling my new commit and installing. I just went through a clean install and fixed all the dependency problems I ran into, so it should work. Most significantly, I now specify the numba and transformers version that's compatible with everything else.

[holdengand]

Oh right, that makes sense!

I took a nap and you seem to have fixed all my issues with the dependencies as far as I can tell. However I still can't get the main.py to run.

I did not understand the errors I was getting, so I will provide the explanation Codeium is giving me for them.

The error message indicates that there is an issue with importing the log variable from the torch.distributed.elastic.agent.server.api module.

Looking at the provided code snippet from .aienv/Lib/site-packages/torch/distributed/elastic/agent/server/api.py, I see that log is not defined as a variable that can be imported. Instead, logger is defined as a variable that can be used for logging.

It seems that the deepspeed library is trying to import log from torch.distributed.elastic.agent.server.api, but it doesn't exist.

To fix this issue, you may need to update the deepspeed library to use the correct logging variable, which is logger. Alternatively, you can try to modify the torch.distributed.elastic.agent.server.api module to define log as an alias for logger.

However, without more information about your codebase and the specific requirements of your project, it's difficult to provide a more detailed solution.

I was able to resolve this issue;
I was first worried I was just not properly connecting to an LLM Endpoint and went ahead and resolved my issues with text-webui, and switching between it and kobold made no difference. But then suspected it was DeepSpeed again; I had assumed I was meant to be using that .14 wheel version that was produced by AllTalk. Double checked the freeze and saw it was the newer .15 version listed above. Pip installed that and that error was fixed. nope...

However I still have a series of issues related to Deepspeed and it thinking I am trying to use the CPU rather then GPU I think. I am not exactly sure where in this setup it is thinking that. But there are also a few other errors. I had Codium produce a simplified list of the errors I was getting:

Error 1:
Error message: "This op had not been implemented on CPU backend."
File: deepspeed\ops\op_builder\cpu\no_impl.py
Line: 21
Related files:
deepspeed\model_implementations\transformers\ds_gpt.py (line 20)
deepspeed\module_inject\containers\gpt2.py (line 20)
deepspeed\module_inject\replace_module.py (line 385, 634, 694)
RealtimeTTS\engines\coqui_engine.py (line 501, 477)

Error 2:
Error message: "You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly."
File: TTS\tts\layers\xtts\xtts_manager.py
Line: 6
Related files:
TTS\utils\io.py (line 54)

Error 3:
Error message: "Config parameter replace_method is deprecated."
File: deepspeed\config_utils.py
Line: 70
Related files:
deepspeed\inference\engine.py (line 421)

Error 4:
Error message: "Config parameter mp_size is deprecated."
File: deepspeed\config_utils.py
Line: 70
Related files:
deepspeed\inference\engine.py (line 421)

Error 5:
Error message: "Setting accelerator to CPU. If you have GPU or other accelerator, we were unable to detect it."
File: real_accelerator.py
Line: 162

Anyway, I am quite pleased the dependencies all seem to work now! The requirements.txt is super straight forward now. At some point you might want to update the install instructions to direct to the newer deepspeed wheel though. I'm an idiot so ignore that.

I will look into this more on my own later this evening. If you happen to know what I am doing wrong though that would be nice. But at this point I am single-handedly going to be the source of all the troubleshooting getting this project started XD. Hopefully all these issues listed here will help anyone else at least for people who aren't developers or programmers, less familiar with these hand on installations like myself.

[holdengand]

Also, I just double checked to make sure my GPU was working within this venv;
nvcc --version
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

python -c "import torch; print(torch.cuda.is_available())"
False

Seems like its a CUDA issue? Related to torch?

[kimjammer]

Yup, I would double check that the torch version didn't get overriden. This might be a dumb question, but you have an NVIDIA gpu, right?

[kimjammer]

Not sure with what you mean with deepspeed, I'm still using 0.14.