Feature Request: Integration w/ Conversation Framework Pipecat

[Rodaviva29]

Integrate F5-TTS to a well-known framework for voice and multimodal conversational AI, pipecat: https://github.com/pipecat-ai/pipecat.

Currently it supports a lot of Text-to-Speech services (https://docs.pipecat.ai/server/services/supported-services), but F5-TTS has gain a lot of popularity and would be amazing to have it integrated with another framework.

Integration in services, maybe similar to XTTS as a streaming alike? https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/xtts.py.

I would to get this into reality but I'm not very into this type of programming, so it would be amazing if anyone could contribute ❤️

[SWivid]

https://github.com/SWivid/F5-TTS/tree/main/src/f5_tts/infer#socket-realtime-client from #803

There may be some first packet delay as it's a NAR TTS, but there will be no delay if the LLM part prefill in advance some text content when the user is about to finish speaking (in that case, may even have larger chunk size for better consistency).

[Rodaviva29]

Hey! Gotcha and thanks for the reply, it means a lot!

They told me that I need to write a custom service in Pipecat that connects to the web socket API (or client) from F5-TTS. Probably check out the available TTS services in Pipecat to find a similar example and get started (https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/xtts.py).

Would be very difficult to do? I never worked with web sockets so not sure If for now by myself I can integrate a service like this. 😅