My Demo POC is completed everything is working as expected but not able to run same on server in docker container #961

nik0811 · 2024-10-15T20:17:05Z

nik0811
Oct 15, 2024

I am using speech to text and text to speech service of deepgram using python sdk, everything work as expected when running on localhost but the time when i am deploying on server inside docker, speech to text stoped working, i thought might be the issue of docker so i tried running on server itself there also i am facing same issue, Don't know what's the issue, ran in debug mode too deepgram api reeives bytes but unable to transcribe, I need to take my poc to the client but i am stuck. Also I raised the concern on dicord but haven't got any response from them, what to do i don't know now, the only solution i am seeing is to change the service provider but that also will force me to do the rework on stt and tts side, which i am avoiding, curently i am using free trial but we will be purchasing plan soon if everything will work as expected.

jkroll-deepgram · 2024-10-15T21:02:17Z

jkroll-deepgram
Oct 15, 2024
Collaborator

Hi @nik0811, what Deepgram errors are you seeing? Do you have any differences in networking, API keys, request volume, etc between Docker and your localhost? Can you provide Deepgram request IDs from any failed requests?

10 replies

nik0811 Oct 15, 2024
Author

@jkroll-deepgram here is the issue where i haven't got any response back from deepgram while running on server, where as on localhost i am seeing the transcription.

Sending KeepAlive...
2024-10-15 22:19:32,158 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:32,158 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:32,158 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:32,158 - NOTICE - keep_alive succeeded
2024-10-15 22:19:34,193 - INFO - Processed 10 seconds of audio so far
Sending KeepAlive...
2024-10-15 22:19:35,032 - NOTICE - Sending KeepAlive...
keep_alive succeeded
2024-10-15 22:19:35,032 - NOTICE - keep_alive succeeded
2024-10-15 22:19:35,800 - INFO - Voice command detected
2024-10-15 22:19:35,801 - INFO - Sent data of type vad to WebSocket
Sending KeepAlive...
2024-10-15 22:19:37,190 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:37,190 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:37,190 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:37,190 - NOTICE - keep_alive succeeded
2024-10-15 22:19:37,517 - INFO - Processed 20 seconds of audio so far
2024-10-15 22:19:39,139 - INFO - Voice command detected
2024-10-15 22:19:39,139 - INFO - Sent data of type vad to WebSocket
Sending KeepAlive...
2024-10-15 22:19:40,032 - NOTICE - Sending KeepAlive...
keep_alive succeeded
2024-10-15 22:19:40,032 - NOTICE - keep_alive succeeded
2024-10-15 22:19:40,876 - INFO - Processed 30 seconds of audio so far
Sending KeepAlive...
2024-10-15 22:19:42,195 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:42,195 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:42,195 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:42,195 - NOTICE - keep_alive succeeded
2024-10-15 22:19:44,216 - INFO - Processed 40 seconds of audio so far
Sending KeepAlive...
2024-10-15 22:19:45,034 - NOTICE - Sending KeepAlive...
keep_alive succeeded
2024-10-15 22:19:45,035 - NOTICE - keep_alive succeeded
Sending KeepAlive...
2024-10-15 22:19:47,223 - NOTICE - Sending KeepAlive...
send() exiting gracefully: 1000
2024-10-15 22:19:47,224 - NOTICE - send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
2024-10-15 22:19:47,224 - DEBUG - AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2024-10-15 22:19:47,224 - NOTICE - keep_alive succeeded
2024-10-15 22:19:47,555 - INFO - Processed 50 seconds of audio so far
Text data received
2024-10-15 22:19:48,239 - DEBUG - Text data received
AsyncListenWebSocketClient._process_text ENTER
2024-10-15 22:19:48,239 - DEBUG - AsyncListenWebSocketClient._process_text ENTER
Text data received
2024-10-15 22:19:48,239 - DEBUG - Text data received
response_type: Metadata, data: {'type': 'Metadata', 'transaction_key': 'deprecated', 'request_id': 'c4bf9f35-0464-4912-8ed6-010866b2cc99', 'sha256': '22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b', 'created': '2024-10-15T22:19:30.018Z', 'duration': 0.0, 'channels': 0}
2024-10-15 22:19:48,239 - DEBUG - response_type: Metadata, data: {'type': 'Metadata', 'transaction_key': 'deprecated', 'request_id': 'c4bf9f35-0464-4912-8ed6-010866b2cc99', 'sha256': '22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b', 'created': '2024-10-15T22:19:30.018Z', 'duration': 0.0, 'channels': 0}
MetadataResponse: {
    "type": "Metadata",
    "transaction_key": "deprecated",
    "request_id": "c4bf9f35-0464-4912-8ed6-010866b2cc99",
    "sha256": "22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b",
    "created": "2024-10-15T22:19:30.018Z",
    "duration": 0.0,
    "channels": 0
}
2024-10-15 22:19:48,239 - VERBOSE - MetadataResponse: {
    "type": "Metadata",
    "transaction_key": "deprecated",
    "request_id": "c4bf9f35-0464-4912-8ed6-010866b2cc99",
    "sha256": "22b21813156e21f63b7beecefd155e40eb5e4cd6a00820102348e53cea14bc2b",
    "created": "2024-10-15T22:19:30.018Z",
    "duration": 0.0,
    "channels": 0
}
AsyncListenWebSocketClient._emit ENTER

nik0811 Oct 15, 2024
Author

from my frontend i am sending the data via sendAudioMessage(event.data)

const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
        const mimeType = MediaRecorder.isTypeSupported('audio/webm') ? 'audio/webm' : 'audio/mp4';

        closeChatConnection();
        const token = localStorage.getItem('access_token') || '';
        initializeAudioWebSocket(token);
        initializeVadWebSocket(token);

        const audioContext = new AudioContext();
        const source = audioContext.createMediaStreamSource(stream);
        const processor = audioContext.createScriptProcessor(4096, 1, 1);

        source.connect(processor);
        processor.connect(audioContext.destination);

        processor.onaudioprocess = (e) => {
          const inputData = e.inputBuffer.getChannelData(0);
          const pcmData = new Int16Array(inputData.length);
          for (let i = 0; i < inputData.length; i++) {
            pcmData[i] = Math.max(-1, Math.min(1, inputData[i])) * 0x7FFF;
          }
          const blob = new Blob([pcmData.buffer], { type: 'application/octet-stream' });
          // Only send VAD message if the blob size is greater than zero
          if (blob.size > 0) {
            sendVadMessage(blob);
          }
        };

        mediaRecorderRef.current = new MediaRecorder(stream, { mimeType });
        mediaRecorderRef.current.ondataavailable = (event) => {
          if (event.data.size > 0) {
            sendAudioMessage(event.data);
          }
        };

@jkroll-deepgram

jkroll-deepgram Oct 15, 2024
Collaborator

@nik0811, for the request ID in your log (c4bf9f35-0464-4912-8ed6-010866b2cc99), I see on our backend that you sent Deepgram invalid/corrupt audio data. I recommend debugging your audio source and transmission in the containerized environment.

nik0811 Oct 15, 2024
Author

@nik0811, for the request ID in your log (c4bf9f35-0464-4912-8ed6-010866b2cc99), I see on our backend that you sent Deepgram invalid/corrupt audio data. I recommend debugging your audio source and transmission in the containerized environment.

Thanks, at least i got the clue, i will have a look, today and will update you with the result accordingly.

nik0811 Oct 16, 2024
Author

It started working but now facing the issue where i am not able to receive complete audio, it is getting closed before the entire response arrives. I am using the below config,

options = SpeakWSOptions(
            model="aura-luna-en",
            encoding="linear16",
            sample_rate=16000,
        )

and also in speech to text the utterance end is not working properly here i am using,

        encoding="linear16",
        channels=1,
        sample_rate=16000,
        model="nova-2",
        language="en-US",
        smart_format=True,
        punctuate=True,
        interim_results=True,
        utterance_end_ms=2000,
        vad_events=True,
        endpointing=300

@jkroll-deepgram

fatemehkhoramdel · 2024-11-26T06:40:56Z

fatemehkhoramdel
Nov 26, 2024

I am using deepgram to transcript voice and with some device i got error like sumsung a13 or Xiaomi. even i tried to convert voice and then send to deepgram but it doesn't work

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deepgram

My Demo POC is completed everything is working as expected but not able to run same on server in docker container #961

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 10 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Deepgram

My Demo POC is completed everything is working as expected but not able to run same on server in docker container #961

nik0811 Oct 15, 2024

Replies: 2 comments · 10 replies

jkroll-deepgram Oct 15, 2024 Collaborator

nik0811 Oct 15, 2024 Author

nik0811 Oct 15, 2024 Author

jkroll-deepgram Oct 15, 2024 Collaborator

nik0811 Oct 15, 2024 Author

nik0811 Oct 16, 2024 Author

fatemehkhoramdel Nov 26, 2024

nik0811
Oct 15, 2024

Replies: 2 comments 10 replies

jkroll-deepgram
Oct 15, 2024
Collaborator

nik0811 Oct 15, 2024
Author

nik0811 Oct 15, 2024
Author

jkroll-deepgram Oct 15, 2024
Collaborator

nik0811 Oct 15, 2024
Author

nik0811 Oct 16, 2024
Author

fatemehkhoramdel
Nov 26, 2024