Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jumps in recorded file. #54

Closed
robertio opened this issue May 30, 2022 · 5 comments · Fixed by #72
Closed

Jumps in recorded file. #54

robertio opened this issue May 30, 2022 · 5 comments · Fixed by #72
Assignees
@chenxiaolong

Comments

@robertio
Copy link

robertio commented May 30, 2022 •
edited
Loading

Hi,
Thanks for this project efforts!

Recorded files with (all version including Version 1.6) sounds have random small jumps (sounds like hiccups)
It seems there is no problem with the BCR-1.4.r3.gaa21633-debug-dump_raw_wave_16kHz.zip you provided in other issue#39.

I have Redmi K30 Pro (Poco F2 Pro) - with LOS 19.1 (Android 12)
Sample file having jumps - errors- in it:
sample error from recorded sound
Original recorded file is 1 minutes 10 sec - this sample is from ~27seconds where the error started, and as you can hear after a 5 seconds sounds good again as it was earlier and after.

This is the Codec debug info:
`OMX.qcom.video.encoder.heic:

  • Canonical name: OMX.qcom.video.encoder.heic
  • Is alias: false
  • Is hardware accelerated: true
  • Is software only: false
  • Is vendor: true
  • Supported types:
    • image/vnd.android.heic

OMX.qcom.video.encoder.avc:

  • Canonical name: OMX.qcom.video.encoder.avc
  • Is alias: false
  • Is hardware accelerated: true
  • Is software only: false
  • Is vendor: true
  • Supported types:
    • video/avc

OMX.qcom.video.encoder.h263sw:

  • Canonical name: OMX.qcom.video.encoder.h263sw
  • Is alias: false
  • Is hardware accelerated: true
  • Is software only: false
  • Is vendor: true
  • Supported types:
    • video/3gpp

OMX.qcom.video.encoder.hevc:

  • Canonical name: OMX.qcom.video.encoder.hevc
  • Is alias: false
  • Is hardware accelerated: true
  • Is software only: false
  • Is vendor: true
  • Supported types:
    • video/hevc

OMX.qcom.video.encoder.hevc.cq:

  • Canonical name: OMX.qcom.video.encoder.hevc.cq
  • Is alias: false
  • Is hardware accelerated: true
  • Is software only: false
  • Is vendor: true
  • Supported types:
    • video/hevc

OMX.qcom.video.encoder.mpeg4sw:

  • Canonical name: OMX.qcom.video.encoder.mpeg4sw
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: false
  • Is vendor: true
  • Supported types:
    • video/mp4v-es

OMX.qcom.video.encoder.vp8:

  • Canonical name: OMX.qcom.video.encoder.vp8
  • Is alias: false
  • Is hardware accelerated: true
  • Is software only: false
  • Is vendor: true
  • Supported types:
    • video/x-vnd.on2.vp8

c2.android.aac.encoder:

  • Canonical name: c2.android.aac.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/mp4a-latm
      • Bitrate range: [8000, 510000]
      • Sample rate ranges: [[8000, 8000], [11025, 11025], [12000, 12000], [16000, 16000], [22050, 22050], [24000, 24000], [32000, 32000], [44100, 44100], [48000, 48000]]
      • Sample rates: [8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000]

OMX.google.aac.encoder:

  • Canonical name: c2.android.aac.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/mp4a-latm
      • Bitrate range: [8000, 510000]
      • Sample rate ranges: [[8000, 8000], [11025, 11025], [12000, 12000], [16000, 16000], [22050, 22050], [24000, 24000], [32000, 32000], [44100, 44100], [48000, 48000]]
      • Sample rates: [8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000]

c2.android.amrnb.encoder:

  • Canonical name: c2.android.amrnb.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/3gpp
      • Bitrate range: [4750, 12200]
      • Sample rate ranges: [[8000, 8000]]
      • Sample rates: [8000]

OMX.google.amrnb.encoder:

  • Canonical name: c2.android.amrnb.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/3gpp
      • Bitrate range: [4750, 12200]
      • Sample rate ranges: [[8000, 8000]]
      • Sample rates: [8000]

c2.android.amrwb.encoder:

  • Canonical name: c2.android.amrwb.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/amr-wb
      • Bitrate range: [6600, 23850]
      • Sample rate ranges: [[16000, 16000]]
      • Sample rates: [16000]

OMX.google.amrwb.encoder:

  • Canonical name: c2.android.amrwb.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/amr-wb
      • Bitrate range: [6600, 23850]
      • Sample rate ranges: [[16000, 16000]]
      • Sample rates: [16000]

c2.android.flac.encoder:

  • Canonical name: c2.android.flac.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/flac
      • Bitrate range: [1, 21000000]
      • Sample rate ranges: [[7350, 192000]]
      • Sample rates: null

OMX.google.flac.encoder:

  • Canonical name: c2.android.flac.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/flac
      • Bitrate range: [1, 21000000]
      • Sample rate ranges: [[7350, 192000]]
      • Sample rates: null

c2.android.opus.encoder:

  • Canonical name: c2.android.opus.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • audio/opus
      • Bitrate range: [6000, 510000]
      • Sample rate ranges: [[8000, 8000], [12000, 12000], [16000, 16000], [24000, 24000], [48000, 48000]]
      • Sample rates: [8000, 12000, 16000, 24000, 48000]

c2.android.avc.encoder:

  • Canonical name: c2.android.avc.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/avc

OMX.google.h264.encoder:

  • Canonical name: c2.android.avc.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/avc

c2.android.h263.encoder:

  • Canonical name: c2.android.h263.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/3gpp

OMX.google.h263.encoder:

  • Canonical name: c2.android.h263.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/3gpp

c2.android.hevc.encoder:

  • Canonical name: c2.android.hevc.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/hevc

c2.android.mpeg4.encoder:

  • Canonical name: c2.android.mpeg4.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/mp4v-es

OMX.google.mpeg4.encoder:

  • Canonical name: c2.android.mpeg4.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/mp4v-es

c2.android.vp8.encoder:

  • Canonical name: c2.android.vp8.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/x-vnd.on2.vp8

OMX.google.vp8.encoder:

  • Canonical name: c2.android.vp8.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/x-vnd.on2.vp8

c2.android.vp9.encoder:

  • Canonical name: c2.android.vp9.encoder
  • Is alias: false
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/x-vnd.on2.vp9

OMX.google.vp9.encoder:

  • Canonical name: c2.android.vp9.encoder
  • Is alias: true
  • Is hardware accelerated: false
  • Is software only: true
  • Is vendor: false
  • Supported types:
    • video/x-vnd.on2.vp9
      `

How can i provide debug info to you?
@chenxiaolong chenxiaolong self-assigned this May 31, 2022
@chenxiaolong
Copy link
Owner

It seems there is no problem with the BCR-1.4.r3.gaa21633-debug-dump_raw_wave_16kHz.zip you provided in other issue#39.

Hmm, if that's the case, I suspect this is caused by the same issue as in #39. I'm still working on troubleshooting that issue.

By the way, with stable version 1.7, it's possible to select WAV/PCM output + 16kHz so you don't have to use the debug build.

chenxiaolong added a commit that referenced this issue Jun 1, 2022
@chenxiaolong
RecorderThread: Send input timestamps to MediaCodec encoders
fb920f0 
This was previously disabled due to incorrect troubleshooting of an
issue where the `c2.android.flac.encoder` component would crash during
recording. It turns out the crash was caused by incorrect math:

    inputTimestamp += frames * 1000000 / audioRecord.sampleRate

`frames` and `1000000` are both 32-bit integers, so the multiplication
result overflowed and caused `inputTimestamp` (a 64-bit integer) to
accumulate negative values, which would crash the encoder. This bug was
inadvertently fixed during refactorying in commit
3d3eb82.

This commit reenables the submission of timestamps to MediaCodec. To
avoid accumulated error due to integer division when the microseconds
per sample count is not perfectly divisible, the recording loop will
keep track of the total frame count and only calculate the timestamp
during buffer submission.

This should fix issues where the encoded output files had overlapping
audio or other weird audible artifacts. These were caused by
MediaCodec's multithreaded encoding, where the lack of timestamps would
cause encoded samples to be produced out of order.

Fixes: #39 #54
Signed-off-by: Andrew Gunnerson <chillermillerlong@hotmail.com>
chenxiaolong added a commit that referenced this issue Jun 1, 2022
@chenxiaolong
RecorderThread: Send input timestamps to MediaCodec encoders
39767d3 
This was previously disabled due to incorrect troubleshooting of an
issue where the `c2.android.flac.encoder` component would crash during
recording. It turns out the crash was caused by incorrect math:

    inputTimestamp += frames * 1000000 / audioRecord.sampleRate

`frames` and `1000000` are both 32-bit integers, so the multiplication
result overflowed and caused `inputTimestamp` (a 64-bit integer) to
accumulate negative values, which would crash the encoder. This bug was
inadvertently fixed during refactoring in commit
3d3eb82.

This commit reenables the submission of timestamps to MediaCodec. To
avoid accumulated error due to integer division when the microseconds
per sample count is not perfectly divisible, the recording loop will
keep track of the total frame count and only calculate the timestamp
during buffer submission.

This should fix issues where the encoded output files had overlapping
audio or other weird audible artifacts. These were caused by
MediaCodec's multithreaded encoding, where the lack of timestamps would
cause encoded samples to be produced out of order.

Fixes: #39 #54
Signed-off-by: Andrew Gunnerson <chillermillerlong@hotmail.com>
@chenxiaolong chenxiaolong mentioned this issue Jun 1, 2022
Merged
@chenxiaolong
Copy link
Owner

chenxiaolong commented Jun 1, 2022 •
edited
Loading

Can you give the test build in #59 a try? I think I may have found the issue 🙂

EDIT: Version 1.8 has been released with the fix included. Please give that one a try instead.

chenxiaolong added a commit that referenced this issue Jun 1, 2022
@chenxiaolong
RecorderThread: Send input timestamps to MediaCodec encoders (#59)
1c95846 
This was previously disabled due to incorrect troubleshooting of an
issue where the `c2.android.flac.encoder` component would crash during
recording. It turns out the crash was caused by incorrect math:

    inputTimestamp += frames * 1000000 / audioRecord.sampleRate

`frames` and `1000000` are both 32-bit integers, so the multiplication
result overflowed and caused `inputTimestamp` (a 64-bit integer) to
accumulate negative values, which would crash the encoder. This bug was
inadvertently fixed during refactoring in commit
3d3eb82.

This commit reenables the submission of timestamps to MediaCodec. To
avoid accumulated error due to integer division when the microseconds
per sample count is not perfectly divisible, the recording loop will
keep track of the total frame count and only calculate the timestamp
during buffer submission.

This should fix issues where the encoded output files had overlapping
audio or other weird audible artifacts. These were caused by
MediaCodec's multithreaded encoding, where the lack of timestamps would
cause encoded samples to be produced out of order.

Fixes: #39 #54
Signed-off-by: Andrew Gunnerson <chillermillerlong@hotmail.com>
@robertio
Copy link
Author

robertio commented Jun 1, 2022 via email

@robertio
Copy link
Author

robertio commented Jun 1, 2022 via email •
edited
Loading

@robertio
Copy link
Author

robertio commented Jun 1, 2022 •
edited
Loading

BCR V1.8 tests:
Unfortunately ALL compressed format -OGG,M4A,FLAC- has error on my Xiaomi K30 Pro.
I tried different sample rates. (bitrate was not changed in any codec)

WAV records seems OK. Sample rate, tried 8000Hz.

I plan to try ffmpeg -post process- conversion from wav to mp3 in Termux. https://www.reddit.com/r/tasker/comments/iyttlf/convert_recorded_audio_to_wave_nonroot/
ffmpeg -i /media/storage/emulated/0/Recordings/Tasker.mp4 -f wav media/storage/emulated/0/Recordings/Tasker.wav

@chenxiaolong chenxiaolong mentioned this issue Jun 2, 2022
Merged
chenxiaolong added a commit that referenced this issue Jun 4, 2022
@chenxiaolong
RecorderThread.kt: Set priority to THREAD_PRIORITY_URGENT_AUDIO
e82e46d 
Issue: #39 #54
Signed-off-by: Andrew Gunnerson <chillermillerlong@hotmail.com>
This was referenced Jun 4, 2022
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chenxiaolong chenxiaolong

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

RecorderThread: Record to internal storage first chenxiaolong/BCR
2 participants
@chenxiaolong @robertio