LiveEffect: Problem while using AAudio on Huawei Mate 20 Pro

[miqouser]

I had made some changes in recording/playback mechanism. I am doing some audio processing between record and playback. I am using callbacks for both input and playback audio, then the input data is being written into ring buffer. Then it is being processed by a dedicated thread and passed to another ring buffer which is being read by playback callback.
Everything is good on Samsung (S7, S9+, S10+), Google Pixel 2, One Plus 5T. The problem is on Huawei Mate 20 Pro device, Have you ever tested oboe on that device ? What can be the problem ? With AAudio the sound is broken, with OpenSL ES there is huge latency.

Here is the code of LiveEffectEngine.cpp:

`//other code I didn't touch

void LiveEffectEngine::adjustVolume(void *audioSamples, int sampleCount) {
short *shortSamples = (short *) audioSamples;
float data;
for (int i = 0; i < sampleCount; i++) {
data = static_cast(shortSamples[i]) * mVolumeLevel;
shortSamples[i] = std::min(32767.,std::max(-32768., data));
}
}

oboe::DataCallbackResult
LiveEffectEngine::onAudioReady(oboe::AudioStream *oboeStream, void *audioData, int32_t numFrames){
static bool skip_read = false;

if (oboeStream == mPlayStream) {
    int32_t framesRead = 0;
    framesRead = 0;

    if (m_out_ringbuf.numSamples() > 2000) {
        m_out_ringbuf.purgeData(1000);
    }

    if (m_out_ringbuf.numSamples() < numFrames) {
        skip_read = true;
    } else if (m_out_ringbuf.numSamples() > numFrames + 500) {
        skip_read = false;
    }

    if (!skip_read) {
        framesRead = m_out_ringbuf.getData((int16_t *) audioData, numFrames);
    }

    if (framesRead < numFrames) {
        int32_t bytesPerFrame = mRecordingStream->getChannelCount() * oboeStream->getBytesPerSample();
        uint8_t *padPos = static_cast<uint8_t *> (audioData) + framesRead * bytesPerFrame;
        memset(padPos, 0, static_cast<size_t>((numFrames - framesRead) * bytesPerFrame));
    }

    mProcessedFrameCount += numFrames;
} else {
    assert (oboeStream == mRecordingStream);

    if (m_in_ringbuf.numSamples() <= 5000) {
        m_in_ringbuf.putData((int16_t *) audioData, numFrames);
    }
}

return oboe::DataCallbackResult::Continue;

}

void LiveEffectEngine::NC_processing() {
static int16_t temp_buf[960], temp_buf1[960];
std::this_thread::sleep_for(ms(1000));

// Access the native tracing functions.
if (lib != NULL) {
// Use dlsym() to prevent crashes on devices running Android 5.1
// (API level 22) or lower.
    ATrace_beginSection = reinterpret_cast<fp_ATrace_beginSection>(
            dlsym(lib, "ATrace_beginSection"));
    ATrace_endSection = reinterpret_cast<fp_ATrace_endSection>(
            dlsym(lib, "ATrace_endSection"));
}


int prio = getpriority(PRIO_PROCESS, 0);
LOGE("priority: %d" , prio);


cpu_set_t my_set;        /* Define your cpu_set bit mask. */
CPU_ZERO(&my_set);       /* Initialize it all to 0, i.e. no CPUs selected. */
CPU_SET(1, &my_set);     /* set the bit that represents core 2. */
CPU_SET(2, &my_set);     /* set the bit that represents core 2. */
//CPU_SET(3, &my_set);     /* set the bit that represents core 3. */
prio = sched_setaffinity(gettid(), sizeof(cpu_set_t), &my_set); /* Set affinity of tihs process to */
/* the defined mask, i.e. only 7. */

while (!m_stop_nc) {
    //TODO change the hardcoded value
    if ( m_in_ringbuf.numSamples() >= 720) {
        int read_count = m_in_ringbuf.getData(temp_buf, 720);

        assert(read_count == 720);

        if (mProcessingEnabled) {
            THz_NC_CleanAmbientNoiseInt16(mSession, temp_buf, 720, temp_buf1, 720);
        }

        adjustVolume(mProcessingEnabled ? temp_buf1 : temp_buf, 720);

        m_out_ringbuf.putData(mProcessingEnabled ? temp_buf1 : temp_buf, 720);
    } else {
            std::this_thread::yield();
    }
}`

[philburk]

On Fri, May 24, 2019, 9:37 AM miqouser ***@***.***> wrote: The problem is on Huawei Mate 20 Pro device, Have you ever tested oboe on that device ? What can be the problem ?

What problem are you seeing? What did you expect to happen? What actually happened? Did it crash, make noise, crackle, sound too quiet or what? The mate 20 may be using the mmap data path which is a different data path than the normal audio flinger data path. There was a problem with the mono mode. This was recently corrected by Huawei. It would cause corruption of the mono signal. Can you reproduce the problem using the OboeTester app in the github repository?

[miqouser]

Just tested with OboeTester. I hear my voice like I am a monster, that's the problem. On Pixel 2 voice is distorted with one of built in mics, the other one is ok. But as I know we can't choose microphone using OpenSL ES, right ? We can do it only with AAudio which works only on Pie, right ?

[philburk]

Please try OboeTester "RECORD AND PLAY"
1 channel vs 2 channels
and "Exclusive" mode on and off.

Do they all sound the same? You can use the SHARE button to email yourself a WAV file of the recording.

On Pixel 2 voice is distorted with one of built in mics, the other one is ok.

I tried to reproduce that with OboeTester by recording in 2 channel and 3 channel mode. In 2 channel mode I get the same signal on both channels. In 3 channels mode I get 3 distinct signals. They all sound clear on my Pixel 2 XL.

But as I know we can't choose microphone using OpenSL ES, right ?

You can record 3 channels to get 3 mics and then only keep the one you want.

[miqouser]

Good, will test today. What 3 microphones do you mean ? Do you mean I should change this param to 3 ?

int32_t mInputChannelCount = oboe::ChannelCount::Mono;

I see the enum has only 2 types mono and stereo.

enum ChannelCount : int32_t {
/**
* Audio channel count definition, use Mono or Stereo
*/
Unspecified = kUnspecified,

  /**
   * Use this for mono audio
   */
  Mono = 1,

  /**
   * Use this for stereo audio.
   */
  Stereo = 2,
};

[miqouser]

Hi again. Tested 1 vs 2 channels on Pixel 2. There is all right with one channel. With 2 or 3 channels it sounds like monotonic signal. It's strange.
You haven't answered yet, so let me ask my question more detailed.

If wired headphones with microphone are plugged (USB Type-C or 3.5 Jack), is there any way to switch to one of built in microphones instead of using headphones microphone on devices under API level 26 ? With API level 26,27 and 28 it is ok (I guess AAudio does the trick). Google has "Sound Amplifier" accessibility application which is available on Play Store only for devices with API 28 (Android P). And I guess it's because the same thing I said above.

Thanks in advance, I will be happy if you clearly answer if it is possible or not, so I will not try different ways on devices under API 26.

[philburk]

Do you mean I should change this param to 3 ?
int32_t mInputChannelCount = oboe::ChannelCount::Mono;
I see the enum has only 2 types mono and stereo.

The channel count parameter is just an int32_t so you can pass 3.
We defined the Mono and Stereo enums just to make the examples more clear.

With 2 or 3 channels it sounds like monotonic signal.

"Monotonic" or "Monophonic"? On some devices you will get the same signal on every channel.

is there any way to switch to one of built in microphones instead of using headphones microphone on devices under API level 26 ?

Unfortunately no. Oboe builder.setDeviceId() does not work with OpenSL ES.

oboe/include/oboe/AudioStreamBuilder.h

Line 283 in f8ab39a

* Note that when using OpenSL ES, this will be ignored and the created

[miqouser]

Thank you for help, closing