Feature/mixer

[leiradel]

Added an audio mixer.

To test, add this to audio_driver.c, inside function audio_driver_flush (around line 615). You should have both the WAV and the OGG repeating while still hearing the core audio.

Regular operation needs only the line audio_mixer_mix(audio_driver_output_samples_buf, output_frames); in audio_driver_flush.

[inactive123]

Thanks a lot @leiradel for delivering this to us on time. We owe you big time.

[RobLoach]

The diff I used is over at https://hastebin.com/buhogaqahi.cpp .... Getting the following error:

CC audio/audio_driver.c
audio/audio_driver.c:147:11: error: unknown type name ‘audio_mixer_sound_t’
    static audio_mixer_sound_t* wav = NULL;
           ^
audio/audio_driver.c:148:11: error: unknown type name ‘audio_mixer_sound_t’
    static audio_mixer_sound_t* ogg = NULL;

[inactive123]

Tried loading a 33MB Ogg file using that example code snippet. Unfortunately it slows everything down to a crawl and it seems to not play back the audio anyway. Does it need to be all copied into RAM before it plays any music? Is it performed on a task/thread?

[inactive123]

Alright, I understand that by doing this I am kinda overloading the sound system since this is spawned for every audio_driver_flush call so a 33mb Ogg file would indeed result in this kind of crawling performance loss. I guess we can dispatch these audio load calls from anywhere in the code instead of them being spammed from here right?

@RobLoach tells me that this sound file works for him -

http://www.kozco.com/tech/piano2.wav

however this one crashes for him -

https://files.andressm.org/f/b099dfeb99/

[RobLoach]

Worked with the following audio files:

• http://www.kozco.com/tech/piano2.wav
• http://docpiper.com/tests/audio/music/SumertimeBlues.ogg

The following wav file crashed it:

• https://files.andressm.org/f/b099dfeb99/

[inactive123]

Here is the stack trace of nav.wav -

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `retroarch'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f97b2ab68f6 in _int_free () from /usr/lib/libc.so.6
[Current thread is 1 (Thread 0x7f97bb05dd40 (LWP 869))]
(gdb) bt
#0 0x00007f97b2ab68f6 in _int_free () from /usr/lib/libc.so.6
#1 0x000000000047da82 in resampler_sinc_free (data=0x2a925c0)
at libretro-common/audio/resampler/drivers/sinc_resampler.c:379
#2 0x0000000000453152 in one_shot_resample (samples_out=,
out=, rate=, samples_in=,
in=0x462e4c0) at audio/audio_mixer.c:212
#3 audio_mixer_load_wav (
path=path@entry=0x70cf60 "/home/squarepusher/Downloads/nav.wav")
at audio/audio_mixer.c:264
#4 0x0000000000440496 in audio_driver_flush (samples=1470,
data=) at audio/audio_driver.c:609
#5 audio_driver_sample_batch (data=, frames=735)
at audio/audio_driver.c:690
#6 0x00007f97a4a24055 in PsndGetSamples ()
from /home/squarepusher/libretro-super/dist/unix/picodrive_libretro.so
#7 0x00007f97a49fb25a in PicoFrame ()
from /home/squarepusher/libretro-super/dist/unix/picodrive_libretro.so
#8 0x00007f97a49f9c02 in retro_run ()
from /home/squarepusher/libretro-super/dist/unix/picodrive_libretro.so
#9 0x000000000041bc48 in core_run () at core_impl.c:437
#10 0x000000000042c6af in runloop_iterate (
sleep_ms=sleep_ms@entry=0x7fff6acc314c) at runloop.c:1155
#11 0x000000000041ab82 in rarch_main (argc=1, argv=, data=0x0)
---Type to continue, or q to quit---q
atQuit

[inactive123]

Mixing the two audio streams together with the core's audio works very well, congratulations on that!

Guess it's just a matter of ironing out some of the rough edges.

[leiradel]

Tried loading a 33MB Ogg file using that example code snippet. Unfortunately it slows everything down to a crawl and it seems to not play back the audio anyway. Does it need to be all copied into RAM before it plays any music? Is it performed on a task/thread?

Are you calling one of the audio_mixer_load functions every frame? The only function that must be called every frame is audio_mixer_mix.

I'm not loading files using tasks, and I don't think the audio mixer should bother with tasks. That is missing are functions that take memory buffers instead of file paths, so the caller can load the audio data however it wants (i.e. loading synchronously, using tasks, by embedding the data into the executable) and just pass it to the audio mixer. I'll take a look at it today or tomorrow (I'll travel with the family next Saturday and we have to make preparations.)

[leiradel]

Alright, I understand that by doing this I am kinda overloading the sound system since this is spawned for every audio_driver_flush call so a 33mb Ogg file would indeed result in this kind of crawling performance loss. I guess we can dispatch these audio load calls from anywhere in the code instead of them being spammed from here right?

Hm, I believe you're calling the load function every frame. Load audio data only once, the only function that must be called every frame is audio_mixer_mix.

[leiradel]

The following wav file crashed it:

https://files.andressm.org/f/b099dfeb99/

It looks like a regular WAV file, I'll have to take a look:

$ file ~/Downloads/nav.wav 
/Users/andre.leiradella/Downloads/nav.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 44100 Hz

[inactive123]

OK, the audio mixer should indeed not bother with tasks.

I was talking more about creating a 'task' inside 'tasks' for creating a sound effect. But we can take a look at that instead now that we know how it works.

[leiradel]

OK, the audio mixer should indeed not bother with tasks.

I believe we should not add a specific task for that. It's just a matter of loading the file data asynchronously, which the tasks already do, so all that is missing is an audio mixer API that takes memory buffers instead of file paths.

Also, I believe we should stream OGG data from the file system using a RFILE instead of having everything in memory.

[inactive123]

You might be right.

Try to figure out why the crash happens with that wav file that was posted, and I will start thinking of ways to integrate this into RA.

[leiradel]

nav.wav now works with the mixer. The resampler was overwriting memory because we don't know how much memory it needs, so I've added a safeguard.

#4712