Expose audio mixers to mods

[hlysine]

Re: #16740 (reply in thread)

The new IApplicableToTrackMixer interface mainly provides a way for mods to apply audio filters in song select. During gameplay, this interface is a shortcut to the already possible way of adding a drawable via IApplicableToDrawableRuleset, then getting the AudioManager.TrackMixer via DI.

This PR is a pre-requisite to changing muted mod to use a low-pass filter only, instead of adjusting track volume.

This is a relatively simple diff, but there are some points I'm not super sure about:

• Should IApplicableToTrack provide the mixer instead of a separate interface?
  @frenzibyte originally proposed to include the mixer in IApplicableToTrack.ApplyToTrack, but there are many cases where a mixer isn't available (e.g. when using this interface to calculate new track rate in diff calc), so I decided to split this into a new interface.

• Should IApplicableToSampleMixer exist?
  Yes for symmetry, but muted mod probably won't be needing this since it requires separate control of hit samples and metronome samples. Feel free to revert 016e0c3 if this is deemed redundant.

• Should local mixers be provided to ApplyTo*Mixer instead?
  I don't think any local mixers exist yet. Just providing the global mixers should be the simplest working implementation IMO.

For reference, this is how the muted mod could look like using this interface:

diff

Index: osu.Game/Rulesets/Mods/ModMuted.cs
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/osu.Game/Rulesets/Mods/ModMuted.cs b/osu.Game/Rulesets/Mods/ModMuted.cs
--- a/osu.Game/Rulesets/Mods/ModMuted.cs	(revision 016e0c320745dae61a9c273d4bbb46d382392baa)
+++ b/osu.Game/Rulesets/Mods/ModMuted.cs	(date 1643959558362)
@@ -1,13 +1,15 @@
 // Copyright (c) ppy Pty Ltd <contact@ppy.sh>. Licensed under the MIT Licence.
 // See the LICENCE file in the repository root for full licence text.
 
+using System;
 using System.Linq;
 using osu.Framework.Audio;
-using osu.Framework.Audio.Track;
+using osu.Framework.Audio.Mixing;
 using osu.Framework.Bindables;
 using osu.Framework.Graphics;
 using osu.Framework.Graphics.Sprites;
 using osu.Framework.Localisation;
+using osu.Game.Audio.Effects;
 using osu.Game.Configuration;
 using osu.Game.Graphics.UserInterface;
 using osu.Game.Overlays.Settings;
@@ -28,13 +30,14 @@
         public override double ScoreMultiplier => 1;
     }
 
-    public abstract class ModMuted<TObject> : ModMuted, IApplicableToDrawableRuleset<TObject>, IApplicableToTrack, IApplicableToScoreProcessor
+    public abstract class ModMuted<TObject> : ModMuted, IApplicableToDrawableRuleset<TObject>, IApplicableToScoreProcessor, IApplicableToTrackMixer
         where TObject : HitObject
     {
         private readonly BindableNumber<double> mainVolumeAdjust = new BindableDouble(0.5);
         private readonly BindableNumber<double> metronomeVolumeAdjust = new BindableDouble(0.5);
 
         private BindableNumber<int> currentCombo;
+        private AudioFilter lowPassFilter;
 
         [SettingSource("Enable metronome", "Add a metronome beat to help you keep track of the rhythm.")]
         public BindableBool EnableMetronome { get; } = new BindableBool
@@ -69,11 +72,11 @@
         protected ModMuted()
         {
             InverseMuting.BindValueChanged(i => MuteComboCount.MinValue = i.NewValue ? 1 : 0, true);
-        }
-
-        public void ApplyToTrack(ITrack track)
-        {
-            track.AddAdjustment(AdjustableProperty.Volume, mainVolumeAdjust);
+            mainVolumeAdjust.BindValueChanged(i =>
+            {
+                if (lowPassFilter != null)
+                    lowPassFilter.Cutoff = (int)(Math.Pow(i.NewValue, 3) * AudioFilter.MAX_LOWPASS_CUTOFF);
+            });
         }
 
         public void ApplyToDrawableRuleset(DrawableRuleset<TObject> drawableRuleset)
@@ -107,6 +110,11 @@
         }
 
         public ScoreRank AdjustRank(ScoreRank rank, double accuracy) => rank;
+
+        public void ApplyToTrackMixer(AudioMixer mixer)
+        {
+            lowPassFilter = new AudioFilter(mixer) { Cutoff = 1000 };
+        }
     }
 
     public class MuteComboSlider : OsuSliderBar<int>

[frenzibyte]

• Should IApplicableToTrack provide the mixer instead of a separate interface?
  @frenzibyte originally proposed to include the mixer in IApplicableToTrack.ApplyToTrack, but there are many cases where a mixer isn't available (e.g. when using this interface to calculate new track rate in diff calc), so I decided to split this into a new interface.

I still don't really see the reason behind these existing, you can add the mixer but as an optional parameter for such cases:

public void ApplyToTrack(ITrack track, IAudioMixer mixer = null);

And avoid all the extra additions of IApplicableToTrackMixer/IApplicableToSampleMixer/IApplicableToMixers.

[bdach]

Why would the mixer be nullable in the above proposal? What would a mod implementation do if it is supplied a null mixer while it wants to apply effects, just give up?

I would even go a step further and propose that maybe ITrack should itself provide a reference to the mixer, rendering all of this unnecessary. But I don't feel confident on that proposal, so I'd advise not to take it too seriously.

In fact note that IAudioChannel already exposes mixers (albeit only internally to the frammework).

[frenzibyte]

Why would the mixer be nullable in the above proposal? What would a mod implementation do if it is supplied a null mixer while it wants to apply effects, just give up?

It's nullable because it's irrelevant in cases like diffcalc. And, correct, I don't see that being a problem, we could not make it an optional parameter if your concern is about it being forgotten in future usages?

[bdach]

Yeah I dunno. I think diffcalc doing stuff on tracks in general is kinda dodgy already, and it's currently worked around by passing a virtual track. I'd rather follow that lead and pass a dummy mixer or something, but the perfect scenario would be to decouple diffcalc from audio stuff.

[frenzibyte]

Having a VirtualAudioMixer sounds workable, sure.

[hlysine]

Created a framework PR for that: ppy/osu-framework#5013