Implement MIDI input support #2804

holly-hacker · 2019-08-28T21:44:29Z

Closes #2723.

Still WIP, uses managed-midi for cross-platform MIDI input.

Remaining issues:

~~Running status is not handled yet, which is used by Linux when multiple keys are pressed at once.~~
~~Linux midi input stops when too many keys are pressed at once~~
~~Disconnecting a device on Windows throws an unhandled exception (Exception on device disconnect on Windows atsushieno/managed-midi#49)~~
~~Does not yet take velocity into account~~

Tested on:

osu.Framework/Input/Handlers/MidiInputHandler.cs

osu.Framework/Input/MidiKey.cs

holly-hacker · 2019-08-29T17:52:45Z

Point raised in private discussion: Should MidiKey be part of InputKey? Right now it's a separate enum which makes conversion to and from integers easy, but it probably isn't usable as an input in projects like lazer.

peppy · 2019-08-29T21:52:18Z

It should be available as normal input, yes.

holly-hacker · 2019-08-29T23:42:16Z

That CodeFactor failure complains about KeyBindingContainer.Handle(UIEvent) being too complex, but it contains only a switch statement so I don't know if it's logical to split that up into different methods.

holly-hacker · 2019-09-05T13:30:38Z

PR should be ready for review and testing on Android, iOS and MacOS. I've tried testing on Android but my device doesn't seem to support MIDI input.

smoogipoo · 2019-09-06T05:26:58Z

Unfortunately this library seems to crash on macOS (atsushieno/managed-midi#50), and the developer seems to be "unable to continue development" (atsushieno/managed-midi#42).

May be best to wait for ManagedBass to be fixed...

Code-wide it looks great.

atsushieno · 2019-09-06T07:11:10Z

the developer seems to be "unable to continue development"

True, but this applies to primary development (on Linux) and not for MacOS-specific issues. I will try fixing this on weekend.

smoogipoo · 2019-09-10T07:16:06Z

Have made a few fixes for other hard crashes in managed-midi, but at this point I'm not completely sold on the stability of the framework. Gonna delay merging this one for a few weeks while the managedbass guy responds (they're generally pretty responsive).

holly-hacker · 2020-03-28T11:15:57Z

@smoogipoo ManagedBass got updated to v3 and I got a response in ManagedBass/ManagedBass#54, and it seems Bass does not support MIDI input on Android. Managed-midi has received a few bugfixes since your last message so could you give it another look when you have a spare moment?

smoogipoo · 2020-04-27T04:48:35Z

I'm able to trigger the sequence:

0x000000F0
0x0000000D

By refreshing devices on macOS. Based on how MIDI input is implemented on macOS it's probably doing something really silly, but this should probably be guarded against otherwise it crashes the app. Causes:

Unhandled exception. [runtime] 2020-04-27 04:45:45 [error]: An unhandled error has occurred.
[runtime] 2020-04-27 04:45:45 [error]: System.Collections.Generic.KeyNotFoundException: The given key '__Source0' was not present in the dictionary.
[runtime] 2020-04-27 04:45:45 [error]: at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
[runtime] 2020-04-27 04:45:45 [error]: at osu.Framework.Input.Handlers.Midi.MidiInputHandler.readEvent(Byte[] data, String senderId, Int32& i, Byte& eventType, Byte& key, Byte& velocity) in /Users/smoogipoo/Repos/osu-framework/osu.Framework/Input/Handlers/Midi/MidiInputHandler.cs:line 120
[runtime] 2020-04-27 04:45:45 [error]: at osu.Framework.Input.Handlers.Midi.MidiInputHandler.onMidiMessageReceived(Object sender, MidiReceivedEventArgs e) in /Users/smoogipoo/Repos/osu-framework/osu.Framework/Input/Handlers/Midi/MidiInputHandler.cs:line 108
[runtime] 2020-04-27 04:45:45 [error]: at Commons.Music.Midi.CoreMidiApi.CoreMidiInput.OnMessageReceived(Object sender, MidiPacketsEventArgs e)
[runtime] 2020-04-27 04:45:45 [error]: at CoreMidi.MidiPort.CallMessageReceived(IntPtr pktlist, IntPtr readProcRefCon, IntPtr srcConnRefCon)

smoogipoo · 2020-04-27T04:49:15Z

osu.Framework/Input/InputManager.cs

@@ -539,6 +560,10 @@ protected virtual void HandleMouseButtonStateChange(ButtonStateChangeEvent<Mouse

        private bool handleScroll(InputState state, Vector2 lastScroll, bool isPrecise) => PropagateBlockableEvent(PositionalInputQueue, new ScrollEvent(state, state.Mouse.Scroll - lastScroll, isPrecise));

+        private bool handleMidiKeyDown(InputState state, MidiKey key, byte velocity) => PropagateBlockableEvent(NonPositionalInputQueue, new MidiDownEvent(state, key, velocity));
+
+        private bool handleMidiKeyUp(InputState state, MidiKey key) => PropagateBlockableEvent(NonPositionalInputQueue, new MidiUpEvent(state, key));


This should be updated to use the new ButtonEventManager stuff. See how it's implemented for joysticks / mouse buttons / keys for reference.

smoogipoo · 2020-04-27T04:50:19Z

osu.Framework/Graphics/Drawable.cs

@@ -2192,6 +2198,10 @@ public bool TriggerEvent(UIEvent e)
        /// <param name="e">The <see cref="JoystickReleaseEvent"/> containing information about the input event.</param>
        protected virtual void OnJoystickRelease(JoystickReleaseEvent e) => Handle(e);

+        protected virtual bool OnMidiDown(MidiDownEvent e) => Handle(e);
+
+        protected virtual bool OnMidiUp(MidiUpEvent e) => Handle(e);


Should return void (and false in TriggerEvent()).

osu.Framework/osu.Framework.csproj

smoogipoo · 2020-04-27T04:59:53Z

Also I assume this isn't supposed to support channels at this point? It's coded in such a way that it seems like it should support them, but it isn't for some reason.

atsushieno · 2020-04-27T05:23:32Z

In case you are trying to use managed-midi's own Core MIDI support, note that its quality is super low and not quite ready for general consumption yet. I cannot provide solid implementation for Xamarin.Mac alternatives up to CoreFoundation level. On Mac managed-midi via Xamarin.Mac is the only viable approach (managed-midi has particular build for the profile). Or you might want to try RtMidiSharp which might work better.

smoogipoo · 2020-04-27T05:36:15Z

Ah... We're avoiding using Xamarin on desktop platforms in favour of netcore, so that isn't really an option for us at this point. Maybe NET5 will change that.

I'm not super concerned about macOS support for the time being though, just as long as it doesn't crash the app. If the reason for channels not working is a result of being on macOS then I'm fine with that.

atsushieno · 2020-04-27T05:49:21Z

Sorry but I'm not really tracking this project issue and have no idea what kind of issue the "channels" stuff is. If there is something reproducible at my side let me know and I might be able to help.

holly-hacker · 2020-05-02T14:27:27Z

Updated according to feedback, there's a few things I want to mention:

MidiKeyInput does not extend ButtonInput<TKey>, but implements IInput directly. ButtonInput<TKey>'s purpose seems to be to handle multiple input changes at once, but MidiKeyInput is only directly used in MidiInputHandler so it doesn't seem useful.
I changed how velocity gets stored. MidiState now stores an internal Dictionary<MidiKey, byte>, which gets updated in MidiStateChangeEvent's ctor and read in MidiKeyEventManager.HandleButtonDown. It's not the prettiest solution, but the only other way I could think of involved refactoring ButtonEventManager which seems a bit silly if it's only for MIDI input. I can try to look for something better if this isn't good enough.
MIDI input is currently accepted even when the game is not focused, though most other programs share this behavior.

smoogipoo · 2020-05-03T16:27:19Z

The way you've implemented it is fine :) Made one minor change to make the velocities public - may come in useful if someone's handling input manually via InputManager.CurrentState.

Thanks!

resolved

holly-hacker added 3 commits August 28, 2019 23:23

Implement MIDI input

748dd68

Don't throw if MIDI input cannot be initialized (eg. missing asound)

8f4ff56

Enable MidiInputHandler for Android and iOS

d1d88ff

pull-request-size bot added the size/L label Aug 28, 2019

frenzibyte previously requested changes Aug 28, 2019

View reviewed changes

osu.Framework/Input/Handlers/MidiInputHandler.cs Outdated Show resolved Hide resolved

holly-hacker mentioned this pull request Aug 28, 2019

Exception on device disconnect on Windows atsushieno/managed-midi#49

Closed

holly-hacker added 7 commits August 29, 2019 13:00

Handle Running Status messages

b67a97b

Handle midi event edge cases

5e7cab0

Include velocity in Midi events

0c43156

Move MidiInputHandler to correct namespace

14d2d11

Manually (and properly) parse MIDI messages

0a16a99

Fix code style issues

068b07f

Remove remaining TODO comment

a22e62e

frenzibyte reviewed Aug 29, 2019

View reviewed changes

osu.Framework/Input/MidiKey.cs Show resolved Hide resolved

pull-request-size bot added size/XL and removed size/L labels Aug 29, 2019

Fix missing imports in Android and iOS projects

21d6ada

holly-hacker added 3 commits August 30, 2019 00:24

Add Midi keys to InputKey

c11b145

Remove accidental using statement

78b1c63

Track Midi events in FrameStatistics

c669b13

Update managed-midi to 1.9.6

6f588aa

pull-request-size bot added size/XXL and removed size/XL labels Sep 5, 2019

Merge remote-tracking branch 'origin/master' into midi-input

0a408fd

Update managed-midi

feeda0a

bdach added the area:input label Oct 10, 2019

smoogipoo added 2 commits April 27, 2020 13:32

Merge branch 'master' into midi-input

dd1ad39

Remove unnecessary clone

3c154a8

smoogipoo requested changes Apr 27, 2020

View reviewed changes

holly-hacker added 5 commits May 2, 2020 14:21

Update managed-midi

c633835

Fix return parameter of OnMidiUp, add doc comments

fc54002

Use ButtonEventManager instead of manually calling handleMidiKey*

aac4366

Improve error handling in case of invalid MIDI messages

cc44d6c

Fix up debug logging

6769cd1

smoogipoo added 2 commits May 4, 2020 01:21

Merge branch 'master' into midi-input

b7fcba7

Make velocities public

89ad557

smoogipoo approved these changes May 3, 2020

View reviewed changes

smoogipoo merged commit 0f73c6d into ppy:master May 3, 2020

holly-hacker deleted the midi-input branch May 3, 2020 18:09

holly-hacker mentioned this pull request May 3, 2020

MIDI devices support ppy/osu#4328

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement MIDI input support #2804

Implement MIDI input support #2804

holly-hacker commented Aug 28, 2019 •

edited

Loading

holly-hacker commented Aug 29, 2019

peppy commented Aug 29, 2019

holly-hacker commented Aug 29, 2019

holly-hacker commented Sep 5, 2019

smoogipoo commented Sep 6, 2019

atsushieno commented Sep 6, 2019

smoogipoo commented Sep 10, 2019

holly-hacker commented Mar 28, 2020

smoogipoo commented Apr 27, 2020 •

edited

Loading

smoogipoo Apr 27, 2020

smoogipoo Apr 27, 2020

smoogipoo commented Apr 27, 2020

atsushieno commented Apr 27, 2020

smoogipoo commented Apr 27, 2020

atsushieno commented Apr 27, 2020

holly-hacker commented May 2, 2020

smoogipoo commented May 3, 2020

Implement MIDI input support #2804

Implement MIDI input support #2804

Conversation

holly-hacker commented Aug 28, 2019 • edited Loading

holly-hacker commented Aug 29, 2019

peppy commented Aug 29, 2019

holly-hacker commented Aug 29, 2019

holly-hacker commented Sep 5, 2019

smoogipoo commented Sep 6, 2019

atsushieno commented Sep 6, 2019

smoogipoo commented Sep 10, 2019

holly-hacker commented Mar 28, 2020

smoogipoo commented Apr 27, 2020 • edited Loading

smoogipoo Apr 27, 2020

Choose a reason for hiding this comment

smoogipoo Apr 27, 2020

Choose a reason for hiding this comment

smoogipoo commented Apr 27, 2020

atsushieno commented Apr 27, 2020

smoogipoo commented Apr 27, 2020

atsushieno commented Apr 27, 2020

holly-hacker commented May 2, 2020

smoogipoo commented May 3, 2020

holly-hacker commented Aug 28, 2019 •

edited

Loading

smoogipoo commented Apr 27, 2020 •

edited

Loading