Skip to content
This repository has been archived by the owner on Apr 5, 2023. It is now read-only.

Bidirectional MIDI controls #19

Merged
merged 8 commits into from
Apr 14, 2020
Merged

Bidirectional MIDI controls #19

merged 8 commits into from
Apr 14, 2020

Conversation

houz
Copy link
Contributor

@houz houz commented Apr 9, 2020

Hello,

this PR adds support for reacting to OBS messages and sending MIDI messages. Currently it only reacts to changes of the scene (current and preview), but adding other feedback should be easy.

I also fixed some bugs in the registration of WebSocketApp callbacks.

I tested this with a KORG nanoKONTROL2.

Related to issue #16 .

houz added 3 commits April 9, 2020 17:01
@houz
Fix WebSocketApp callbacks
c5c555a 
The callbacks need to be wrapped, otherwise they either don't get called
at all or with the wrong arguments
@houz
React to OBS state changes with MIDI messages
f138cb8 
This will add a "bidirectional" option for buttons that change the
current or preview scene. When we see an update from OBS that the scene
changed we try to inform the MIDI controls that are associated with that
scene about it if it's bidirectional. In other words: If your scene
changing button has an LED it will turn on/off.
I tested this with a KORG nanoKONTROL2 where it works beautifully.
@houz
Fix last commit
9ecd097 
It seems I had forgotten to add most of my changes. :-/
@lebaston100 lebaston100 self-assigned this Apr 9, 2020
@lebaston100
Copy link
Owner

Looks fine to me.
I'm actually amazed that you can send to an input port. I've looked though the ports documentation a few times and it's never clearly mentioned that it does.
Also i can't test it as mentioned in the issue, still lacking the hardware.
I will give your branch a test if anything broke for non-bidirectional devices but other then that i'll merge that in the near future.

@houz
Copy link
Contributor Author

houz commented Apr 9, 2020

Thank you. I don't think you can send to an input port, that's why I changed it to an io port.

@lebaston100
Copy link
Owner

Oh right, i apparently missed that while reading though it.

@me-vlad
Copy link

me-vlad commented Apr 10, 2020

Hi,
Changing mido.open_input to mido.open_ioport without checking device compatibility will cause connect issues like with my Behringer X-Touch Mini

Python 3.7.5 (tags/v3.7.5:5c02a39a0b, Oct 15 2019, 00:11:34) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mido
>>> mido.get_ioport_names()
[]
>>> mido.get_input_names()
['X-TOUCH MINI 0'] 
>>> mido.get_output_names()
['Microsoft GS Wavetable Synth 0', 'X-TOUCH MINI 1'] 
>>>
C:\Users\Vlad\Desktop\MIDItoOBS-bidirectional>python main.py
[2020-04-10 13:20:45,935] (INFO) T8960 : Successfully parsed config file
--- Logging error --- 
Traceback (most recent call last):
  File "main.py", line 76, in __init__
    self._port = mido.open_ioport(name=self._devicename, callback=self.callback, autoreset=True)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\backend.py", line 159, in open_ioport
    self.module.Output(output_name, **kwargs))
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 265, in __init__
    BasePort.__init__(self, name, **kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 86, in __init__
    self._open(**kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 190, in _open
    virtual=virtual, api=self.api)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 93, in _open_port
    raise IOError('unknown port {!r}'.format(name))
OSError: unknown port 'X-TOUCH MINI 0'

@lebaston100
Copy link
Owner

Thanks for testing that @me-vlad .
I though that might happen that's why i said i need to test it first with a unidirectional device before merging.
Best case i think would be to add a flag into the device object inside the database itself that indicates that it supports receiving too and then use that information in the DeviceHandler init before opening the device.
Otherwise we could just try with ioport first and in case that goes into the except try again with only input device.
I wonder why the try except doesn't catch that exception.

@me-vlad
Copy link

me-vlad commented Apr 10, 2020

Full output

C:\Users\Vlad\Desktop\MIDItoOBS-bidirectional>python main.py
[2020-04-10 13:20:45,935] (INFO) T8960 : Successfully parsed config file
--- Logging error --- 
Traceback (most recent call last):
  File "main.py", line 76, in __init__
    self._port = mido.open_ioport(name=self._devicename, callback=self.callback, autoreset=True)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\backend.py", line 159, in open_ioport
    self.module.Output(output_name, **kwargs))
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 265, in __init__
    BasePort.__init__(self, name, **kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 86, in __init__
    self._open(**kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 190, in _open
    virtual=virtual, api=self.api)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 93, in _open_port
    raise IOError('unknown port {!r}'.format(name))
OSError: unknown port 'X-TOUCH MINI 0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1025, in emit
    msg = self.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 869, in format
    return fmt.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 608, in format
    record.message = record.getMessage()
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "main.py", line 388, in <module>
    handler = MidiHandler()
  File "main.py", line 139, in __init__
    self._portobjects.append((DeviceHandler(device, device.doc_id), device.doc_id))
  File "main.py", line 78, in __init__
    self.log.critical("\nCould not open", self._devicename)
Message: '\nCould not open'
Arguments: ('X-TOUCH MINI 0',)
--- Logging error ---
Traceback (most recent call last):
  File "main.py", line 76, in __init__
    self._port = mido.open_ioport(name=self._devicename, callback=self.callback, autoreset=True)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\backend.py", line 159, in open_ioport
    self.module.Output(output_name, **kwargs))
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 265, in __init__
    BasePort.__init__(self, name, **kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 86, in __init__
    self._open(**kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 190, in _open
    virtual=virtual, api=self.api)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 93, in _open_port
    raise IOError('unknown port {!r}'.format(name))
OSError: unknown port 'X-TOUCH MINI 0'

Traceback (most recent call last):
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1025, in emit
    msg = self.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 869, in format
    return fmt.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 608, in format
    record.message = record.getMessage()
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "main.py", line 388, in <module>
    handler = MidiHandler()
  File "main.py", line 139, in __init__
    self._portobjects.append((DeviceHandler(device, device.doc_id), device.doc_id))
  File "main.py", line 78, in __init__
    self.log.critical("\nCould not open", self._devicename)
Message: '\nCould not open'
Arguments: ('X-TOUCH MINI 0',)
[2020-04-10 13:20:45,976] (CRITICAL) T8960 : The midi device might be used by another application/not plugged in/have a different name.
[2020-04-10 13:20:45,977] (CRITICAL) T8960 : Please close the device in the other application/plug it in/select the rename option in the device management menu and restart this script.

@cpyarger
Copy link
Contributor

cpyarger commented Apr 10, 2020 via email •
edited by lebaston100
Loading

@lebaston100
Copy link
Owner

Full output

C:\Users\Vlad\Desktop\MIDItoOBS-bidirectional>python main.py
[2020-04-10 13:20:45,935] (INFO) T8960 : Successfully parsed config file
--- Logging error --- 
Traceback (most recent call last):
  File "main.py", line 76, in __init__
    self._port = mido.open_ioport(name=self._devicename, callback=self.callback, autoreset=True)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\backend.py", line 159, in open_ioport
    self.module.Output(output_name, **kwargs))
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 265, in __init__
    BasePort.__init__(self, name, **kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 86, in __init__
    self._open(**kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 190, in _open
    virtual=virtual, api=self.api)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 93, in _open_port
    raise IOError('unknown port {!r}'.format(name))
OSError: unknown port 'X-TOUCH MINI 0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1025, in emit
    msg = self.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 869, in format
    return fmt.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 608, in format
    record.message = record.getMessage()
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "main.py", line 388, in <module>
    handler = MidiHandler()
  File "main.py", line 139, in __init__
    self._portobjects.append((DeviceHandler(device, device.doc_id), device.doc_id))
  File "main.py", line 78, in __init__
    self.log.critical("\nCould not open", self._devicename)
Message: '\nCould not open'
Arguments: ('X-TOUCH MINI 0',)
--- Logging error ---
Traceback (most recent call last):
  File "main.py", line 76, in __init__
    self._port = mido.open_ioport(name=self._devicename, callback=self.callback, autoreset=True)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\backend.py", line 159, in open_ioport
    self.module.Output(output_name, **kwargs))
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 265, in __init__
    BasePort.__init__(self, name, **kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\ports.py", line 86, in __init__
    self._open(**kwargs)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 190, in _open
    virtual=virtual, api=self.api)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\backends\rtmidi.py", line 93, in _open_port
    raise IOError('unknown port {!r}'.format(name))
OSError: unknown port 'X-TOUCH MINI 0'

Traceback (most recent call last):
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1025, in emit
    msg = self.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 869, in format
    return fmt.format(record)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 608, in format
    record.message = record.getMessage()
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "main.py", line 388, in <module>
    handler = MidiHandler()
  File "main.py", line 139, in __init__
    self._portobjects.append((DeviceHandler(device, device.doc_id), device.doc_id))
  File "main.py", line 78, in __init__
    self.log.critical("\nCould not open", self._devicename)
Message: '\nCould not open'
Arguments: ('X-TOUCH MINI 0',)
[2020-04-10 13:20:45,976] (CRITICAL) T8960 : The midi device might be used by another application/not plugged in/have a different name.
[2020-04-10 13:20:45,977] (CRITICAL) T8960 : Please close the device in the other application/plug it in/select the rename option in the device management menu and restart this script.

Well, that should now be fixed with ae6de7f

@houz
Copy link
Contributor Author

houz commented Apr 10, 2020 •
edited
Loading

Thanks for your feedback. I added a check if the device is capable to do IO. Of course, that will still fail if the device lies and isn't actually supporting it. I am not sure if that is the case with the X-Touch Mini.

Edit: Actually, I just noticed that there are 2 X-Touch Mini reported, one for input and one for output. I assume that's the same physical device though?

@me-vlad
Copy link

me-vlad commented Apr 10, 2020

Edit: Actually, I just noticed that there are 2 X-Touch Mini reported, one for input and one for output. I assume that's the same physical device though?

Yes it is the same device listed in input and output devices but not in ioport list

houz added 2 commits April 11, 2020 16:41
@houz
Change shutdown sequence
52ed359 
This should call self.close() exactly once, no matter if the program was
terminated with ctrl-c or because OBS died.
I am not 100% sure about this one.
@houz
Separate in and out ports
2fc7124 
If the same midi device can be used for input and output, or if it's an
ioport, then the "bidirectional" key in the config should be enough, if
you have a separate device for output, then add it as a 2nd device and
add "out_deviceID": <deviceID> to the config. It's not done in setup.py
for you.
@houz
Copy link
Contributor Author

houz commented Apr 11, 2020

I tried to fix the case that different logical midi devices are needed for input and output. You have to add the 2nd device in your config, and add a key out_deviceID to the key mapping, analogous to deviceID.

@me-vlad
Copy link

me-vlad commented Apr 11, 2020 •
edited
Loading

Hi Tobias, thanks for your help.
Just added manually to my config.json my midi device as output,
It works fine as an input device (without errors in console) but leds on buttons not inicate preview/current scene.

My mido device names

>>> import mido
>>> mido.get_input_names()
['X-TOUCH MINI 0']
>>> mido.get_output_names()
['Microsoft GS Wavetable Synth 0', 'X-TOUCH MINI 1']

My test config (two buttons for "preview" and two another - for "program")

{
    "_default": {},
    "keys": {
        "1": {
            "msg_type": "note_on",
            "msgNoC": 16,
            "input_type": "button",
            "action": "{\"request-type\": \"SetPreviewScene\", \"message-id\" : \"1\",\"scene-name\" : \"CAMERA1 - 1\"}",
            "deviceID": 1,
            "out_deviceID": 2,
            "bidirectional": true
        },
        "2": {
            "msg_type": "note_on",
            "msgNoC": 17,
            "input_type": "button",
            "action": "{\"request-type\": \"SetPreviewScene\", \"message-id\" : \"1\",\"scene-name\" : \"CAMERA2 - 2\"}",
            "deviceID": 1,
            "out_deviceID": 2,
            "bidirectional": true
        },
        "3": {
            "msg_type": "note_on",
            "msgNoC": 8,
            "input_type": "button",
            "action": "{\"request-type\": \"SetCurrentScene\", \"message-id\" : \"1\", \"scene-name\" : \"CAMERA1 - 1\"}",
            "deviceID": 1,
            "out_deviceID": 2,
            "bidirectional": true
        },
        "4": {
            "msg_type": "note_on",
            "msgNoC": 9,
            "input_type": "button",
            "action": "{\"request-type\": \"SetCurrentScene\", \"message-id\" : \"1\", \"scene-name\" : \"CAMERA2 - 2\"}",
            "deviceID": 1,
            "out_deviceID": 2,
            "bidirectional": true
        }
    },
    "devices": {
        "1": {
            "devicename": "X-TOUCH MINI 0"
        },
        "2": {
            "devicename": "X-TOUCH MINI 1"
        }
    }
}

@houz
Copy link
Contributor Author

houz commented Apr 11, 2020 •
edited
Loading

Does the following code turn on a light?

import mido
port = mido.open_output('X-TOUCH MINI 1')
port.send(mido.Message(type="note_on", channel=0, control=8, value=127))

Edit: https://stackoverflow.com/questions/39435550/changing-leds-on-x-touch-mini-mackie-control-mc-mode sounds as if it's not as simple with that device

@me-vlad
Copy link

me-vlad commented Apr 11, 2020

Xtouch Mini has 2 modes - standart midi (in this mode device works with miditoobs) and mackie control mode (MC).

Buttons led in standart mode on my device turns on with
port.send(mido.Message(type="note_on", channel=0, note=8, velocity=1))
and turns off with
port.send(mido.Message(type="note_on", channel=0, note=8, velocity=0))

>>> port.send(mido.Message(type="note_on", channel=0, control=8, value=127))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\messages\messages.py", line 88, in __init__
    check_msgdict(msgdict)
  File "C:\Users\Vlad\AppData\Local\Programs\Python\Python37\lib\site-packages\mido\messages\checks.py", line 97, in check_msgdict
    '{} message has no attribute {}'.format(spec['type'], name))
ValueError: note_on message has no attribute control
>>>
>>>
>>> mido.Message('note_on')
<message note_on channel=0 note=0 velocity=64 time=0>
>>>
>>>
>>>
>>> # led on
... port.send(mido.Message(type="note_on", channel=0, note=8, velocity=1))
>>>
>>>
>>> # len off
... port.send(mido.Message(type="note_on", channel=0, note=8, velocity=0))

@houz
Bidirectional: Try to support X-Touch Mini ...
27ae42f 
... or probably devices that are sending NOTE_ON in general.
@houz
Copy link
Contributor Author

houz commented Apr 11, 2020

Next round of testing. :-)

@me-vlad
Copy link

me-vlad commented Apr 12, 2020

Thanks @houz ,
Just tested you branch and found one more nuance - buttons (notes) numbering for input and output device does not match. For example: input device configured by setyp.py with buttons from 8 to 23, in same time for output device buttons numbered from 0 to 15.
So, to flush button 8 I need to ineract with button 0 on the output device.
May be we need to add manually not only out_deviceID but also something like out_msgNoC ?

@houz
bidirectional: Add support for out_msgNoC in config
0a63c68 
Some MIDI devices like the X-Touch Mini have different notes for sending
and receiving. In those cases you can specify the output note with
out_msgNoC in your config.json.
@houz
Copy link
Contributor Author

houz commented Apr 12, 2020

Thank you again for your feedback. I added out_msgNoC.

@me-vlad
Copy link

me-vlad commented Apr 12, 2020 •
edited
Loading

@houz, now it works after adding out_msgNoC to buttons config.
Minor issue - buttons led turns on not right after pressing a button to select a scene, but only after scene transition (takes scene status info from OBS websocket?)

@houz
Copy link
Contributor Author

houz commented Apr 12, 2020

Yes, that is on purpose as it makes the code easier – we have to react to scene change events from OBS anyway. Of course it wouldn't be hard to set it immediately when sending the command to OBS. The subsequent message received would just set it again.

@lebaston100
Copy link
Owner

I would leave it like this by design. What might be confusing at first is actually kind of handy and how i would expect it to work.
Right now it's listening to SwitchScenes and PreviewSceneChanged events. If you wanted to light up both buttons at the same time when using a transition that is not a cut, you would need to additionally listen to the TransitionBegin event.

@houz
Copy link
Contributor Author

houz commented Apr 12, 2020

Reacting to TransitionBegin is easy of course, but what would be the right thing to show? Keeping the preview button unaffected and lighting both current scene buttons? This is one of those cases where finding a good design is way harder than writing the code.

@lebaston100
Copy link
Owner

So for traditional hardware video mixer that can display red and green the concept if simple: if a source is visible it's red. So when there is a transition the green instantly changes to red (so both buttons are red at that moment). And when the transition is done the now-preview scene changes to green.
That's exactly how it's implemented in https://github.com/lebaston100/OBS-to-XAir and https://github.com/lebaston100/OBSliveTally
But for the sake of simplicity here i suggest we just leave it how it is now.

@houz
Copy link
Contributor Author

houz commented Apr 12, 2020

I don't really care, making both buttons light up while the transition happens would be easy. Tell me what you prefer as a user experience and I will implement it. I don't think simplicity of code should be the determining factor here. 😃

@cpyarger
Copy link
Contributor

cpyarger commented Apr 12, 2020 via email

@lebaston100
Copy link
Owner

For now i'm perfectly fine with it. From a UX side i don't really know which is right but i could imagine some confusion with the technical correct version. But on the other hand the entire script is tui-based so what do i know about UX :D
This whole thing needs a complete rewrite anyway sometime in the very far future.
I'm thinking about having device template files that can be selected and loaded that define the midi values needed to light up the buttons or move faders. But that's just a potential idea.

@houz
Copy link
Contributor Author

houz commented Apr 12, 2020

Good idea, having a description of a concrete device would even allow to write a gui that shows an image of the hardware with overlays in setup. Just thinking out loud here.

@lebaston100
Copy link
Owner

Yea exactly that's what i was thinking too. Having a online docs page for every device with a svg of the device with numbers that links to the right entry in the configuration file.

@me-vlad
Copy link

me-vlad commented Apr 12, 2020

IMHO something like "device database" with tipical mapping for different hw models + ability to edit config file manually (for advanced users) - a good solution.

@houz
Copy link
Contributor Author

houz commented Apr 14, 2020

I don't think those advanced features are in the scope of this PR. So, is there anything else I should do to get this merged?

@lebaston100
Copy link
Owner

No, nothing from your end. I wasn't expecting you to add all that stuff. Now it's my turn. Thanks again.

@lebaston100 lebaston100 merged commit 94b1e18 into lebaston100:master Apr 14, 2020
@houz houz deleted the bidirectional branch April 14, 2020 18:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees

@lebaston100 lebaston100

Labels
enhancement (idea)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@houz @lebaston100 @me-vlad @cpyarger