feat/neon_transformers

[NeonJarbas]

integrates https://github.com/NeonGeckoCom/neon-transformers

functionality originally developed over at neon-core

to use add a section to your mycroft.conf, any plugin present will be loaded, config options depend on the plugin itself

example utterance plugins

• OpenVoiceOS/ovos-classifiers@525641b
• OpenVoiceOS/ovos-classifiers@e8b87b0
• feat/opm plugin ovos-lingua-franca#57
• https://github.com/NeonGeckoCom/neon-utterance-plugin-cancel

sample config

{
    "utterance_transformers": {
            "neon_utterance_cancel_plugin": {},
            "ovos-utterance-normalizer": {}
     }
}

utterance transformer plugins base class

from ovos_plugin_manager.templates.transformers import UtteranceTransformer

class MyUtteranceHandler(UtteranceTransformer):

    def initialize(self):
        """ perform any initialization actions """
        pass

    def transform(self, utterances: List[str],
                  context: dict = None) -> (list, dict):
        """
        Optionally transform passed utterances and/or return additional context
        :param utterances: List of str utterances to parse
        :param context: existing Message context associated with utterances
        :returns: tuple of (possibly modified utterances, additional context)
        """
        return utterances, {}

    def default_shutdown(self):
        """ perform any shutdown actions """
        pass

metadata transformer plugin template

from ovos_plugin_manager.templates.transformers import MetadataTransformer

class MyMessageContextHandler(MetadataTransformer):

    def initialize(self):
        """ perform any initialization actions """
        pass

    def transform(self, context: dict = None) -> (dict):
        """
        Optionally transform passed context
        eg. inject default values or convert metadata format
        :param context: existing Message context from all previous transformers
        :returns: dict of possibly modified or additional context
        """
        context = context or {}
        return context

    def default_shutdown(self):
        """ perform any shutdown actions """
        pass

an equivalent for audio is already available in ovos-dinkum-listener, a PR is needed in ovos-listener

audio transformer plugin template

from ovos_plugin_manager.templates.transformers import AudioTransformer


class MyAudioHandler(AudioTransformer):
    """process audio data and optionally transform it before STT stage"""

    def initialize(self):
        """ perform any initialization actions """
        pass

    def on_audio(self, audio_data):
        """ Take any action you want, audio_data is a non-speech chunk
        """
        return audio_data

    def on_hotword(self, audio_data):
        """ Take any action you want, audio_data is a full wake/hotword
        Common action would be to prepare to received speech chunks
        NOTE: this might be a hotword or a wakeword, listening is not assured
        """
        return audio_data

    def on_speech(self, audio_data):
        """ Take any action you want, audio_data is a speech chunk (NOT a
        full utterance) during recording
        """
        return audio_data

    def on_speech_end(self, audio_data):
        """ Take any action you want, audio_data is the full speech audio
        """
        return audio_data

    def transform(self, audio_data):
        """ return any additional message context to be passed in
        recognize_loop:utterance message, usually a streaming prediction
        Optionally make the prediction here with saved chunks from other handlers
        """
        return audio_data, {}

    def default_shutdown(self):
        """ perform any shutdown actions """
        pass

[codecov]

Codecov Report

Merging #163 (41e160e) into dev (6ceb058) will decrease coverage by 36.22%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##              dev     #163       +/-   ##
===========================================
- Coverage   50.35%   14.13%   -36.22%     
===========================================
  Files         119       11      -108     
  Lines       10077     1493     -8584     
===========================================
- Hits         5074      211     -4863     
+ Misses       5003     1282     -3721     

see 93 files with indirect coverage changes

[JarbasAl]

an example of lang tagging in STT can be found here https://github.com/OpenVoiceOS/ovos-stt-plugin-fasterwhisper/blob/dev/ovos_stt_plugin_fasterwhisper/__init__.py#L30