Viam Module Migration - API Mapping

[danic85]

We have discussed with Viam the opportunity to map existing modules from this framework into the Viam registry, and there is a question as to whether the correct API package exist for each. Below is a list of the existing modules and a proposal to map to an existing API.

I took the liberty of assuming a viam.services.generic subpackage as we discussed this may be an option, but if there is a better fit I'm happy to consider it.

Servo - Handle servo control - viam.components.servo
Behaviour - Dynamic behaviour depending on external stimuli and internal state - viam.services.generic
Sentiment Analysis - Parse text and return sentiment rating - viam.services.generic
Coral Face & Object Detection - viam.services.vision
OpenCV Face & Object Detection - viam.services.vision
Robust Serial - Serial communication with Arduino or other device - viam.services.generic
Snowboy - Hotword detection to enable voice recognition - viam.services.generic / viam.components.audio_input
Animate - Scripted animations of robot - viam.service.generic
Battery Monitor - viam.components.power_sensor
Braillespeak - Audio output converting text to tones - viam.services.generic
Buzzer - Handles buzzer output - viam.component.generic
Chirp - Converts text to OTA audio data - viam.services.generic
Neopixel - Controls Neopixels via I2C or GPIO - viam.components.generic
SpeechInput - viam.services.generic / viam.components.audio_input
Translator - Translates text to other language - viam.services.generic
TTS - Text to speech - viam.services.generic
Log Manager - viam.services.generic
Gemini.ai Integration - viam.services.mlmodel
OpenBCI Integration - viam.components.input

Full list of modules for reference:
https://github.com/makerforgetech/modular-biped/tree/feature/viam/modules

The plan is to refactor existing modules for more generic use cases and migrate these into viam modules within the namespace makerforgetech, repo:
https://github.com/makerforgetech/viam-modules

[mcvella]

Hi Dan-

Thanks for this, all look correct except maybe:

Robust Serial - Serial communication with Arduino or other device - I wonder if this might instead be a board implementation?

Also, note that I have created an "action" API that might work for some of your modules (like Animate). I used it for a "docking" service I created. If you feel this API or something similar would be useful, let me know and perhaps I can give input to the product team on making it more official.

Also check out the "speech" service I created - I does some of the things you are looking to do, and could potentially be improved, added to , or amended based on your needs. It also has a custom API that I'd want to decouple.

On the Generic Service API, we did just have our 6 week planning cycle and that was included, so hopefully we'll have that soon!

Matt

[danic85]

Thanks @mcvella - the speech service and action API would definitely have some overlap with some of my modules, so I'll take a look at using them. Can I use them as-is or does there need to be any changes on your side first? (I'll read their docs before I get started)

Good news on the generic service API, quite a few seem to fit in that category.

I'll also take a look at the board API as I had assumed that was more to define the parameters of the boards, rather than facilitate communication between two boards.

A few actions for me, then I'll update this post!

[mcvella]

On the modules I mentioned, I am open to any feedback or pull requests! I want to ensure they work for a number of use-cases!

[danic85]

A list of resources that may have parity with existing features:

• camera.viam:camera:csi-pi
• camera.transform, ML Model.tflite_cpu, Vision.mlmodel (already implemented)
• viam-labs:camera:overlay-fps
• viam:vision:deepface-identification
• viam-labs:audioout:pygame
• viam:vision:motion-detector
• viam-labs:speech:speechio
• viam-labs:detector:facial-detector
• zaporter:timelapse:v1
• viam-soleng:raspi:temperatures
  Useful in future:
• jeremyrhyde:npl:chat-gpt-module
• ianwhalen:ianwhalen:neopixel - likely not i2c, but a good example
• biotinker:component:display - i2c display may be useful in future
• viam-labs:classifier:verification-system
• viam-labs:sensor:sysfs-hcsr04
• viam-labs:camera:image-dir
• viam-labs:sensor:battery-level
• zack:random:animalsoundspammer
• viam:wifi_sensor:linux
• viam-soleng:weather-api:current

@mcvella some feedback from me for the team on the registry after having investigated the above:

• It isn't clear whether the registry search includes the standard modules from viam, and if not, why some viam modules are included and some aren't.
• The capabilities look great, but I get the feeling from some of these community created modules that they are not production ready. As far as I'm aware there isn't a way of filtering on 'in development', alpha, beta builds vs production. It would be good if we could mark our modules as such, and even better if there was some automated check against a module that tested the functionality against it's documented capabilities.
• There is functionality created by viam that I know exist, and know how to use them because of an example guide, but short of checking every tutorial for functionality I want, there isn't an easy way to find how to configure and get started with some piece of functionality, or even if that functionality exists. I don't have a great solution here, but for an example see the camera.transform, ML Model.tflite_cpu, Vision.mlmodel configuration which is explained in the tutorial here but since I only needed part of this functionality I had to deconstruct the guide.
• I'd love to get updates on new functionality available on the platform, either community created or created by Viam - I could be building a feature from scratch that didn't exist when I started and does now, and I wouldn't know.
• A small thing, but the Platform -> Registry -> Find modules page doesn't actually let you find modules, that's a level higher. I consistently go to the wrong page because of the naming.
• The action API and speech service you mentioned aren't listed, presumably because they're not released on the registry yet - Is that something that will happen in time, or will there always be a few modules that are in that state? It would be good to be able to find those too.

[mcvella]

Overall, thanks - this is some good product feedback, I will make sure the team sees it. Some responses inline below.

• It isn't clear whether the registry search includes the standard modules from viam, and if not, why some viam modules are included and some aren't.

"built in" resources don't show on the main registry listing page or the search tool in the docs. They do of course show in the dropdown for components or services on the configuration pages.

I can see why it might be confusing as to why built-in resources don't show and why they are different.

• The capabilities look great, but I get the feeling from some of these community created modules that they are not production ready. As far as I'm aware there isn't a way of filtering on 'in development', alpha, beta builds vs production. It would be good if we could mark our modules as such, and even better if there was some automated check against a module that tested the functionality against it's documented capabilities.

Agree that some indicator of state could be helpful.

• There is functionality created by viam that I know exist, and know how to use them because of an example guide, but short of checking every tutorial for functionality I want, there isn't an easy way to find how to configure and get started with some piece of functionality, or even if that functionality exists. I don't have a great solution here, but for an example see the camera.transform, ML Model.tflite_cpu, Vision.mlmodel configuration which is explained in the tutorial here but since I only needed part of this functionality I had to deconstruct the guide.

Have you tried the Viam AI assistant on the help docs? I am curious to know if that helps you locate the content you might need - trying it with a few example questions and letting us know would be helpful!

• I'd love to get updates on new functionality available on the platform, either community created or created by Viam - I could be building a feature from scratch that didn't exist when I started and does now, and I wouldn't know.

Thanks, I'll let our product team that it might be helpful to include not only internal feature releases, but highlights of community contributions!

• A small thing, but the Platform -> Registry -> Find modules page doesn't actually let you find modules, that's a level higher. I consistently go to the wrong page because of the naming.

Good call, will let the docs team know.

• The action API and speech service you mentioned aren't listed, presumably because they're not released on the registry yet - Is that something that will happen in time, or will there always be a few modules that are in that state? It would be good to be able to find those too.

If I search for 'speech' or 'action' I do find them, do you not see them listed?

[danic85]

I haven't tried the AI, I will admit I immediately discounted it as a sales chatbot or something similar. I'll take a look.

If I search for 'speech' or 'action' I do find them, do you not see them listed?

Yes you're right, I can see those modules with those keywords, never mind!

[danic85]

I have made some limited progress on testing out existing modules (limited because my mic and speaker aren't supported on the pi 5 yet, nothing to do with Viam).
One of the community created registry modules doesn't seem to work, but there isn't an easy way to see if others have experienced that, or ask questions. A comments or Q&A section on a registry module might be useful so we can interact with the author.

I'll look at the micro-rdk next and then come back to this when the pi 5 compatibility issues are fixed.

[mcvella]

Curious which specific modules gave you trouble and how specifically. There's some common issues that might be easy to resolve.

Also on the mic and speaker not being supported by the pi5 - curious which ones? In theory its just running linux so I am surprised there would be pi-5 specific issues.

[danic85]

It was more of a general comment for feedback, but I remember the timelapse module doesn't seem to work, but I didn't dig into it very much. The mic and speaker issues are because they are i2s and use adafruit software that is having issues on bookworm. I actually wrote a post about this: https://www.makerforge.tech/posts/pi5-build-log-4/#adafruit-support But if you have any suggestions for a work around I'd love to hear it, it's a shame to be blocked because of this.

…

________________________________ From: Matt Vella ***@***.***> Sent: Friday, January 26, 2024 9:40:17 PM To: makerforgetech/modular-biped ***@***.***> Cc: Dan Nicholson ***@***.***>; Author ***@***.***> Subject: Re: [makerforgetech/modular-biped] Viam Module Migration - API Mapping (Discussion #72) Curious which specific modules gave you trouble and how specifically. There's some common issues that might be easy to resolve. Also on the mic and speaker not being supported by the pi5 - curious which ones? In theory its just running linux so I am surprised there would be pi-5 specific issues. — Reply to this email directly, view it on GitHub<#72 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABSHFZG7756VUXBLFHXKH23YQQPEDAVCNFSM6AAAAABBR562SCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DENRRHA3TM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[mcvella]

For the i2s speaker, did you try this module? I know we used it with a different i2s speaker previously. It uses pygame (no adafruit deps) so it it is more portable. I don't know if there's a module for generic mic usage. The speech module uses pygame for mics as well - but I need to find the time to decouple the speech module so it can use another module for audio in, and the audioout module for playback... In general when I create modules I try to avoid making them with Adafruit deps as they tend to not be very portable...

I've not used the timelapse module, but the author actually is an engineer at Viam. I also created this one, which could be used for similar functionality: https://app.viam.com/module/viam-labs/image-dir-cam