diff --git a/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/MicroGraphics/index.md b/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/MicroGraphics/index.md index 4aadc1d19..0916f05e5 100644 --- a/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/MicroGraphics/index.md +++ b/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/MicroGraphics/index.md @@ -151,7 +151,7 @@ graphics.DrawTriangle( filled: true); ``` -The code sample above draws a red rectangle: +The code sample above draws a red triangle: ![meadow micrographics line](micrographics_triangle.png) diff --git a/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/TensorFlowLite/index.md b/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/TensorFlowLite/index.md new file mode 100644 index 000000000..ce447f682 --- /dev/null +++ b/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/TensorFlowLite/index.md @@ -0,0 +1,91 @@ +--- +layout: Meadow +title: TensorFlowLite for Microcontrollers Library +subtitle: Using the Meadow.Foundation TensorFlow integration for running machine learning models on Meadow devices. +--- + + +The Meadow.TensorFlow library provides Meadow users with the ability to run machine learning models directly on microcontrollers using TensorFlow Lite. It is designed to enable inference from pre-trained models efficiently, using minimal resources, while being optimized for IoT environments. + +With this library, you can load and run models for tasks like image recognition, gesture detection, and other forms of inference. This documentation will walk you through how to integrate TensorFlow into your Meadow application, load models, and run inferences. + +# Using TensorFlow on Meadow + +TensorFlow Lite for Microcontrollers on Meadow provides a modern .NET API, allowing you to run machine learning inference directly on your Meadow devices. This enables you to build intelligent IoT applications that can process data locally without relying on cloud services. + +## Integrating TensorFlow into Your Meadow Application + +To get started, you'll need to add the `Meadow.TensorFlowLiteForMicrocontrollers` library to your Meadow application. You can do this by adding the NuGet package to your project: + +```csharp +dotnet add package Meadow.TensorFlowLiteForMicrocontrollers +``` + +## Loading a TensorFlow Lite Model + +First, include your TensorFlow Lite model (.tflite file) in your Meadow project. Set its Build Action to EmbeddedResource so it gets embedded into your application assembly. + +To load the model, read it as a byte array from the embedded resources: + +```csharp +// Load the model from embedded resources +var assembly = Assembly.GetExecutingAssembly(); + +using (var stream = assembly.GetManifestResourceStream("YourNamespace.Models.gesture_model.tflite")) +{ + byte[] modelData = new byte[stream.Length]; + stream.Read(modelData, 0, modelData.Length); + + // Create the GestureModel object with the model data + gestureModel = new GestureModel(modelData); +} +``` + +Replace "YourNamespace.Models.gesture_model.tflite" with the actual namespace and resource path of your model file. + +## Preparing Input Data + +The `GestureModel` class provides a `ModelInput` property to manage inputs. You need to prepare your input data according to the model's expected input format. + +For example, if your model expects a float array of a specific size: + +```csharp +// Prepare your input data (e.g., sensor readings) +float[] inputData = new float[] { /* your input data */ }; + +// Set the input data +gestureModel.ModelInput.SetData(inputData); +``` + +## Running Inference + +With the model loaded and input data prepared, you can run inference by calling the `Predict` method: + +```csharp +// Run the prediction +var output = gestureModel.Predict(); +``` + +## Interpreting the Results + +The `ModelOutput` object contains the output data from the model, which you can interpret according to your application's logic. + +For example, if the model outputs probabilities for different classes: + +```csharp +// Assuming output contains a float array of probabilities +float[] probabilities = output.GetOutputData(); + +// Find the class with the highest probability +int predictedClass = Array.IndexOf(probabilities, probabilities.Max()); + +// Use the predicted class in your application +Console.WriteLine($"Predicted Class: {predictedClass}"); +``` + +# Best Practices + +* **Model Optimization** - Optimize your model for size and performance using quantization and pruning techniques. +* **Resource Management** - Meadow devices have limited resources. Keep your models small and efficient. +* **Input Validation** - Ensure that the input data matches the model's expected format and data types. +* **Error Handling** - Implement error handling to manage issues with model loading and inference execution. diff --git a/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/index.md b/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/index.md index 1f5a2295f..d84868c51 100644 --- a/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/index.md +++ b/docs/Meadow/Meadow.Foundation/Libraries_and_Frameworks/index.md @@ -11,4 +11,5 @@ Meadow.Foundation includes a number of libraries and frameworks to make working * [MicroGraphics](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/MicroGraphics) - General purpose graphics library. * [Maple Server](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/Maple%2EServer) - Ultra-lightweight web API server. * [TextDisplayMenu](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/TextDisplayMenu/) - General purpose menu library for character and graphic displays. -* [GPS/GNSS NMEA Processor Library](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/Gps_Gnss_Nmea_Processor/) - Global Positioning System (GPS) and Global Navigation Satellite System (GNSS) sentence processor for reading data from GPS receivers. \ No newline at end of file +* [GPS/GNSS NMEA Processor Library](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/Gps_Gnss_Nmea_Processor/) - Global Positioning System (GPS) and Global Navigation Satellite System (GNSS) sentence processor for reading data from GPS receivers. +* [TensorFlowLite for Microcontrollers](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/TensorFlowLite/) - TensorFlowLite for Microcontrollers library for Meadow F7. \ No newline at end of file diff --git a/docs/Meadow/Meadow.OS/AI/index.md b/docs/Meadow/Meadow.OS/AI/index.md new file mode 100644 index 000000000..4755eea0b --- /dev/null +++ b/docs/Meadow/Meadow.OS/AI/index.md @@ -0,0 +1,33 @@ +--- +layout: Meadow +title: Artificial Intellegence (AI) +subtitle: Artifical Intelligence (AI) at the edge with Meadow. +--- + +AI @ the edge is a critically important part of many IoT workflows and growing exponentially. IoT devices routinely not only provide sensory input to model training in the cloud but in turn, execute those models locally for efficient, low-latency outcomes: + +* **Sensory Input** - ML training pipelines and modern LLMs are gobbling up data at an unprecedented pace. The edge provides a wealth of data from the real world that can only be gathered from field-deployed devices. Digital twins would be impossible without the edge sensory input that feeds them, and the insights from not only individual devices, but in aggregate, they can unlock insights that can massively increase overall efficiency and productivity. +* **Local Model Execution** - While it’s generally not practical to train models on edge, these devices have outsized capabilities to execute models locally. For instance, realtime defect detection via an attached camera can run easily on a microcontroller (MCU), and object detection in multi-channel video can be done on a multi-GPU Jetson NX rather efficiently. This seminal post by Pete Warden (Useful Sensors) explains why non-LLM AI/ML models run so efficiently on low-powered devices. + +# Benefits of Running @ the Edge + +Additionally, running AI at the edge rather than in the cloud has a number of important efficiencies and advantages: +* **Lower Latency** - By executing locally, model execution outcomes are available immediately, not requiring a round-trip to the server. +* **Reduced Bandwidth Cost** - It’s magnitudes more efficient to send the output of a model than the raw data that fed it (once training has occurred). For instance, if you’re creating a smart outlet that tracks appliance usage, all that the cloud cares about is what appliances are running and if there are anomalies. It doesn’t need the raw waveform data of actual electrical usage which may not even be possible to feed over a low-bandwidth IoT connection. +* **Reduced TCO** - By outsourcing ML processing to the edge and running in situ, server loads (and costs) can be reduced and devices can use available processing cycles as part of their normal operating lifecycle. + +# AI on Meadow Devices + +Meadow provides a productive and powerful, unified, multi-platform AI experience at the edge for .NET developers, whether it be an MCU-based device like a Meadow F7, Single-Board-Computer (SBC) such as a Raspberry Pi or Multi-GPU Jetson Nano, or desktop machine. + +Today Meadow enables AI across the following vectors: + +* **[TensorFlowLite for Microcontrollers](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/TensorFlowLite/)** - TensorFlowLite is an model execution runtime that is baked into Meadow.OS as a first-class feature. Simply add the nuget package, load your model, feed it data, and get the results. For more information see the [TensorFlowLite for Microcontrollers guide](/Meadow/Meadow.Foundation/Libraries_and_Frameworks/TensorFlowLite/). + +* **Useful Sensors Drivers** - We have Meadow.Foundation drivers for Useful Sensors, which are purpose built, commodity AI sensors for doing person detection and QR code reading. See the [Meadow.Foundation Peripheral driver list page](/Meadow/Meadow.Foundation/Peripherals/) for the latest Useful Sensors drivers. + +* **Single-Board-Computer (SBC) Meadow Support** - Meadow.Linux has full support for a number of the most popular SBCs, including Raspberry Pi boards, Beagle Bone, Nvidia Jetson, and more. With Meadow's first class hardware and industrial protocol support, it's super easy to get sensor data into the SBC AI workflows. For more information, see the [Getting Started guides](/Getting_Started/) on how to get up and running with Meadow on SBCs. + +# Future Integrations + +* **Edge Impulse** - Edge Impulse provides not only pre-trained, optimized models for embedded scenarios, but they also provide a powerful workflow for importing training data, labeling it, and then training, tuning, and outputting a useful model. We're currently in the process of integrating Edge Impulse into Meadow.OS as a first class feature. \ No newline at end of file diff --git a/docs/Meadow/Meadow.OS/index.md b/docs/Meadow/Meadow.OS/index.md index 68012af3f..22be68292 100644 --- a/docs/Meadow/Meadow.OS/index.md +++ b/docs/Meadow/Meadow.OS/index.md @@ -6,6 +6,7 @@ subtitle: List of features and documentation of Meadow.OS Meadow.OS offers a comprehensive operating system framework that includes features for wireless network and device connectivity, file and database operations, and application lifecycle monitoring and management. +* **[Artificial Intelligence (AI)](AI)** * **[Automatic Restarts](Automatic_Restarts)** * **[Bluetooth](Bluetooth)** * **[Cellular](Cellular)** diff --git a/docs/Meadow/Release_Notes/v1/index.md b/docs/Meadow/Release_Notes/v1/index.md index a763381f2..4c01d8916 100644 --- a/docs/Meadow/Release_Notes/v1/index.md +++ b/docs/Meadow/Release_Notes/v1/index.md @@ -10,6 +10,30 @@ subtitle: Release Notes * [Meadow.CLI](/Meadow/Meadow_Tools/Meadow_CLI/) * [Meadow.OS](/Meadow/Getting_Started/Deploying_Meadow%2EOS/) +## v1.15.0.0 + +This release delivers some key optimizations to Meadow networking and sleep. + +### Meadow.OS + +* Fixed a critical issue that was affecting the reliability of Meadow Sleep/Wake +* Added support for the Quectel EG21 modem +* Minor improvements to edge-case network reliability + +### Meadow.Core + +* OtA download progress reporting fix +* Improved reporting of deadlock conditions +* Improved network compatibility + +### Meadow.Foundation + +* Renamed PCanBasic assembly to prevent OEM assembly name conflict +* Added ADS1263 Driver +* Added ChromaTek WS2812 momentary and latching button drivers +* Improved WS2812 driver + + ## v1.14.0.0 This release delivers a major memory management upgrade that's the result of months of work. This new version of Meadow OS both reduces the total memory used and more efficiently manages freed memory. diff --git a/src/sidebars/meadowOsSidebar.js b/src/sidebars/meadowOsSidebar.js index 3e6f7ea8a..e6be726d5 100644 --- a/src/sidebars/meadowOsSidebar.js +++ b/src/sidebars/meadowOsSidebar.js @@ -171,6 +171,7 @@ const meadowOsSidebar = [ id: "Meadow/Meadow.OS/index", }, items: [ + "Meadow/Meadow.OS/AI/index", "Meadow/Meadow.OS/Automatic_Restarts/index", "Meadow/Meadow.OS/Bluetooth/index", { @@ -244,6 +245,7 @@ const meadowOsSidebar = [ "Meadow/Meadow.Foundation/Libraries_and_Frameworks/Maple.Server/index", "Meadow/Meadow.Foundation/Libraries_and_Frameworks/TextDisplayMenu/index", "Meadow/Meadow.Foundation/Libraries_and_Frameworks/Gps_Gnss_Nmea_Processor/index", + "Meadow/Meadow.Foundation/Libraries_and_Frameworks/TensorFlowLite/index", ], }, {