Skip to content

Latest commit

 

History

History
 
 

arm_plugin

README.md

OpenVINO™ Toolkit - ARM CPU plugin

OpenVINO™ ARM CPU plugin is developed in order to enable deep neural networks inference on ARM CPUs, using OpenVINO™ API. The plugin uses ARM Compute Library* as a backend.

Supported Platforms

OpenVINO™ ARM CPU plugin is supported and validated on the following platforms:

Host OS
Raspberry Pi* 4 Model B Debian* 9 (32-bit)
Raspberry Pi* 4 Model B Debian* 10.3 (32-bit)
Raspberry Pi* 4 Model B Ubuntu* 18.04 (64-bit)
Raspberry Pi* 4 Model B Ubuntu* 20.04 (64-bit)
Apple* Mac mini with M1 macOS 11.1 (64-bit)
Samsung* Galaxy S20 FE Android 10 (64-bit)

Distribution

OpenVINO™ ARM CPU plugin is not included into Intel® Distribution of OpenVINO™. To use the plugin, it should be built from source code.

Get Started

  1. Build ARM plugin
  2. Prepare models
  3. Run IE samples
  4. Run OMZ demos

Supported Mutable Properties

The plugin supports mutable properties listed below. All parameters must be set before calling ov::Core::compile_model() in order to take effect.

Parameter name Parameter values Default Description
ov::streams::num ov::streams::NUMA, ov::streams::AUTO, or non negative integer values 1 Specifies number of CPU "execution" streams for the throughput mode. Upper bound for the number of inference requests that can be executed simultaneously. All available CPU cores are evenly distributed between the streams.
ov::affinity ov::Affinity::CORE/NUMA/NONE ov::Affinity::NONE Binds inference threads to CPU cores. Enabled only if OpenVINO™ is built with TBB that supports affinity configuration
ov::inference_num_threads positive integer values Limit #threads that are used by Inference Engine for inference on the CPU
ov::enable_profiling true/false Enable profiling information collection

Supported Layers and Limitations

The plugin supports IRv10 and higher. The list of supported layers and its limitations are defined here.

Supported Model Formats

  • FP32 – Supported and preferred
  • FP16 – Supported
  • I8 – Experimental support

Supported Input Precision

  • FP32 - Supported
  • FP16 - Supported
  • U8 - Supported
  • U16 - Supported
  • I8 - Not supported
  • I16 - Not supported

Supported Output Precision

  • FP32 – Supported
  • FP16 - Supported

Supported Input Layout

  • NCDHW – Not supported
  • NCHW - Supported
  • NHWC - Supported
  • NC - Supported

License

OpenVINO™ ARM CPU plugin is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

How to Contribute

We welcome community contributions to openvino_contrib repository. If you have an idea how to improve the modules, please share it with us. All guidelines for contributing to the repository can be found here.

* Other names and brands may be claimed as the property of others.