Skip to content

Latest commit

 

History

History
 
 

inference_generator

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 

Inference Generator

caffe2openvx: Convert a pre-trained CAFFE model into a C library for use by applications.

  • Extract neural network model from deploy.prototxt
    • generate C code that instantiates OpenVX kernels from vx_nn module
    • generate build scripts that package C code into a library
    • the generated C code or library can be easily integrated into an application for running inference
  • Extract weights and biases from weights.caffemodel into separates folders for use by the C library during initialization
  • Also generate a GDF for quick prototyping and kernel debugging

The generated C code will have two functions in annmodule.h:

void annGetTensorDimensions(
        vx_size dimInput[4],    // input tensor dimensions
        vx_size dimOutput[4]    // output tensor dimensions
    );

vx_graph annCreateGraph(
        vx_context context,     // OpenVX context
        vx_tensor input,        // input tensor
        vx_tensor output,       // output tensor
        const char * dataFolder // folder with weights and biases
    );
or
vx_graph annCreateGraphWithInputImage(
        vx_context context,     // OpenVX context
        vx_image input,         // input image (RGB or U8)
        vx_tensor output,       // output tensor
        const char * dataFolder // folder with weights and biases
    );
or
vx_graph annCreateGraphWithInputImageWithArgmaxTensor(
        vx_context context,     // OpenVX context
        vx_image input,         // input image (RGB or U8)
        vx_tensor output,       // output tensor
        const char * dataFolder // folder with weights and biases
    );
or
vx_graph annCreateGraphWithInputImageWithArgmaxImage(
        vx_context context,     // OpenVX context
        vx_image input,         // input image (RGB or U8)
        vx_image output,        // output image (U8)
        const char * dataFolder // folder with weights and biases
    );
or
vx_graph annCreateGraphWithInputImageWithArgmaxImageWithLut(
        vx_context context,     // OpenVX context
        vx_image input,         // input image (RGB or U8)
        vx_image output,        // output image (RGB)
        const char * dataFolder // folder with weights and biases
    );
  • annGetTensorDimensions: allows an application to query dimensions of input and output tensors
  • annCreateGraph (or another variant above): creates and initializes a graph with trained neural network for inference

Command-line Usage

  % caffe2openvx
        [options]
        <net.prototxt|net.caffemodel>
        [n c H W [type fixed-point-position [convert-policy round-policy]]]
option description
--(no-)error-messages do/don't enable error messages (default: ON)
--(no-)virtual-buffers do/don't use virtual buffers (default: ON)
--(no-)generate-gdf do/don't generate RunVX GDF with weight/bias initialization (default: ON)
--(no-)generate-vx-code do/don't generate OpenVX C Code with weight/bias initialization (default: ON)
--output-dir specify output folder for weights/biases, GDF, and OpenVX C Code (default: current)
--input-rgb convert input from RGB image into tensor using (a*x+b) conversion: rev=(BGR?1:0)
--input-u8 convert input from U8 image into tensor using (a*x+b) conversion
--argmax-tensor u8/u16 k return argmax output with specified tensor type and top_k
--argmax-image u8/u16 return argmax output with specified image type
--argmax-lut <rgbLut.txt> argmax color table: one R G B entry per label
--flags specify custom flags (default: 0)

Example

Make sure that all executables and libraries are in PATH and LD_LIBRARY_PATH environment variables.

% export PATH=$PATH:/opt/rocm/mivisionx/bin
% export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/mivisionx/lib

Below log outlines a simple use-case with inference generator.

% caffe2openvx weights.caffemodel 1 3 32 32
% caffe2openvx deploy.prototxt 1 3 32 32
% ls
CMakeLists.txt   annmodule.txt   cmake              weights
annmodule.cpp    anntest.cpp     deploy.prototxt    weights.caffemodel
annmodule.h      bias            net.gdf
% mkdir build
% cd build
% cmake ..
% make
% cd ..
% ls build
CMakeCache.txt  Makefile        cmake_install.cmake
CMakeFiles      anntest         libannmodule.so
% ./build/anntest
OK: annGetTensorDimensions() => [input 32x32x3x32] [output 1x1x10x32]

The anntest.cpp is a simple program to initialize and run neural network using the annmodule library.