OpenVINO - Workflow
In this chapter, we will learn about the OpenVINO workflow and method of reading, compiling, and converting OpenVINO.
OpenVINO Workflow
The following is the step-by-step procedure (workflow) for the OpenVINO toolkit −
Step 1: Planning and Setup
Determine the necessary configurations. Then, identify the model type and framework.
Step 2: Select a Model
In this step, find or train a model from Open Model Zoo, then check if the model is accurate enough. If the model is not accurate, retrain it.
Step 3: Modify
Run the Model Optimizer to convert the model into Intermediate Representation and check if the model converted successfully. If not, fix the errors or create a custom layer. If that doesnt work, escalate to Intel support or try alternative models. Now, we will move on to the next step: Tune.
Step 4: Tuning
Now run the inference engine on the model and check if it is fast enough with acceptable accuracy. If not, try advanced tuning of the model. If advanced tuning also does not work, try either hardware-based interventions or training extensions.
Step 5: Deployment
In this step, integrate the model into a pipeline or application. Once the package is ready, simply deploy the app and model.
Models for Reading, Compiling, and Converting
The following are the models for Converting, Reading, and compiling.
Read_model
- Constructs an ov.Model from a file.
- File formats Supported: ONNX, OpenVINO IR,PaddlePaddle, TensorFlow Lite, and TensorFlow. PyTorch files are not supported directly.
- OpenVINO files are read directly, whereas other formats are converted automatically.
Compile_model
- Constructs an ov.CompiledModel from a file or ov.Model object.
- File formats Supported: OpenVINO IR, ONNX,
- PaddlePaddle, TensorFlow, and TensorFlow Lite. PyTorch files are not supported directly.
- OpenVINO files are read directly, whereas other formats are converted automatically.
Convert_model
- Constructs an ov.Model from a file or Python memory object.
- File formats supported are PaddlePaddle, ONNX, TensorFlow Lite, and TensorFlow.
- Framework objects supported: TensorFlow, PaddlePaddle, and PyTorch.
- This process is only available in the Python API.
- Saves an ov.Model to OpenVINO IR format.
- Compresses weights to FP16 by default.
- This process is only available in the Python API.