Yolov5s on Openvino

Question

I have trained a model using yolov5 and it is working just fine:

My ultimate goal is to use a model that I have trained on custom data (to detect the hook and bucket) in the Openvino framework.

To achieve this, I first exported the best version of the model to the appropriate Openvino format, using the following command:

!python export.py --weights runs/train/yolov5s24/weights/best.pt --include openvino  --dynamic --simplify

The export performed successfully generated me 3 files: best.xml, best.bin, best.mapping;

Now I would like to load it using the Openvino framework and to to that I am following this pipeline:

1. Create Core object
2. 1.1. (Optional) Load extensions
3. Read a model from a drive
4. 2.1. (Optional) Perform model preprocessing
5. Load the model to the device
6. Create an inference request
7. Fill input tensors with data
8. Start inference
9. Process the inference results

---

1 Create Core

import numpy as np
import openvino.inference_engine as ie
core = ie.IECore()

---

2 Read a model from a drive

path_to_xml_file = 'models/best_openvino_model/best.xml'
path_to_bin_file = 'models/best_openvino_model/best.bin'
network = core.read_network(model=path_to_xml_file, weights=path_to_bin_file)

---

3 Load the Model to the Device

# Load network to the device and create infer requests
exec_network = core.load_network(network, "CPU", num_requests=4)

And here I am getting an error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-21-7c9ba5f53484> in <module>
      1 # Load network to the device and create infer requests
----> 2 exec_network = core.load_network(network, "CPU", num_requests=4)

ie_api.pyx in openvino.inference_engine.ie_api.IECore.load_network()

ie_api.pyx in openvino.inference_engine.ie_api.IECore.load_network()

RuntimeError: Check 'std::get<0>(valid)' failed at inference/src/ie_core.cpp:1414:
InferenceEngine::Core::LoadNetwork doesn't support inputs having dynamic shapes. Use ov::Core::compile_model API instead. Dynamic inputs are :{ input:'images,images', shape={?,3,?,?}} 

I am using the OpenVINO™ Development Tools - release 2022.1; The files to reproduce the error are here;

Answer 1

This error is expected as the model contains a dynamic shape. This model can be executed using the ov::Core:compile_model API in OpenVINO 2022.1.

You can refer to class ov::CompiledModel and Dynamic Shape for more information.