Are object detection models supported in optimum?

🤗Optimum
1

are object detection models supported in optimum? I don’t see any tasks defined for OD here optimum/optimum/onnxruntime/trainer.py at main · huggingface/optimum · GitHub

2

Hi @sak99! What is your use case (training, inference, ONNX, BetterTransformer,…)? For instance it is possible to export models like DETR to the ONNX format.

3

Hi @regisss I am looking at Faster R-CNN, YOLO, SparseRCNN

4

DETR and YOLOS are the only object-detection models that can be exported to ONNX with Optimum. The models you mentioned are not supported by the Transformers library so they are not by Optimum either.

5

hello,i am begin learning huggingface , can u tell me how load YOLOS onnx in huggingface

6

@chaosbasicly Here is @merve’s great notebook showing how to do it with DETR: smol-vision/Reduce_any_model_to_fp16_using_🤗_Optimum_DETR.ipynb at main · merveenoyan/smol-vision · GitHub
It should be easy to adapt it to YOLOS. Let me know if you encounter any issues.

1 Like
7

Many thanks <3

1 Like
8

Hi regisss, thank you for your help. Currently, I’ve successfully inferred the YOLO model with ONNX, but its speed is too slow, so I want to convert it to an engine format. Do you know how to do this? I’ve tried to convert it using TensorRT 8.6.
import tensorrt as trt
import onnx

Load ONNX model

onnx_model_path = ‘/home/chaos/Documents/ChaosAIVision/reseach/xenova.onnx’
onnx_model = onnx.load(onnx_model_path)

Create TensorRT builder and network

EXPLICIT_BATCH = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
builder = trt.Builder(trt.Logger(trt.Logger.WARNING))
network = builder.create_network(EXPLICIT_BATCH)

Parse ONNX model to create network definition

parser = trt.OnnxParser(network, builder.logger)
parser.parse(onnx_model.SerializeToString())

Create TensorRT builder configuration

config = builder.create_builder_config()
config.max_workspace_size = 1 << 30 # Set maximum workspace size

Build TensorRT engine

engine = builder.build_engine(network, config)

Save TensorRT engine to file

with open(‘your_model.engine’, ‘wb’) as f:
f.write(engine.serialize())

However, it encountered an error:

convert_tensorrt.py
[06/22/2024-01:33:04] [TRT] [W] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/22/2024-01:33:04] [TRT] [W] onnx2trt_utils.cpp:400: One or more weights outside the range of INT32 was clamped
/home/chaos/Documents/ChaosAIVision/reseach/convert_tensorrt.py:18: DeprecationWarning: Use set_memory_pool_limit instead.
config.max_workspace_size = 1 << 30 # Set maximum workspace size
/home/chaos/Documents/ChaosAIVision/reseach/convert_tensorrt.py:21: DeprecationWarning: Use build_serialized_network instead.
engine = builder.build_engine(network, config)
[06/22/2024-01:33:04] [TRT] [E] 4: [network.cpp::validate::3047] Error Code 4: Internal Error (Network has dynamic or shape inputs, but no optimization profile has been defined.)
Traceback (most recent call last):
File “/home/chaos/Documents/ChaosAIVision/reseach/convert_tensorrt.py”, line 25, in
f.write(engine.serialize())
AttributeError: ‘NoneType’ object has no attribute ‘serialize’

hope your answers