In the end I found out that I was not encountering a bug and my GPU was truly OOM.
Novice misunderstanding as I am new to ONNX. For those who are new to ONNX the memory requirements are actually 4-5x that of running a model in the PyTorch format.
In the end I found out that I was not encountering a bug and my GPU was truly OOM.
Novice misunderstanding as I am new to ONNX. For those who are new to ONNX the memory requirements are actually 4-5x that of running a model in the PyTorch format.