Hi @AlekseyIvashov, If you are using AWS INF1, this issue might interests you: [Hugging Face] neuron compiler fails on tracing DeBERTa v1 and v2 models on INF1 · Issue #642 · aws-neuron/aws-neuron-sdk · GitHub
And btw, with AWS INF2 and the latest neuronx compiler, the XSoftmax operator shall not be an issue. You can export your custom deberta-like model optimum-neuron with a one-liner like this:
optimum-cli export neuron --model hf-internal-testing/tiny-random-DebertaModel --sequence_length 16 --batch_size 4 test/