Debiasing models by HEX projection

Research
1

I am interested in implementing the orthogonality portion of Towards Robustifying NLI Models Against Lexical Dataset Biases from ACL 2020 in Pytorch.

The overall idea seems simple. Have a primary model and a sub-model (like BOW) to detect superficial features, and then use HEX projection (Wang et al., 2019a) to project the representation of the original primary model to the orthogonal space of the representation of the BoW model.

In this case, I would use a transformer as the primary model. I’m not sure about the implementation of HEX projection. If someone is familiar with it, it would be really helpful if they can share the snippet responsible for projecting the representation orthogonally.

Additionally, adding a debiasing example in Transformers repo would be a good addition which I’m happy to add, once I implement myself.

1 Like
2

I figured out myself from the equations in the paper (Wang et al.). My implementation seems to be working. Will share the link for my repo once I open source the code.

1 Like