Models for eye gaze data

Models
1

Hi,

I was wondering if anyone knows any (transformer) models that work well for eye gaze data classification?
My input would be a time series of x/y coordinates (and potentially some pupil dilation values). The output would be a classification label, e.g., “calm”/“confused”.
As I understand, one possible model would be something like Deepmind’s PerceiverIO, however, I only have a few 100 training samples. Thus, I think I would require a much smaller model.

Does anyone have any experience with this? Current research seems to prefer RNNs; however, I do not see any reasons why transformers should not outperform them here as well. I am thankful for any hints or guidance.

2 Likes
2

Hi Lewa I researched alot because i need that kinda model too did you find anything relevant to that if yes then kindly reply me i shall be thankful to you

3

Hey man did you find something?

1 Like
4

Hi!

You can find what I ended up doing in this publication: https://kclpure.kcl.ac.uk/ws/portalfiles/portal/272361658/TAC_When_to_Explain_2024.pdf

The accompanying code can be found here:

1 Like
5

I also made some improvements to the classification pipeline described in the above paper, which you can find here: https://dl.acm.org/doi/pdf/10.1145/3678957.3688386

However, the first paper is specific to gaze data, while this one deals with other social/non-verbal signals.

1 Like