Layoutlmv2 for classifying sequence of pages


I have been using LayoutLMv2 for classifying pages of a document as either “First page”, “Middle page”, and “Last page”. I have fine-tuned the model with examples, however the performance is suboptimal.
As it is now, each page gets a prediction independent from surrounding pages, however the predictions should truly depend on each other. I was thinking of adding another classifiers after LayoutLM to incorporate this dependency. I came across bilstm-crf models for NER and think it might be relevant for my case. Any suggestions if I am on the right track and how to implement?