Thanks that makes sense - although I used the value of 512 for the position embedding size based on PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES from source - this is also the default for BertConfig from what I can see.
Plus it still crashes when max_position_embeddings = model_max_length + padding_index, I think it should be max_position_embeddings = model_max_length + padding_index + 1 (since the the indices start from 1 not zero)