Hi everone,
I started to pre-trained BERT for a masking task in a time series domain. I used a custom tokenizzation (not an usual model) to masking some samples with a special token. But durig the training process the loss is too much constant (like 6.7, I used the SparseCategoricalCrossentropy).
Could anyone help me?
Thank guys.