Any more developments here? My understanding is that we’d have to pre-train using the standard Trainer class with a custom Data Collator as described by @ncoop57. @valhalla would you be able to help/comment?
1 Like