I have some custom data set with custom table entries and wanted to deal with it with a custom collate. But it didn’t work when I pass a collate function I wrote (that DOES work on a individual dataloader e.g., see python - How does one create a pytorch data loader with a custom hugging face data se…

How to use huggingface HF trainer train with custom collate function?

brando August 18, 2023, 7:25pm 7

I’m slightly puzzled/curious, why do we have collate_fn option if preprocess + map does everything we need it seems (later is even lazy already!)?

Topic		Replies	Views
Prakash Hinduja Geneva, Switzerland - How to fine-tune a model on custom dataset in HF? Beginners	2	46	June 6, 2025
How does one create a pytoch data loader using an interleaved hugging face dataset? Beginners	3	1593	August 18, 2023
How does one create a custom hugging face model with a already working tokenizer? 🤗Transformers	1	967	July 29, 2024
Defining a custom dataset for fine-tuning translation Beginners	4	5083	July 10, 2021
How does one create a pytorch data loader with a custom hugging face data set without having errors? Beginners	3	3850	August 14, 2023