I’ve been working on a side project that uses phonetic English language models for text generation. Since I’m not aware of any existing phonetic English datasets, I’ve been preprocessing existing English text datasets with my phonemization script to give myself enough training data. Mainly OSCAR for…

Understanding set_transform

jncasey February 21, 2021, 9:20pm 5

Thanks, @lhoestq! That’s an incredibly cool and useful feature. Can’t wait to play with it.

Topic		Replies	Views
Using load_dataset.set_transform() function along with Trainer class 🤗Datasets	4	2616	April 26, 2021
How to use set_transform when map becomes unfeasible? Intermediate	2	135	June 19, 2024
Pipeline with custom dataset tokenizer: when to save/load manually 🤗Datasets	18	5636	September 18, 2020
Transformed dataset to_json saves cache dataset Beginners	3	370	January 3, 2023
Set batch instead of full train dataset on Trainer 🤗Transformers	1	372	March 11, 2024