I think your answer can easily load if the arrow_file is single. But when I process the data using multiple processes, the datasets are stored into several arrow files. Can I simply load each of them and concatenate them?
I think your answer can easily load if the arrow_file is single. But when I process the data using multiple processes, the datasets are stored into several arrow files. Can I simply load each of them and concatenate them?