Does Hugging Face Datasets Support Efficient Referencing of Images to Avoid Duplication?

If you are primarily concerned with preventing duplication, it may be better to save files by URL or file name, but this may not be very convenient for large datasets. @lhoestq