The original post: /r/datahoarder by /u/cabriofritolay on 2024-12-09 20:22:44.

TLDR: Trying to combine somewhere south of 20TB of data to a server and in that process automatically merge duplicates files.

I’ve been saving a lot of my video, photos etc. to portable hard drives through the years (like one still uses USB 1.0). I’ve recently purchased a server and essentially want to collect all of the data on those drive and put it in one place. However, I want to get rid of duplicate media. Is there any way you would recommend going about this?