The original post: /r/datahoarder by /u/slippy_3 on 2024-12-25 04:34:16.

Hello!

I’m hoping to get some feedback on my new backup/archival solution. I’d appreciate any thoughts y’all have. I’m trying to see if there’s anything I’ve overlooked or could improve.

Background

I’m a new filmmaker and have a few projects under my belt, so about 4TB of data, and I have will be growing my video footage archive. The plan is to hang onto the footage for now at least, and will be adding to it with subsequent projects.

Equipment/Software

  • Macbook M1 Pro
    • I’m keeping a copy of most of my files in iCloud
  • 4 Bay Terramaster DAS with support for USB 3.1 Gen2
  • 3x 8TB Toshiba N300
  • 1x 6TB older WD Elements drive
    • I’ve been using this as my storage drive until recently
  • 1x 6TB new WD MyBook drive
  • Apple Time Machine
  • Carbon Copy Cloner
  • 2TB iCloud plan

Storage/Archival Plan

Drive One - Archive 8TB

  • Using the DAS, I’ll be storing all my footage for completed projects on one of the 8TB Toshiba drives.

Drive Two - Backup #1 8TB

  • A second 8TB drive will be in the DAS but only serves as a backup to the first drive.
  • Using Carbon Copy Cloner, I run a task to copy all data from the Archive drive to the Backup #1 and have it check the files to make sure everything is copied correctly.

Drive Three - Backup #2 8TB

  • A third drive also dedicated to backing up the Archive drive will with Carbon Copy Cloner, but after this initial backup, it will be moved to an offsite location.

Personal Backup Plan

  • Both external WD drives will be backed up using Time Machine.
  • One of the external drives will be moved offsite.
  • Most files on my personal computer will also be synced to iCloud.

Offsite Plan

  • Periodically, after making sure everything is up to date with the active drives, I will take them ( the backup of the Archive drive from the DAS, as well as the active external WD drive) to the offsite location where I store the second Archive backup and Time Machine backup drives, and swap them out.
  • Leaving the most updated copy offsite, I’ll bring the out of sync drives back to my house and update them, then use them as the new active Archive backup and Time Machine backup drives.
  • Rinse and repeat.

This means I’ll always have the Archive drive at home with my personal data on the computer + iCloud, even when moving the secondary drives to the offsite location.

Thoughts on cloud storage, RAID, and NAS

  • I was using Backblaze rather than using the offsite location to store the backup drives, but after some testing, I hated the lack of detail the software provided on the current status of my backup. I also was able to confirm that things were not backed up to the cloud after the initial backup, even though I gave them overnight to be backed up (with a Gigabit connection over ethernet). I read other users concerns around Backblaze not updating, and after my own personal experience, decided to stop using their service.
  • I started using IDrive, who’s software I liked much better (though that’s not saying much), and I appreciated knowing I am paying for a specific amount of storage, and not feeling like I’m getting away with something like I did on the Backblaze personal plan. However, I really don’t like the fact that IDrive charges an exorbitant amount when you go over your capacity, and they don’t keep the external drives in sync, i.e. if you make changes to your external drives, it only ever copies over new files, it doesn’t remove deleted files or rename existing files, creating duplicates, with no way to automatically sync them. This would lead to it being much easier to go over your storage allotment I’ve paid for and end up with surprise charges.
  • I’ve never used RAID, and in research for my situation, I’m not sure it would really benefit me in any way, as I’m not looking for speed, just resiliency of my data. Plus I couldn’t use the snapshot feature integrated with Carbon Copy Cloner.
  • I ultimately decided that I’d rather be completely in control of my data, and not have to trust a cloud company with (in my experience) mediocre customer support.
  • As cool as a NAS would be, I don’t really have any use to access the data over the network. It’s much faster to plug it into my computer directly to transfer files. Plus I didn’t like the idea of wearing out the drives by leaving them on all the time, not to mention the increased risk of a power outage or surge damaging my equipment.

Concerns with my setup

  • Since the Archive and Backup drives are offline the majority of the time, is this going to cause any issues with the drives? I read something about them losing magnetism or something when not in use. But I’m assuming that’s for long periods of time, like years. Whereas with me they will be off at most a couple of months.
  • I’m using a hard case with foam inserts to transport the drives to the offsite location, but will the packing and unpacking of the drives increase chances of damage?
  • The offsite location is still within the same state and not that far from me, so I know it won’t completely protect my data if there was some major disaster. But if that happens, I don’t think I’ll care much about some old footage. I’m also using iCloud for my personal data, so I’ll have a copy in the cloud somewhere at least.

I know that was a lot! So if you made it this far, I appreciate it. Please let me know if you see any flaws in my plan or ways to increase efficiency. Thanks so much!