What is the best way to store image files (including RAWs and edited files for printing) for long-term storage?
People used to use gold CD/DVDs. What is the best option now?
I currently have all my files backed up to 3 hard drives. A combination of Seagate and Western Digital. I keep these drives unplugged and unhooked from my computer, except when I am downloading to them or retrieving something off them. But these have a limited life, correct?
I am not very interested in cloud storage at this time, for several reasons. I just really don’t like the idea o someone else having absolute control of, and access to, my images. I suppose I could change my mind on that if the argument was strong enough, but given my location, and general lack of good internet access, I would rather have my files here. Case in point: all of today, our satellite internet was non-functional because of heavy snow.
Thoughts?
I’m not an expert on spinning hard drives, but I do know that the “limited life” is due to the mechanical breakdown of the moving parts, so if the drives are rarely used they should be good for a long time. I would guess that the major limiting factor for “long term storage” is that today’s technology probably won’t be accessible in ten years, maybe less. I had tons of family videos on VCRs, then when VCRs went out of production I made digital copies and put them on DVDs. Now I don’t have a DVD player. And on and on. So however you store them, you will need to constantly change the format to keep up with changing technology. The format will be the limiting factor.
Genny, I assume your 3 hard drives contain 3 complete duplicates - not a single copy stored over 3 drives? I also use your multiple hard drive approach for backup storage. However, I keep one copy offsite, at my son’s house in case of fire, etc.
I also use multiple hard drives, though I keep a primary array attached to the computer for quick access. I’m not too worried about changing formats as there is usually some way of copying as the technology transitions. It may be tedious, but it can be done.
I know Mike Ramy deletes his culls in camera and saves his flash cards as they fill up as a separate archive of raw files.
I’m on the cusp of recommending an even better way for long term storage, depending on your view of its particular reliability.
Having just gone through the frustrations of searching for old images on old technology storage devices, it’s easy for me to picture the day when CDs, DVDs, and even portable drives are passe in our ability to read and access them.
I’m about 2 steps away from moving my most critical images onto the cloud. At the very least they’ll be more accessible when I travel, but they’ll also be accessible without burning myself with arrays of storage devices and readers. They’ll also be more immune to things such as fires, earthquakes, floods, mudslides, home burglaries and hurricanes.
I understand the appeal of the cloud, but I live in a rural area and the best internet access is a marginal connection through a cellular modem. The data plan says unlimited but is severely throttled at 15 GB and it’s amazing how quickly we can burn through that for a couple of old fogies. Backing up very many large files to the cloud is pretty much a non option.
Regardless of the technology used to store backups, I believe it is quite important to have them off site or protected in some way. I store my backup at work. Of course, that is where the “cloud” comes in handy for those with decent broadband. If one can’t store them in the cloud or off site, maybe use a small fire proof safe someone hidden from intruders?
Aha! You nailed one of my “…2 steps away from moving my most critical images onto the cloud…” In Alaska we’re rural too, with even more serious limits. Meanwhile for 6 months each winter we have access to better service in Florida. We’re looking into local service down here, allowing us to bring files south on each migration. Step 2 is picking WHICH service. 
That might work well, Hank. I guess, living in Washington, with sometimes two Starbucks in the same strip mall, I could use their wifi to upload everything, but that feels like a bit of an abuse of their system.
Well, it sounds like I am in the ball park, anyway!
Yes, Bill, I have complete copies. It is actually more complex than that, as I still have some older files on small drives, and I am working through them to get everything organized, and re-copied, again, onto new, big drives.
I will have to look into what Hank says about having critical files in the cloud. I am not sure how many I would put there, or which ones. I think my primary method will still be hard drives. Are SSD drives different/better?
We had a real lesson several years ago, when we had a forest fire get within a mile of our place. I had a Rubbermaid tub full of hard drives that I when they told us to evacuate.
Genny: I recently went through the same process. Now my primary storage is in a 5-drive bay. I can grab the whole thing and go. With 4 TB drives it should be good for several more years. What I don’t have is a good off-site place for an additional back-up. Right now it’s the detached garage, but that’s perilously close in case of fire.
Genny, yes, I would consider the SSD drives to be better - no moving parts, and the file transfers would be much faster. However, the storage cost per megabyte would be significantly higher than conventional drives.
There is no such thing as a single best long term solution. Redundancy is key and diversification. I personally have a NAS unit with 4 drives, a bunch of externals some of which are off site and also on Photoshelter.
I had to look up NAS drives. Interesting. That might be a possibility. I need something that I can access fairly quickly, so I can get to at least the recent (within the last year) RAW files.
I keep WAY to much “stuff” on my computer hard drive…personal photos that I want to access easily, and jpgs of most of my recent images. It all comes down to that, doesn’t it, too much STUFF!
Thanks for help, everyone.
I have about 3.5 TB of images. My preservation strategy is to have an external drive array attached to my computer but only turned on when I need it. I call this back up 1. I then back up the external back up 1 to an additional back up drive (6 TB single drive) called my off site drive and store it in my pre-retirement office about 5 miles away. Cloud storage for this large a library is not practical at this time. Maybe in the future, but not now.
Just so happens I spent most of my August dealing with this very situation, and honestly Ive been haphazardly stumbling down this path since 2010. I’ve been doing video production for longer than photography so I’ve amassed roughly 18 tbs over 6 years over multiple size/shape HDDs. Things were a mess and it was time to clean house. I’ve actually been starting to put together a more substantial blog post/walkthrough of my experience, but I’ll give you a “short” run down.
Firstly, is the 3-2-1 rule, 3 copies, 2 onsite, 1 off site. This is the only way to guarantee when an act of god occurs, and your home is completely destroyed, you have copies somewhere.
When I started to take my storage somewhat seriously (lost a few important drives), I started buying two 3-4 TB 7200 every year, and manually syncing them. I would also purchase 1 tb 7200rpm 2.5" laptop drives for individual projects and do the same. I keep all my primarys and all my backups in separate pelican cases, one eventually off site. I also keep all my finished Movs/JPEGS/TIFFS on my dropbox (also accessible from anywhere)
This had been working though manually syncing really blows and If I ever wanted to access another years archive (especially working on video projects ect) it started to get really messy. Not to mention I/O speeds for external hard drives are pretty horrible compared to SSDs and Raided HDDs when on a modern interface (TB2/TB3/PCIe SATA ect) Once you start editing 4k footage, and d850 Raw Timelapses, those spinning 7200rpm drives feel like dial-up networking. But if you’re not needing the same amount of space and performance I need, this might be a great options for you for some time.
Where did this leave me though? Well SSDs are more or less fast (SATA 3 = ~400-500) but exorbitantly expensive beyond 1 tb. You could buy a whole mess of them but you’d be swapping even more drives ect. This only really left a Raided Enclosure of somesort.
So up until recently, I/O interfacing with an external Direct Attached Storage (DAS) raid array has been pretty lackluster unless you were on MAC and could utilize TB2, or you built a Windows machine with an internal PCIe RAID card to actually utilize the Read/Write capabilities of an array. There are some USB3 options, but you are limited to USB3s max I/O 5 Gpbs, which is theoretically 500 MB/s (really like 100-150 MB/s) Thunderbolt 2 is 20Gbps (1500 MB/s) which is significant;y faster, but now with TB3 (40 Gbps) offered on Windows machines the market has opened quite a bit.
Quick bit about RAIDs - RAIDs combine the write speed of multiple drives as well as can offer redundancy depending on how the drives are combined. RAIDS however do not count as you second copy - consider a RAIDed array a single hard drive with SOME redundancy. You still need to adhere to the 3-2-1 rule if you need 100% uptime. The main benefit of a raided array is accessibility to a large data pool, increases I/O speeds., and uptime (access to data)
So - onto the Raid shopping. There are a few manufactures who are making DAS TB3 storage devices albeit pricey or affordable and lackluster performance. This is when I stumbled about Photography Lifes’ Article on the Synology NAS systems (Network Attached Storage) NAS drives are typically pretty low performance as their main interface is Gigabit Networking (1 Gbps - less than the theoretical USB3 5 Gpbs speeds) but with new tech increases, you can now upgrade to 10 Gbps Networking (equivalent to TB1 speeds) fairly affordably, especially with the Sonnet Solo TB3 to 10 Gbps adapter which basically allow you to turn a NAS into a DAS while retaining the benefits of a Network Attached Device.
So I bit the bullet and went with their 8 Bay NAS, and eight 10 TB HDDs. I’m running a RAID 10 (Raid 0 plus Raid 1 Redundancy) which allows for maximum speed with full striped backup that doesn’t take as long to rebuild if there are any errors (at the expense of losing half of your data pool storage) This has left me with 40 TBs of storage (28ish once my archive has been added) which should give me enough storage for 5-6 years. I also get around 900-1000 MBs write speeds, 2-3x that of SSD, and all of my computers can access all of my archive at the same time (as well as have access to it it all remotely outside of my home)
All in all - I’m pretty stoked of my setup and performance, not to mention that I can now have access to all of my photography over the years in one master catalog, or easily switch between yearly catalogs. I can also schedule a backup to external HDDs, which I still do to keep a set off site.
Phew… ok sorry that wasn’t that short, but there is more if you want to know lol.
WOW.
WOW again.
I am really glad you posted this, because I will have to read it several more times to fully absorb it. You packed a LOT of information in there.
I don’t have nearly the amount of data that you have. I think I have about 4-5 TB total, but it has been increasing almost exponentially in the last couple years.
The reason I am not sure exactly how much data I have is because the old data; all my photos from 2000 (when I started using digital images) to 2009 are on one set of external drives, and the newer stuff is on a different set of drives, and I don’t have it all cleaned up yet.
Some of my best photographs are in the old stuff, although they were done with a Minolta and a Canon 20d, and are pretty limited in resolution.
Thanks for all the information. I will have to go back through it SEVERAL times!!!
I tell ya Fritz, after doing individual drives and manually backing up/transporting ect for a couple of years, it is a BLESSING to have a large central storage array and be able to access/search any and all of my data. It was just becoming such a hassle to find which hardrive I needed or dealing with plugging multiple drives via USB3 ect. (Not to mention the speeds you get with an array with TB3)
If you can afford to invest into your storage, it’s well worth it just for his fact.
I do use cloud storage. Backblaze is a reasonable choice. I used to keep an offsite HD at work, but now that I am retired I have changed that to Backblaze. San Diego has ample fast internet fortunately. Other than that, I keep my images on a 10 TB external thunderbolt drive that backs up to a RAID 1 drive. I also use Chronosync to keep them backed up and in sync.