Long-term storage (archiving) requires a very different approach to backup and recovery, where throughput and deduplication are primary concerns. Archiving requires storing data for long periods of time without being corrupted, so when retrieved it is exactly what was stored 10 or 20 years ago.
For most organizations that grow to a certain size, standard open linear tape (LTO) magnetic tape is the best choice. But for those who can’t justify the cost or think tape is a thing of the past, there are three viable alternatives: cloud object storage, on-premises disk storage, and optical media.
Object storage is specifically designed for long-term storage because the checksums used to identify each object can also be used to verify that its contents have not changed. The system can rerun the checksum and compare it to the previous checksum used as the unique identifier (UID) of the object. This allows constant verification of data integrity, even decades after storage.
It’s also inexpensive. AWS, Azure, and Google will all store 100TB of data in cold storage for around $100/month, so you can have two copies in two storage providers for around $200/month. Assuming you only download data and never retrieve it, it would be very profitable. If you ever need to get it back, it will be expensive, but it’s probably worth it. Note that you will pay two fees: a get fee for each object and a bandwidth fee per gigabyte. If you delete the archive earlier, you may also pay other charges as the pricing is based on storing data for a long time. Make sure you know what you will pay if and when you actually recover the data.
On-premises disk drives
If you plan to use on-premises disk for the archive, you have three choices: standard disk arrays, deduplicated disk targets, and on-premises object storage.
Standard disk arrays and network attached storage systems tend to be designed for primary data and have a higher cost that reflects this design. Therefore, most environments consider them too expensive for long-term storage.
Deduplicated disk systems can help reduce this cost for backups, but long-term storage is something quite different. Over time, you will end up storing entirely new data for the deduplication system and the deduplication rate will drop. Therefore, the cost of a target disk system will end up being more than a standard disk system because you are paying extra for a deduplication rate that you don’t actually get due to the nature of the data .
Standard and deduplicated disk systems also have the problem that the disk is not very good at retaining data for more than five years; after that he will suffer corruption – rot. Much like cloud object storage, on-premises object storage systems can help by constantly checking data integrity on each system and replacing corrupt data with good data stored in a different disk system. . However, disks still tend to be more expensive than just using cold cloud storage. In short, any type of disc is generally not the best choice for archiving.
There are three optical disc options: standard DVD and Blu-Ray recordable discs, archive quality DVD-R and M-disc.
DVD and Blu-Ray discs use an organic matrix that changes phase when struck by a laser. Data written to DVD and Blu-Ray should be good for decades. Archive-quality DVD-Rs use double reflective layers and an extra-hard coating to prevent scratches, and they’re 10 times more expensive than DVD-Rs. M-disc is specially designed for long term storage and uses an inorganic layer designed to last 100 years. They are expensive, but are about 25% of the cost of an archive-quality DVD-R per gigabyte. Most modern optical drives can write to all three types of media.
DVD discs have a very small capacity—less than 5 GB. Blu-ray discs have a capacity of 25 GB or 50 GB with a double-sided disc. M-disc is available in 25 GB, 50 GB and 100 GB. Storing data on these devices is quite slow compared to tape or disk because the phase change process on physical media is slow.
It is also important to point out that optical media also have a very low uncorrected bit error rate (UBER), some as low as 10-8most at 10-ten. For Serial Advanced Technology Attachment (SATA) disk, the UBER is 10-14 and for LTO-9 tape is 10-19. This means that optical media does not write data as reliably as SATA disk or tape.
I’ve spoken to media and entertainment companies that use them to make long-term storage copies of movies, in addition to their LTO copies. Their thinking is that the Blu-Ray disc will be easier to read in 50 years than an LTO tape. One thing that goes in that direction is that the most modern Blu-Ray device can read the oldest CDs and DVDs, which means it’s much more backwards compatible than the typical tape drive, which usually only reads a generation or two back.
Companies wishing to use optical as long-term storage should look to optical libraries. They look like tape libraries, but with optical drives instead of tape drives, and provide a nearly unlimited amount of long-term storage without too much manual intervention.
Archives store data that you will keep for decades, so take the time to do your research. Know what you’re getting into with each option and make an informed choice.
Copyright © 2022 IDG Communications, Inc.