In recent blogs, we have touched on backup and archive strategies utilising Linear Tape Open (LTO), or a combination of LTO with on-premise disk and public cloud. We have also talked about the current generation of LTO, LTO-9, migration strategies for LTO, and the LTO roadmap going forward and how this compares to hard drives. In this blog, the aim is to expand on where the various digital storage technologies such as hard drives (HDDs) and LTO compete with each other and the future of the various technologies.
The recent pandemic years of 2020 and 2021 saw a growing demand for digital storage, especially HDDs driven by data centre expansion and client computing to support remote working. LTO also did extremely well in 2021 growing 40% over the previous year with 148 Exabytes (EB) shipped, versus 114 EB in 2019 and 105 EB in 2020 (note these figures are for compressed capacities, but still impressive).
2022 for HDDs and SSDs saw a slowdown, especially in the second half of that year, as economic uncertainty increased, demand dropped, which led to significant oversupply in the market. Trendfocus, Inc. released preliminary results for CQ4 2022 showing HDD shipments decreased between 11.5 and 12.5 million units or around 25% quarter on quarter. That said, SSD is now dominating in many areas that were traditionally the preserve of the hard disk, such as laptops and desktops, portable storage, and high-performance storage solutions which are becoming increasingly dominated by SSD media. This has relegated the HDD to higher capacity nearline storage applications where the price per TB for SSD is too high. The increasing use of SSD media is contributing significantly to the decreasing sales of HDDs, particularly in conjunction with the global economic slowdown that occurred in the latter half of 2022.
2022 saw the release of 20TB HDDs from Seagate, Toshiba, and WDC, with WDC going one better and releasing a 22TB HDD. Let’s also not forget that LTO-9 was released in 2021, admittedly a year late and with not quite the promised capacity, with 18TB native and 45TB compressed per cartridge. What is driving these very large capacities both on HDD and LTO? In a nutshell, it is largely the cloud hyperscalers, who are demanding, along with other countless data centres, ever higher capacities on both HDD and LTO.
This demand for ever-higher capacities has caused problems for the HDD vendors and the problem is density. Consider that an 18TB hard drive has a density of 1022 Gbit/sq inch versus the LTO-9 cartridge which has a density of only 12Gbit/sq inch. Put simply, this means that LTO-9 tape can achieve the same capacity with only 1/85th of the areal density as that of the same capacity disk. This is why it is possible for LTO technology to keep increasing capacity at historical rates, with capacity doubling every generation (see LTO roadmap).
The latest LTO roadmap features LTO-14 with a native capacity of 576TB per data cartridge, which you can expect to be available in the mid 2030s, assuming a 3-year gap between generations, with LTO-10 rumoured to be released in the latter half of 2023. These higher capacities have already been demonstrated in a lab, with IBM and Fujifilm demonstrating 220TB on a tape as early as April 2015, using barium ferrite (BaFe) technology. Then in December 2017, IBM and Sony achieved a 330TB native tape. Not to be outdone, Fujifilm resumed its partnership with IBM and in 2020 achieved a capacity of 580TB a density of 317Gbit/sq inch (remember an 18TB HDD has a density of 1022 Gbit/sq).
The road to 576TB LTO tape is unlikely to be without its problems. LTO-8 ended the backwards read compatibility of two generations of LTO, due to changes in the head technology. There was a problem with media supply due to the patent dispute between Sony and Fujifilm that disrupted the supply of LTO-8 media in the early days of that generation. This led to the rather ill-conceived LTO-8M technology stopgap, the less said the better (in our opinion). Currently, LTO-8 and LTO-9 media uses BaFe technology, but it is likely that future LTO media will use new magnetic particles, such as epsilon ferric oxide (ε-Fe2O3) the most likely candidate for LTO-10, and then further down the line, Strontium Ferrite (SrFe) which was used to achieve that 580 TB on a tape in 2020. Changes in the magnetic particles used will have a knock-on effect on the new generation of drive heads and vice versa, which will inevitably lead to changes and restrictions in compatibility between generations. The low cost per TB to store data on LTO, combined with the fact that Fujifilm estimates that the carbon emissions from LTO is 43% less compared to HDDs, could beg the question: Do these speculated changes to compatibility between generations even matter?
As we have just seen, areal density on media does not affect LTO, but it certainly affects HDDs. The current 20TB and 22TB HDDs use Conventional Magnetic Recording (CMR), utilising perpendicular magnetic recording on the disk platters, as their forebears have done for a decade or more. But they are very likely to be the last of their kind. To gain higher track densities on the disk platters, manufacturers are turning to Shingling Magnetic Recording (SMR) and Energy-Assisted Magnetic Recording (EAMR) technologies. Shingling offers a relatively easy capacity gain of about 10% by overlaying tracks on the disks, like roof shingles or tiles(hence the name), the downside of this is that the process to write the tracks is more complicated and therefore slower, especially if data has to be changed or deleted. To replace a single shingle (tile) on a roof, many other shingles have to be removed to change the desired shingle, and the principle is the same with shingled HDD technology. So SMR hard drives are traditionally more suited to data sets that are changed infrequently, but frequently read. Indeed, the hyperscalers and data centre customers have in place software strategies to take advantage of these higher capacity drives.
WDC introduced, what they call, ultra-SMR drives in 2022, which feature their OptiNAND technology. Essentially this is an additional NAND flash layer incorporated into the drive and used for key drive housekeeping functions that can take advantage of the speed of the NAND flash to store more metadata which enhances the performance of the drive along with freeing up capacity on the disk itself that would otherwise be taken up by the metadata. This means the same drive heads and media used in the WDC 22TB CMR HDD can be stretched to provide 26TB in the ultra-SMR format.
Seagate has SMR HDDs as well, but Seagate has also bet big on Heat Assisted Magnetic Recording (HAMR), which is a form of energy-assisted magnetic recording to build the next generations of high-capacity hard drives. HAMR technology allows for even high densities on the disk platters without using shingling. This is where looking at WDC and Seagate, the digital storage strategy differs for the two companies, with WDC much more heavily invested in NAND Flash. Although to be fair to WDC, they pioneered Helium-filled drives in 2013 and announced their energy-assisted offering ePMR back in 2020. But it is Seagate, with their implementation of HAMR technology, that looks set to break free from the 1022 Gbit/sq inch density shackles and ship 30TB+ HDDs in 2023. Indeed, Seagate is confident in its ability to deliver a 50TB hard drive by 2026.
At Symply we are also fans of another Seagate hard drive technology, called MACH.2, which we use in our products. These drives feature dual independent actuators that can transfer data to storage systems concurrently, essentially doubling the performance of a standard single actuator drive, providing up to 480MB/sec of sustained throughput. This is on par with SATA SSDs, but at a much lower price per TB than SSD. Currently available in 14TB, 16TB and 18TB capacities, it will be interesting to see how HAMR and MACH.2 technologies converge to solve one of the key problems of higher-density hard drives… the reduction of performance as destiny increases. By enabling the storage system to request and receive data from two areas of the drive in parallel, MACH.2 doubles the IOPS performance of each individual hard drive, more than offsetting any issues of reduced data availability that would otherwise arise with higher capacities.
Toshiba, who along with WDC and Seagate make up the three manufacturers of HDDs, are pursuing their own energy-assisted magnetic recording technology, microwave-assisted magnetic recording (MAMR). Toshiba is targeting a 30TB HDD by 2024, with a plan to have HAMR HDDs available in 2026 with capacities in the range of 40TB. Industry associations, such as the Advanced Storage Research Consortium, an HDD industry association, believe that a 200TB HDD should be possible by the mid-2030s. Interestingly around the same time that the LTO consortium might be shipping a 576TB LTO tape.
So where does that all leave us? As the world’s demand for data will continue to explode, the IDC projects a cumulative annual growth rate of 23%. A Seagate-commissioned report (again from IDC) put the amount of digital data in the world in 2025 at 163 ZB. Of course, it is not all going to be stored long term, but IDC predicts that a capacity of 11.1 ZB will have to be stored in 2025 and 60% to 80% of that data will be cold or archival data.
It is clear that NAND Flash technology is going to continue to eat away at hard drive sales. In the media and entertainment industries most of the portable HDD-based single and dual-drive systems, and mobile hard drives have been replaced by SSD. Undoubtedly, how to store that 11.1 ZB of data is going to be down to HDDs and LTOs. HDD shipments in terms of capacity are inevitably going to be picking up again after the current economic uncertainty passes. What is certain is that with the HDD vendors pushing the 30TB+ capacities, it is unlikely that NAND Flash devices are going to get near the price per TB of HDDs, so they will remain the preserve of high-performance storage. As the preferred nearline digital storage solution, the HDD solidifies its position not just among hyperscalers and data centres, but also among establishments seeking to store their cold or archived data for 5 to 7 years, with the added convenience of swift and effortless access. Although LTO sales have increased in terms of capacity, the hard drive remains far ahead in terms of scale, with disk capacity in 2021 reaching 1,269EB compared to tape capacity of 148EBIt will be interesting to see towards the end of the decade when tape capacity could be double that of HDDs, what the balance between HDDs and LTOs will be, given the additional benefits of data security, media longevity, cost per TB and a significantly lower carbon footprint.
