There are multiple media options for storing archival data, and each has its strengths and weaknesses. For shorter term archive periods, magnetic disk arrays may provide an appropriate hosting platform. Serial ATA arrays can leverage the characteristics of their component disk drives to provide considerable capacity at an inexpensive per GB price. But is magnetic disk appropriate for longer term archives?

At least one vendor is insisting that it is. In late 2006, EMC Corporation initiated a marketing and sales initiative called “Shatter the Platter” that was intended to convince consumers to replace other forms of archival media such as magnetic tape and magnetic/optical disc with EMC-branded disk arrays such as Centera. The arguments of EMC and other advocates of disk-as-archive solutions sound compelling on their face, but they run afoul of a growing body of evidence about the questionable durability of SATA disk drives and arrays composed of them, as well as any rational calculus of associated energy and management costs.

According to studies authored in 2007 by Google and by Carnegie Mellon University, the annual failure rates (AFRs) of SATA drives are well above manufacturer-specified statistics. Increased frequencies of drive failure suggest that magnetic disk arrays, if used to host archival data, may place the data assets at greater risk than previously thought.

Higher than specified disk failure rates impose the need to protect data by introducing RAID (redundant array of independent disks) technology for use in rebuilding data if and when a drive in a RAID set fails. RAID is a set of schemes formulated by researchers at the University of California Berkeley in 1977 for grouping disk drives into sets with data protection provided by mirroring or striping data across multiple disks in the set.

The most popular RAID technology at present is RAID 5, but its efficacy is generally believed to be waning in the face of SATA drive capacities that are accelerating beyond 1 Terabyte per disk platter. At such capacities, RAID set recovery is a very protracted and time consuming undertaking should any member drive fail. To address this problem, most major array vendors are moving away from RAID 5 and implementing more proprietary RAID schemes – ostensibly for improved data protection.

However, the more proprietary the RAID scheme, the more limited the consumer’s choices in the design and implementation of archival storage infrastructure. RAID n technologies are too often being leveraged by array vendors to lock in consumers to expensive disk platform products, while locking out competitors.

Case in point: an individual 1 TB disk, when these are introduced to the market in late 2007, will carry a street price of approximately $750. By contrast, a proprietary RAID array from a name brand manufacturer offering 1 Terabyte of usable storage capacity and featuring a proprietary RAID scheme typically costs between $53 and $83 thousand. At such prices, the costs of archiving using this media would be prohibitive.

Add to the cost of disk-based archive, the price for electrical power both to power disk drives continuously and to cope with the heat generated by disk arrays in the data center or equipment room. Dell Computer Corporation, for example, recently disclosed that 37 percent of the electrical power used by its data center is consumed by data storage components. It is anticipated that the power requirements of storage would shortly exceed the power consumption of servers, which now account for 40 percent of the total power consumed by the Dell data center.

Using always-on disk drive arrays to store long term archival data also carries a heat management burden: the more arrays you field, the more power they will consume and the more heat they will generate. So, in addition to powering drive arrays themselves, organizations may well confront the requirement to purchase additional power to meet increased heating ventilation and air conditioning (HVAC) requirements imposed by additional disk array hardware.

With power in increasingly short supply in key metropolitan areas of the US and Europe, with per Megawatt prices on the climb for utility power services, and with mounting concerns about the “carbon footprint” associated with power generation in many parts of the world and its impact on climate change, the costs associated with archiving exclusively on magnetic disks can be daunting – both in the short and longer term.

In addition to data protection and power issues, the efficacy of long term archive on disk arrays also imposes management burdens. Given failure rates in disk drive repositories, a practical requirement for repository management will be a program of data migration between disk arrays about every 24 months.

Disk is an acceptable target for short term data archives, but on a cost basis, tape or magnetic/optical disc might provide more bang for the buck, especially for longer term archives. “Shatter the platter” and similar initiatives should be viewed for what they really are: the triumph of marketecture over architecture.

Copyright (C) 2007 by Jon William Toigo for the AMO Journal.  All Rights Reserved.