A handful of storage start-ups are developing software that will let users guarantee the integrity, authenticity and rapid accessibility of large amounts of data at a fraction of the cost of traditional storage arrays.
Cluster File Systems Inc., Panasas Inc., Permabit Inc. and Reference Information Systems Inc. (RIS) are among the vendors creating software that runs on industry-standard server clusters, and uses objects to track and store content-addressable and other data, and make chunks of it readily available.
All four are focusing on hash-based or object-oriented storage, in which data is stored based on its contents rather than its location or a specific data block.
In object-oriented storage, each piece of data is represented as an object and automatically is assigned a unique digital identifier or fingerprint, which is used to retrieve it. The fingerprint, which a hash algorithm creates, is used not only to retrieve the object, irrespective of its location and placement on tape or spinning disk, but also to maintain the integrity of the data against changes or deletion. Often called metadata, the fingerprint tracks the data's location, so that it can be retrieved quickly and so that related data objects, such as X-rays and test results for a patient, can be correlated and retrieved coherently.
For instance, multiple X-rays and diagnostic test results for a patient could be stored as individual objects, which could be retrieved together by searching on the employee's name or Social Security number.
Other companies also are going after this market for quickly accessing primary or secondary data. EMC Corp.'s hardware and software-based Centera system focuses on storing what is called content-addressable storage - data that doesn't change and needs to be retained for a long period of time. Network Appliance Inc.'s NearStore appliance and SnapLock software store data in regulation-compliant mode. Start-up Avamar focuses on backing up and retrieving object-based data. And start-up Persist Technologies Inc.'s AppStor software is designed to archive object-tagged e-mail records.
The market is ripe, analysts say.
"There's a push that would suggest that moving out of content-addressable storage to object-based storage is a more broad-scale play," says Steve Kenniston, a senior analyst with Enterprise Storage Group Inc. "End users want to have object-oriented solutions that can focus not only on applications like e-mail or backup, but also CAD files, diagnostic images, etc."
Permabit's Permeon software is installed on a cluster of industry-standard Intel servers equipped with as much as 40 terabytes of storage. The cluster communicates with Permeon portal software, which runs native on application servers in the network or on blade-server gateway computers positioned so they provide access to application servers.
The portal software then breaks data into information objects and uses TCP/IP to communicate it to the cluster. The servers in the cluster route, store and replicate the data blocks as needed. Users can see the data via the Common Information File System or Network File System interface used in network-attached and file server-based storage.
Bob McKie, systems manager for the Surgical Planning Laboratory at Brigham and Women's Hospital in Boston, is implementing Permabit's clustered software to protect his secondary storage, which is made up of diagnostic images, each of which could be as large as 100 megabytes.
"We have two portals and five storage units for storing two terabytes of storage," McKie says. "We are looking at Permabit for secondary, modular storage to protect our primary storage. The primary data is such that it needs to be kept for a while, and since research results are based on it, it needs to be safe and be possible to reproduce if it's accidentally deleted from primary storage."
McKie says he put in Permabit to protect his repository of secondary medical-image data after Brigham and Women's suffered water damage from a burst pipe at its primary storage.
In Cluster File Systems' open source Lustre File System, the clustered file system is separated from the actual storage of the object. Client software communicates with a Linux cluster of metadata servers and Linux storage servers called object storage targets, which are attached to traditional storage arrays. The metadata servers, which manage and create files, communicate with the object storage targets, which in turn communicate with client computers and the physical storage devices to access data.
Sources say that there are other companies, including Panasas and RIS, that will introduce products in the next few months. Panasas is building Linux clusters and an object-oriented file system in which file data moves from centrally shared storage onto local disks, where it is then divided among the compute nodes for processing and finally sent back to shared storage. RIS is building cluster-based archives for digital data. Panasas and RIS declined to comment.
Industry groups such as the Storage Networking Industry Association (SNIA) and the American National Standards Institute (ANSI) also say they are hoping to capitalize on objectoriented storage. SNIA has formed a committee to investigate the technology; it is working on a specification with the Object-Based Storage Device working group within the ANSI's T10 SCSI Storage Interfaces Group. EMC, Hewlett-Packard Co., IBM Corp., Intel Corp., Seagate Technology LLC and Veritas Software Corp. are members of this group.
Permabit says its product, while similar to EMC's Centera, Hitachi Ltd.'s Open LDEV Guard and Network Appliance's Snaplock software, is less expensive and more reliable. Permeon will be available this month starting at US$40,000 for a two-terabyte configuration. By contrast, EMC's Centera starts at US$205,000 for five terabytes.
Cluster File Systems' Lustre is being used by DataDirect Networks Inc., HP, Lawrence Livermore National Laboratory and the Pacific Northwest National Laboratory, the company says. It is expected to be available late this year at no charge from www.lustre.org.