Fibre channel storage means business

In 1997 we saw average storage capacities double on x86 servers, and the forecast for '98 is no less bullish. In business applications, growth in the number of users of information sharing applications such as Lotus Notes, e-mail, Web and datamart servers are one driver of growth. Another driver is the average file size or database size that these applications store out on the server.

The ease of adding graphical, animated and sound content means that even mail messages are now adorned with pictures, driving up storage needs. For knowledge workers, these are critical applications with stringent uptime requirements.

Traditional mission-critical applications migrating from mainframe environments to client server networks also add to the data growth and strain MIS staff's ability to deliver mainframe-class availability, management and backup.

To address these needs, client/server storage vendors are rallying around a new high-speed interconnect standard called fibre channel.

At last count, over two-thirds of storage vendors had fibre channel-based products in their product pipelines. SCSI, the dominant storage interface today, will continue to be useful for many applications and offer the best price and performance.

Apples to apples price comparisons between SCSI and fibre channel are complicated by the fact that as vendors introduce fibre-based products they often add new features and new components that don't exist in the SCSI world, such as optical-electrical converters and fibre hubs. Some of the electronics involved in fibre channel will cost more due to more features, newness of the technology and lower volumes than equivalent SCSI components. Obviously vendors that are already charging near the high end of the market in terms of dollars per MB may introduce fibre with minimal price premium to their already high-priced SCSI offerings. Vendors that price closer to their real costs in the low end or middle of the market will likely charge a premium correlated with their increased costs. Fibre channel-based storage systems are expected to initially be used where SCSI-based systems leave off, and as volume increases and cost reductions occur, they will become as ubiquitous as SCSI.

Fibre channel, an output of the ANSI X3T11 committee, was conceived of as a next generation transport mechanism that could support the multiple protocols that exist today, including SCSI, Internet Protocol (IP), HIPPI, and IPI. The five layers of the fibre channel standard define a physical layer, encode/decode scheme, framing protocol, common services and include channel and network protocols.

Fibre channel uses the same physical media as Gigabit Ethernet -- offering the options of fibre optic or copper connections. Both fibre channel and Gigabit Ethernet offer 100Mbps transfer speed. However the two interconnects require different electronics (they do not share hubs, switches or controllers) and are designed for different purposes -- one for a communications network, and one for a low-latency storage network. Thus Gigabit Ethernet-based communications networks are expected to remain distinct from fibre channel storage networks.

Fibre-optic connections allow distances of 500m with multi-mode fibre and up to 10km with single-mode fibre. Industry standard connectors, such as the electro-optic GBIC standard supported by Compaq, Sun, Vixel and others, ensure that devices from diverse vendors can be attached with the same hot-pluggable connector. By supporting these various physical media, storage users are no longer restricted to 4-metre cable lengths of single-ended SCSI. Mirrored storage configurations that place copies of data in different buildings also become possible without resorting to today's more expensive alternatives. Physical consolidation of storage also becomes easier, even if the servers are located at different corners of a data centre or building.

Another benefit of fibre channel is its topology independence. Possible topologies include point-to-point, crosspoint-switched and arbitrated loop. Topology selection is influenced by performance requirements, packaging options, and capacity/scalability requirements. Storage solutions are expected to implement primarily FC-AL (fibre channel -- arbitrated loop) topologies, then add switched fabrics as bandwidth requirements grow and fibre channel switches are offered. Many FC-AL implementations, such as Compaq's fibre channel Storage System, consist of a loop expansion module or hub between servers and storage devices that allow star cabling configurations while maintaining a logical loop between the connected devices. The FC-AL topology allows up to 127 devices to be attached to a single loop, but actual implementations will attach fewer devices due to slot limitations and the need to maintain high performance.

Communication between these devices may occur using any of the above mentioned protocols, but due to the great number of existing storage devices that utilise SCSI, the SCSI command set running on top of the fibre channel transport is expected to be used in many implementations. Devices connected to the loop may consist of disk drives, array controllers with disk drives behind them, tape backup devices, hubs, switches or servers.

This greatly expanded connectivity enables users to build storage area networks -- high- speed networks of inter-communicating storage devices. Storage backbones will be distinguished from communications backbones by their use of high-speed, low-latency protocols. Gigabit Ethernet implementations will continue to use CSMA/CD to arbitrate network usage among clients, and will not be optimised for the low- latency, high-reliability requirements of storage. The advantages of having a separate storage backbone are that your communications network will not be burdened by storage traffic and vice versa, and each backbone can be optimised for its specific purpose.

Fibre channel storage applications

Fibre channel is not inherently an application-specific technology. Like SCSI, it has the capability to cover a wide range of applications and is expected to be ubiquitous after a normal industry adoption period. Fibre channel opens up some new possibilities, however, that make it the best choice for particular environments and applications, justifying any cost premium that may exist in the near term. The discussion below will focus on general business applications rather than speciality applications such as video.

High capacity applications

Applications that are driving higher capacities include datamarts, groupware such as Lotus Notes, Web servers, e-mail servers, and database applications. Occurring in tandem with the implementation of these applications is the trend at larger companies to consolidate the hundreds of small server platforms distributed in workgroups, into larger servers deployed in departments or data centres.

Even basic file and print servers are affected -- the consolidation is driving some companies to build servers in the hundreds of gigabytes range that provide "file and print utilities" to thousands of clients.

Fibre channel storage systems provide specific benefits for these high-capacity applications. While most servers today may not be in the hundreds of gigabytes range, it's easy to see that many servers will be in that range within the lifecycle of the server's storage system, given the growth rates of today's applications and data stores.

Decision support applications, in particular, require storage systems with a new level of scalability due to their rapid accumulation of historical information and functions that allow users to store multiple versions of their analyses. An often quoted rule for datamart rollouts is "plan for three times the capacity you think you'll need".

Fibre channel's connectivity and higher speed enables 3+ terabyte configurations using 6 PCI slots and 9GB HDD with an industry standard x86 server, but allows the user to start with configurations of under 50GB. The result is that providing a path to multi-terabyte storage systems becomes simpler and more cost effective. No longer are over a dozen cables and server slots necessary to connect a server to terabytes of storage.

Some vendors have circumvented SCSI's limitations by building large external controllers that have many SCSI connections on the HDD side. In these architectures, the controller has the potential of becoming a bottleneck.

These vendors compensate for this by engineering large amounts of cache and processing power into the controller, increasing total system cost, and establishing a high initial purchase price.

Fibre channel eliminates the need for a large monolithic controller, allowing the use of more cost efficient modular and scalable components. Fibre channel HDD do not need to be used to achieve these capacities. Fibre channel between the host and storage subsystem is sufficient, protecting the customer's investment in SCSI HDD.


For the same reasons that fibre channel increases storage connectivity, server connectivity is improved. FC-AL hardware enables multiple servers to be attached to a storage network in high availability clusters. In the industry standard server world, hardware vendors and their software partners will be collaborating heavily to integrate their parts of the solution.

A strong initial focus is releasing fibre channel storage systems that work with Microsoft NT 4.0 Enterprise Edition's Cluster Server (also known as Wolfpack), which is a two-node failover cluster. Fibre channel's architecture allows multiple servers and storage devices to be attached to the same loop via a hub or switch. As Microsoft releases future versions of N-node clusters or users use N-node Unix clusters, this feature will be critical. It's important to note, however, that with every new O/S release, the task of integrating a highly complex O/S with a highly complex storage network has to be completed, requiring seamless partnerships between storage vendor, server vendor and O/S vendor.

Transaction environments

In Online Transaction Processing environments, disk I/O is often the most significant determinant of performance. Removing the disk subsystem as a bottleneck is done by adding more HDD spindles and the needed peripheral processing power. Fibre channel's connectivity makes it possible to attach many more disk drives and storage controllers to single servers or clusters of servers, scaling I/O performance beyond what was previously possible. For servers with limited peripheral slots, fibre channel maximises usage of those slots.

Migration issues

The fibre channel interface can be implemented as an interconnect between server and storage enclosure, and/or inside the storage enclosure connecting the hard drives.

In the next one to two years, the majority of fibre channel-based storage devices are expected to emphasise the server to storage interconnect over the HDD interconnect. This is due to the fact that all the benefits mentioned above can be achieved without using fibre channel HDD. Another big part of the reason is a desire to protect the customer's substantial investment in SCSI HDD.

Many fibre-attached storage subsystems will allow customers to move the HDD in their SCSI- attached storage subsystem to fibre attached systems, usually with a simple backup and restore of their data. (The backup is heartily recommended regardless of what the storage vendor claims). Lastly, SCSI HDDs are manufactured by multiple vendors, resulting in cost-effectiveness and availability.

In contrast, today there is only one manufacturer of fibre channel hard drives, and they are priced at a premium. Towards the end of 1998, more vendors are expected to begin offering fibre channel HDD, and prices are expected to come down in 1999 to the point where the fibre interface will become common at the HDD level. Fibre channel disk drives are not compatible with SCSI storage enclosures, and SCSI drives cannot be used with fibre channel backplanes and enclosures.


From a technical perspective, fibre channel delivers some exciting features: higher transfer speeds, greater connectivity, simpler cabling, and longer distance. However the user benefits that will result from building storage backbones is where fibre channel will really shine:

Scaling storage capacity and performance from a few gigabytes to terabytes without starting from scratch with new equipmentHigh-speed backups that transfer data directly between tape and disk over the storage backboneFlexibility to locate your hard disk subsystems, backup devices, and servers from anywhere, without giving up the benefits of centralised backup and management.

Jan Grassick is options manager at Compaq Australia and is a special guest writer this week in ARN.