Cisco’s done it again. Packed with 112 ports of 2Gbps Fibre Channel, the latest version of Cisco’s MDS 9509 delivers a feature set, management interface and performance that earned it our Clear Choice designation.
We first viewed this Cisco storage-area network switch early last year. The latest software (Version 1.3) supports new quality-of-service (QoS) traffic classes and routing between virtual SAN (VSAN) groups. Additionally, new optional modules deliver storage virtualisation and caching capabilities. The switch hardware base is the same and earned it a near-perfect performance score, although this rating dipped a bit because this round of testing was more extensive and a tad more critical.
The 9509 remains a top performer in our high-end SAN switch tests. Cisco showed up, and all the other SAN switch marketplace leaders — including Brocade Communications Systems and McData — stayed at home, in spite of our invitation to compete head-to-head with Cisco.
Brocade and McData hinted at major new director-class architectures due out in the coming months, and we’ll test them when they deliver their new versions.
The 9509 supports an array of interface modules. Up to seven hot-swappable line cards can be any mixture of 16- or 32-port, 2Gbps Fibre Channel Switching Modules. Then there’s an eight-port Gigabit Ethernet IP Storage Module, which lets users directly integrate popular storage-over-IP connections with the Fibre Channel fabric. The module supports both iSCSI and Fibre Channel-over-IP links. This connectivity and conversion was not verified in the testing.
The Cisco switch also delivers the survivability users expect at the core of their SAN fabric. Each 9509 ships with redundant, hot-swappable management/fabric-control cards, called supervisors, and redundant power supplies.
There’s nothing quite like a good command-line interface (CLI) to manage a Cisco network device, unless there is an even better GUI. The 9509 has both. The CLI has the standard Cisco IOS look and feel. And the GUI delivers effective central management, featuring dynamic topology mapping.
The Cisco Fabric Manager GUI is impressive. Extensive configuration capabilities are accessible, which is helpful because these capabilities can seem imposing to a first-time user. The main GUI screen offers a directory tree on the left side for selecting the management topic, an auto-discovered fabric topology map on the right. Multiple tables for configuration and statistics are accessed through tabs at the top.
Most impressive is the copy-and-paste configuration, which lets the user select any configured switch and apply all the same settings to any other switch. Locating particular devices or links also has been simplified: If the IP address of a switch or label of an inter-switch link (ISL) is not enough, you can select the component you want from a configuration table, and its image is highlighted instantly in the fabric topology map.
The Fabric Manager also can readily push new software images out onto one or a group of switches. And we confirmed that new code could be loaded and activated under full operational load — without dropping a bit.
The 9509 brings a smorgasbord of features to the table.
Consider the capabilities offered for Fibre Channel diagnostics. The 9509 includes a built-in protocol analyser, driven from the CLI, for control traffic, which is very effective for diagnosing Fibre Channel issues.
Cisco also supports a mirrored-port capability to which frames between any two ports in the fabric can be replicated, without disrupting ongoing traffic. Fibre Channel frames can be encapsulated into Ethernet frames, using the Cisco Port Analyzer Adapter, and captured in ‘libpcap’ format — a popular format for storing packet traffic. The resulting dump can be analysed within Ethereal, a popular open source analyser application, for which Cisco has developed a Fibre Channel decode plug-in.
Cisco also offers its proprietary storage equivalent to virtual LANs (VLAN) — VSANs.
VSANs separate groups of ports into discrete “virtual fabrics”, up to 1000 per switch. This isolates each VSAN group from the disruptive effects of fabric reconvergence that may occur in another VSAN. And, as with VLANs, routing is used to forward frames between initiator and target (SAN source and destination) pairs in different VSANs. Cisco has integrated VLANs and VSANs effectively: The IP Storage Services Module, which extends the SAN fabric into an IP network, can map 802.11q VLAN tags to VSAN identifiers.
Cisco also offers an effective QoS solution that uses a traffic-distribution algorithm and four output queues. Three queues are assignable by the user for prioritising traffic, while the fourth queue is reserved for Fibre Channel control traffic.
Storage virtualisation is a buzzword in the SAN industry that implies storage volume management, mirroring and replication across physical locations, which is transparent to users and applications. Cisco offers two specialised module options that support these virtualisation functions: The Advanced Services Module, produced jointly with Veritas, and the Caching Services Module (CSM), co-developed with IBM.
Users seeking as close to wire-speed performance as they can get, under maximum load on all ports, will want to use only the 16-port modules in the 9509. That’s because the 32-port modules introduce over-subscription — a SAN euphemism for bottlenecks, a condition Cisco documents.
We ran the switch through both torturous and more typically realistic tests, all at 100 per cent offered load. It performed nearly flawlessly — that is, delivering theoretical maximum throughput except for a few worst-case load scenarios. For example, in the full-mesh test with a very small frame size — an absolutely worst-case scenario — the switch dropped to 54 per cent of theoretical line rate. However, through every test, even with congestion, the switch maintained fair and evenly distributed throughput. We noted too that, in the absence of congestion, latency — the time it takes frames to move through the switch — ranged from 10 to 250 microseconds (ms), depending on frame size. This is a normal and acceptable range, given variable-length frames traversing one or more modules and the internal switching fabric.
The 9509 also has a link-aggregation feature. We built a “port channel”, Cisco’s term for a group of aggregated ISLs connecting two 9509s, and we saw no degradation in throughput across the aggregated switch-to-switch trunk links, compared to the same load sent between ports on one switch. When we failed one of the ISLs in a trunk group, the switches dutifully reallocated streams from the failed link to the others in the group. The total time for this reconvergence, where throughput on affected streams is temporarily halted but no data was lost, was 115 milliseconds.
To abuse the switch, we pulled the active supervisor module and tried upgrading the software with all 112 ports transferring SAN traffic over 12,432 unique flows. Neither condition had any degrading effect on throughput performance because of the failover redundancy of the two supervisors. Boot time after a power failure was a very respectable two minutes, 32 seconds.
Cisco also provides an abundance of security features for its management and the SAN fabric. With the use of a RADIUS or Tacacs+ authentication servers, administrators can be assigned very tailorable access and configuration rights.
Additionally, IP-based Access Control Lists can be applied to management access, whether the administrator is accessing via an Ethernet management interface (out-of-band) or from another switch using IP over Fibre Channel (in-band).
What’s more, all management traffic is encrypted — using SNMPv3 for the GUI, Secure Shell for the CLI and secure file transfer for moving files to and from the supervisor.
The SAN fabric itself is secured through hardware-enforced zoning, which is performed at ingress, read-only zones, fixed port types and device authentication via the Fibre Channel Security Protocol.
The 9505 is a powerful director-class SAN switch that sets a high bar for the industry in terms of features and management. While we can’t call it perfect, we can say it’s the one the competition has to beat.
How we did it
Spirent Communications provided all the performance testing equipment we used. We employed five SmartBits SMB-6000B chassis, fully populated with FBC-3602A 1G and 2Gbps Fibre Channel modules. Spirent’s SmartFabric test application, Version 1.31, provided port-by-port results.
Cisco submitted an MDS 9509 switch populated with two DS-X9530-SF1-K9 Supervisor 1 cards running firmware Version 1.3(3) and seven DS-X9016, 16-port 1G and 2Gbps Fibre Channel cards.
All tests were run for 30 seconds using small (60-byte) and large (2148-byte) frames at 100 pe cent load and were repeated several times to note variability. Latency was measured while applying less than maximum loads.
With the high-stress, full-mesh throughput test, we configured the 112 SmartBits ports to send frames to, and receive from, each other port.
In the reboot, we cut off and restored power to see how quickly it could resume normal operation.
To test non-disruptive code load, we did a full-mesh test of large frames across all 112 ports. A code load sequence was initiated and completed while the traffic flows continued.
A fabric failure was simulated during a full-mesh test by removing the active Supervisor module while traffic was passing.
The product is distributed in Australia by Express Data, IT Wholesale, and Lynx Technologies.
RRP: The price per port for a fabric switch is $1000. The price per port for a director class is $1500
Top 3 Cisco MDS 9509 features
Cisco’s business development manager for A/NZ, Dylan Morison, said resellers could pitch three top-selling features including multi-protocol; diagnostics and virtual SAN functionality.
“With the multi-protocol, it does Fibre Channel; iSCSI; and Fibre Channel-over IP in the single box,” Morison said.
With this functionality, customers and resellers can choose the protocol, thereby customising the design of the SAN to help lower the total cost of the solution.
“This gives them the capability to pick the right protocol for the right application,” he said.
With diagnostics, Morison said resellers could help ease the pain of congestion within a SAN, offering tools such as a Fibre Channel Pin and an in-built Fibre Channel analyser.
The virtual SAN functionality gave resellers the capability to use a single infrastructure to logically partition the technology, he said.
“With this, the evolution of the data centre — and consolidation of storage — is a good thing, offering a TCO and simplification,” Morison said.