Brocade Communications Systems FCX Series User manual

DATA CENTER

Configuring an iSCSI Storage Area

Network Using Brocade FCX Switches

A step-by-step guide describing how to install and configure Brocade FCX

Switches in an iSCSI Storage Area Network (SAN).

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 2

CONTENTS

Introduction........................................................................................................................................................................................................................................ 3

Getting Ready ................................................................................................................................................................................................................................... 4

Installation.......................................................................................................................................................................................................................................... 4

Cabling Infrastructure.................................................................................................................................................... 4

Installation Location ...................................................................................................................................................... 4

Rack Installation ............................................................................................................................................................ 5

Power Connection.......................................................................................................................................................... 7

Cabling............................................................................................................................................................................ 8

Connecting switches in a stack ............................................................................................................................. 8

Cabling requirements for redundancy.................................................................................................................10

Basic Switch Setup and Configuration...................................................................................................................................................................................11

Powering On the System..............................................................................................................................................11

Connecting to a PC or Terminal...................................................................................................................................11

Using the CLI ................................................................................................................................................................12

Configuring IP Addresses.............................................................................................................................................12

Assigning Passwords ...................................................................................................................................................13

Attaching to the LAN ....................................................................................................................................................14

Configuring the Stack ..................................................................................................................................................15

Configuring Dynamic Link Aggregation between switches.................................................................................16

Configuring the Switch for SAN.................................................................................................................................................................................................18

Network Loop Prevention ............................................................................................................................................18

Symmetric Flow Control...............................................................................................................................................18

Jumbo Frames .............................................................................................................................................................20

Rate Limiting ................................................................................................................................................................22

Switch Configuration Dump.......................................................................................................................................................................................................22

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 3

INTRODUCTION

This document describes the steps for configuring the Brocade FCX switch in an iSCSI SAN environment. In

this document, you will find recommendations and instructions for specific commands to optimize the

switch for an iSCSI SAN network.

Internet Small Computer Systems Interface (iSCSI) is an IP-based protocol that links data storage facilities

by ensuring reliable data transfers over long distances. It allows IT staff to utilize a single underlying

infrastructure (Ethernet) to meet IT and storage networking needs. The behavior of iSCSI traffic differs from

regular IP traffic. IP traffic is generally composed of packets ranging in different sizes, requiring various

amounts of bandwidth from the network. On the other hand, iSCSI traffic encapsulates SCSI traffic (SCSI is

commonly used in data centers to connect disks and tape drives) and sends large blocks of data in bursts.

Carrying block SCSI traffic places greater demands on the network compared to regular IP traffic. In order to

run high-performance SCSI traffic over an IP infrastructure, dedicated iSCSI networks are recommended to

ensure traffic flows freely and quickly between initiators (clients) and targets (iSCSI arrays) without

competition from general IP traffic.

In order to provide reliable operation in a mission-critical iSCSI SAN environment, an ideal switch must offer

the following switch functionality:

•Non-Blocking backplane design. A switch should be able to provide the same amount of backplane

bandwidth required to support full duplex communication on ALL ports simultaneously. The Brocade

FCX, with its non-blocking line-rate architecture, is an ideal switch for such environments.

•Support for Inter-Switch Linking (ISL) or Dedicated Stacking Architecture. ISL support is required to

link all switches in a SAN infrastructure together. For non-stacking switches, the switch should support

designating one or more (through Link Aggregation Groups) ports for inter-switch links. Brocade FCX

stand-alone switches provide up to 40Gbps full-duplex bandwidth for ISL. For stacking switches, the

use of stacking ports for ISL is assumed, and each stacking switch should provide at least 20Mbps full-

duplex bandwidth. The Brocade FCX offers up to 64Gbps of stacking bandwidth.

•Support for creating Link Aggregation Groups (LAG). For non-stacking switches, the ability to bind

multiple physical ports into a single logical link for use as an ISL is required. These switches should

support creating LAGs of at least 8x 1Gbps ports or 1x 10Gbps port. In a non-stacking configuration, a

Brocade FCX offers 8x 1Gbps ports or 4x 10Gbps for LAG.

•Support for active or passive Flow Control (802.3x) on ALL ports. Switches must be able to actively

manage “pause” frames received from hosts, or they must passively pass all “pause” frames through

to the target arrays. Brocade FCX switches actively respond to “pause” frames received from hosts and

also generate “pause” frames offering a backpressure mechanism when the receiving port (such as

initiator) is not able to process data as fast as it is being sent to it.

•Support for Rapid Spanning Tree Protocol (R-STP). For SAN infrastructures consisting of more than

two non-stacking switches, R-STP must be enabled on all ports used for ISLs. All non-ISL ports should

be marked as “edge” ports or they should be set to “fast port-span.”

•Support for Jumbo Frames. Many storage implementations are able to take advantage of Jumbo

Frames. Whether or not Jumbo frames are able to provide performance increases depends on the

application and data characteristics.

•Ability to disable Unicast Storm Control. In General, iSCSI can send packets in a very “bursty” profile

that many switches misdiagnose as a viral induced packet storm. Since the SAN should be isolated

from general Ethernet traffic, the viral possibilities are non-existent. Switches need to always pass

Ethernet packets regardless of bandwidth utilization. The default behavior of a Brocade FCX is such

that iSCSI packets sent in bursts will not be diagnosed as a packet storm.

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 4

•Adequate Buffer Space per switch port. Switch buffer mechanism also has an impact on the

performance of the switch. The Brocade FCX offers capabilities to optimize the buffers that will provide

the best possible performance in iSCSI SAN.

GETTING READY

Before you install and configure the Brocade FCX switch and create an iSCSI SAN, ensure you have the

following ready:

1. Download Brocade FCX switch documentation from https://kp.foundrynet.com.

a. If you are a first time visitor to the site, click the First Time Registration link to register.

b. Once logged-in, click Product Manuals, then FastIron.

2. A box cutter to unpack the device.

3. A #2 Phillips screw driver for rack installation.

4. Four #2 Phillips screws (per switch) to mount the device in the rack.

5. Acquire the IP address (or multiple addresses if configuring more than one switch), subnet mask, and

default gateway IP address from the networking group. The IP address will be assigned to the management

port on Brocade FCX switches.

6. Ethernet cable(s) with RJ-45 connecter that will connect to the management port on the Brocade FCX

switch. Only one cable is required per switch or one per stack in a Brocade FCX stack. The Ethernet cable

must be connected to a corporate or data center network that allows the administrator to manage the

switch remotely via telnet, web, or SNMP based management application.

INSTALLATION

Prior to installation, verify the contents of the shipping container. If any items are missing, contact the place

of purchase. Brocade FCX switches ship with the following:

•Brocade FCX switch

•115V AC power cable (for AC sourced devices)

•Rack mount brackets

•Warranty card

•A straight-through EIA or TIA DB-9 serial cable (F/F). The serial cable can be ordered separately from

Brocade.

NOTE: Hardware mounting screws are a customer-provided item and are not included in the rack mount kit.

Cabling Infrastructure

Ensure proper cabling is installed at the site. Refer to Brocade’s Best Practices Guide: Cabling the Data

Center (PN: GA-BP-036-01) for a summary of supported cabling types and their specifications. Brocade

recommends the use of Cat 6/Cat 6a cables to connect the Brocade FCX switch with the servers and

storage devices in an iSCSI SAN.

Installation Location

Before installing the switch, plan its location and orientation relative to other devices and equipment.

Switches can be mounted in a standard 19-inch equipment rack or on a flat surface. Be sure to follow the

guidelines below when choosing a location.

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 5

The site should meet the following requirements:

•Maintain temperatures within 0 to 40οC (32 to 104οF) and humidity levels within 5% to 95%, non-

condensing.

•Allow a minimum of 3 in. of space between the sides and the back of the device and walls or other

obstructions for proper air flow.

•Allow at least 3 in. of space at the front and back of the device for the twisted-pair, fiber-optic, and

power cabling.

•Allow accessibility for installing, cabling, and maintaining the devices.

•Allow the status LEDs to be clearly visible.

•Allow for twisted-pair Ethernet cables to always be routed away from power lines, fluorescent lighting

fixtures, and other sources of electrical interference, such as radios and transmitters.

•Allow for the unit to be connected to a separate grounded power outlet that provides 110 to 240 VAC,

50 to 60 Hz, is within 2 m (6.6 feet) of each device, and is powered from an independent circuit

breaker. As with any equipment, a filter or surge suppressor is recommended.

Rack Installation

As noted in the previous section, Brocade FCX switches can be installed on a desktop or in an equipment

rack. Due to the density of devices and associated power, cooling, and safety requirements, Brocade

recommends installing the switches in a rack. Follow the step-by-step instructions in this section for proper

rack installation.

Before mounting a switch in a rack, pay particular attention to the following factors:

•Temperature. Since the temperature within a rack assembly may be higher than the ambient room

temperature, ensure that the rack-environment temperature is within the specified operating

temperature range. For details, refer to the “Operating Environment” section in the Brocade FCX

Installation Guide.

•Mechanical loading. Do not place any equipment on top of a rack-mounted unit.

•Circuit overloading. Ensure that the supply circuit to the rack assembly is not overloaded.

•Grounding. Rack-mounted equipment must be properly grounded. Particular attention should be given

to supply connections rather than direct connections to the mains.

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 6

To mount a Brocade FCX in a rack:

1. Ensure you have a #2 Phillips screwdriver handy for the installation.

2. Remove the rack mount kit from the shipping carton. The kit contains two L-shaped mounting brackets and

mounting screws.

3. Attach the mounting brackets to the each side of the device using the provided screws.

4. Mount the device to the rack. Rack mount screws are not included in the rack mount kit.

5. If installing multiple switches, mount them in the rack, one below the other, and in any order.

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 7

Power Connection

A secondary power supply provides backup power in case of a failure and for load-balancing when both

power supplies are operational. Load-balancing gives the power supplies a longer life span. The Brocade

FCX PSUs are hot-swappable. In addition, power for each PSU should be routed from two independent

power sources to increase availability. The same power considerations should be made for the servers and

storage arrays.

BEST PRACTICE:Brocade highly recommends that redundant power supplies be installed in the switch. An

integral component of an effective SAN is reliability. A redundant power supply connected to a separate

source of power ensures no downtime in the event of a circuit failure.

The Brocade FCX has two power receptacles at the rear of the unit. Each device ships with one power

supply installed, and a redundant power supply can be purchased and installed separately. Each power

supply has one standard power receptacle for the AC power cable, and includes AC and DC status LEDs for

easy monitoring and troubleshooting.

When only one PSU is installed, both "AC OK" and "DC OK" LEDs on the installed PSU must be green for the

Brocade FCX device to function normally. When two PSUs are installed, both "AC OK" and "DC OK" LEDs for

one of the installed PSUs must be green for the FCX device to function normally.

To install a power supply unit in the switch:

1. Remove the blank metal plate (or a previously installed PSU) from the appropriate slot by removing the two

screws with a flat-head screwdriver.

2. Before opening the package that contains the PSU, touch the bag to the switch casing to discharge any

potential static electricity. Brocade recommends using an ESD wrist strap during installation.

3. Remove the PSU from the anti-static shielded bag.

4. Holding the PSU level, guide it into the carrier rails on each side and gently push it all the way into the slot,

ensuring that it firmly engages with the connector.

5. When you are sure the PSU has properly engaged with the connector, tighten the retainer screws to secure

the PSU in the slot. When the device is powered on, the AC and DC LEDs on the PSU back panel should turn

green to confirm that the PSU is correctly installed and is supplying power.

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 8

Cabling

A separate, isolated network should be configured to handle iSCSI traffic for performance and reliability of

the iSCSI SAN. Proper cabling is also necessary to provide redundancy in the iSCSI SAN environment. This

section reviews required cabling practices.

NOTE: If you only installed one switch, proceed to “Powering On the System” on page 11.

Connecting switches in a stack

NOTE: If you are creating an iSCSI SAN based on non-stackable switches, skip this section.

The Brocade FCX switch can be stacked to increase the available port count. To stack multiple Brocade FCX

648-S switches, use the dedicated 16Gbps stacking ports found on the rear of each unit, see Figure 1. To

stack multiple Brocade FCX 648-E switches, use the 10 Gbps SFP+ front-facing ports with LC-LC MM Fiber

cables, see Figure 2. Note that Brocade FCX 648-E switches must have a 4-port 10 Gbps SFP+ module

(optional) installed to operate in a stack. Up to eight Brocade FCX switches may be stacked, although a

typical iSCSI solution will not require this many ports.

In ring stack topology, an extra cable is connected between the top and bottom switches forming a “ring” or

“closed-loop.” The closed-loop cable provides a redundant path for the stack link, so if one link fails, stack

communications can be maintained. Figure 1 illustrates a ring-topology stacking configuration. In a

redundant iSCSI deployment of stacked Brocade FCX switches, the closed-loop topology should be

deployed.

BEST PRACTICE:

Brocade highly recommends using the ring topology stack. This topology will ensure that all

switches have a redundant path in the stack.

Connecting Brocade FCX-S Switches

A stack can contain up to eight Brocade FCX-S switches. To connect Brocade FCX-S switches in a stack:

1. Plug one end of a stack cable into one of the top unit’s CX4 stacking ports.

2. Plug the other end of the stack cable into the next unit’s CX4 stacking ports.

3. Repeat steps 1 and 2 for each unit in the stack. Form a simple chain starting with a stacking port on the top

unit and ending at a stacking port on the bottom unit (stacking up to eight units).

4. (Optional) To form a ring stack topology, plug one end of a stack cable into the remaining stacking port on

the bottom unit and the other end into the remaining stacking port on the top unit.

One device in the stack will operate as the Active Controller, one will operate as standby Controller, with

the rest of the units operating as stack members. For information about how to configure your stack,

see the Brocade FCX Configuration Guide.

BEST PRACTICE:

If you are approaching the eight-switch stack limitation, contact Brocade for

recommendations on how to effectively scale your iSCSI SAN for the future.

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 9

Figure 1. Connecting Brocade FCX-S switches in a ring topology stack

Connecting Brocade FCX-E Switches

NOTE: Brocade FCX 648-E switches must have a 4-port 10 Gbps SFP+ module (optional) installed to

operate in a stack.

A stack can contain up to eight Brocade FCX-E switches. To connect Brocade FCX-E switches in a stack:

1. Plug one end of an LC-LC MM Fiber cable into one of the top unit’s SFP+ stacking ports.

2. Plug the other end of the cable into one of the stacking ports of the next unit.

3. Repeat steps 1 and 2 for each unit in the stack. Form a simple chain starting with a stacking port on the top

unit and ending at a stacking port on the bottom unit (stacking up to eight units).

4. (Optional) To form a ring stack topology, plug one end of an LC-LC MM Fiber cable into the remaining

stacking port on the bottom unit and the other end into the remaining stacking port on the top unit.

Figure 2. Connecting Brocade FCX-E switches in a ring topology stack

DATA CENTER CONFIGURATION GUIDE

Configuring an iSCSI SAN Using Brocade FCX Switches 10

Cabling requirements for redundancy

Most iSCSI storage arrays have two storage controllers, with two to four Ethernet ports per controller. At

least two Ethernet ports should be cabled from each of the controllers for redundancy. Each controller is

then connected to multiple switches in the iSCSI SAN.

To build a highly available iSCSI SAN, ensure the following requirements are met:

•Contains at least two Brocade FCX switches, each with two PSUs, and each PSU uses a separate power

source.

•For Brocade FCX switches configured in a stack, create two redundant stacks. Each stack is managed

as a logical switch with one active management node. Each switch in the stack should have two PSUs,

and each PSU should be supplied by a separate power source.

•Ensure there are two separate power sources for each iSCSI storage array.

•For each iSCSI storage array, ensure there are at least two Ethernet ports on each controller with

connections to multiple switches in the stack.

•For greater redundancy, each initiator may be configured with multiple adapters or dual-port adapters

to connect to the iSCSI SAN.

Figure 3.Redundant cabling example using Brocade FCX-S stacks

In Figure 3, switches in each Brocade FCX stack are connected via link aggregation to both switches in the

opposing stack. Initiators have redundant connections to each stack, and both the active and passive iSCSI

controllers on each array are connected to each switch for full redundancy.

Other manuals for FCX Series

This manual suits for next models

Table of contents

Other Brocade Communications Systems Switch manuals