Mellanox Technologies M2401G User manual

Mellanox Technologies

M2401G 24 Port 20Gb/s InfiniBand Switch

for the Dell M1000e Chassis User Manual

Rev 1.0

Rev 1.0 MellanoxTechnologies

Mellanox Technologies, InfiniScale®is a registered trademark of Mellanox Technologies, Inc. All

other marks and names mentioned herein may be trademarks of their respective companies.

M2401G 24 Port 20Gb/s InfiniBand Switch

Document Number: 2948

Mellanox Technologies, Inc.

2900 Stender Way

Santa Clara, CA 95054

U.S.A.

www.Mellanox.com

Tel: (408) 970-3400

Fax: (408) 970-3403

Mellanox Technologies Ltd

PO Box 586 Hermon Building

Yokneam 20692

Israel

Tel: +972-4-909-7200

Fax: +972-4-959-3245

NOTE:

THIS INFORMATION IS PROVIDED BY MELLANOX FOR INFORMATIONAL PURPOSES ONLY AND ANY

EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF

MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL

MELLANOX BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSE-

QUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR

SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON

ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLI-

GENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS HARDWARE, EVEN IF ADVISED

OF THE POSSIBILITY OF SUCH DAMAGE.

M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual

MellanoxTechnologies Rev 1.0

Contents

Contents 3

About this Manual 4

Revision History 6

Chapter 1 Overview 7

Chapter 2 Installation and Basic Operation 8

2.1 Switch Platform Hardware Overview 8

2.1.1 InfiniBand Connectors 8

2.1.2 Switch Status Lights 9

2.2 Switch Platform Installation and Operation 10

2.2.1 Installation Safety Warnings 10

2.2.2 Mechanical Installation 11

2.2.2.1 Removing an Old Switch From the Chassis 12

2.2.2.2 Removing a Blank Cover Plate From the Chassis 12

2.2.2.3 Installing the Mellanox M2401G Switch Into the Dell Chassis 12

2.2.3 Power Connections and Initial Power On 13

2.2.4 InfiniBand Cable Installation 13

2.2.5 InfiniBand Optical Cable Installation 13

Chapter 3 Management Tools Overview 14

3.1 Updating Firmware 14

Chapter 4 Troubleshooting 15

Appendix A Specification 16

Appendix B Mechanical Drawing 17

Appendix C EMC Certification Statements 18

Appendix D InfiniBand Connector Pinout 20

Appendix E Local Authorization Form 21

Rev 1.0 MellanoxTechnologies

About this Manual

This manual describes the electrical and physical specifications of Mellanox Technologies’

M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting

up the switch platforms listed above.

The manual assumes familiarity with the InfiniBand® Architecture Specification.

Related Documentation

The documentation set accompanying the Mellanox M2401G includes the following:

Online Resources

• Mellanox Technologies Web pages: http://www.mellanox.com

• Dell Support Web pages: http://support.dell.com

Conventions

Throughout this manual, the name M2401G and the term switch are used to describe the 24-port

20Gb/s InfiniBand switch for the Dell M1000e, unless explicitly indicated otherwise.

Trademarks and Service Marks

The following are Mellanox Technologies registered trademarks:

InfiniHost®, ConnectX®, and InfiniScale®

Table 1 - Reference Documents

InfiniBand Architecture Specifica-

tion Volume 1 Release 1.2.1 and

Volume 2 release 1.2.1

InfiniBand architecture specification descriptions

Mellanox Firmware Tools (MFT)

User’s Manual

Document # 2329

The MFT (Mellanox Firmware Tools) package is a set of firmware

tools. The manual supplied with this package provides an overview of

the firmware its installation and replacement. The MFT can be down-

loaded with its documentation at:

http://www.mellanox.com under “Firmware” downloads.

M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual

MellanoxTechnologies Rev 1.0

This product does not require Advanced Management.

Table 2 - Switch Products Covered in this User’s Manual

Product Number Description

M2401G 24 Port 20 Gb/s unmanaged switch including 8 external ports for the Dell M1000e chassis.

Rev 1.0 MellanoxTechnologies

Revision History

Table 3 - Revision History Table

Date Revision Description

June 2008 Rev 1.0 First Draft

M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual

MellanoxTechnologies Rev 1.0

1Overview

This User’s Manual provides an overview of the M2401G 24-port 20Gb/s InfiniBand switch for the

Dell M1000e. The switch supports 16 switch ports towards the midplane and 8 external ports. In

addition to supporting passive copper InfiniBand cables, the external ports can provide power for

supporting active cables and optical cables through the use of media adapters.

The switch supports 16 switch ports towards the midplane and 8 external ports. In addition to sup-

porting passive copper InfiniBand cables, the external ports can provide power for supporting

active cables and optical cables through the use of media adapters.

The switch platform comes pre-installed with all necessary firmware, and configured for standard

operation with InfiniBand fabric, and requires an InfiniBand compliant Subnet Manager running

from one of the hosts. All that is required for normal operation is to follow the usual precautions for

installation and connection from the switch to the HCAs or other switches. Once connected, the

Subnet Management software automatically configures and begins utilizing the switch.

Maintenance and configuration of the switch is done in-band over the InfiniBand fabric using the

Mellanox Technologies InfiniBand Administration (IBADM) tools package. This package provides

the ability to upgrade firmware and monitor the temperature, voltage, port utilization, and other sta-

tus parameters in the switch.

Basic installation, hot-swapping components and hardware maintenance is covered in “Installa-

tion and Basic Operation” on page 8.

The M2401G switch has a Hot Swap controller and a PSOC Management IC

Rev 1.0 MellanoxTechnologies

2Installation and Basic Operation

2.1 Switch Platform Hardware Overview

Figure 1 shows the front panel view of the switch.

Figure 1: Switch Front Panel and Locking Mechanism

2.1.1 InfiniBand Connectors

InfiniBand connectivity has 8 CX4 microGiGa connectors through the front panel. The remaining

16 interfaces are through the AirMax Midplane Connector out the back of the switch. Figure 1

shows the front 8 ports.

Each of the front panel connectors has two LEDs located next to the connector. The green LED,

when lit, indicates that the InfiniBand driver is running and a valid physical connection between

nodes exists. The yellow LED when lit, indicates a valid data activity link, this is the logical link.

The yellow LED illuminates when the InfiniBand network is discovered over the physical link. A

valid data activity link without data transfer is designated by a constant yellow LED indication. A

valid data activity link with data transfer is designated by a blinking yellow LED indication. If the

LEDs are not active, either the physical link or the logical link (or both) connections has not been

established.

Figure 2: Physical and Logical Link Indications LEDs

Locking Mechanism Switch Status Lights

LED Name

Physical Link - Green

Data Activity - Yellow

Blinking indicates Data Transfer

Constant on indicates no Data Transfer

M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual

MellanoxTechnologies Rev 1.0

2.1.2 Switch Status Lights

The switch Status lights indicate whether the switch is receiving power from the chassis.

Figure 3: Indicator LEDs

Table 4 - IOM states and LED configurations:

IOM State Source LED

Blue Amber Green

OFF N/A OFF OFF OFF

BOOT_IN_PROGRESS IOM OFF OFF ON

NORMAL/ STACK_MASTER IOM ON OFF ON

STACK_SLAVE Stackable IOM OFF OFF ON

IOM_FAULT (self-diagnosed) IOM OFF BLINK

1HZ ON

IOM_FAULT (Misconfig or other CMC-detected fault) CMC OFF BLINK

1HZ OFF

MODULE_IDENTIFY CMC BLINK

1HZ OFF ON

When both the blue and

green LEDs are lit this indi-

cates that the IO Module is

on and ready.

This LED can be Blue or

Amber. If it is blinking

Amber it indicates a Fault.

See Table 4.

Port number

Rev 1.0 MellanoxTechnologies

2.2 Switch Platform Installation and Operation

Installation and initialization of the switch platform are straightforward processes, requiring atten-

tion to the normal mechanical, power, and thermal precautions for rack-mounted equipment. The

switch platform does not require any programming or configuration to operate as a basic Infini-

Band switch and includes all of the necessary functionality to operate with external standard Infini-

Band Subnet Management software.

This section describes the installation process and basic operation of the switch platform. Please

first read the warnings sub-section carefully before carrying on with installation procedures.

2.2.1 Installation Safety Warnings

1. Installation Instructions

Read all installation instructions before connecting the equipment to the power

source.

2. Over-temperature

This equipment should not be operated in an area with an ambient temperature

exceeding the maximum recommended: 50°C (122°F). Moreover, to guarantee proper

air flow, allow at least 8cm (3 inches) of clearance around the ventilation openings.

3. Stacking the Chassis

The chassis should not be stacked on any other equipment. If the chassis falls, it can

cause bodily injury and equipment damage.

4. During Lightning - Electrical Hazard

During periods of lightning activity, do not work on the equipment or connect or dis-

connect cables.

5. Copper InfiniBand Cable Connecting/Disconnecting

Copper InfiniBand cables are heavy and not flexible, as such they should be carefully

attached to or detached from the connectors. Refer to the cable manufacturer for spe-

cial warnings and instructions.

Table of contents

Other Mellanox Technologies Switch manuals

Mellanox Technologies

Mellanox Technologies Vantage 8500 User manual

Mellanox Technologies

Mellanox Technologies Spectrum SN2700 Instructions for use

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 User manual

Mellanox Technologies

Mellanox Technologies MetroDX MTX6000-2SFS User manual

Mellanox Technologies

Mellanox Technologies TX6000 Instructions for use

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 User manual

Mellanox Technologies

Mellanox Technologies MSX6536-10R Manual

Mellanox Technologies

Mellanox Technologies SB7700 User manual

Mellanox Technologies

Mellanox Technologies MSX6536-10R Manual

Mellanox Technologies

Mellanox Technologies TX6100 MetroX Series Instructions for use

Mellanox Technologies

Mellanox Technologies MIS5300 Manual

Mellanox Technologies

Mellanox Technologies Grid Director 4700 User manual

Mellanox Technologies

Mellanox Technologies InfiniScale III User manual

Mellanox Technologies

Mellanox Technologies Spectrum SN2100 User manual

Mellanox Technologies

Mellanox Technologies SX67X0 Instructions for use

Mellanox Technologies

Mellanox Technologies Spectrum SN2700 Instructions for use

Mellanox Technologies

Mellanox Technologies Spectrum SN2100 Instructions for use

Mellanox Technologies

Mellanox Technologies MIS5022Q-1BFR User manual

Mellanox Technologies

Mellanox Technologies SX1012 User manual

Mellanox Technologies

Mellanox Technologies MSX6506-3R Manual

Mellanox Technologies

Mellanox Technologies MSX6512-NR Manual

Mellanox Technologies

Mellanox Technologies MSX6512-4R Manual

Mellanox Technologies

Mellanox Technologies MIS5022Q-1BRR User manual

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX6005F-1BFS Instructions for use

Popular Switch manuals by other brands

URC

URC Total Control MFS-8 installation manual

Azbil

Azbil HPQ-D11 user manual

Sony

Sony DFS-900M operating instructions

SICK

SICK i17S operating instructions

F&F

F&F SZR-277 quick start guide

Shure

Shure UAMS/BK user guide

Dräger

Dräger X-zone Switch Off Instructions for use

Asco

Asco Joucomatic 349 Series manual

BT Mini Hub user guide

D-Link

D-Link DES-1226G manual

Johnson Controls

Johnson Controls FS80-C installation instructions

Alcatel-Lucent

Alcatel-Lucent 7450 ESS-12 Configuration guide

MicroNet

MicroNet SP1200A user manual

Ruijie

Ruijie RG-PF2920 Series Hardware installation and reference guide

Altinex

Altinex DA1914SX user guide

MiLAN

MiLAN MIL-SM2401M user guide

RJM

RJM MASTERMIND PBC/6X user manual

Atlas

Atlas MMK-KVM8 owner's manual