Mellanox Technologies M2401G User manual
Mellanox Technologies
M2401G 24 Port 20Gb/s InfiniBand Switch
for the Dell M1000e Chassis User Manual
Rev 1.0
Rev 1.0 MellanoxTechnologies
2
Copyright 2008. Mellanox Technologies, Inc. All Rights Reserved.
Mellanox Technologies, InfiniScale®is a registered trademark of Mellanox Technologies, Inc. All
other marks and names mentioned herein may be trademarks of their respective companies.
M2401G 24 Port 20Gb/s InfiniBand Switch
Document Number: 2948
Mellanox Technologies, Inc.
2900 Stender Way
Santa Clara, CA 95054
U.S.A.
www.Mellanox.com
Tel: (408) 970-3400
Fax: (408) 970-3403
Mellanox Technologies Ltd
PO Box 586 Hermon Building
Yokneam 20692
Israel
Tel: +972-4-909-7200
Fax: +972-4-959-3245
NOTE:
THIS INFORMATION IS PROVIDED BY MELLANOX FOR INFORMATIONAL PURPOSES ONLY AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
MELLANOX BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSE-
QUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLI-
GENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS HARDWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.
M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual
MellanoxTechnologies Rev 1.0
3
Contents
Contents 3
About this Manual 4
Revision History 6
Chapter 1 Overview 7
Chapter 2 Installation and Basic Operation 8
2.1 Switch Platform Hardware Overview 8
2.1.1 InfiniBand Connectors 8
2.1.2 Switch Status Lights 9
2.2 Switch Platform Installation and Operation 10
2.2.1 Installation Safety Warnings 10
2.2.2 Mechanical Installation 11
2.2.2.1 Removing an Old Switch From the Chassis 12
2.2.2.2 Removing a Blank Cover Plate From the Chassis 12
2.2.2.3 Installing the Mellanox M2401G Switch Into the Dell Chassis 12
2.2.3 Power Connections and Initial Power On 13
2.2.4 InfiniBand Cable Installation 13
2.2.5 InfiniBand Optical Cable Installation 13
Chapter 3 Management Tools Overview 14
3.1 Updating Firmware 14
Chapter 4 Troubleshooting 15
Appendix A Specification 16
Appendix B Mechanical Drawing 17
Appendix C EMC Certification Statements 18
Appendix D InfiniBand Connector Pinout 20
Appendix E Local Authorization Form 21
Rev 1.0 MellanoxTechnologies
4
About this Manual
This manual describes the electrical and physical specifications of Mellanox Technologies’
M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis.
Intended Audience
This manual is intended for users and system administrators responsible for installing and setting
up the switch platforms listed above.
The manual assumes familiarity with the InfiniBand® Architecture Specification.
Related Documentation
The documentation set accompanying the Mellanox M2401G includes the following:
Online Resources
• Mellanox Technologies Web pages: http://www.mellanox.com
• Dell Support Web pages: http://support.dell.com
Conventions
Throughout this manual, the name M2401G and the term switch are used to describe the 24-port
20Gb/s InfiniBand switch for the Dell M1000e, unless explicitly indicated otherwise.
Trademarks and Service Marks
The following are Mellanox Technologies registered trademarks:
InfiniHost®, ConnectX®, and InfiniScale®
Table 1 - Reference Documents
InfiniBand Architecture Specifica-
tion Volume 1 Release 1.2.1 and
Volume 2 release 1.2.1
InfiniBand architecture specification descriptions
Mellanox Firmware Tools (MFT)
User’s Manual
Document # 2329
The MFT (Mellanox Firmware Tools) package is a set of firmware
tools. The manual supplied with this package provides an overview of
the firmware its installation and replacement. The MFT can be down-
loaded with its documentation at:
http://www.mellanox.com under “Firmware” downloads.
M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual
MellanoxTechnologies Rev 1.0
5
This product does not require Advanced Management.
Table 2 - Switch Products Covered in this User’s Manual
Product Number Description
M2401G 24 Port 20 Gb/s unmanaged switch including 8 external ports for the Dell M1000e chassis.
Rev 1.0 MellanoxTechnologies
6
Revision History
Table 3 - Revision History Table
Date Revision Description
June 2008 Rev 1.0 First Draft
M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual
MellanoxTechnologies Rev 1.0
7
1Overview
This User’s Manual provides an overview of the M2401G 24-port 20Gb/s InfiniBand switch for the
Dell M1000e. The switch supports 16 switch ports towards the midplane and 8 external ports. In
addition to supporting passive copper InfiniBand cables, the external ports can provide power for
supporting active cables and optical cables through the use of media adapters.
The switch supports 16 switch ports towards the midplane and 8 external ports. In addition to sup-
porting passive copper InfiniBand cables, the external ports can provide power for supporting
active cables and optical cables through the use of media adapters.
The switch platform comes pre-installed with all necessary firmware, and configured for standard
operation with InfiniBand fabric, and requires an InfiniBand compliant Subnet Manager running
from one of the hosts. All that is required for normal operation is to follow the usual precautions for
installation and connection from the switch to the HCAs or other switches. Once connected, the
Subnet Management software automatically configures and begins utilizing the switch.
Maintenance and configuration of the switch is done in-band over the InfiniBand fabric using the
Mellanox Technologies InfiniBand Administration (IBADM) tools package. This package provides
the ability to upgrade firmware and monitor the temperature, voltage, port utilization, and other sta-
tus parameters in the switch.
Basic installation, hot-swapping components and hardware maintenance is covered in “Installa-
tion and Basic Operation” on page 8.
The M2401G switch has a Hot Swap controller and a PSOC Management IC
Rev 1.0 MellanoxTechnologies
8
2Installation and Basic Operation
2.1 Switch Platform Hardware Overview
Figure 1 shows the front panel view of the switch.
Figure 1: Switch Front Panel and Locking Mechanism
2.1.1 InfiniBand Connectors
InfiniBand connectivity has 8 CX4 microGiGa connectors through the front panel. The remaining
16 interfaces are through the AirMax Midplane Connector out the back of the switch. Figure 1
shows the front 8 ports.
Each of the front panel connectors has two LEDs located next to the connector. The green LED,
when lit, indicates that the InfiniBand driver is running and a valid physical connection between
nodes exists. The yellow LED when lit, indicates a valid data activity link, this is the logical link.
The yellow LED illuminates when the InfiniBand network is discovered over the physical link. A
valid data activity link without data transfer is designated by a constant yellow LED indication. A
valid data activity link with data transfer is designated by a blinking yellow LED indication. If the
LEDs are not active, either the physical link or the logical link (or both) connections has not been
established.
Figure 2: Physical and Logical Link Indications LEDs
1
2
7
6
5
4
3
8
Locking Mechanism Switch Status Lights
LED Name
Physical Link - Green
Data Activity - Yellow
Blinking indicates Data Transfer
Constant on indicates no Data Transfer
M2401G 24 Port 20Gb/s InfiniBand Switch for the Dell M1000e Chassis User Manual
MellanoxTechnologies Rev 1.0
9
2.1.2 Switch Status Lights
The switch Status lights indicate whether the switch is receiving power from the chassis.
Figure 3: Indicator LEDs
Table 4 - IOM states and LED configurations:
IOM State Source LED
Blue Amber Green
OFF N/A OFF OFF OFF
BOOT_IN_PROGRESS IOM OFF OFF ON
NORMAL/ STACK_MASTER IOM ON OFF ON
STACK_SLAVE Stackable IOM OFF OFF ON
IOM_FAULT (self-diagnosed) IOM OFF BLINK
1HZ ON
IOM_FAULT (Misconfig or other CMC-detected fault) CMC OFF BLINK
1HZ OFF
MODULE_IDENTIFY CMC BLINK
1HZ OFF ON
8
When both the blue and
green LEDs are lit this indi-
cates that the IO Module is
on and ready.
This LED can be Blue or
Amber. If it is blinking
Amber it indicates a Fault.
See Table 4.
Port number
Rev 1.0 MellanoxTechnologies
10
2.2 Switch Platform Installation and Operation
Installation and initialization of the switch platform are straightforward processes, requiring atten-
tion to the normal mechanical, power, and thermal precautions for rack-mounted equipment. The
switch platform does not require any programming or configuration to operate as a basic Infini-
Band switch and includes all of the necessary functionality to operate with external standard Infini-
Band Subnet Management software.
This section describes the installation process and basic operation of the switch platform. Please
first read the warnings sub-section carefully before carrying on with installation procedures.
2.2.1 Installation Safety Warnings
1. Installation Instructions
Read all installation instructions before connecting the equipment to the power
source.
2. Over-temperature
This equipment should not be operated in an area with an ambient temperature
exceeding the maximum recommended: 50°C (122°F). Moreover, to guarantee proper
air flow, allow at least 8cm (3 inches) of clearance around the ventilation openings.
3. Stacking the Chassis
The chassis should not be stacked on any other equipment. If the chassis falls, it can
cause bodily injury and equipment damage.
4. During Lightning - Electrical Hazard
During periods of lightning activity, do not work on the equipment or connect or dis-
connect cables.
5. Copper InfiniBand Cable Connecting/Disconnecting
Copper InfiniBand cables are heavy and not flexible, as such they should be carefully
attached to or detached from the connectors. Refer to the cable manufacturer for spe-
cial warnings and instructions.
Table of contents
Other Mellanox Technologies Switch manuals
Mellanox Technologies
Mellanox Technologies MIS5022Q-1BRR User manual
Mellanox Technologies
Mellanox Technologies SX6720 Instructions for use
Mellanox Technologies
Mellanox Technologies InfiniScale MTS3600Q-1BNC User manual
Mellanox Technologies
Mellanox Technologies Switch-IB User manual
Mellanox Technologies
Mellanox Technologies MIS5035Q-2SFC User manual
Mellanox Technologies
Mellanox Technologies InfiniScale III User manual
Mellanox Technologies
Mellanox Technologies MSX6512-NR Manual
Mellanox Technologies
Mellanox Technologies InfiniScale MTS3600Q-1BNC User manual
Mellanox Technologies
Mellanox Technologies SwitchX-2 MSX6005F-1BFS Instructions for use
Mellanox Technologies
Mellanox Technologies MIS5300 Manual
Mellanox Technologies
Mellanox Technologies Spectrum SN2700 User manual
Mellanox Technologies
Mellanox Technologies SB7700 User manual
Mellanox Technologies
Mellanox Technologies MIS5600Q-10DNC User manual
Mellanox Technologies
Mellanox Technologies MSX6518-6R Manual
Mellanox Technologies
Mellanox Technologies Vantage 8500 User manual
Mellanox Technologies
Mellanox Technologies MSX6512-4R Manual
Mellanox Technologies
Mellanox Technologies IS5025D-1BRC User manual
Mellanox Technologies
Mellanox Technologies Spectrum SN2700 Instructions for use
Mellanox Technologies
Mellanox Technologies SX1710 User manual
Mellanox Technologies
Mellanox Technologies SX1012 User manual
