Western Digital Ultrastar Serv60+8 User manual

User Guide

Ultrastar®Serv60+8

Regulatory Model: H4060-S

December 2018

Rev. 1.3

1ET1110

Long Live Data ™| www.hgst.com

Contents

Revision History..............................................................................................7

Chapter 1 Overview....................................................................................9

1.1 Ultrastar®Serv60+8 Description........................................................9

1.2 System Architecture Overview.........................................................10

1.3 System Level Block Diagram...........................................................11

1.4 Ultrastar Serv60+8 Specification Summary.....................................12

1.5 Ultrastar Serv60+8 Layout...............................................................15

1.6 List of Customer Replaceable Units (CRUs)....................................17

1.7 Supported Operating Systems.........................................................19

1.8 LEDs................................................................................................20

1.8.1 Front and Rear IO LEDs...............................................................20

1.8.2 IOM LEDs.....................................................................................21

1.8.3 PSU LED......................................................................................22

1.8.4 Drive Assembly LED....................................................................23

1.9 List of Compatible Drives.................................................................24

Chapter 2 System Management Overview.............................................34

2.1 Storage Subsystem Firmware Features Overview...........................34

2.2 OOBM Management Overview........................................................35

2.3 SCSI Enclosure Services Page 02...................................................35

Chapter 3 Component Overviews...........................................................36

3.1 Chassis Description.........................................................................36

3.1.1 Chassis Specifications.................................................................36

3.1.2 Chassis Layout.............................................................................37

3.2 IOM Description...............................................................................38

Contents

User Guide

3.2.1 IOM Specifications.......................................................................38

3.2.2 IOM Layout...................................................................................39

3.2.3 IOM Blank Description.................................................................39

3.3 PSU Description...............................................................................40

3.3.1 PSU Specifications.......................................................................40

3.3.2 PSU Layout..................................................................................41

3.4 System Fan Description...................................................................42

3.5 Rails Description..............................................................................42

3.5.1 Rails Specifications......................................................................43

3.5.2 Rails Layout..................................................................................43

3.5.3 Rear Cover Alignment Bracket Description..................................44

3.6 CMA Description..............................................................................44

3.6.1 CMA Specifications......................................................................45

3.6.2 CMA Layout..................................................................................45

3.7 Drive Assembly Description.............................................................46

3.7.1 Drive Assembly Specifications.....................................................46

3.7.2 Drive Assembly Layout.................................................................47

3.7.3 2.5" Drive Carrier Description.......................................................48

3.7.3.1 Operating the 2.5" Drive Carrier...............................................48

3.7.4 Drive Blank Description................................................................50

3.8 System SSD Description..................................................................52

3.8.1 System SSD Specifications..........................................................52

Chapter 4 Part Replacement...................................................................53

4.1 Part Replacement Service Window.................................................53

4.2 IOM Replacement............................................................................53

4.3 PSU Replacement............................................................................57

4.4 System Fan Replacement................................................................61

4.5 Drive Assembly Replacement..........................................................64

4.6 System SSD Replacement...............................................................69

4.7 DIMM Replacement.........................................................................72

iii

Contents

User Guide

4.8 CPU and Heat Sink Module Replacement.......................................99

4.9 Heat Sink Replacement.................................................................128

4.10 M.2 Replacement.........................................................................161

4.11 Add-in Card Replacement............................................................178

4.12 CMA Replacement.......................................................................196

4.13 Rails Replacement.......................................................................201

4.14 Chassis Replacement..................................................................225

Chapter 5 System Management............................................................277

5.1 Storage Subsystem Firmware Upgrade.........................................277

5.1.1 Downloading Firmware from the Support Portal........................277

5.1.2 Linux Upgrade Preparation........................................................279

5.1.3 Linux Upgrade to New Firmware................................................279

5.1.4 Non-Automatic Firmware Activation in Linux..............................281

5.1.5 Windows Upgrade Preparation...................................................282

5.1.6 Windows Upgrade to New Firmware..........................................283

5.1.7 Non-Automatic Firmware Activation in Windows........................284

5.1.8 Upgrading Firmware with OOBM...............................................285

5.2 BIOS Firmware Update..................................................................286

5.2.1 Updating BIOS Firmware using EFI Boot Mode.........................287

5.2.2 Updating BIOS Firmware using the Legacy Boot Mode.............289

5.3 HGST BMC Web GUI....................................................................291

5.3.1 BMC Web GUI Overview............................................................291

5.3.1.1 Dashboard..............................................................................292

5.3.2 User Management......................................................................293

5.3.2.1 User Privileges.......................................................................294

5.3.2.2 Changing the Default Administrator Credentials....................296

5.3.2.3 Logging into the BMC Web GUI.............................................303

5.3.2.4 Creating a New User..............................................................304

5.3.2.5 Modifying a User....................................................................307

5.3.2.6 Deleting a User.......................................................................310

5.3.2.7 Ending a User Session...........................................................312

Contents

User Guide

5.3.3 Security Features.......................................................................313

5.3.3.1 Enabling Security Settings.....................................................314

5.3.4 Network Settings........................................................................317

5.3.4.1 Configuring the IPv4 Network Settings...................................317

5.3.4.2 Configuring the IPv6 Network Settings...................................320

5.3.4.3 Configuring VLAN Settings.....................................................323

5.3.5 BMC Web GUI Firmware............................................................324

5.3.5.1 Updating BMC Web GUI Firmware........................................324

5.3.5.2 Updating the Server BMC Firmware Using Redfish OOBM....329

5.3.6 Event Logs.................................................................................331

5.3.6.1 Viewing Event Logs................................................................332

5.4 Partially Populated Enclosures.......................................................335

5.4.1 Partial Population Configurations...............................................335

5.4.2 Installing Drives..........................................................................338

Chapter 6 Safety.....................................................................................342

6.1 Electrostatic Discharge..................................................................342

6.2 Optimizing Location........................................................................342

6.3 Power Connections........................................................................342

6.4 Power Cords...................................................................................343

6.5 Rackmountable Systems...............................................................343

6.6 Safety and Service.........................................................................343

6.7 Safety Warnings and Cautions.......................................................344

Chapter 7 Disclaimers............................................................................345

7.1 Restricted Access Location............................................................345

7.2 Safety Compliance.........................................................................345

7.3 Electromagnetic Compatibility (EMC) Class A Compliance ..........345

7.4 Country Certifications....................................................................346

Chapter 8 Regulatory Statements.........................................................347

Contents

User Guide

8.1 Europe (CE Declaration of Conformity)..........................................347

8.2 FCC Class A Notice.......................................................................347

8.3 ICES-003 Class A Notice—Avis NMB-003, Classe A....................347

8.4 Japanese Compliance Statement, Class A ITE.............................347

8.5 Taiwan Warning Label Statement, Class A ITE.............................348

Chapter 9 Points of Contact..................................................................349

Appendix A Site and System Operation Requirements............................350

A.1 Power Requirements.....................................................................350

A.2 Supported Operating Systems......................................................350

A.3 List of Supported Server Components..........................................351

A.4 Ultrastar Serv60+8 Rack Requirements........................................352

A.4.1 Compatible Hardware Configuration..........................................354

A.5 ESD...............................................................................................355

A.6 Enclosure Cooling..........................................................................355

A.7 Host Connectivity...........................................................................356

Contents

User Guide

Revision History

CommentRevisionDate

Initial ReleaseRevision 1.0June 2018

Added the following sections:Revision 1.1July 2018

•DIMM Replacement on page 72

•CPU and Heat Sink Module Replacement on page 99

•Heat Sink Replacement on page 128

•M.2 Replacement on page 161

•Add-in Card Replacement on page 178

Updated the following sections:Revision 1.2August 2018

•List of Customer Replaceable Units (CRUs) on page 17

•CPU and Heat Sink Module Replacement on page 99

•Heat Sink Replacement on page 128

•Storage Subsystem Firmware Upgrade on page 277

Updated the following sections:Revision 1.3December2018

•List of Customer Replaceable Units (CRUs) on page 17

•List of Compatible Drives on page 24

•LEDs on page 20

Added the following sections:

•BIOS Firmware Update on page 286

•Updating the Server BMC Firmware Using Redfish

OOBM on page 329

Revision History

User Guide

The following paragraph does not apply to the United Kingdom or any country where such provisions

are inconsistent with local law: HGST a Western Digital company PROVIDES THIS PUBLICATION "AS

IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT

LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR

PURPOSE. Some states do not allow disclaimer or express or implied warranties in certain transactions,

therefore, this statement may not apply to you.

This publication could include technical inaccuracies or typographical errors. Changes are periodically made

to the information herein; these changes will be incorporated in new editions of the publication. HGST may

make improvements or changes in any products or programs described in this publication at any time.

It is possible that this publication may contain reference to, or information about, HGST products (machines

andprograms),programming,orservicesthat are notannounced in yourcountry. Such referencesorinformation

must not be construed to mean that HGST intends to announce such HGST products, programming, or

services in your country.

Technical information about this product is available by contacting your local HGST representative or on the

Internet at: support.hgst.com

HGST may have patents or pending patent applications covering subject matter in this document. The furnishing

of this document does not give you any license to these patents.

Western Digital

5601 Great Oaks Parkway

San Jose, CA 95119

Long Live Data™is a trademark of HGST, Inc. and its affiliates in the United States and/or other countries.

HGST trademarks are authorized for use in countries and jurisdictions in which HGST has the right to use,

market and advertise the brands.

Other product names are trademarks or registered trademarks of their respective owners.

One MB is equal to one million bytes, one GB is equal to one billion bytes, one TB equals 1,000GB (one trillion

bytes) and one PB equals 1,000TB when referring to storage capacity. Usable capacity will vary from the raw

capacity due to object storage methodologies and other factors.

References in this publication to HGST products, programs or services do not imply that HGST intends to

make these available in all countries in which HGST operates.

Product information is provided for information purposes only and does not constitute a warranty.

Information is true as of the date of publication and is subject to change. Actual results may vary. This publication

is for general guidance only. Photographs may show design models.

User Guide

Chapter

1 Overview

This section provides a high level overview of the features of the Ultrastar Serv60+8.

1.1 Ultrastar®Serv60+8 Description

Figure 1: Ultrastar Serv60+8

The Ultrastar®Serv60+8 is a 4U form factor, high density, rack-mounted storage enclosure that is capable

of hosting up to 60 HDD SAS or SATA drives. The maximum data storage capacity of the Ultrastar Serv60+8

is720TB using 12TB HGST Ultrastar®He12 HDDs plus an additional 61.44 TB using HGST Ultrastar®SS300

SSDs in the system SSD slots. (For a full list of compatible drives and total storage capacities, see the List

of Compatible Drives on page 24.) The Ultrastar Serv60+8 also integrates an Intel Skylake based server

front end. The server is built around an Intel Sawtooth Pass S2600ST motherboard and supports 2 LGA3647-0

(Socket P) processor sockets CPUs, 16 (8 per CPU) slots of DDR4 ECC DIMM memory, and leverages an

AST2500 BMC for out-of-band management of the server subsystem.The enclosure runs on an input voltage

of 200 - 240 VAC and consumes ~1800W of power under typical conditions. It requires a maximum of ~2000W

at full load.

Note: Max and Typical Power Consumption values represent the output power to the system. Input

power will vary depending on the PSU efficiency and load sharing between PSUs

The system contains three externally facing Half-Height, Half-Length (HHHL) PCIe x16 slots, and 3 externally

facing HHHL PCIe x8 slots.

It is designed to fit within a 4U rack space and requires 1200 mm (47.24in.) of usable rack space, frame to

frame. A fully loaded system will add 95.25 kg / 210 lbs. of static load when fully loaded with drives.

• 4U Storage Server

• Supports up to 60 HDD Drives

• Can support 3.5” drives and 2.5” SSD drives (2.5" requires an adapter) in the 60 available drive bays.

Supports an additional 8 2.5" drives in the system SSD slots located in the center channel

• Up to 12W per drive slot for the 60 HDD data storage drives (Cannot exceed 85A on the 5V rail) and 25W

per slot for the 8 system SSD slots

• House and control six (6) N+1 redundant 80mm fans

• Powered by two (2) redundant 2000W PSUs

• Toolless replacement of all Customer Replaceable Units (CRUs)

• Fits within a standard EIA-310 rack including all necessary cable management

• Supports Highline Input Power

1 Overview

User Guide 1.1 Ultrastar®Serv60+8 Description

• Dual Skylake LGA3647 Socket Processors

• 16 DDR4 ECC DIMM slots

• ASPEED AST2500 BMC

• Lewisburg PCH C624 Chipset

1.2 System Architecture Overview

Storage Subsystem Overview

The Ultrastar Serv60+8 IOM uses a cascaded expander design to allow for connection to all 60 HDD drives.

The minimum configuration comes with two of the six PCIe slots pre-populated with a combination of internal

SAS HBAs and RAID adapters that act as an interconnect between the server subsystem and the storage

subsystem by connecting to the system SSDs and the IOM. The exact configuration of the PCIe slots may

vary depending the specific version of the Ultrastar Serv60+8. The eight system SSDs are connected via one

of these SAS HBAs.

The storage subsystem out-of-band management microprocessor provides an Ethernet connection using a

Redfish/RESTful API to access the various enclosure services. All the SES enclosure information can be

obtainedthrough the out-of-bandmanagement port.Major use casesforthisfeature includeobtaining storage

subsystemhealth information, locating enclosurecomponents using theIDENT LEDs,and updating expander

and storage OOBM firmware.

The system FPGAs control and report the states of the four storage subsystem fans, drive LEDs, and T10

drive power disable signals on the 60 HDD data storage drives.

Note: To use T10 power disable, the drives installed must also support this

feature.

The I2C architecture is designed to support only one single master on any given bus. The primary expander

will be the master on each I2C bus. There are eight I2C buses used in the Ultrastar Serv60+8 enclosure. The

number of devices on each bus are balanced to allow communication to peripheral devices and not overload

any one bus. The devices connected on the I2C buses include the enclosure VPDs, temp sensors, baseboard

FPGAs, and SAS connectors among others.

Four of the six total fans are controlled by the storage subsystem. Status and control for these four fans is

handled by the in-band SES management interface called the SEP and the Redfish OOBM.

Server Subsystem Overview

The server subsystem is situated in an individual compartment at the rear of the unit. It is comprised of a dual

socket Intel Sawtooth Pass S2600ST that supports LGA3647 socket Intel Xeon processors. The Ultrastar

Serv60+8 has 16 DDR4 ECC DIMM slots for memory that support a clock speed of 2666 MHz, and it leverages

the Intel C624 chipset. The motherboard also hosts an ASPEED AST2500 BMC for out-of-band management

of the server components. The server motherboard has a total of six PCIe slots that support PCIe 3.0, two

M.2 ports that can host compatible SSDs for the system OS.

1 Overview

User Guide 1.2 System Architecture Overview

Other manuals for Ultrastar Serv60+8

This manual suits for next models

Table of contents

Other Western Digital Storage manuals

Western Digital

Western Digital WDBMWV0020BBK User manual

Western Digital

Western Digital WDBGJA0010BBK User manual

Western Digital

Western Digital Purple PR1000M Product manual

Western Digital

Western Digital WDXB1200JBR - Dual-option External Hard Drive 120... User manual

Western Digital

Western Digital WDBAAR3200ABK - Elements Portable User manual

Western Digital

Western Digital WD4000YS - RE2 Installation instructions

Western Digital

Western Digital WD4001FAEX User manual

Western Digital

Western Digital WDBAAH0015HCH - My Book Elite User manual

Western Digital

Western Digital WD Blue 3D NAND SATA SSD User manual

Western Digital

Western Digital VelociRaptor VR200M WD6000HLHX User manual

Western Digital

Western Digital MY PASSPORT AIR User manual

Western Digital

Western Digital Passport Slim WDBGMT0010BAL Installation instructions

Western Digital

Western Digital WD RE3 User manual

Western Digital

Western Digital WD AV-GP User manual

Western Digital

Western Digital My Book Studio User manual

Western Digital

Western Digital My Book WDH1U10000 User manual

Western Digital

Western Digital WDBGCH0010BSL User manual

Western Digital

Western Digital Ultrastar Data60 User manual

Western Digital

Western Digital WDG2TP10000N - My Book Pro Edition II Hard Drive... User manual

Western Digital

Western Digital My Book World Edition WDH1NC10000 User manual

Western Digital

Western Digital My Book World Edition User manual

Western Digital

Western Digital WDBAAC2500ACH - My Passport Elite User manual

Western Digital

Western Digital My Book Studio User manual

Western Digital

Western Digital Scorpio WD2500BPVT User manual

Popular Storage manuals by other brands

QNAP

QNAP TS 32PXU Series user guide

LaCie

LaCie RUGGED RAID user manual

Freecom

Freecom CLS DOCK Quick install guide

PNY

PNY Attache P-FD64MBATT3 install guide

HP StorageWorks MSL2024 Getting started

OLT

OLT 6x6 Maximizer Assembly manual

Transcend

Transcend JetFlash 160 Specifications

LaCie

LaCie RUGGED MINI user manual

Thunderbolt

Thunderbolt 600 user manual

Panasonic

Panasonic LB-DH8 series Operator's guide

Intel

Intel RAID High Availability user guide

LaCie

LaCie 12big Thunderbolt 3 Quick install guide

Spectra Logic

Spectra Logic T-Series Spectra T50e quick start guide

Panasonic

Panasonic LKM-F931-1 instruction manual

Tabernus

Tabernus Enterprise Erase E2400 user guide

Rocstor

Rocstor COMMANDER 3F series user manual

HP P9000 user guide

Dell

Dell PowerVault MD3600f Series Getting started