Broadcom Stingray PS225 User manual

Broadcom Confidential 5880X-PS225-UG101

December 12, 2018

Stingray™ PS225

Quick Start Guide

User Guide

Broadcom Confidential

Broadcom, the pulse logo, Connecting everything, Stingray, TruFlow, Avago Technologies, Avago, and the A logo are among

the trademarks of Broadcom and/or its affiliates in the United States, certain other countries, and/or the EU.

The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. For more information, please visit www.broadcom.com.

Broadcom reserves the right to make changes without further notice to any products or data herein to improve reliability,

function, or design. Information furnished by Broadcom is believed to be accurate and reliable. However, Broadcom does

not assume any liability arising out of the application or use of this information, nor the application or use of any product or

circuit described herein, neither does it convey any license under its patent rights nor the rights of others.

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

Table of Contents

1 Introduction ..................................................................................................................................................................4

1.1 Block Diagram......................................................................................................................................................4

1.2 Board Features ....................................................................................................................................................5

1.3 Package Contents................................................................................................................................................6

1.4 PS225 Card Variants ...........................................................................................................................................7

2 Checklist .......................................................................................................................................................................8

3 Initial Board Setup and Connectivity .......................................................................................................................10

3.1 PS225 Connector Locations ..............................................................................................................................10

3.2 Installing the PS225 ...........................................................................................................................................10

3.3 Setting Up the Serial Console (Optional)...........................................................................................................11

3.4 Accessing the PS225 from an x86 Host ............................................................................................................12

3.4.1 Installing L2 Driver on the x86 Host.........................................................................................................12

3.4.2 Assigning an IP Address to the First New Host Interface........................................................................13

3.4.3 Establishing an SSH Connection to the PS225.......................................................................................16

4 Upgrading the PS225 Software.................................................................................................................................17

5 Setting Up a Linux Distribution rootfs on the PS225..............................................................................................19

5.1 Running Ubuntu on a PS225 .............................................................................................................................19

5.1.1 Setting Up Ubuntu on a PS225................................................................................................................19

5.2 Running CentOS on a PS225............................................................................................................................25

5.2.1 Setting Up CentOS on the PS225 ...........................................................................................................25

Revision History............................................................................................................................................................ 30

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

1 Introduction

Broadcom®PS225 SmartNIC adapters, based on Broadcom's latest data center SoC technology, offer groundbreaking

network and compute power in a compact package. The PS225 adapters are PCI Express low-profile, half-length adapters,

and provide two 25-Gigabit Ethernet (25GbE) ports along with eight 64-bit ARMv8 processors running at an unmatched

3.0 GHz clock rate. The adapters are available in a range of on-board DDR4 memory sizes. The PS225-H04, PS225-H08,

and PS225-H16 adapters include 4 GB, 8 GB, and 16 GB of DDR4 memory, respectively.

1.1 Block Diagram

Figure 1 shows the main functional blocks of the PS225 SmartNIC adapter cards.

Figure 1: PS225 Block Diagram

PCIe 3.0

CPU

Subsystem

Ethernet 25GbE SerDes

DDR4

Ch. 1

72b

L3$

DDR

x16

DDR

x16 DDR

x16

DDR

x16

DDR

x16

SPI

8 MB

SMBus

NC-SI 20-pin

connector

PCIe Edge Connector

BCM58802H DDR4

Ch. 0

DDR

x16

DDR

x16 DDR

x16

DDR

x16

DDR

x16

eMMC

16 GB

UART 3.5 mm

connector

Accelerators

72b

VPD

FRU

SFP28

connector

SFP28

connector

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

1.2 Board Features

PCIe Interface:

– x8 PCI Express 3.0 compliant

– Single-Root I/O Virtualization (SR-IOV) with up to 128 VFs

– Function Level Reset (FLR) support

Processing Subsystem:

– 8-core ARMv8 A72 64-bit processor subsystem at 3.0 GHz

– 16 MB total cache (8 MB L2 + 8 MB L3)

– Two channels (64-bit + ECC) of DDR4 memory at 2,400 MT/s

Hardware Accelerators:

– TruFlow™ configurable flow accelerator engine

– Line-rate Crypto engine with single-pass hashing and encryption

– RAID 5/6 engine

Network Interface:

– Dual-port SFP28 pluggable media interface, supporting 25GbE or 10GbE optical transceiver or direct-attach copper

(DAC) cables

– RDMA over Converged Ethernet (RoCE) v1 and v2

– Advanced congestion avoidance

– Virtual network termination – VXLAN, NVGRE, Geneve, GRE

– Multiqueue, NetQueue, and VMQ

– Tunnel-aware stateless offloads:

IPv4 and IPv6

TCP, UDP, and IP checksum

Large send offload (LSO)

Large receive offload (LRO)

TCP segmentation offload (TSO)

Receive-side scaling (RSS)

Transmit-side scaling (TSS)

VLAN insertion/removal

– DCB support: PFC, ETS, QCN, DCBx

– Jumbo frames (up to 9 KB)

– Network boot (PXE, UEFI)

Security and Manageability:

– Secure Boot

– Secure Key Storage

– ARM PKA Engine

– NC-SI over MCTP (SMBus, PCIe VDM)

– NC-SI over RMB (via separate connector)

Form Factor:

– PCI Express CEM specification for half-height, half-length adapters

– I/O brackets available in both low profile and full height

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

1.3 Package Contents

The PS225 package contains the following components:

PS225 adapter with low-profile bracket installed

Full-height bracket

The PS225 card with a low-profile bracket installed is shown in Figure 2.

Figure 2: PS225 with Low-Profile Bracket

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

1.4 PS225 Card Variants

The PS225 card comes in the following three variants based on the amount of DDR memory:

PS225-H04 (4 GB DDR4, part number BCM958802A8024C/BCM958802A8044C)

PS225-H08 (8 GB DDR4, part number BCM958802A8028C/BCM958802A8048C)

PS225-H16 (16 GB DDR4, part number BCM958802A8021C)

The back of the card contains a sticker that indicates the board type. Figure 3 shows an example of the 4G card label on a

preproduction adapter.

Figure 3: Example Label



Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

2 Checklist

The following list of major steps must be completed to complete the setup of the PS225:

1. Install the PS225 card in a host machine (see Initial Board Setup and Connectivity).

2. Obtain the documentation listed in Table 1.

3. Install the latest binary/image package for the PS225. Upgrade if the PS225 card received has an old image.

NOTE: To check the image version information, refer to docSAFE (the Broadcom documentation portal) to download the

binary package required for the specific card type.

Package name format : NXS_SN-<release type>_<release version>_<card type>_bin.tar.gz

Example: NXS_SN-BM-Beta_1.1.8.0_H08GB_bin.tar

4. Become familiar with the following PS225 basic information:

– PS225 software version:

On PS225 Stingray (MAIA) console:

root@bcm958802a8048c:~# uname -a

Linux bcm958802a8048c 4.14.79+g5bb1623 #1 SMP Tue Nov 27 01:48:50 UTC 2018 aarch64

aarch64 aarch64 GNU/Linux

– PS225 board/card type (8 GB, 4 GB, 16 GB):

On PS225 Stingray (MAIA) console:

root@bcm958802a8048c:~# hostname

bcm958802a8048c

– Check the Nitro profile configuration.

NOTE: By default, the card is upgraded with 2 x 25G port mode and 8 + 8 PFs profile configuration with eight PFs

on the Stingray (MAIA) side and eight PFs on the host (x86) side.

Table 1: List of Reference Documentation

Document name Document number Description

SmartNIC Configuration Guide 5880X-PS225-AN1Xx Application note for configuring OVS+DPDK on the

PS225.

BCM5880X High Performance Ethernet System on Chip 5880X-DS1Xx BCM5880X Data Sheet

BCM5880X Hardware Design Guide 5880X-DG1Xx Hardware design guide

BCM5880X SmartNIC Solution 5880X-UG3Xx BCM5880X SmartNIC solution

PS225 SmartNIC Adapters Dual-Port 25 Gb/s PCI

Express 5880X-PS225-DS1Xx PS225 Advanced Data Sheet

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

Supported Nitro profile configurations in the PS225 released image package are as follows:

stingray_4p.pkg

stingray_2p.pkg

stingray_1p.pkg

BCM958802A8048C.pkg

bcm958802a8048_8+8_pf_rdma.pkg

bcm958802a8048_2xAN.pkg

bcm958802a8048_2xAN_8+5_pf_rep.pkg

bcm958802a8048_2xAN_8+4_pf_rep.pkg

bcm958802a8048_2xAN_8+4_pf.pkg

bcm958802a8048_2xAN_8+3_pf.pkg

bcm958802a8048_2xAN_8+2_baremetal.pkg

bcm958802a8048_2x25g.pkg

bcm958802a8048_2x25g_8+5_pf_rep.pkg

bcm958802a8048_2x25g_8+4_pf_rep.pkg

bcm958802a8048_2x25g_8+4_pf.pkg

bcm958802a8048_2x25g_8+3_pf.pkg

bcm958802a8048_2x25g_8+2_baremetal.pkg

bcm958802a8048_2x10g.pkg

bcm958802a8048_2x10g_8+5_pf_rep.pkg

bcm958802a8048_2x10g_8+4_pf_rep.pkg

bcm958802a8048_2x10g_8+4_pf.pkg

bcm958802a8048_2x10g_8+3_pf.pkg

bcm958802a8048_2x10g_8+2_baremetal.pkg

bcm958802a8048_1xAN.pkg

bcm958802a8048_1x25g.pkg

bcm958802a8048_1x10g.pkg

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101

PS225 User Guide Quick Start Guide

3 Initial Board Setup and Connectivity

This section describes the initial board setup and connectivity.

3.1 PS225 Connector Locations

There are two SFP28 ports and a UART console port for the ARM CPU subsystem located on the I/O panel of the board

(see Figure 4).

Figure 4: PS225 Connector Locations

NOTE: The UART console is accessible via the round 3.5 mm jack on the I/O panel. The interface uses TTL-level signaling

and can be accessed using a widely available USB-to-TTL cable of type TTL-232R-3V3-AJ (based on FTDI chip).

3.2 Installing the PS225

To install the PS225:

1. Power down the host system and remove the AC power.

2. Insert the PS225 card into a PCIe 3.0 slot (at least x8 size) in the host system.

3. Connect one end of the SFP cable to port 0 or port 1 on the PS225 (see Figure 5).

4. Connect the other end of the SFP cable to an Ethernet switch that supports 10G or 25G Ethernet (see Figure 5).

NOTE: Both Ethernet ports must be connected to link partners operating at the same speed: either both 25GbE or both

10GbE. The ports do not support mixing 25GbE and 10GbE links.

5. Optionally:

a. Connect the other port of the PS225 card to an Ethernet switch (see Figure 5).

b. Insert the 3.5 mm plug of a USB serial cable to the 3.5 mm socket on the front panel (see Figure 5).

ARM A72

UART Console Port 0 Port 1

Broadcom Confidential

Table of contents

Other Broadcom PCI Card manuals

Broadcom

Broadcom BCM94321MC User manual

Broadcom

Broadcom AirForce 54g Intensi-fi BCM94313HMG2L User manual

Broadcom

Broadcom BCM943162ZP User manual

Broadcom

Broadcom HBA 9400-16e User manual

Broadcom

Broadcom BCM94318MPG User manual

Broadcom

Broadcom BCM94319SDB User manual

Broadcom

Broadcom 95 Series User manual

Broadcom

Broadcom BCM94356Z User manual

Broadcom

Broadcom BRCM1002 User manual

Broadcom

Broadcom BCM94352Z User manual

Broadcom

Broadcom BCM943225HMB User manual

Broadcom

Broadcom BCM94321CB2 User manual

Broadcom

Broadcom BCM94322HM8L User manual

Broadcom

Broadcom BCM94311MCG User manual

Broadcom

Broadcom BCM943225HM User manual

Broadcom

Broadcom BCM94321COEX2 User manual

Broadcom

Broadcom BRCM1020 User manual

Broadcom

Broadcom BCM94312HMG User manual

Broadcom

Broadcom BCM943142HM User manual

Broadcom

Broadcom BCM94312HMGB User manual

Broadcom

Broadcom BCM943602CS User manual

Broadcom

Broadcom BCM94350ZAE User manual

Broadcom

Broadcom BCM943228Z User manual

Broadcom

Broadcom BCM943227HM4L User manual

Popular PCI Card manuals by other brands

ERMA

ERMA SSI 1417 instruction manual

SIIG

SIIG CyberSerial Dual PCI Software installation

Moxa Technologies

Moxa Technologies CP-102E Quick installation guide

I-Bus

I-Bus IBC2602 user guide

Exsys

Exsys EX-60102 manual

Chelsio Communications

Chelsio Communications T5 Series Installation and user guide

Conrad Electronic

Conrad Electronic 97 46 71 operating instructions

Micro Innovations

Micro Innovations USB650A Quick installation guide

KTI Networks

KTI Networks KF-310 user guide

Asus

Asus SpaceLink WL230 user manual

SIIG

SIIG SC2475 Quick installation guide

TRENDnet

TRENDnet TEW-703PI Quick installation guide

Belkin

Belkin F5D7000F user manual

Pericom

Pericom PI7C7100 manual

ADLINK Technology

ADLINK Technology cPCI-3548 user manual

Digium

Digium TE200 Series user manual

Allo.com

Allo.com 2nd Gen E1/T1/J1 Quick installation guide

Western Digital

Western Digital FireWire PCI Adapter installation guide