SuperBlade SBI-611E-1C2N - Server Supermicro - Free user manual and instructions

USER MANUAL SuperBlade SBI-611E-1C2N Supermicro

Internal view of a server rack with cooling fans, drive bays, and internal circuit boards (no visible text or labels)

USER'S MANUAL

Revision 1.0a

The information in this User's Manual has been carefully reviewed and is believed to be accurate. The vendor assumes no responsibility for any inaccuracies that may be contained in this document, and makes no commitment to update or to keep current the information in this manual, or to notify any person or organization of the updates. Please Note: For the most up-to-date version of this manual, please see our website at www.supermicro.com.

Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product described in this manual at any time and without notice. This product, including software and documentation, is the property of Supermicro and/or its licensors, and is supplied only under a license. Any use or reproduction of this product is not allowed, except as expressly permitted by the terms of said license.

IN NO EVENT WILL Super Micro Computer, Inc. BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE USE OR INABILITY TO USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. IN PARTICULAR, SUPER MICRO COMPUTER, INC. SHALL NOT HAVE LIABILITY FOR ANY HARDWARE, SOFTWARE, OR DATA STORED OR USED WITH THE PRODUCT, INCLUDING THE COSTS OF REPAIRING, REPLACING, INTEGRATING, INSTALLING OR RECOVERING SUCH HARDWARE, SOFTWARE, OR DATA.

Any disputes arising between manufacturer and customer shall be governed by the laws of Santa Clara County in the State of California, USA. The State of California, County of Santa Clara shall be the exclusive venue for the resolution of any such disputes. Supermicro's total liability for all claims will not exceed the price paid for the hardware product.

FCC Statement: This equipment has been tested and found to comply with the limits for a Class A or Class B digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in industrial environment for Class A device or in residential environment for Class B device. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the manufacturer's instruction manual, may cause harmful interference with radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case you will be required to correct the interference at your own expense.

California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. "Perchlorate Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate".

WARNING: This product can expose you to chemicals including lead, known to the State of California to cause cancer and birth defects or other reproductive harm. For more information, go to www.P65Warnings.ca.gov.

The products sold by Supermicro are not intended for and will not be used in life support systems, medical equipment, nuclear facilities or systems, aircraft, aircraft devices, aircraft/emergency communication devices or other critical systems whose failure to perform be reasonably expected to result in significant injury or loss of life or catastrophic property damage. Accordingly, Supermicro disclaims any and all liability, and should buyer use or sell such products for use in such ultra-hazardous applications, it does so entirely at its own risk. Furthermore, buyer agrees to fully indemnify, defend and hold Supermicro harmless for and against any and all claims, demands, actions, litigation, and proceedings of any kind arising out of or related to such ultra-hazardous use or sale.

Manual Revision 1.0a

Release Date: February 21, 2024 mk

Unless you request and receive written permission from Super Micro Computer, Inc., you may not copy any part of this document. Information in this document is subject to change without notice. Other products and companies referred to herein are trademarks or registered trademarks of their respective companies or mark holders.

Copyright © 2024 by Super Micro Computer, Inc.

All rights reserved.

Printed in the United States of America

Preface

About this Manual

This manual is written for professional system integrators and technicians. It provides information for the installation and use of the blade server. Installation and maintenance should be performed by experienced technicians only.

Please refer to the SBI-611E-1C2N/1T2N/5T2N server specifications page on our website for updates on supported memory, processors and operating systems.

Notes

For your system to work properly, please follow the links below to download all necessary drivers/utilities and the user's manual for your server.

• Supermicro product manuals: https://www.supermicro.com/support/manuals/
- Product drivers and utilities: https://www.supermicro.com/wdl
- Product safety info: https://www.supermicro.com/about/policies/safety_information.cfm

If you have any questions, please contact our support team at: [e-mail]

This manual may be periodically updated without notice. Please check the Supermicro website for possible updates to the manual revision level.

Secure Data Deletion

A secure data deletion tool designed to fully erase all data from storage devices can be found on our website: https://www.supermicro.com/about/policies/disclaimer.cfm?url=/wdl/utility/Lot9_Secure_Data_Deletion_Utility/

Warnings

Special attention should be given to the following symbols used in this manual.

Warning! Indicates important information given to prevent equipment/property damage or personal injury.

Warning! Indicates high voltage may be encountered when performing a procedure.

Contents

Chapter 1 Introduction

1.1 Overview....8 Eligible Enclosures....8

1.2 System Features....9 Front View....9 Drive Carrier Indicators....10 Control Panel....11 Top view....12

1.3 System Architecture ....14 1.4 Motherboard Layout ....15 Quick Reference ....16 Motherboard Block Diagram ....17

Chapter 2 Installation and Setup

2.1 Unpacking the System 18
2.2 Installing or Removing the Blade Unit....19 Installing a Blade Unit into the Enclosure....19 Removing a Blade Unit from the Enclosure....19

2.3 Powering Up or Down 20

2.4 Processor and Heatsink Installation....20 The Processor Carrier Assembly....21 The Processor Heatsink Module (PHM)....23 Installing the PHM into the CPU Socket....24 Removing the PHM from the CPU Socket....27 Removing the Processor Carrier Assembly from the PHM....28 Removing the Processor from the Carrier Assembly....29

2.5 Memory....30 Memory Support....30 Max Series CPUs....32 Memory Population Guidelines....32 Guidelines Regarding Mixing DIMMs....32 Installing Memory....33

2.6 Replacing the Battery....34 2.7 Storage Drives....35

Installing Drives....35

M.2 Solid State Drives ....38

Checking the Temperature of an NVMe Drive....39

Hot-Swap for NVMe Drives....40

2.8 System Cooling 41

Installing the Air Shrouds 41

Checking the Server Air Flow 42

Overheating....42

2.9 Installing the GPU 43

2.10 Installing the Operating System....44

2.11 Configuring RAID 44

2.12 Driver Installation 45

2.13 Installing SuperCap....46

Chapter 3 Motherboard Connections

3.1 Headers and Connectors ....49

3.2 Jumpers....52

Explanation of Jumpers....52

3.3 LED Indicators....54

Chapter 4 Software

4.1 Microsoft Windows OS Installation....55

4.2 Driver Installation....57

4.3 SuperDoctor ^® 5....58

4.4 BMC....59

BMC ADMIN User Password ....59

Chapter 5 Optional Components

5.1 TPM Security Module....60

5.2 Intel Virtual RAID on CPU (VROC)....61

Requirements and Restrictions....61

Supported SSDs and Operating Systems 61

Additional Information 62

Hardware Key 62

Configuring Intel VMD....63

Configuring VMD Manually....63

Creating NVMe RAID Configurations....67

Status Indications....68

Hot-Swap Drives 68

Related Information Links 68

Chapter 6 Troubleshooting and Support

6.1 Information Resources....69

Website 69

Direct Links for the SBI-611E-1C2N/1T2N/5T2N System 69

Direct Links for General Support and Information 69

6.2 BMC Interface 70

6.3 Troubleshooting Procedures 71

General Technique....71

No Power 71

No Video 71

System Boot Failure 72

Memory Errors 72

Losing the System Setup Configuration 72

When the System Becomes Unstable....72

6.4 Crash Dump Using the BMC Dashboard....74

6.5 UEFI BIOS Recovery ....75

Overview 75

Recovering the UEFI BIOS Image....75

Recovering the Main BIOS Block with a USB Device....75

6.6 CMOS Clear....80

6.7 BMC Reset....80

6.8 Where to Get Replacement Components....81

6.9 Reporting an Issue....81

Technical Support Procedures 81

Returning Merchandise for Service....81

Vendor Support Filing System 82

6.10 Feedback....82

6.11 Contacting Supermicro....83

Appendix A BIOS POST Codes

Appendix B Standardized Warning Statements for AC Systems

Appendix C Specifications and Compliance

Contacting Supermicro

Headquarters

Address: Super Micro Computer, Inc.

980 Rock Ave.

San Jose, CA 95131 U.S.A.

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Sales Inquiries)

[e-mail] (Gov. Sales Inquiries)

[e-mail] (Technical Support)

[e-mail] (RMA Support)

[e-mail] (Webmaster)

Website: www.supermicro.com

Europe

Address: Super Micro Computer B.V.

's-Hertogenbosch, The Netherlands

Tel: [phone]

Fax: [phone]

Email: [e-mail] (Sales Inquiries)

[e-mail] (Technical Support)

[e-mail] (RMA Support)

Website: www.supermicro.nl

Asia-Pacific

Address: Super Micro Computer, Inc.

3F, No. 150, Jian 1st Rd.

Zhonghe Dist., New Taipei City 235

Taiwan (R.O.C)

Tel: [phone]

Fax: [phone]

Email: [e-mail] (Sales Inquiries)

[e-mail] (Technical Support)

[e-mail] (RMA Support)

Website: www.supermicro.com.tw

Chapter 1

Introduction

1.1 Overview

The SBI-611E-1C2N/1T2N/5T2N blade is a compact self-contained server that connects into a pre-cabled enclosure that provides power, cooling, management and networking functions. One enclosure can hold up to ten blades. Each blade contains one computing node.

In this manual, “blade” or “blade unit” refers to a single blade, and “blade enclosure” is the chassis that houses the blades, power supplies and other modules. “Blade system” refers to the enclosure, blades units, and various management and networking modules.

Models
System Model	Blades per Enclosure	Max CPU TDP (Air cooled)	Storage Types
SBI-611E-1C2N 10 250W			SAS/SATA/NVMe drive ports with Broadcom 3108
SBI-611E-1T2N 10 250W SATA/NVMe drive ports
SBI-611E-5T2N (double width) 5 350W SATA/NVMe drive ports

System Overview
Motherboard	B13SEE-CPU-25G
SuperBlade Sled	MCP-680-61006-0N
Processor	4th or 5th Generation Intel Xeon Scalable processor in socket E (LGA4677)
Memory	Supports up to 4TB of DDR5 ECC 3Ds RDIMM/3DS LRDIMM/Registered R/LR/3DSLR memory with speeds of up to 4800 MT/s (4th Gen) or 5600 MT.s (5th Gen); DIMM size up to 256GB
Storage Drives	Two hot-swap U.2 2.5" bays:SBI-611E-1C2N, two U.2 SAS/SATA/NVMe; one M.2 SATA/NVMeSBI-611E-1T2N, two U.2 SATA/NVMe; three M.2 NVMe; two E1.SSBI-611E-5T2N, two U.2 SATA/NVMe; three M.2 NVMe; two E1.S
LAN	Two 25GbE connections
Expansion Slots	One PCIe Gen5 x16 slot, one PCIe Gen5 x8 slot, support one FHFL double-width GPU or two single-width PCIe cards
System Cooling	A set of air shrouds, one CPU heatsink
Form Factor	6U height; 9.75" x 1.75" x 23.5" (HxWxD) (24.8 x 4.4 x 59.7 cm)SBI-611E-5T2N width, 3.5" (8.8cm)

Eligible Enclosures

• SBE-610J2-822
• SBE-610J2-622
• SBE-610J2-422

1.2 System Features

The following views of the system display the main features.

Front View

Control Panel Storage Drive Storage Drive Latch Lever Service Tag Latch Lever

Figure 1-1. Front View, SBI-611E-1C2N, SBI-611E-1T2N

Control Panel Storage Drive Storage Drive Latch Lever Service Tag Latch Lever

Figure 1-2. Front View, SBI-611E-5T2N

Chassis Front Features
Features Descriptions
Control Panel	Power button and status indicators
Storage Drives	Two hot-swap 2.5" U.2 drive bays; the drive carriers display status lights
Latch Levers	Levers for two latches that secure the blade in the enclosure
Service/Asset Tag	Pull-out identifier (with BMC ADMIN default password sticker)

Drive Carrier Indicators

Each drive carrier has two LED indicators: an activity indicator and a status indicator. For RAID configurations using a controller, the meaning of the status indicator is described in the table below. For OS RAID or non-RAID configurations, some LED indications are not supported, such as hot spare.

Drive Carrier LED Indicators
	Color Blinking Pattern Behavior for Device
Activity LED	Blue Solid On Idle SAS/NVMe drive installed
	Blue Blinking I/O activity
	Off Idle SATA drive
Status LED	Red Solid On Failure of drive with RSTe support
	Red Blinking at 1 Hz Rebuild drive with RSTe support
	Red Blinking with two blinks and one stop at 1 Hz	Hot spare for drive with RSTe support
	Red On for five seconds, then off	Power on for drive with RSTe support
	Red Blinking at 4 Hz Identify drive with RSTe support

Control Panel

KVM/UID LEDSystem Fail Power LEDNIC LED

Figure 1-2. Control Panel

Control Panel Features
Feature State	Description
Power button		The main power switch applies or removes primary power from the power supply to the server but maintains standby power.
Power LED	Green Power on
	Amber steady Power off
	Amber blinks Before the BMC is ready, blinks until every node is ready
KVM/UID LED	Steady blue Indicates that KVM has been initialized
	Flashing blue	Unit Identifier indicator (The UID function is activated with a management program.)
NIC LED	Flashing green	Indicates traffic (Tx and RX data) on the LAN connection to this blade module
	Flashing orange Indicates traffic over the network (when present in the system)
System Fail LED	Red	Indicates a fatal error. This may be a memory error, a VGA error or any other fatal error that prevents the operating system from booting.

Top view

M.2 Socket DIMMs Processor and Heatsink M.2 Sockets LSI SAS Controller Disk Cage Latch Levers GPU

Figure 1-3. Components Labeled (SBI-611E-1C2N model shown)

Blade Components
Item	Feature Description
1	Drives Two 2.5" storage drives
2	CPU Processors and heatsinks
3	Memory DIMM slots
4	Cover Partial chassis cover
5	Mezzanine Network mezzanine add-on card
6	Latch Levers Levers for two latches that secure the blade in the enclosure
7	LSI SAS Controller For SAS support
8	GPU Single width (double width in SBI-611E-5T2N)

Exploded view diagram of a server rack with numbered components for identification

Figure 1-4. Components, SBI-611E-1C2N

1.3 System Architecture

This section shows system block diagrams.

SBI-611E-1T2N/5T2N SBI-611E-1C2N

graph TD
    subgraph_Left_Side["PCI-E Gen.5 x16"]
        A["CPU"] --> B["Mezz2"]
        B --> C["2600"]
        C --> D["C741"]
        D --> E["E810"]
        F["PCI-E Gen.5 x16"] --> G["M.2 22280/22110"]
        H["PCI-E Gen.5 x16"] --> I["M.2 22280/22110"]
        J["PCI-E Gen.5 x16"] --> K["M.2 22280/22110"]
    end
    subgraph_Right_Side["PCI-E Gen.5 x16"]
        L["CPU"] --> M["Mezz2"]
        M --> N["2600"]
        N --> O["C741"]
        O --> P["E810"]
        Q["PCI-E Gen.5 x16"] --> R["M.2 22280/22110"]
        S["PCI-E Gen.5 x16"] --> T["M.2 22280/22110"]
        U["PCI-E Gen.5 x16"] --> V["M.2 22280/22110"]
    end
    Left_Side --> Right_Side
    style Left_Side fill:#f9f,stroke:#333
    style Right_Side fill:#bbf,stroke:#333

Figure 1-5. System Block Diagrams

1.4 Motherboard Layout

Below is a layout of the B13SEE-CPU-25G motherboard with jumper, connector and LED locations shown. See the table on the following page for descriptions. For detailed descriptions, pinout information and jumper settings, refer to Chapter 3 or the Motherboard Manual.

Jumper Description Default Setting

JBRE1 BIOS Recovery Pins 1-2 (Normal)

JPG1 VGA Enable/Disable Pins 1-2 (Enable)

JPME1 ME Recovery Mode Pins 1-2 (Normal)

JPME2 Manufacturing Mode Pins 1-2 (Normal)

JWD1 Watchdog Timer Pins 1-2 (Reset)

Connector Description

BT1 Onboard CMOS Battery
J1 Chassis Backplane Connector
J5 Front Panel Connector
J6, J7, J8, J9	PCIe 5.0 x8 MCIO Connector
J10	PCIe 4.0 x16 SIOM Connector for one SAS card or two M.2 x4 or two PCIe 4.0 NVMe
JKVM1	VGA/USB Module Connector
JGPU1	12V GPU Power Connector (ATX 8-pin)
JRC1	Proprietary Riser Power Connector
JRC2	Proprietary Riser Power Connector
JREK1	Intel RAID Key Header
JMEZZ1	PCIe 4.0 x16 Mezzanine Card Connector for AOM
JTPM1	Trusted Platform Module (TPM)/Port 80 Connector
M.2-HC	M.2 M-Key PCIe 3.0 for SATA 3.0 Connector (2280/22110)
PWR1	Power Receptacle to Chassis Backplane
PWR2	Power Receptacle to Chassis Backplane
USB0	Internal USB 2.0/3.0 Type-A Connector
MH_G1 – MH_G2	Mounting Holes
MH1 – MH9	Mounting Holes

LED	Description	State: Status
LE6	Power Indicator	On: Power is normal
LED1	BMC Heartbeat LED	Blinking Green: BMC Normal

Motherboard Block Diagram

graph TD
    A["CPU"] --> B["IMC"]
    A --> C["PEG"]
    A --> D["PEG (ICX-80L)"]
    A --> E["DMI"]
    A --> F["PCH"]
    A --> G["Chassis Backplane"]

    subgraph CPU
        H["2x MCIO x8"] --> I["PCIe 5.0 x16 32 GT/s"]
        J["2x MCIO x8"] --> K["PCIe 5.0 x16 32 GT/s"]
        L["Intel E810XXVAM2"] --> M["PCIe 4.0 x8 16.0 GT/s"]
        N["2x 25GBASE.KX 25Gbps"] --> O["PCIe 4.0 x8 16.0 GT/s"]
        P["PCIe 5.0 x16"] --> Q["PCIe 5.0 x16 32 GT/s"]
        R["PCIe 4.0 x8"] --> S["PCIe 4.0 x8 16.0 GT/s"]
        T["PCIe 4.0 x4"] --> U["PCIe 4.0 x4 16.0 GT/s"]
        V["PCIe 4.0 x4"] --> W["PCIe 4.0 x4 16.0 GT/s"]
        X["LSI300V HDD1"] --> Y["Storage Module"]
        Z["SATAO-2"] --> AA["DMI"]
        AB["USB TYPE-A"] --> AC["PCIe 3.0 x4 8.0 GT/s"]
        AD["M.2 SATA/NVMe"] --> AE["PCIe 3.0 x4 8.0 GT/s"]
        AF["eMMC"] --> AG["PCIe 3.0 x4 8.0 GT/s"]
        AH["PCIe 1.0 x1 2.5 GT/s"] --> AI["PTA5"]
        AJ["eSPI"] --> AK["PTA5"]
        AL["PTA5"] --> AM["PTA5"]
        AN["PTA5"] --> AO["PTA5"]
        AP["PTA5"] --> AQ["PTA5"]
        AR["PTA5"] --> AS["PTA5"]
        AT["PTA5"] --> AU["PTA5"]
        AV["PTA5"] --> AW["PTA5"]
        AX["PTA5"] --> AY["PTA5"]
        AZ["PTA5"] --> BA["PTA5"]
        BB["PTA5"] --> BC["PTA5"]
        BD["PTA5"] --> BE["PTA5"]
        BF["PTA5"] --> BG["PTA5"]
        BH["PTA5"] --> BI["PTA5"]
        BJ["PTA5"] --> BK["PTA5"]
        BL["PTA5"] --> BM["PTA5"]
        BN["PTA5"] --> BO["PTA5"]
        BP["PTA5"] --> BQ["PTA5"]
        BR["PTA5"] --> BS["PTA5"]
        BT["PTA5"] --> BU["PTA5"]
        BV["PTA5"] --> BW["PTA5"]
        BX["PTA5"] --> BY["PTA5"]
    end

    subgraph PCH
        Z["SATAO-2"] --> AA
        AA --> AB["PCE8_11 SATA5"]
        AB --> AC["PCE5"]
        AC --> AD
        AD --> AE
        AE --> AF
        AF --> AG["eMMC"]
        AG --> AH
        AI["JKVM1"] --> AJ["RGB (JAR)"]
        AJ --> AK["RGB (JAR)"]
        AL["AST2600"] --> AM["PCE8_11 SATA5"]
        AM --> AN["PCE5"]
        AN --> AO["eSPI 50 MHz"]
        AP["PCE8_11 SATA5"] --> AQ["eSPI 1x USB 2.0 480 Mbps"]
        AR["PCE8_11 SATA5"] --> AS["eSPI 1x USB 2.0 480 Mbps"]
        AT["PCE8_11 SATA5"] --> AU["eSPI 1x USB 2.0 480 Mbps"]
        AV["PCE8_11 SATA5"] --> AW["eSPI 1x USB 2.0 480 Mbps"]
        AX["PCE8_11 SATA5"] --> AY["eSPI 1x USB 2.0 480 Mbps"]
        AZ["PCE8_11 SATA5"] --> BA["eSPI 1x USB 2.0 480 Mbps"]
        BB["PCE8_11 SATA5"] --> BB["eSPI 1x USB 2.0 480 Mbps"]
        BC["PCE8_11 SATA5"] --> AD
    end

    subgraph AST2600
        AD["PCE8_11 SATA5"] --> AE["PCE5"]
        AE --> AF
        AF["PCE5"] --> AG["eSPI 50 MHz"]
        AH["PCE8_11 SATA5"] --> AI["eSPI 1x USB 2.0 480 Mbps"]
        AJ["PCE8_11 SATA5"] --> AK["eSPI 1x USB 2.0 480 Mbps"]
        AL["PCE8_11 SATA5"] --> AM["eSPI 1x USB 2.0 480 Mbps"]
        AN["PCE8_11 SATA5"] --> AN["eSPI 1x USB 2.0 480 Mbps"]
        AO["PCE8_11 SATA5"] --> AO["eSPI 1x USB 2.0 480 Mbps"]
        AP["PCE8_11 SATA5"] --> AP["eSPI 1x USB 2.0 480 Mbps"]
        AQ["PCE8_11 SATA5"] --> AQ["eSPI 1x USB 2.0 480 Mbps"]
        AR["PCE8_11 SATA5"] --> AR["eSPI 1x USB 2.0 480 Mbps"]
        AS["PCE8_11 SATA5"] --> AS["eSPI 1x USB 2.0 480 Mbps"]
        AT["PCE8_11 SATA5"] --> AT["eSPI 1x USB 2.0 480 Mbps"]
        AU["PCE8_11 SATA5"] --> AU["eSPI 1x USB 2.0 480 Mbps"]
        AV["PCE8_11 SATA5"] --> AV["eSPI 1x USB 2.0 480 Mbps"]
        AW["PCE8_11 SATA5"] --> AW["eSPI 1x USB 2.0 480 Mbps"]
        AX["PCE8_11 SATA5"] --> AX["eSPI 1x USB 2.0 480 Mbps"]
        AY["PCE8_11 SATA5"] --> AZ["eSPI 1x USB 2.0 480 Mbps"]
        BA["PCE8_11 SATA5"] --> BB["eSPI 1x USB 2.0 480 Mbps"]
        CA["PCE8_11 SATA5"] --> CA["eSPI 1x USB 2.0 480 Mbps"]
        AD["PCE8_11 SATA5"] --> AD["eSPI 1x USB 2.0 480 Mbps"]
        AE["PCE8_11 SATA5"] --> AE["eSPI 1x USB 2.0 480 Mbps"]
        AF["PCE8_11 SATA5"] --> AF["eSPI 1x USB 2.0 480 Mbps"]
        AG["PCE8_11 SATA5"] --> AG["eSPI 1x USB 2.0 480 Mbps"]
        AH["PCE8_11 SATA5"] --> AH["eSPI 1x USB 2.0 480 Mbps"]
        AI["PCE8_11 SATA5"] --> AI["eSPI 1x USB 2.0 480 Mbps"]
        AJ["PCE8_11 SATA5"] --> AJ["eSPI 1x USB 2.0 480 Mbps"]
        AK["PCE8_11 SATA5"] --> AK["eSPI 1x USB 2.0 480 Mbps"]
        AL["PCE8_11 SATA5"] --> AL["eSPI 1x USB 2.0 480 Mbps"]
        AM["PCE8_11 SATA5"] --> AM["eSPI 1x USB 2.0 480 Mbps"]
        AN["PCE8_11 SATA5"] --> AN["eSPI 1x USB 2.0 480 Mbps"]
        AO["PCE8_11 SATA5"] --> AO["eSPI 1x USB 2.0 480 Mbps"]
        AP["PCE8_11 SATA5"] --> AP["eMPIC"]
    end

    subgraph CPU
    B
    C
    D
    E
    F
    G
    H
    I
    J
    K
    L
    M
    N
    O
    P
    Q
    R
    S
    T
    U
    V
    W
    X
    Y
    Z
    AA
    AB
    AC
    AD
    AE
    AF
    AG
    AH
    AI
    AJ
    AK
    AL
    AM
    AN
    AO
    AP
    AQ
    AR
    AS
    AT
    AU
    AV
    AW
    AX
    AZ
    BA
    BB
    AC
    AD
    AE
    AF
    AG
    AH
    AI
    AJ
    AK
    AL
    AM
    AN
    AO
    AP
    AQ
    AR
    AS
    AT
    AU
    AV
    AW
    AX
    AZ
    BA
    BB
    AC
    AD
    AE
    AF
    AG
    AH
    AI
    AJ)

Figure 1-9. Motherboard Block Diagram

Chapter 2

Installation and Setup

This chapter provides instructions on installing and replacing main system components. To prevent compatibility issues, only use components that match the specifications or part numbers.

Up to ten blade modules may be installed into a blade enclosure, depending upon your enclosure and blade. Blade modules with Windows and Linux operating systems may be mixed together in the same blade enclosure.

2.1 Unpacking the System

Inspect the box the system was shipped in and note if it was damaged in any way. If any equipment appears damaged, please file a damage claim with the carrier who delivered it.

Decide on a suitable location for the rack unit that will hold the enclosure. It should be situated in a clean, dust-free area that is well ventilated. Avoid areas where heat, electrical noise and electromagnetic fields are generated. It will also require a grounded AC power outlet nearby. Be sure to read the precautions and considerations noted in Appendix B.

2.2 Installing or Removing the Blade Unit

Installing a Blade Unit into the Enclosure

1. Pull the locking levers out and apart, and push the blade into its bay. Caution: Insert the blade carefully so the rear connectors are not damaged.

2. As the blade is seated in the enclosure, push the levers into their locked position.

Locking Levers

Figure 2-1. Blade Locking Levers

Removing a Blade Unit from the Enclosure

A blade can be removed from the enclosure while other blades continue to operate.

Note: When a blade is removed, please ensure the slot is covered with a dummy blade/chassis to ensure the neighboring blade does not overheat.

Removing a Blade Unit from the Enclosure

1. Power down the blade unit.
2. Push and hold the two locking levers out and away from each other.
3. Use the locking levers to pull the blade from the enclosure.

2.3 Powering Up or Down

Each blade unit may be powered on and off independently from the rest of the blades in the enclosure.

A blade unit may be powered down or up in any of the following ways:

• Press the power button on the blade unit.
• Use IPMIView or the browser based management utility to power down; requires Operator or Admin privileges on the CMM.
• Use SMCIPMItool when connected to the CMM to power down; requires Operator or Admin privileges on the CMM.
• Use IPMI view or a browser connected to the onboard BMC chip to power down.
• Use SMCIPMltool to use a Command Line Interface to the onboard BMC chip; requires Operator or Admin privileges.

2.4 Processor and Heatsink Installation

The processor (CPU) and processor carrier should be assembled together first to form the processor carrier assembly. This will be attached to the heatsink to form the processor heatsink module (PHM) before being installed onto the CPU socket.

Notes:

• Use ESD protection.
• Unplug the AC power cord from all power supplies.
• Check that the plastic protective cover is on the CPU socket and that none of the socket pins are bent. If they are, contact your retailer.
• When handling the processor, avoid touching or placing direct pressure on the land grid array (gold contacts).
• Improper installation or socket misalignment can cause serious damage to the processor or the socket and may require manufacturer repairs.
• Thermal grease is pre-applied on new heatsinks. No additional thermal grease is needed.
  • Refer to the Supermicro website for updates on processor support.
• Graphics in this manual are for illustration only. Your components may look different.

The Processor Carrier Assembly

The processor carrier assembly is comprised of the processor and the processor carrier.

1. Hold the processor with the land grid array (LGA, gold contacts) facing down. Locate the gold triangle at the corner of the processor and the corresponding hollowed triangle on the processor carrier as shown below. These triangles indicate the location of pin 1.

Pin 1

Processor

Pin 1

Carrier

1. Turn the processor over (with the gold LGA up). Locate the CPU keys on the processor and the four latches on the carrier as shown below.

Processor (Reverse Side Up) Latch Carrier (Top Side Up) CPU Key Latch Latch Latch Latch CPU Key

1. Locate the lever on the carrier and, if necessary, press it down as shown below.

Lever

1. Align the CPU keys on the processor (A & B) with those on the carrier (a & b) as shown below.

CPU Key (on the processor) CPU Key (on the carrier) Latch 1 B b A 2 a Latch CPU Key (on the processor) CPU Key (on the carrier)

1. Carefully place one end of the processor under latch 1 on the carrier, and then press the other end down until it snaps into latch 2 and is properly seated on the carrier.

Processor Carrier Assembly (Top View) Processor Carrier Assembly (Underside view)

The Processor Heatsink Module (PHM)

After creating the processor carrier assembly, mount the heatsink onto the carrier assembly to form the processor heatsink module (PHM).

Note: If this is a new heatsink, the thermal grease has been pre-applied. Otherwise, apply the proper amount of thermal grease to the underside of the heatsink.

1. Turn the heatsink over with the thermal grease facing up. Note the two triangle cutouts (A, B) located at the diagonal corners of the heatsink as shown in the drawing below.

2. On the processor carrier assembly, find pin 1, as noted by the triangles. Hold the processor carrier assembly over so that the gold LGA is facing up.

3. Align clip "a" (pin 1) on the carrier assembly with the triangular cutout A on the heatsink and b, c, d on the carrier assembly with B, C, D on the heatsink.

4. Push the carrier assembly onto the heatsink, making sure that all four clips on each corner are properly secured.

Processor Carrier Assembly a b d Pin1 C D B A C Heatsink

Installing the PHM into the CPU Socket

1. Remove the plastic protective cover from the CPU socket. Gently squeeze the grip tabs then pull the cover off.

CPU Socket with Plastic Protective Cover Grip Tabs

1. Locate four threaded fasteners (a, b, c, d) on the CPU socket.

CPU Socket Threaded Fastener a b c d (a, b, c, d: Threaded Fasteners) CPU Socket Pin1

1. Locate four PEEK nuts (A, B, C, D) and four rotating wires (1, 2, 3, 4) on the heatsink as shown below.

A, B, C, D: Peek Nut 1, 2, 3, 4: Rotating Wire a, b, c, d: Threaded Fastener Heatsink Rotating Wire 2 B D Rotating Wire 4 Rotating Wire 3 Rotating Wire 1 Peek Nut CPU Socket Rotating Wire Peek Nut (Unlatched) (latched)

1. Check that the rotating wires (1, 2, 3, 4) are in the unlatched position as shown.

Unlatched State Rotating Wire Side View Top View Peek Nut

1. Align nut A (next to the triangles and pin 1) on the heatsink with threaded fastener "a" on the CPU socket. Also align nuts B, C, D on the heatsink with threaded fasteners b, c, d on the CPU socket.

2. Gently place the heatsink on the CPU socket, making sure that each nut is properly aligned with its corresponding threaded fastener.

A, B, C, D: Peek Nut on the Heatsink B D C A b c d a, b, c, d: Threaded Fastener on the GPU socket

1. Press all four rotating wires outward to latch the PHM onto the CPU socket.

Rotating Wire Rotating Wire Latched State Top View

1. With a t30-bit screwdriver, tighten all PEEK nuts in the sequence of A, B, C, and D with even pressure not greater than 12 lbf-in.

Technical diagram showing a mechanical assembly with labeled components and directional arrow indicating transformation or assembly.

Removing the PHM from the CPU Socket

Be sure the system is shut down and all AC power cords are unplugged.

1. Use a t30-bit screwdriver to loosen the four PEEK nuts on the heatsink in the sequence of A, B, C, and D.

B D Peak Nut C

1. Press the four rotating wires inward to unlatch the PHM as shown below.

Unlatched State Rotating Wire Side View Peek Nut

1. Gently lift the PHM upward to remove it from the CPU socket.

Technical line drawing of a mechanical housing with mounting holes and internal components, showing two views with red arrows indicating movement or assembly (no text or symbols present)

Removing the Processor Carrier Assembly from the PHM

Detach the four plastic clips (a, b, c, d) on the processor carrier assembly from the four corners of the heatsink (A, B, C, D) as shown below, and lift off the processor carrier assembly.

Removing the Processor from the Carrier Assembly

Unlock the lever from its locked position and push it upwards to disengage the processor from the carrier as shown below right. Carefully remove the processor from the carrier.

Processor Carrier Assembly

Isometric technical drawing of a microchip or integrated circuit board (no text or symbols visible)

Technical diagram showing a mechanical assembly with an inset view labeled 'Lever' indicating a lever mechanism.

Note: Handle the processor with care to avoid damage.

Isometric technical diagram of a computer motherboard with a highlighted component and red arrow indicating upward motion (no text or symbols present)

2.5 Memory

Memory Support

The system supports up to 4TB of DDR5 ECC 3Ds RDIMM/3DS LRDIMM/Registered R/LR/3DSLR memory with speeds of up to 4800 MT/s (4th Gen) or 5600 MT.s (5th Gen); DIMM size up to 256GB. For validated memory, use our Product Resources page.

DDR5 Memory Support for the 4th Generation Intel Xeon Scalable Processors-SP
Type	Ranks Per DIMM and Data Width (Stack)	DIMM Capacity (GB)		Speed (MT/s)
				One DIMM per Channel1	Two DIMMs per Channel
		16 Gb Chip	24Gb 2 Chip	1.1 Volts
RDIMM	SRx8 (RC D) 16GB	24GB		4800* 4400*
	SRx4 (RC C) 32GB	48GB
	SRx4 (RC F) 9x4 32GB NA
	DRx8 (RC E) 32GB	48GB
	DRx4 (RC A) 64GB	96GB
	DRx4 (RC B) 9x4 64GB NA
RDIMM 3DS (4R/8R) x4 (RC A)		2H-128 GB4H-256 GB	NA

*Memory speed and capacity support depends on the processors used in the system.
Note 1: 1DPC applies to 1SPC or 2SPC implementations (SPC - sockets per channel).
Note 2: 24Gb XCC only with limited configs: 1DPC all DIMM types, 2DPC 96GB only. Only 8 and 16 DIMM configs, no failbacks.

DDR5 Memory Support for the 5th Generation Intel Xeon Scalable Processors

Type	Ranks Per DIMM and Data Width (Stack)	DIMM Capacity (GB)		Speed (MT/s)
				One DIMM per Channel1	Two DIMMs per Channel
		Memory Density 16 Gb	Memory Density 24Gb2	1.1 Volts
RDIMM	SRx8 (RC D) 16GB	24GB		5600* 5200*
	SRx4 (RC C) 32GB	48GB
	SRx4 (RC F) 9x4 32GB NA
	DRx8 (RC E) 32GB	48GB
	DRx4 (RC A) 64GB	96GB
	DRx4 (RC B) 9x4 64GB NA
RDIMM 3DS (4R/8R) x4 (RC A)		2H-128 GB4H-256 GB	NA

*Memory speed and capacity support depends on the processors used in the system.
Note 1: 1DPC applies to 1SPC or 2SPC implementations (SPC - sockets per channel).
Note 2: 24Gb XCC only with limited configs: 1DPC all DIMM types, 2DPC 96GB only. Only 8 and 16 DIMM configs, no failbacks.
Note 3: For 1DPC 5600MT/s speed, DDR5-5600 DIMMs are required
Note 4: Mixing DRAM density (16 Gb/24 Gb) and/or frequency is not allowed.

P1-DIMMA2 P1-DIMMA1 P1-DIMMB2 P1-DIMMB1 P1-DIMMC2 P1-DIMMC1 P1-DIMMD2 P1-DIMMD1 CPU P1DIMMH1 P1DIMMH2 P1DIMMG1 P1DIMMG2 P1DIMMF1 P1DIMMF2 P1DIMME1 P1DIMME2

Figure 2-2. Memory Slots

Use the DIMM slots listed below for memory modules. This memory population table is based on guidelines provided by Intel to support Supermicro motherboards.

Memory Population for 16 DIMM Slots, MCC or XCC CPUs
DIMMs Slots
1 DIMM	A1 or B1 or E1 or F1
2 DIMMs	A1 and G1, or C1 and E1
4 DIMMs	A1, C1, E1, G1
6 DIMMs	A1, C1, D1, E1, F1, G1 or A1, B1, C1, E1, G1, H1 or B1, C1, D1, E1, F1, H1 or A1, B1, D1, F1, G1, H1
8 DIMMs	A1, B1, C1, D1, E1, F1, G1, H1
12 DIMMs	A1, A2, B1, C1, C2, D1, E1, E2, F1, G1, G2, H1 or A1, B1, B2, C1, D1, D2, E1, F1, F2, G1, H1, H2
16 DIMMs	A1, A2, B1, B2, C1, C2, D1, D2, E1, E2, F1, F2, G1, G2, H1, H2

Other Intel validated memory configurations with an even number of DIMMs are supported, although they may not provide optimal performance. See Intel documentation for more information.

Max Series CPUs

Memory Population for 16 DIMM Slots, Max CPU
DIMMs Slots
1 DIMM	A1 or E1
2 DIMMs	A1 and G1, or C1 and E1
4 DIMMs	A1, C1, E1, G1
8 DIMMs	A1, B1, C1, D1, E1, F1, G1, H1
16 DIMMs	A1, A2, B1, B2, C1, C2, D1, D2, E1, E2, F1, F2, G1, G2, H1, H2

Notes:

- Max Series (HBM) CPU supports 1DPC (4800MT/s) / 2DPC (4400MT/s) to optimize the memory bandwidth. Max Series (HBM) CPU supports 1, 2, 4, 8, or 16 DIMMs in Flat Mode as well as Cache Mode, and 0 DIMMs in HBM-Only mode. HBM-Only mode runs exclusively using HBM memory.

- SPR+HBM supports 4, 8, or 16 DIMMs in all modes (Flat / Cached and Quadrant / SNC4)

4 DIMMs -> populate 1 DIMM/iMC 8 DIMMs -> populate 1 DIMM/Channel, 2 DIMM/iMC 16 DIMMs -> populate 1 DIMM/Channel, 4 DIMM/iMC

- All other configurations not listed above are not supported.

- For 2S design, each socket has to be populated identically.

Memory Population Guidelines

• All DIMMs must be DDR5.

- Balanced memory. Using unbalanced memory topology, such as populating two DIMMs in one channel while populating one DIMM in another channel, reduces performance. It is not recommended for Supermicro systems.

- In single-CPU configurations, memory must be installed in the DIMM slots associated with the installed CPU.

- For MM, NM/FM ratio is between 1:4 and 1:16. The capacity not used for FM can be used for AD. (NM = Near Memory; FM = Far Memory).

Guidelines Regarding Mixing DIMMs

• Populating slots with a pair of DIMM modules of the same type and size results in interleaved memory, which improves memory performance.
• Use memory modules of the same type and speed, as mixing is not allowed.
• x4 and x8 DIMMs can be mixed in the same channel.

Installing Memory

ESD Precautions

Electrostatic Discharge (ESD) can damage electronic components including memory modules. To avoid damaging DIMM modules, it is important to handle them carefully. The following measures are generally sufficient.

• Use a grounded wrist strap designed to prevent static discharge.
• Handle the memory module by its edges only.
• Put the memory modules into the antistatic bags when not in use.

Installing Memory

Begin by removing power from the system as described in Section 3.1. Follow the memory population sequence in the table above.

1. Push the release tabs outwards on both ends of the DIMM slot to unlock it.

Notches Release Tabs

1. Align the key of the DIMM with the receptive point on the memory slot and with your thumbs on both ends of the module, press it straight down into the slot until the module snaps into place.

Key

1. Ensure the release tabs are in the locked position to secure the DIMM module into the slot.

Caution: Exercise extreme caution when installing or removing memory modules to prevent damage to the DIMMs or slots.

Removing Memory

To remove a DIMM, unlock the release tabs then pull the DIMM from the memory slot.

2.6 Replacing the Battery

The motherboard uses non-volatile memory to retain system information when system power is removed. This memory is powered by a lithium battery residing on the motherboard.

Replacing the Battery

Begin by removing power from the system.

1. Push aside the small clamp that covers the edge of the battery. When the battery is released, lift it out of the holder.
2. To insert a new battery, slide one edge under the lip of the holder with the positive (+) side facing up. Then push the other side down until the clamp snaps over it.

Note: Handle used batteries carefully. Do not damage the battery in any way; a damaged battery may release hazardous materials into the environment. Do not discard a used battery in the garbage or a public landfill. Please comply with the regulations set up by your local hazardous waste management agency to dispose of your used battery properly.

LITHIUM BATTERY BATTERY HOLDER

Figure 2-3. Installing the Onboard Battery

Warning: There is a danger of explosion if the onboard battery is installed upside down (which reverses its polarities). This battery must be replaced only with the same or an equivalent type recommended by the manufacturer (CR2032).

2.7 Storage Drives

The blade has two 2.5" hot-swap storage drive bays. The drives are mounted in tool-less drive carriers that simplify their removal from the chassis. These carriers also help promote proper airflow. Carriers without drives must remain in the chassis for proper airflow.

Note: Enterprise level storage drives are recommended for use in Supermicro systems. For information on recommended drives, visit the Supermicro website.

Installing Drives

Front view of a server rack with hexagonal mesh back panel and two labeled buttons (1 and 0) on the right side (no readable text or symbols beyond labels)

Figure 2-4. Logical Drive Numbers

Removing a Hot-Swap Drive Carrier from the Chassis

1. Press the release button on the drive carrier, which will extend the drive carrier handle.

2. Use the drive carrier handle to pull the drive out of the chassis.

Diagram of an electronic device with labeled components and green arrows indicating rotation or direction

Figure 2-5. Removing a Drive Carrier

Removing a Hot-Swap Drive Carrier from the Chassis

1. Press the release button on the drive carrier, which will extend the drive carrier handle.
2. Use the drive carrier handle to pull the drive out of the chassis.

Technical diagram showing a computer motherboard with labeled components and directional arrows indicating assembly steps.

Figure 2-6. Removing a Drive Carrier

Installing a Drive

1. Remove the dummy drive, which comes pre-installed in the drive carrier. Pull out the two locking clasps and lift out the dummy drive.

Post Align Back Mounts Hole Post Locking Clasps

Figure 2-7. Removing the Dummy Drive from a Carrier

1. Position the drive into the carrier with the PCB side facing down and the connector end toward the rear of the carrier.
2. Tilt the drive to insert it onto the two posts on the right inside of the carrier.
3. Pull out the two spring locking clasps to allow the drive to sit fully in the carrier, then push in the two locking clasps to secure the drive.
4. Insert the drive carrier into its bay, keeping the release button on the bottom. When the carrier reaches the rear of the bay, the release handle will retract.
5. Push the handle in until it clicks into its locked position

M.2 Solid State Drives

The motherboard supports one M.2 PCIe 3.0 x4 solid state drive (SSD) of length 80mm or 110mm. The SBI-611E-1T2N and SBI-611E-5T2N models support two additional M.2 NVMe SSDs of length 80mm or 110mm.

For each length, there is a hole in the mounting platform for a plastic clasp to secure the M.2 SSD.

M.2 Socket Mounting Holes M.2 Sockets SBI-611E-1T2N and SBI-611E-5T2N only Mounting Holes Mounting Holes

Figure 2-8. M.2 SSDs (SBI-611E-1T2N model shown)

Installing M.2 Drives

1. Power down the system and then remove the top cover as described in Sections 3.1 and 3.2.
2. Locate the plastic clips that will lock the M.2 SSD in place.
3. Determine whether your M.2 SSD is 80mm or 110mm. If the plastic clip is not in the correct hole, move it.

a. To remove the plastic clip, twist it 90 degrees and pull up.

b. To insert the plastic clip, push it into the correct mounting hole and twist 90 degrees.

1. Insert the M.2 SSD sideways into the connector on the motherboard so that it lays flat, then secure it to the motherboard with the plastic clip.

2. Replace the cover and restore the power to the system.

Checking the Temperature of an NVMe Drive

There are two ways to check using the BMC Dashboard.

Checking a Drive

• BMC Dashboard > Server Health > NVMe SSD – Shows the temperatures of all NVMe drives.
• BMC Dashboard > Server Health > Sensor Reading > NVME_SSD – Shows the single highest temperature among all the NVMe drives.

Hot-Swap for NVMe Drives

Supermicro servers support NVMe surprise hot-swap. For even better data security, NVMe orderly hot-swap is recommended. NVMe drives can be ejected and replaced remotely using the BMC Dashboard.

Ejecting a Drive

1. BMC Dashboard > Server Health > NVMe SSD
2. Select Device, Group and Slot, and click Eject. After ejecting, the drive Status LED indicator turns green.
3. Remove the drive.

Note that Device and Group are categorized by the CPLD design architecture.

Slot is the slot number on which the NVMe drives are mounted.

Component Info Health Event Leg Manage Monitoring Configuration Remote Control Maintenance Overview Physical View Logical View Controller Physical View Blank Unidink Reject Insert Slot LED Status Supported Actions Disk Info# Capacity Link Speed Connected Logical Drive Connected Controller 0.0 SAMSUNG MZOLAWROCHMLF- 00050 N/A 8:6 OTIs- N/A N/A White Device0 0.1 SAMSUNG MZOLAWROCHMLF- 00050 N/A 8:6 OTIs- N/A N/A White Device0

Figure 2-9. BMC Dashboard Screenshot

Replacing the Drive

1. Insert the replacement drive.
2. BMC Dashboard > Server Health > NVMe SSD
3. Select Device, Group and slot and click Insert. The drive Status LED indicator flashes red, then turns off. The Activity LED turns blue.

2.8 System Cooling

Installing the Air Shrouds

Air shrouds concentrate airflow to maximize fan efficiency.

- Position the air shrouds as illustrated in the figure below, sliding them over the components.

Shroud Shroud

Figure 2-10. Installing the Standard Air Shrouds

Checking the Server Air Flow

• Make sure there are no objects to obstruct airflow in and out of the server.
• Do not operate the server without drive carriers in the drive bays.
• Use only recommended server parts.
• Make sure no wires or foreign objects obstruct air flow through the chassis. Pull all excess cabling out of the airflow path or use shorter cables.

The control panel LEDs display system heat status. See “Control Panel” in Chapter 1 for details.

Overheating

There are several possible responses if the system overheats.

• Use the LEDs to determine the nature of the overheating condition.
  • Make sure all fans are present and operating normally.
• Check the routing of the cables.
• Verify that the heatsinks are installed properly.

2.9 Installing the GPU

1. Remove the riser card assembly from the blade chassis. Note that one screw is difficult to see, as it is accessed through a hole in the bracket. (marked below as hidden)
2. Insert the GPU PCIe contacts into the riser card slot, while fitting the GPU into the assembly.
3. Re-install the assembly into the blade chassis.
4. Connect the necessary PCIe cables and the power cable. The power cable must be appropriate for the particular GPU model.

Hidden Screw Riser Card Assembly

Figure 2-11. Installing the GPU

2.10 Installing the Operating System

An operating system (OS) must be installed on each blade module. Blades with Microsoft Windows OS and blades with Linux OS can operate within the same blade enclosure. Refer to the SuperMicro web site for a list of supported operating systems.

Installing by using PXE Boot

Preboot Execution Environment (PXE) is used to boot a computer over a network. To install the OS using PXE, the following conditions must be met:

• The PXE BOOT option in BIOS must be enabled.
• A PXE server has been configured; this can be another blade in the system.
• The PXE server must be connected over a network to the blade to be booted.
• The blade has only non-partitioned/unformatted hard drives installed and no bootable devices attached to it.

Once these conditions are met, make sure the PXE server is running. Then turn on the blade on which you wish to install the OS. The BIOS in the blade will look at all bootable devices and finding none, will connect to the PXE server to begin the boot/install.

Installing by using Virtual Media (Drive Redirection)

You can install the OS via Virtual Media through either the IPMIview (Java-based client utility), SuperBladeTool or the Web-based Management Utility. With this method, the OS is installed from an ISO image that resides on another system.

Refer to the manuals on the SuperMicro web site for further details on the Virtual Media (CD-ROM or Drive Redirection) sections of these two utility programs.

2.11 Configuring RAID

For RAID setup, see http://www.supermicro.com/support/manuals/ under RAID Installation Guides.

2.12 Driver Installation

The Supermicro website contains drivers and utilities for your system at www.supermicro.com/wdl/driver. Some of these must be installed, such as the chipset driver.

After accessing the website, go into the CDR_Images (in the parent directory of the above link) and locate the ISO file for your motherboard. Download this file to a USB flash or media drive. (You may also use a utility to extract the ISO file if preferred.)

Another option is to go to the Supermicro website at http://www.supermicro.com/products/. Find the product page for your motherboard, and "Download the Latest Drivers and Utilities". Insert the flash drive or disk and the screenshot shown below should appear.

SUPERMICRO B13SEE-CPU-25G Motherboard Drivers & Tools (Win11) Intel® C741 Chipset B13SEE-CPU-25G SUPERMICRO Computer Inc. Intel Chipset INF files Microsoft .Net Framework 4.8(Optional) ASPEED Graphics Driver Intel Virtual RAID on CPU Realtek High Definition Audio Driver Aquantia AQtion LAN Driver Intel PRO Network Connections Drivers SUPERMICRO SuperDoctor 5 Build driver diskettes and manuals Browse CD Auto Start Up Next Time For more information, please visit SUPERMICRO's web site.

Figure 2-12. Driver & Tool Installation Screen

Note: Click the icons showing a hand writing on paper to view the readme files for each item. Click the computer icons to the right of these items to install each item (from top to the bottom) one at a time. After installing each item, you must re-boot the system before moving on to the next item on the list. The bottom icon with a CD on it allows you to view the entire contents.

2.13 Installing SuperCap

For the SBI-611E-1C2N model, an optional SuperCap kit may be installed to insure preservation of cache data in the event of a power outage.

Note: SuperCap card must be installed on the AOM bridge card before the bridge card and the motherboard are installed. Remove both to install.

SuperCap Card Position SuperCap Bracket SuperCap Cables

Figure 2-13. Installing SuperCap

Installing SuperCap

1. Mount the SuperCap card onto the bridge board using two screws and standoffs.
2. Install the bridge board and motherboard.
3. Mount two metal standoffs and one plastic standoff onto the bridge board as shown.

BPNIO-5CE2 SUPER CE FC 3 Plastic Standoff

Figure 2-14. Installing the SuperCap Standoffs onto the Bridge Board

410-5CE2 Plastic Standoff SUPER CE P 3/12-01 341-0 E222P014 28M1 24

Figure 2-15. Installing the SuperCap Capacitor Bracket

1. Mount the SuperCap capacitor bracket with three screws as shown. Ensure the plastic standoff is pushed into the bracket.
2. Install SuperCap capacitor on the bracket.
3. Connect the cable to the plug on the SuperCap card. Use the cable clips on the GPU bracket.
4. Use a zip tie to arrange the cable and push it beside the capacitor.

AOM-2P1NIO-3CE2 BROADCOM® Model FBUQ2 Rating 8.4F 13.5V PIN: 49571-15 REV C S/N SP92703757 MH52170 6 1 6 7 CHINA

Figure 2-16. SuperCap Card Installed

Chapter 3

Motherboard Connections

This chapter describes the connections on the motherboard and provides pinout definitions. Note that depending on how the system is configured, not all connections are required. The LEDs on the motherboard are also described here. A motherboard layout indicating component locations may be found in Chapter 1. More detail can be found in the Motherboard Manual. Please review the Safety Precautions in Appendix B before installing or removing components.

3.1 Headers and Connectors

Power Receptables to Chassis Backplane

PWR1 and PWR2 are primary power supply connectors that provide power to the motherboard through the chassis backplane.

GPU Power Connector

JGPU1 is the EATX 12V 8-pin power connector.

Riser Power Connector

JRC1 and JRC2 are proprietary riser power connectors with a rated 150W, two 75W CEM specification.

VGA/USB Module Connector

Use JKVM1 to connect to a VGA/USB module.

Front Panel Connector

Connect an FPC cable from J5 to the front panel module for power on/off, KVM, and other system LED notifications. Refer to the table below for LEDs and their functions.

Function State Description
	Power Button N/A	Turns the blade the module on	and off
	Power LED	GreenSolid OrangeFlashing Orange	Indicates power status "On"Indicates power status "Off" (with power cables plugged in)Indicates the node is not ready or not enough power to turn on
	KVM/UID LED	BlueFlashing Blue	Indicates KVM is being utilized by the blade unitIndicates UID activated on the blade module
	Network/IB LED	Flashing GreenFlashing Orage	Inidicates network activity over LANIndicates network activity over the Infiniband module
	System Fault LED	Red	Indicates a memory error, overheat, VGA error or any error that prevents booting

PCIe 5.0 x8 MCIO Connectors

J6, J7, J8, J9 are Mini Cool Edge IO PCIe x16 connectors. Use these connectors for GPU/E1.S/AIOM riser cards.

PCIe 4.0 x16 SIOM Connector

J10 is the SIOM connector for one SAS card or two M.2 x4 or two PCIe 4.0 NVMe.

PCIE 4.0 x16 Mezzanine Card Connector

JMEZZ1 connects to a backplane Ethernet add-on card expansion slot.

RAID Key Header

An Intel VROC RAID Key header is located at JRK1. It supports VMD used in creating optional advanced NVMe RAID configurations.

RAID Key HeaderPin Definitions

Pin# Definition

1 Ground

2 RAID_KEY_PU

3 Ground

4 PCH_RAID_KEY

VROC Key Header (JRK1)

Note: This drawing is for illustration only. Your motherboard may look different.

M.2 SSD Slot

This motherboard has one M.2 slot at M.2-HC. It supports an M-Key PCIe 4.0/5.0 x4 or SATA 3.0 device in the 2280 amd 22110 form factors. Note also the mounting holes located at SRW! and SRW2.

Internal USB 2.0/3.0 Type-A Connector

There is one internal USB 2.0/3.0 port (USB0) on the motherboard.

TPM Header

The JTPM1 header is used to connect a Trusted Platform Module (TPM)/Port 80, which is available from Supermicro. A TPM/Port 80 connector is a security device that supports encryption and authentication in hard drives. It allows the motherboard to deny access if the TPM associated with the storage drive is not installed in the system. for more information on the TPM: https://www.supermicro.com/manuals/other/TPM.pdf.

Trusted Platform Module/Port 80 Header Pin Definitions
Pin# Definition	Pin# Definition
1 P3V3	2 SPI_TPM_CS_N
3 PCI-E_RESET_N#	4 SPI_PCH_MISO
5 SPI_PCH_CLK#	6 Ground
7 SPI_PCH_MOSI	8 N/A
9 JTPM1	P3V3A 10 IRQ_TPM_SPIN_N

3.2 Jumpers

Explanation of Jumpers

To modify the operation of the motherboard, jumpers are used to choose between optional settings. Jumpers create shorts between two pins to change the function associated with it. Pin 1 is identified with a square solder pad on the printed circuit board. See the motherboard layout page for jumper locations.

Note: On a two-pin jumper, "Closed" means the jumper is on both pins and "Open" indicates the jumper is either on only one pin or has been completely removed.

Connector Pins Jumper Setting 3 2 1 3 2 1

CMOS Clear Contacts

JBT1 is used to clear CMOS. Instead of pins, this jumper consists of contact pads. See the CMOS Clear section for more information.

JBT1 contact pads

BIOS Recovery

Close pins 2-3 of jumper JBRE1 for BIOS recovery. The default setting is on pins 1 and 2 for normal operation. The default setting is Normal.

BIOS RecoveryJumper Settings
Jumper Setting Definition
Pins 1-2 Normal
Pins 2-3 BIOS Recovery

VGA Enable/Disable

Use jumper JPG1 to enable or disable the VGA port using the onboard graphics controller. The default setting is Enabled.

VGA Enable/DisableJumper Settings
Jumper Setting Definition
Pins 1-2 Enabled
Pins 2-3 Disabled

Management Engine (ME) Recovery

Use jumper JPME1 to select ME Firmware Recovery mode, which will limit resource allocation for essential system operation only in order to maintain normal power operation and management. In the single operation mode, online upgrade will be available via Recovery mode.

ME Recovery ModeJumper Settings
Jumper Setting Definition
Pins 1-2 Normal
Pins 2-3 ME Recovery

Manufacturing Mode Select

Close pins 2-3 of jumper JPME2 to bypass SPI flash security and force the system to operate in the manufacturing mode, which will allow you to flash the system firmware from a host server for system setting modifications.

Manufacturing ModeJumper Settings
Jumper Setting Definition
Pins 1-2 Normal	(Default)
Pins 2-3 Manufacturing Mode

Watch Dog

JWD1 controls the Watchdog function. Watchdog is a monitor that can reboot the system when a software application hangs. Jumping pins 1-2 will cause Watchdog to reset the system if an application hangs. Jumping pins 2-3 will generate a non-maskable interrupt signal for the application that hangs. Watchdog must also be enabled in BIOS.

Note: When Watchdog is enabled, users must write their own application software to disable it.

WatchdogJumper Settings
Jumper Setting Definition
Pins 1-2 Reset (Default)
Pins 2-3 NMI
Open Disabled

3.3 LED Indicators

BMC Heartbeat LED

LEDM1 is a BMC Heartbeat indicator. It blinks green when the BMC is working properly.

Power On LED

LE6 indicates that the power to the motherboard is on.

Chapter 4

Software

After the hardware has been installed, you can install the Operating System (OS), configure RAID settings and install the drivers.

4.1 Microsoft Windows OS Installation

If you will be using RAID, you must configure RAID settings before installing the Windows OS and the RAID driver. Refer to the RAID Configuration User Guides posted on our website at www.supermicro.com/support/manuals.

Installing the OS

1. Create a method to access the MS Windows installation ISO file. That can be a USB flash or media drive.
2. Retrieve the proper RST/RSTe driver. Go to the Supermicro web page for your motherboard and click on "Download the Latest Drivers and Utilities", select the proper driver, and copy it to a USB flash drive.
3. Boot from a bootable device with Windows OS installation. You can see a bootable device list by pressing F11 during the system startup.

Please select boot device: ATEN Virtual CDROM YSOJ → IPMI virtual drive (Legacy) ASUS SDRW-08D2S-U F601 → USB DVD device (Legacy) USB FLASH DRIVE PMAP → USB flash drive with OS installation (Legacy) IBA 40-10G Slot 1900 v1060 → PXE boot (Legacy) UEFI: ATEN Virtual CDROM YSOJ → IPMI virtual drive (UEFI) UEFI: ASUS SDRW-08D2S-U F601 → USB DVD device (UEFI) UEFI: Built-in EFI Shell Enter Setup ↑ and ↓ to move selection ENTER to select boot device ESC to boot using defaults

Figure 4-1. Select Boot Device

1. During Windows Setup, continue to the dialog where you select the drives on which to install Windows. If the disk you want to use is not listed, click on "Load driver" link at the bottom left corner.

Where do you want to install Windows? Name Total size Free space Type Refresh Delete Format New Load driver Extend We couldn't find any drives. To get a storage driver, click Load driver. Next

Figure 4-2. Load Driver Link

To load the driver, browse the USB flash drive for the proper driver files.

• For RAID, choose the SATA/sSATA RAID driver indicated then choose the storage drive on which you want to install it.

• For non-RAID, choose the SATA/sSATA AHCI driver indicated then choose the storage drive on which you want to install it.

• Once all devices are specified, continue with the installation.

• After the Windows OS installation has completed, the system will automatically reboot multiple times.

4.2 Driver Installation

The Supermicro website contains drivers and utilities for your system at https://www.supermicro.com/wdl/driver. Some of these must be installed, such as the chipset driver.

After accessing the website, go into the CDR_Images (in the parent directory of the above link) and locate the ISO file for your motherboard. Download this file to a USB flash or media drive. (You may also use a utility to extract the ISO file if preferred.)

Another option is to go to the Supermicro website at http://www.supermicro.com/products/. Find the product page for your motherboard, and "Download the Latest Drivers and Utilities". Insert the flash drive or disk and the screenshot shown below should appear.

SUPERMICRO B13SEE-CPU-25G Motherboard Drivers & Tools (Win11) Intel® C741 Chipset B13SEE-CPU-25G SUPERMICRO Computer Inc. Intel Chipset INF files Microsoft .Net Framework 4.8(Optional) ASPEED Graphics Driver Intel Virtual RAID on CPU Realtek High Definition Audio Driver Aquantia AQtion LAN Driver Intel PRO Network Connections Drivers SUPERMICRO SuperDoctor 5 Build driver diskettes and manuals Browse CD Auto Start Up Next Time For more information, please visit SUPERMICRO's web site.

Figure 4-3. Driver & Tool Installation Screen

Note: Click the icons showing a hand writing on paper to view the readme files for each item. Click the computer icons to the right of these items to install each item (from top to the bottom) one at a time. After installing each item, you must re-boot the system before moving on to the next item on the list. The bottom icon with a CD on it allows you to view the entire contents.

4.3 SuperDoctor® 5

The Supermicro SuperDoctor 5 is a program that functions in a command-line or web-based interface for Windows and Linux operating systems. The program monitors such system health information as CPU temperature, system voltages, system power consumption, fan speed, and provides alerts via email or Simple Network Management Protocol (SNMP).

SuperDoctor 5 comes in local and remote management versions and can be used with Nagios to maximize your system monitoring needs. With SuperDoctor 5 Management Server (SSM Server), you can remotely control power on/off and reset chassis intrusion for multiple systems with SuperDoctor 5 or the BMC. SuperDoctor 5 Management Server monitors HTTP, FTP, and SMTP services to optimize the efficiency of your operation.

SuperDoctor® Manual and Resources

SuperDoctor S M000 ML Certificates error localhost:1444/SuperDoctor Loading... Select Language English AOMIN Linked Motherboard: C7S366-CB-ML Voltage VCCP1 Voltage 12V voltage VCCB1 Voltage VCCC1 voltage VCCN1 voltage VCCU1 voltage VCCU2 voltage VCCU2 voltage VCCU2 voltage 2.5V 1.5V 1.5V 1.5V 1.5V 8.5V DC voltage 3V DC 5V DC voltage 3.5V DC voltage Temperature 42/19A UPS Time 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 21M DC 30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/30/42/42 UPS Time 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC 210 DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DC N/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN/A DCN A US M anging Hard Disk total Steatements UL PHYSICALDRIVE (FJX9G4H) Memory

Figure 4-4. SuperDoctor 5 Interface Display Screen (Health Information)

4.4 BMC

The motherboard provides remote access, monitoring and management through the baseboard management controller (BMC) and other management controllers distributed among different system modules. There are several BIOS settings that are related to BMC. For general documentation and information on BMC, visit our website at:

www.supermicro.com/en/solutions/management-software/bmc-resources

BMC ADMIN User Password

For security, each system is assigned a unique default BMC password for the ADMIN user. This can be found on a sticker on the chassis and a sticker on the motherboard. The sticker also displays the BMC MAC address. If necessary, the password can be reset using the Supermicro IPMICFG tool.

BMC AC1F6BC PWD SUOKJ

Figure 4-5. BMC Password Label

The sticker can be found on the pull-out service tag at the front of the chassis. See Chapter 1 for the location.

Chapter 5

Optional Components

This chapter describes optional system components and installation procedures.

5.1 TPM Security Module

SPI capable TPM 2.0 (or 1.2) with Infineon 9670 controller, Horizontal form factor

The JTPM1 header is used to connect a Trusted Platform Module (TPM). A TPM is a security device that supports encryption and authentication in hard drives. It enables the motherboard to deny access if the TPM associated with the hard drive is not installed in the system.

Details and installation procedures are at:

http://www.supermicro.com/manuals/other/TPM.pdf.

5.2 Intel Virtual RAID on CPU (VROC)

Intel® Virtual RAID on CPU (Intel VROC) is an enterprise RAID solution for NVMe SSDs directly attached to Intel Xeon Scalable processors. Intel Volume Management Device (VMD) is an integrated controller inside the CPU PCIe root complex.

• A single processor supports up to 12 NVMe SSDs and up to 6 RAID arrays.
• A dual processor system supports up to 24 NVMe SSDs and 12 RAID arrays.

Stripe sizes are 4K, 8K, 16K, 32K, 64K, 128K.

Requirements and Restrictions

• Intel VROC is only available when the system is configured for UEFI boot mode.
• To enable the mdadm command and support for RSTe, install the patch from

- Linux: https://downloadcenter.intel.com/download/28158/Intel-Virtual-RAID-on-CPU-Intel-VROC-and-Intel-Rapid-Storage-Technology-enterprise-Intel-RSTe-Driver-for-Linux-

- Windows: https://downloadcenter.intel.com/download/28108/Intel-Virtual-RAID-on-CPU-Intel-VROC-and-Intel-Rapid-Storage-Technology-enterprise-Intel-RSTe-Driver-for-Windows-

• To enable Intel VROC, a hardware key must be inserted on the motherboard, and the appropriate processor's Virtual Management Devices must be enabled in the BIOS setup.
• It is possible to enable Intel VROC without a hardware key installed, but only RAID0 will be enabled.
• Intel VROC is not compatible with secure boot. This feature must be disabled.
• When creating bootable OS RAID1 devices, you must have both devices on the same CPU, and a VMD on that CPU.
• Spanning drives when creating RAID devices is not recommended due to performance issues, even though it is supported.

Supported SSDs and Operating Systems

To see the latest support information: https://www.intel.com/content/www/us/en/support/articles/000030310/memory-and-storage/ssd-software.html

Additional Information

Additional information is available on the product page for the Supermicro add-on card and the linked manuals.

www.supermicro.com/products/accessories/addon/AOC-VROCxxxMOD.cfm

Hardware Key

The Intel VROC hardware key is a license key that detects the Intel VROC SKU and activates the function accordingly. The key must be plugged into the Supermicro motherboard (connector JRK1). The key options are:

Intel® VROC Keys
VROC Package Description Part Number Intel MM Number
Standard	RAID 0, 1, 10Supports 3rd party SSDs	AOC-VROCSTNMOD 95	1605
Premium	RAID 0, 1, 5, 10Supports 3rd party SSDs	AOC-VROCPREMOD 95	1606
Intel SSD only	RAID 0, 1, 5, 10Supports Intel SSDs only	AOC-VROCINTMOD 95	822

VROC Key Header, JRK1

Figure 5-1. Intel VROC RAID Key and Motherboard Connector JRK1

Configuring Intel VMD

VMD must be enabled on PCIe ports which have NVMe drives attached to them in order for those drives to be added to a VROC RAID configuration. The default BIOS setting for the NVMe Mode Switch is Auto which automatically enables VMD on all installed NVMe drives.

NVMe Mode Switch :

• Auto Enables VMD for all NVMe ports if VROC Key is installed.
  • VMD Enables VMD for all NVMe ports despite the lack of the VROC Key.
• Manual Allows the user to select specific NVMe ports on which to enable VMD.

The NVMe Mode Switch can be viewed or selected at BIOS > Advanced > Chipset Configuration > North Bridge > IIO Configuration > Intel® VMD Technology.

Note: Without a VROC Key, there is no RAID support with the Auto switch. Only RAID 0 is supported with the VMD and Manual switches.

Configuring VMD Manually

The steps for manually configuring VMD on specific NVMe ports in UEFI BIOS are shown below. This example shows different but similar system with 12 NVMe. Yours will look different.

1. Reboot the server and press [DEL] key to access the BIOS options.
2. Switch to Advanced > Chipset Configuration > North Bridge > IIO Configuration > Intel® VMD Technology.
3. Select VMD Mode Switch, then select Manual.

Aptio Setup - AMI Advanced Intel VMD Technology NVMe Mode Switch [Auto] ► Intel VMD for Volume Management Device on Socket 0 ► Intel VMD for Volume Management Device on Socket 1 NVMe Mode Switch Manual VMD Auto Select NVMe Mode, default Auto mode - enable VMD when VROC key presence

Note that Socket 0 contains CPU1; Socket 1 contains CPU2

Figure 5-2. BIOS, Selecting VMD Mode

Caution: VMD must only be enabled on NVMe port resources. If VMD is enabled on other PCIe ports, the functionality of those ports will be impacted. See the table below.

1. Select "Intel VMD for Volume Management Device on" on Socket 0 (CPU1) or Socket 2 (CPU2) to enable VMD for devices under the respective CPU.

Auto Setup - AMD Advanced VMD Config for PCM parts Enable/Disable VMD [Disable] VMD Config for IDU 8 Enable/Disable VMD [Disable] VMD Config for IDU 1 Enable/Disable VMD [Disable] VMD Config for IDU 2 Enable/Disable VMD [Disable] VMD Config for IDU 3 Enable/Disable VMD [Disable] VMD Config for IDU 4 Enable/Disable VMD [Disable] VMD Config for IDU 5 Enable/Disable VMD [Disable] VMD Config for IDU 6 Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable VMD [Disable] Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/Disable WMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable/DisableWMD Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Disable Enable / Enable

Not 30 Setup - VHD Advanced VHD Config for IDU 0 Disable/Disable VHD (Disable) VHD Config for IDU 1 Disable/Disable VHD (Disable) VHD Config for IDU 2 Disable/Disable VHD (Disable) VHD Config for IDU 3 Disable/Disable VHD (Disable) VHD Config for IDU 4 Disable/Disable VHD (Disable) VHD Config for IDU 5 Disable/Disable VHD (Disable) VHD Config for IDU 6 Disable/Disable VHD (Disable) Enable/Disable VHD (Disable) Enable/Disable VHD in this Stack. +: Select Screen TA: Select Items Enter: Select +/- Change Dot.

Figure 5-3. Intel VMD for Volume Management Device on Socket 0 and Socket 1

1. Choose Enable for "Enable/Disable VMD" for IOU 3 to list the available devices under IOU 3.

Aptio Setup - AMI Advanced VMD Config for IOU 0 Enable/Disable VMD [Disable] VMD Config for IOU 1 Enable/Disable VMD [Disable] VMD Config for IOU 2 Enable/Disable VMD [Disable] VMD Config for IOU 3 Enable/Disable VMD VMD Config for IOU 4 Enable/Disable VMD VMD Config for IOU 5 Enable/Disable VMD [Disable] VMD Config for IOU 6 Enable/Disable VMD [Disable] Enable/Disable VMD in this Stack. Enable/Disable VMD Disable Enable +: Select Screen ↑↓: Select Item Enter: Select +/-: Change Opt.

Figure 5-4. BIOS, Enabling VMD on Socket 1 (CPU2) (Example)

Aptio Setup - AMI Advanced
VMD Config for IOU 0 Enable/Disable VMD [Disable]	Enable/Disable Intel Volume Management Device Technology on specific root port
VMD Config for IOU 1 Enable/Disable VMD [Disable]
VMD Config for IOU 2 Enable/Disable VMD [Disable]
VMD Config for IOU 3 Enable/Disable VMD [Enable] Socket1 IOU3 VMD port E [Disable] Socket1 IOU3 VMD port G [Disable] Hot Plug Capable [Disable]
VMD Config for IOU 4 Enable/Disable VMD [Disable]
VMD Config for IOU 5 Enable/Disable VMD [Disable]
VMD Config for IOU 6 Enable/Disable VMD [Disable]
+:-: Select Screen ↑↓: Select Item Enter: Select +/-: Change Opt. F1: General Help F2: Previous Values F3: Optimized Defaults

Figure 5-5. BIOS, Enabling VMD on Socket 1 (Example)

1. Enable the NVMe port resource according to table above for the NVMe drives that will be used in a RAID configuration.

Aptio Setup - AMI Advanced
VMD Config for IOU 0 Enable/Disable VMD [Disable]	Enable/Disable Intel Volume Management Device Technology on specific root port
VMD Config for IOU 1 Enable/Disable VMD [Disable]
VMD Config for IOU 2 Enable/Disable VMD [Disable]
VMD Config for IOU 3 Enable/Disable VMD [Enable] Socket1 IOU3 VMD port E [Enable] Socket1 IOU3 VMD port G [Enable] Hot Plug Capable [Disable]
VMD Config for IOU 4 Enable/Disable VMD [Disable]
VMD Config for IOU 5 Enable/Disable VMD [Disable]
VMD Config for IOU 6 Enable/Disable VMD [Disable]
+:-: Select Screen ↑↓: Select Item Enter: Select +/-: Change Opt. F1: General Help F2: Previous Values F3: Optimized Defaults

Figure 5-6. BIOS, Enabling Socket 1 (Example)

1. Choose whether to make the NVMe drives in this IOU Hot Plug Capable by selecting Enabled or Disabled.
2. Repeat steps 4 through 7 for each IOU # on each CPU to enable VMD on the desired NVMe ports.

Aptio Setup - AMI Advanced VMD Config for IOU 0 Enable/Disable VMD [Disable] VMD Config for IOU 1 Enable/Disable VMD [Disable] VMD Config for IOU 2 Enable/Disable VMD [Disable] VMD Config for IOU 3 Enable/Disable VMD [Enable] Socket1 IOU3 VMD port E [Enable] Socket1 IOU3 VMD port G [Enable] Hot Plug Capable [Disable] VMD Config for IOU 4 Enable/Disable VMD [Enable] Socket1 IOU4 VMD port A [Enable] Socket1 IOU4 VMD port C [Enable] Socket1 IOU4 VMD port E [Enable] Socket1 IOU4 VMD port G [Enable] Hot Plug Capable [Disable] VMD Config for IOU 5 Enable/Disable VMD [Disable] Enable/Disable VMD in this Stack. +: Select Screen ↑↓: Select Item Enter: Select +/-: Change Opt. F1: General Help F2: Previous Values F3: Optimized Defaults F4: Save & Exit ESC: Exit VMD Config for IOU 6 Enable/Disable VMD [Disable] F1: General Help F2: Previous Values F3: Optimized Defaults F4: Save & Exit ESC: Exit Version 2.22.1286 Copyright (C) 2022 AMI

Figure 5-7. BIOS, Enabling Socket 1 Completed (Example)

1. Press [F4] to save the configuration and reboot the system.

Note: If there is an existing RAID configuration, delete the RAID volume associated with the VMD controller before disabling the controller. Failure to do so may lead to unexpected behavior.

Note: The effects of physically changing or swapping a CPU on the VMD controller have not been thoroughly tested or documented.

Creating NVMe RAID Configurations

1. Open Advanced > Intel(R) Virtual RAID on CPU > All Intel VMD Controllers > Create RAID Volume.

Advanced Create RAID Volume Name: RAID LEVEL: Enable RAID Spaced over VND Controllers: Select Disks: INTEL 100FF2KHOSETZ SWPHAC11040063FORDA, 3576.98GB Port 3/2 CP30 VMD INTEL 100FF2KHOSETZ SWPHAC14730317FORDA, 3576.98GB Port 3/3 CP30 VMD INTEL 100FF2KHOSETZ SWPHAC11040063FORDA, 3576.98GB Port 4/0 CP30 VMD INTEL 100FF2KHOSETZ SWPHAC11040063FORDA, 3576.98GB Port 4/1 CP30 VMD INTEL 100FF2KHOSETZ SWPHAC14720031FORDA, 3576.98GB Port 4/2 CP30 VMD Valued: RAID(Stripe) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] Enter a unique volume name that does not contain any at the beginning or boccrack and is its characters or less. ++ Select Screen File Select Item Enter Select +/- Change Out. Fit General Help For Previous Values FSI Optimized Defaults For Save & Exit ESC Exit

Figure 5-8. Created Volume without enabling RAID spanned over VMD controller

Aptio Setup - Ant Creates RAID Volume Name: RAID Level: Enable RAID Spanned over VMD Controllers: Select Disk: DTEL STOPF2XK08BTZ DHPHNC10400DFDP6NAH, 3576.95GB Port S12 CPU VMD DTEL STOPF2XK08BTZ DHPHNC47200JTFDP6NAH, 3576.95GB Port S13 CPU VMD DTEL STOPF2XK08BTZ DHPHNC10400DFDP6NAH, 3576.95GB Port 413 CPU VMD DTEL STOPF2XK08BTZ DHPHNC10400DFDP6NAH, 3576.95GB Port 413 CPU VMD DTEL STOPF2XK08BTZ DHPHNC47200JTFDP6NAH, 3576.95GB Port 413 CPU VMD VOLUME: [NAID0(Striped)] [X] [X] [X] [X] [X] [X] [1] X - to Select Disk ---: Select Screen M: Select Item Enter: Select +/-: Change Up. F1: General Help. F2: Previous Values F3: Optimized Defaults F4: Save & Exit RSC: Exit

Figure 5-9. Created Volume with enabling RAID spanned over VMD controller

1. Set Name.
2. Set RAID Level.
3. If cross-controller RAID is required, select Enable RAID spanned over VMD Controller.
4. Select specific disks for RAID with an [X].

• RAID0: Select at least two [2 - 24] disks
• RAID1: Select only two disks
• RAID5: Select at least three [3 - 24] disks
• RAID10: Select only four disks

1. Select Strip Size (Default 64KB).
2. Select Create Volume.
3. If another RAID is needed, start again at step 1.

Status Indications

An LED indicator on the drive carrier shows the RAID status of the drive.

Drive Carrier Status LED Indicator
Status State (red)
Normal function OFF
Locating 4 Hz blink
Fault Solid ON
Rebuilding 1 Hz Blink

IBPI SFF 8489 Defined Status LED States

Hot-Swap Drives

Intel VMD enables hot-plug and hot-unplug for NVMe SSDs, whether from Intel or other manufacturers. Under vSphere ESXi, several steps are necessary to avoid potential stability issues. See the information at the link [1] below.

Hot-unplug

1. Prevent devices from being re-detected during rescan:

esxcli storage core claiming autoclaim --enabled=false

1. Unmount the VMFS volumes on the device. Check [2] for details.
2. Detach the device. Check [3] for details.
3. Physically remove the device.

Hot-plug

• Physically install the device.

ESXi will automatically discover NVMe SSDs, but a manual scan may be required in some cases.

Related Information Links

[1] https://kb.vmware.com/s/article/2151404
[2] https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.storage.doc/GUID-1B56EF97-F60E-4F21-82A7-8F2A7294604D.html
[3] https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.storage.doc/GUID-F2E75F67-740B-4406-9F0C-A2D99A698F2A.html

Chapter 6

Troubleshooting and Support

6.1 Information Resources

Website

A great deal of information is available on the Supermicro website, supermicro.com.

SUPERMIC Products Solutions Company News Support High Performance AI Building Blocks Award-Winning System Portfolio Optimized for AI to Supercharge Your Discovery and Innovation

Figure 6-1. Supermicro Website

• Specifications for servers and other hardware are available by clicking the Products option.
• The Support option offers downloads (manuals, BIOS/BMC, drivers, etc.), FAQs, RMA, warranty, and other service extensions.

Direct Links for the SBI-611E-1C2N/1T2N/5T2N System

Web SBI-611E-1C2N specifications page

Web SBI-611E-1T2N specifications page

Web SBI-611E-5T2N specifications page

Web B13SEE-CPU-25G motherboard page for links to the Quick Reference Guide, User Manual, validated storage drives, etc.

Direct Links for General Support and Information

Frequently Asked Questions

TPM User Guide

BMC User Guide

SuperDoctor5 Large Deployment Guide

Direct Links (continued)

For validated memory, use our Product Resources page

Product Matrices page for links to tables summarizing specs for systems, motherboards, power supplies, riser cards, add-on cards, etc.

Security Center for recent security notices

Supermicro Phone and Addresses

6.2 BMC Interface

The system supports a Baseboard Management Controller (BMC) interface. It provides remote access, monitoring and management. There are several BIOS settings related to the BMC.

For general documentation and information on the BMC, please visit our website at: https://www.supermicro.com/manuals/other/BMC_IPMI_X13_H13.pdf.

Dashboard System Component Info Health Event Log Configuration Remote Control Maintenance Advanced Settings Filter x Sensor-specific x Threshold x Genta x OEM x Unspecified Health Event Log Clear Health Event Log Export to Excel Severity x Data/Time Sensor Type Categories Description Event Type 2020-10-15 18:57:36 ACPowerOn [OEM] First AC Power on - Assertion Sensor-specific 2020-10-15 18:56:43 System NC [OEM] Dedicated LAN Link Up - Assertion Sensor-specific 2020-10-15 17:41:37 ACPowerOn [OEM] First AC Power on - Assertion Sensor-specific 2020-10-15 17:41:17 System NC [OEM] Dedicated LAN Link Up - Assertion Sensor-specific 2020-10-15 17:41:14 System NC [OEM] Dedicated LAN Link Down - Assertion Sensor-specific 2020-10-15 17:41:12 System NC [OEM] Dedicated LAN Link Up - Assertion Sensor-specific + < 1 / > + T = 6/15

Figure 6-2. BMC Dashboard Sample

6.3 Troubleshooting Procedures

Use the following procedures to troubleshoot your system. If you have followed all of the procedures below and still need assistance, refer to the Technical Support Procedures or Returning Merchandise for Service sections in this chapter. Power down the system before changing any non hot-swap hardware components.

General Technique

If you experience unstable operation or get no boot response, try:

1. With power off, remove all but one DIMM and other added components, such as add-on cards, from the motherboard. Make sure the motherboard is not shorted to the chassis.
2. Set all jumpers to their default positions.
3. Power up. If the system boots, check for memory errors and add-on card problems.

JPI JPE1 JPE2 JCP1 JSPD1 JMBG3 JWD1 JPR1 JPFR1 JPFR2 USB0 MH3 PWR1 J1 JPME1 JPME2 JBRE1 JBT1 JP1 JVRM1 JTPM1 SUPER BAR CODE LE6 PWR2 MH_G2 LE6 IMEZZ1 JKVM1 MH4 MH5

Figure 6-3. Location of the MB Power LED

No Power

• Check that the power LED on the motherboard is on.
  • Make sure that the power connector is connected to the power supply.
• Check that the motherboard battery still supplies approximately 3VDC. If it does not, replace it.
• Check that the system input voltage is 100-120v or 180-240v.
  • Turn the power switch on and off to test the system

No Video

If the power is on but you have no video, remove all add-on cards and cables.

System Boot Failure

If the system does not display Power-On-Self-Test (POST) or does not respond after the power is turned on, try the following:

- Turn on the system with only one DIMM module installed. If the system boots, check for bad DIMM modules or slots by following the Memory Errors Troubleshooting procedure below.

Memory Errors

• Make sure that the DIMM modules are properly and fully installed.
• Confirm that you are using the correct memory. Also, it is recommended that you use the same memory type and speed for all DIMMs in the system. See Section 3.3 for memory details.
• Check for bad DIMM modules or slots by swapping modules between slots and noting the results.

Losing the System Setup Configuration

• Always replace power supplies with the exact same model that came with the system. A poor quality power supply may cause the system to lose the CMOS setup configuration.
• Check that the motherboard battery still supplies approximately 3VDC. If it does not, replace it.

If the above steps do not fix the setup configuration problem, contact your vendor for repairs.

When the System Becomes Unstable

If the system becomes unstable during or after OS installation, check the following:

• CPU/BIOS support: Make sure that your CPU is supported and that you have the latest BIOS installed in your system.
• Memory: Make sure that the memory modules are supported. Refer to the product page on our website at www.supermicro.com. Test the modules using memtest86 or a similar utility.
• Storage drives: Make sure that all drives work properly. Replace if necessary.

• System cooling: Check that all heatsink fans and system fans work properly. Check the hardware monitoring settings in the BMC to make sure that the CPU and system temperatures are within the normal range. Also check the Control panel Overheat LED.

• Adequate power supply: Make sure that the power supply provides adequate power to the system. Make sure that all power connectors are connected. Refer to the Supermicro website for the minimum power requirements.

• Proper software support: Make sure that the correct drivers are used.

If the system becomes unstable before or during OS installation, check the following:

• Source of installation: Make sure that the devices used for installation are working properly, including boot devices.
• Cable connection: Check to make sure that all cables are connected and working properly.
• Use the minimum configuration for troubleshooting: Remove all unnecessary components (starting with add-on cards first), and use the minimum configuration (but with a CPU and a memory module installed) to identify the trouble areas.

• Identify a bad component by isolating it. Check and change one component at a time.

• Remove a component in question from the chassis, and test it in isolation. Replace it if necessary.

• Or swap in a new component for the suspect one.
• Or install the possibly defective component into a known good system. If the new system works, the component is likely not the cause or the problem.

6.4 Crash Dump Using the BMC Dashboard

In the event of a processor internal error (IERR) that crashes your system, you may want to provide information to support staff. You can download a crash dump of status information using the BMC Dashboard. The BMC manual is available at https://www.supermicro.com/manuals/other/BMC_IPMI_X13_H13.pdf.

Check Error Log

1. Access the BMC web interface.
2. Click the Server Health tab, then Event Log to verify an IERR error.

Health Event Log Clear Health Event Log Export to Excel Severity = Date/Time Sensor Type Categories Description Event Type 2026-10-15 18:57:06 ACPowerOn [OEM] First AC Power on - Assertion Sensor-specific 2026-10-15 18:56:43 System NIC [OEM] Dedicated LAN Link Up - Assertion Sensor-specific 2026-10-15 17:41:37 ACPowerOn [OEM] First AC Power on - Assertion Sensor-specific 2026-10-15 17:41:17 System NIC [OEM] Dedicated LAN Link Up - Assertion Sensor-specific 2026-10-15 17:41:14 System NIC [OEM] Dedicated LAN Link Up - Assertion Sensor-specific 2026-10-15 17:41:12 System NIC [OEM] Dedicated LAN Link Up - Assertion Sensor-specific

Figure 6-5. BMC Event Log

In the event of an IERR, the BMC executes a crash dump. You must download the crash dump and save it.

6.5 UEFI BIOS Recovery

Warning: Do not upgrade the BIOS unless your system has a BIOS-related issue. Flashing the wrong BIOS can cause irreparable damage to the system. In no event shall Supermicro be liable for direct, indirect, special, incidental, or consequential damages arising from a BIOS update. If you do update the BIOS, do not shut down or reset the system while the BIOS is updating to avoid possible boot failure.

Overview

The Unified Extensible Firmware Interface (UEFI) provides a software-based interface between the operating system and the platform firmware in the pre-boot environment. The UEFI specification supports an architecture-independent mechanism that will allow the UEFI OS loader stored in an add-on card to boot the system. The UEFI offers clean, hands-off management to a computer during system boot.

Recovering the UEFI BIOS Image

A UEFI BIOS flash chip consists of a recovery BIOS block and a main BIOS block (a main BIOS image). The recovery block contains critical BIOS codes, including memory detection and recovery codes for the user to flash a healthy BIOS image if the original main BIOS image is corrupted. When the system power is turned on, the recovery block codes execute first. Once this process is complete, the main BIOS code will continue with system initialization and the remaining POST (Power-On Self-Test) routines.

Note 1: Follow the BIOS recovery instructions below for BIOS recovery when the main BIOS block crashes.

Note 2: When the BIOS recovery block crashes, you will need to follow the procedures to make a Returned Merchandise Authorization (RMA) request. Also, you may use the Supermicro Update Manager (SUM) Out-of-Band (https://www.supermicro.com.tw/products/nfo/SMS_SUM.cfm) to reflash the BIOS.

Recovering the Main BIOS Block with a USB Device

This feature allows the user to recover the main BIOS image using a USB-attached device without additional utilities used. A USB flash or media drive can be used for this purpose. However, a USB hard disk drive cannot be used for BIOS recovery at this time.

The file system supported by the recovery block is FAT (including FAT12, FAT16, and FAT32) which is installed on a bootable or non-bootable USB-attached device. However, the BIOS might need several minutes to locate the SUPER.ROM file if the media size becomes too large due to the huge volumes of folders and files stored in the device.

To perform UEFI BIOS recovery using a USB-attached device, follow the instructions below.

1. Using a different machine, copy the "Super.ROM" binary image file into the Root "\" directory of a USB flash or media drive.
   Note 1: If you cannot locate the "Super.ROM" file in your drive disk, visit our website at www.supermicro.com to download the BIOS package. Extract the BIOS binary image into a USB flash device and rename it "Super.ROM" for the BIOS recovery use.
   Note 2: Before recovering the main BIOS image, confirm that the "Super.ROM" binary image file you download is the same version or a close version meant for your motherboard.
2. Insert the USB device that contains the new BIOS image ("Super.ROM") into your USB drive and reset the system when the following screen appears.
3. After locating the healthy BIOS binary image, the system will enter the BIOS Recovery menu as shown below.

SUPERIOR BMC IP:10.132.161.13 PEI--Could Not Find Recovery Image... 87

Note: At this point, you may decide if you want to start the BIOS recovery. If you decide to proceed with BIOS recovery, follow the procedures below.

Option Setup Utility - Copyright (C) 2017 American Megatrends, Inc. Main Advanced Event Log (PVI) Recovery Security Boot Save a Exit Please select blocks you want to update: Boot Black Boot Block Update [Enabled] [Enabled] ▶ Proceed with flash update Get this option to reset WAVM to default values ++ Select Screen T4: Select Item Enter: Select +/- Change Opt. F1: General Help F2: Previous Values F3: Optimized Defaults F4: Save & Exit ESC: Exit Version 2.19.1766 Copyright (C) 2017 American Megatrends, Inc.

1. When the screen as shown above displays, use the arrow keys to select the item "Proceed with flash update" and press the key. You will see the BIOS recovery progress as shown in the screen below.

Note: Do not interrupt the BIOS flashing process until it has completed.

1. After the BIOS recovery process is complete, press any key to reboot the system.
2. Using a different system, extract the BIOS package into a USB flash drive.

Auto Setup Utility - Copyright (C) 2017 American Megatrends, Inc. Recovery WARNING! System firmware is being updated. Keyboard is locked. DO NOT TURN THE POWER OFF!!. Once firmware update is completed press any key to reboot the system Program new data Write new boot block... 178 +: Select Screen T4: Select Item Enter: Select +/-: Change Opt. F1: General Help F2: Previous Values F3: Optimized Defaults F4: Save a Exit DSC: Exit Version 2.19.1266. Copyright (C) 2017 American Megatrends, Inc.

1. Press continuously during system boot to enter the BIOS Setup utility. From the top of the tool bar, select Boot to enter the submenu. From the submenu list, select Boot

Aotio Setup UTILITY - Copyright (CI) 2017 American HegaTrends, Inc. Recovery WARNING: System firmware is being updated. Keyboard is locked. ON NOT TURN THE POWER OFF !!! Once firmware update is completed Press any key to reboot the system Flash update Flash update completed. Press any key to reset the system +: Select Screen +: Select Item enter: Select +/-: Change Opt. F1: General Help F2: Previous Values F3: Optimized Defaults F4: Save a Exit ESG: EXIT Version 2.15.1926. Copyright (CI) 2017 American HegaTrends, Inc.

Option #1 as shown below. Then, set Boot Option #1 to [UEFI AP:UEFI: Built-in EFI Shell]. Press to save the settings and exit the BIOS Setup utility.

1. When the UEFI Shell prompt appears, type fs# to change the device directory path. Go to the directory that contains the BIOS package you extracted earlier from Step 6. Enter flash.nsh BIOSname.### at the prompt to start the BIOS update process.

Auto Setup Utility - Copyright (C) 2017 American Megatrends, Inc. Main Advanced Event Loss JPMS Security Boot Save & Exit Boot Configuration Boot mode select [DUAL] LEGACY to EFI Support [Disabled] FIXED BOOT ORDER Priorities Boot Option #3 [DEF1 HP/DEF1: Bui...] Boot Option #2 [COM/DVD] Boot Option #3 [USB Hard Disk] Boot Option #4 [USB CD/DVI] Boot Option #5 [USB Key/SanDisk] Boot Option #6 [USB Flopy] Boot Option #7 [USB Lan] Boot Option #8 [Network: IBM GE SI...] Boot Option #9 [UEFI Hard Disk] Boot Option #10 [UEFI DD/DVI] Boot Option #11 [UEFI USB Hard Disk] Boot Option #12 [UEFI USB CO/DVI] Boot Option #13 [UEFI USB Key/UEFI...] Boot Option #14 [UEFI USB Flopy] Boot Option #15 [UEFI USB Lan] Boot Option #16 [UEFI Network] Boot Option #17 [Hard Disk] ▶ Hold New Boot Option Sets the system boot order +: Select Screen T4: Select Item Enter: Select +/-: Change Opt. F1: General help F2: Previous Values F3: Optimized Defaults F4: Save & Exit ESC: Exit Version 2.19.1265 Copyright (C) 2017 American Megatrends, Inc.

Note: Do not interrupt this process until the BIOS flashing is complete.

UIFI Interactive Shell v2.1 E6K 11 UIFI v2.50 (American Regatrends, 0x008900C) Mapping table FS0: FILE0(3):H0D0(1):BLK1; Ps:Root(0x0)/PCI(0x14,0x0)/USB(0x11,0x0)/NOI,HDR,0xG7901D2,0x806,0x1 OR3592) BLK0: File(s): Ps:Root(0x0)/PCI(0x14,0x0)/USB(0x11,0x0) Press ESR to 1 seconds to skills startup.msh or any other key to continue. Shell: FSD FS0: VAPLDOS FS0:VAPLDOS> CD SAPMHE2_0362012 FS0:VAPLDOS\SAPMHE2_0362017\flash.msh X13P07.SX4

1. The screen above indicates that the BIOS update process is complete. When you see the screen above, unplug the AC power cable from the power supply, clear CMOS, and plug

Done. [ Access OXOS Port Ex ] Readout INDEX 0x51: 0x18 Done. ********************************************************************** * Program BIOS and ME (including RDT) regions... ********************************************************************** AVI Firmware Update Utility v5.49.01.1317 Copyright ©2017 American Registrends Inc. All Rights Reserved. OPUD = 50652 Reading Flash ....... done - ME Data Size checking . ok - PFS checksums ....... ok - Check BookLayout ....... ok Erasing Boot Block ....... done Updating Boot Block ....... done Verifying Boot Block ....... done Froiling Main Block ....... 0x00152000 (00)

the AC power cable in the power supply again to power on the system.

1. Press continuously to enter the BIOS Setup utility.

Verifying HCB Block ....... done - Update success for FDR - Update success for SE. - Successful Update Recovery Loader to (PRx!! - Successful Update MFSB11- - Successful Update RTM11- - Successful Update NFX, [081 and 19021] - Successful Update FLOD and UTOX11 - ML entire image update success!! WAVRNDG : System must power-off to have the changes take effect! Moving FS01:AFUD005/SKJPME2_03162027/40x64,efi -> FS01:AFUD005/SKJPME2_03162017/f df.sac - [cA] Moving FS01:AFUD005/SKJPME2_03162027/aufemfix4,efi -> FS01:AFUD005/SKJPME2_0316201 ?afuefu1.sac - [cA] ****************************************************************************************** * * Please ignore this 'shell cannot read from file - device error' * Warning message due to it does not impact flashing process. * ****************************************************************************************** initating ' ' Delete successful. FSb.>

1. Press to load the default settings.
2. After loading the default settings, press to save the settings and exit the BIOS Setup utility.

6.6 CMOS Clear

JBT1 is used to clear CMOS, which will also clear any passwords. Instead of pins, this jumper consists of contact pads to prevent accidentally clearing the contents of CMOS.

To Clear CMOS

1. First power down the system completely.
2. Remove the blade from the enclosure to access the motherboard.

3. Remove the onboard battery from the motherboard.

4. Short the CMOS pads with a metal object such as a small screwdriver for at least four seconds.

JBT1 contact pads

1. Remove the screwdriver or shorting device.

2. Replace the cover, reconnect the power cords and power on the system.

Notes: Clearing CMOS will also clear all passwords.

Do not use the PW_ON connector to clear CMOS.

6.7 BMC Reset

The BMC can be reset using the UID button.

- Reset – Press and hold the button. After six seconds, the LED blinks at 2Hz. The BMC resets and the reset duration is approximately 250 ms. Then the BMC starts to boot.

- Restore factory default configuration – Hold the button for twelve seconds. The LED blinks at 4Hz while defaults are configured. Note: All BMC settings including username and password will be removed except the FRU and network settings.

Firmware update – When the BMC firmware is being updated, the UID LED blinks at 10Hz.

BMC Reset Options
Event UID LED	BMC Heartbeat LED
Reset Blue, Blinks at 2Hz Green, solid
Restore Defaults	Blue, Blinks at 4Hz Off
Update Blue, Blinks at 10Hz

6.8 Where to Get Replacement Components

If you need replacement parts for your system, to ensure the highest level of professional service and technical support, purchase exclusively from our Supermicro Authorized Distributors/System Integrators/Resellers. A list can be found at: http://www.supermicro.com. Click the "Where to Buy" tab.

6.9 Reporting an Issue

Technical Support Procedures

Before contacting Technical Support, please take the following steps. If your system was purchased through a distributor or reseller, please contact them for troubleshooting services. They have the best knowledge of your specific system configuration.

1. Please review the Troubleshooting Procedures in this manual and Frequently Asked Questions on our website before contacting Technical Support.
2. BIOS upgrades can be downloaded from our website. Note: Not all BIOS can be flashed depending on the modifications to the boot block code.

3. If you still cannot resolve the problem, include the following information when contacting us for technical support:

4. System, motherboard, and chassis model numbers and PCB revision number

5. BIOS release date/version (this can be seen on the initial display when your system first boots up)
6. System configuration

An example of a Technical Support form is posted on our website. Distributors: For immediate assistance, please have your account number ready when contacting our technical support department by email.

Returning Merchandise for Service

A receipt or copy of your invoice marked with the date of purchase is required before any warranty service will be rendered. You can obtain service by calling your vendor for a Returned Merchandise Authorization (RMA) number. When returning to the manufacturer, the RMA number should be prominently displayed on the outside of the shipping carton, and mailed prepaid or hand-carried. Shipping and handling charges will be applied for all orders that must be mailed when service is complete.

For faster service, RMA authorizations may be requested online (http://www.supermicro.com/support/rma/).

Whenever possible, repack the chassis in the original Supermicro carton, using the original packaging material. If these are no longer available, be sure to pack the chassis securely, using packaging material to surround the chassis so that it does not shift within the carton and become damaged during shipping.

This warranty only covers normal consumer use and does not cover damages incurred in shipping or from failure due to the alteration, misuse, abuse or improper maintenance of products.

During the warranty period, contact your distributor first for any product problems.

Vendor Support Filing System

For issues related to Intel, use the Intel IPS filing system:

https://www.intel.com/content/www/us/en/design/support/ips/training/welcome.html

For issues related to Red Hat Enterprise Linux, since it is a subscription based OS, contact your account representative.

6.10 Feedback

Supermicro values your feedback as we strive to improve our customer experience in all facets of our business. To provide feedback on our manuals, please email us at [e-mail].

6.11 Contacting Supermicro

Headquarters

Address: Super Micro Computer, Inc.

980 Rock Ave.

San Jose, CA 95131 U.S.A.

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Sales Inquiries)

[e-mail] (Gov. Sales Inquiries)

[e-mail] (Technical Support)

[e-mail] (RMA Support)

[e-mail] (Webmaster)

Website: www.supermicro.com

Europe

Address: Super Micro Computer B.V.

's-Hertogenbosch, The Netherlands

Tel: [phone]

Fax: [phone]

Email: [e-mail] (Sales Inquiries)

[e-mail] (Technical Support)

[e-mail] (RMA Support)

Website: www.supermicro.nl

Asia-Pacific

Address: Super Micro Computer, Inc.

3F, No. 150, Jian 1st Rd.

Zhonghe Dist., New Taipei City 235

Taiwan (R.O.C)

Tel: [phone]

Fax: [phone]

Email: [e-mail] (Sales Inquiries)

[e-mail] (Technical Support)

[e-mail] (RMA Support)

Website: www.supermicro.com.tw

Appendix A

BIOS POST Codes

A.1 BIOS POST Messages

During the Power-On Self-Test (POST), the BIOS will check for problems. If a problem is found, the BIOS will activate an alarm or display a message. The following is a list of such BIOS messages.

BIOS POST Messages
BIOS Message Description
Failure Fixed Disk	Fixed disk is not working or not configured properly. Check to see if fixed disk is attached properly. Run Setup. Find out if the fixed-disk type is correctly identified.
Stuck key Stuck key on keyboard.
Keyboard error Keyboard not working.
Keyboard Controller Failed Keyboard controller failed test. May require replacing keyboard controller.
Keyboard locked - Unlock key switch	Unlock the system to proceed.
Monitor type does not match CMOS - Run SETUP	Monitor type not correctly identified in Setup
Shadow Ram Failed at offset: nnnn	Shadow RAM failed at offset nnnn of the 64k block at which the error was detected.
System RAM Failed at offset: nnnn	System RAM failed at offset nnnn of in the 64k block at which the error was detected.
Extended RAM Failed at offset: nnnn	Extended memory not working or not configured properly at offset nnnn.
System battery is dead - Replace and run SETUP	The CMOS clock battery indicator shows the battery is dead. Replace the battery and run Setup to reconfigure the system.
System CMOS checksum bad - Default configuration used	System CMOS has been corrupted or modified incorrectly, perhaps by an application program that changes data stored in CMOS. The BIOS installed Default Setup Values. If you do not want these values, enter Setup and enter your own values. If the error persists, check the system battery or contact your dealer.
System timer error The timer test	failed. Requires repair of system board.
Real time clock error Real-Time	Clock fails BIOS hardware test. May require board repair.
Check date and time settings	BIOS found date or time out of range and reset the Real-Time Clock. May require setting legal date (1991-2099).
Previous boot incomplete - Default configuration used	Previous POST did not complete successfully. POST loads default values and offers to run Setup. If the failure was caused by incorrect values and they are not corrected, the next boot will likely fail. On systems with control of wait states, improper Setup settings can also terminate POST and cause this error on the next boot. Run Setup and verify that the waitstate configuration is correct. This error is cleared the next time the system is booted.
Memory Size found by POST differed from CMOS	Memory size found by POST differed from CMOS.
Diskette drive A error
Diskette drive B error	Drive A: or B: is present but fails the BIOS POST diskette tests. Check to see that the drive is defined with the proper diskette type in Setup and that the diskette drive is attached correctly.
Incorrect Drive A type - run SETUP	Type of floppy drive A: not correctly identified in Setup.
Incorrect Drive B type - run SETUP	Type of floppy drive B: not correctly identified in Setup.
System cache error - Cache disabled	RAM cache failed and BIOS disabled the cache. On older boards, check the cache jumpers. You may have to replace the cache. See your dealer. A disabled cache slows system performance considerably.
CPU ID: CPU socket number for	Multi-Processor error.
EISA CMOS not writeable Server	BIOS2 test error: Cannot write to EISA CMOS.
DMA Test Failed	ServerBIOS2 test error: Cannot write to extended DMA (Direct Memory Access) registers.
Software NMI Failed	ServerBIOS2 test error: Cannot generate software NMI (Non-Maskable Interrupt).
Fail-Safe Timer NMI Failed Server	BIOS2 test error: Fail-Safe Timer takes too long.
device Address Conflict Address	conflict for specified device.
Allocation Error for: device	Run ISA or EISA Configuration Utility to resolve resource conflict for the specified device.
CD ROM Drive CD ROM Drive identified.
Entering SETUP... Starting Setup program
Failing Bits: nnnn	The hex number nnnn is a map of the bits at the RAM address which failed the memory test. Each 1 (one) in the map indicates a failed bit. See errors 230, 231, or 232 above for offset address of the failure in System, Extended, or Shadow memory.
Fixed Disk n Fixed disk n (0-3) identified.
Invalid System Configuration Data	Problem with NVRAM (CMOS) data.
I/O device IRQ conflict I/O device	IRQ conflict error.
PS/2 Mouse Boot Summary Screen:	PS/2 Mouse installed.
nnnn kB Extended RAM Passed	Where nnnn is the amount of RAM in kilobytes successfully tested.
nnnn Cache SRAM Passed Where nnnn is the amount of system cache in kilobytes successfully tested.
nnnn kB Shadow RAM Passed Where nnnn is the amount of shadow RAM in kilobytes successfully tested.
nnnn kB System RAM Passed Where nnnn is the amount of system RAM in kilobytes successfully tested.
One or more I2O Block Storage Devices were excluded from the Setup Boot Menu	There was not enough room in the IPL table to display all installed I2O block-storage devices.
Operating system not found	Operating system cannot be located on either drive A: or drive C:. Enter Setup and see if fixed disk and drive A: are properly identified.
Parity Check 1 nnnn	Parity error found in the system bus. BIOS attempts to locate the address and display it on the screen. If it cannot locate the address, it displays ???? . Parity is a method for checking errors in binary data. A parity error indicates that some data has been corrupted.
Parity Check 2 nnnn	Parity error found in the I/O bus. BIOS attempts to locate the address and display it on the screen. If it cannot locate the address, it displays ???? .
Pressto resume,to Setup,for previous	Displayed after any recoverable error message. Pressto start the boot process orter Setup and change the settings. Pressto display the previous screen (usually an initialization error of an Option ROM, i.e., an add-on card). Write down and follow the information shown on the screen.
Pressto enter Setup Optional message displayed during POST. Can be turned off in Setup.
PS/2 Mouse: PS/2 mouse identified.
Run the I2O Configuration Utility	One or more unclaimed block storage devices have the Configuration Request bit set in the LCT. Run an I2O Configuration Utility (e.g. the SAC utility).
System BIOS shadowed System BIOS copied to shadow RAM.
UMB upper limit segment address: nnnn	Displays the address nnnn of the upper limit of Upper Memory Blocks, indicating released segments of the BIOS which can be reclaimed by a virtual memory manager.
Video BIOS shadowed Video BIOS successfully copied to shadow RAM.

A.2 BIOS POST Codes

This section lists the POST (Power-On Self-Test) codes for the AMI BIOS. POST codes are divided into two categories: recoverable and terminal.

Recoverable POST Errors

When a recoverable type of error occurs during POST, the BIOS will display an POST code that describes the problem. BIOS may also issue one of the following beep codes:

One long and two short beeps – video configuration error

One repetitive long beep – no memory detected

Terminal POST Errors

If a terminal type of error occurs, BIOS will shut down the system. Before doing so, BIOS will write the error to port 80h, attempt to initialize video and write the error in the top left corner of the screen.

The following is a list of codes that may be written to port 80h.

Terminal POST Errors
Code Description
02h Verify Real Mode
03h Disable Non-Maskable Interrupt (NMI)
04h Get CPU type
06h Initialize system hardware
07h Disable shadow and execute code from the ROM.
08h Initialize chipset with initial POST values
09h Set IN POST flag
0Ah Initialize CPU registers
0Bh Enable CPU cache
0Ch Initialize caches to initial POST values
0Eh Initialize I/O component
0Fh Initialize the local bus IDE
10h Initialize Power Management
11h Load alternate registers with initial POST values
12h Restore CPU control word during warm boot
13h Initialize PCI Bus Mastering devices
14h Initialize keyboard controller
16h 1-2-2-3 BIOS ROM checksum
17h Initialize cache before memory Auto size
18h 8254 timer initialization
1Ah 8237 DMA controller initialization
1Ch Reset Programmable Interrupt Controller
20h 1-3-1-1 Test DRAM refresh
22h 1-3-1-3 Test 8742 Keyboard Controller
28h Auto size DRAM
29h Initialize POST Memory Manager
2Ah Clear 512 kB base RAM
2Ch 1-3-4-1 RAM failure on address line xxxx*
2Eh 1-3-4-3 RAM failure on data bits xxxx* of low byte of memory bus
2Fh Enable cache before system BIOS shadow
32h Test CPU bus-clock frequency
33h Initialize Phoenix Dispatch Manager
36h Warm start shut down
38h Shadow system BIOS ROM
3Ah Auto size cache
3Ch Advanced configuration of chipset registers
3Dh Load alternate registers with CMOS values
41h Initialize extended memory for RomPilot
42h Initialize interrupt vectors
45h POST device initialization
46h 2-1-2-3 Check ROM copyright notice
47h Initialize I20 support
48h Check video configuration against CMOS
49h Initialize PCI bus and devices
4Ah Initialize all video adapters in system
4Bh QuietBoot start (optional)
4Ch Shadow video BIOS ROM
4Eh Display BIOS copyright notice
4Fh Initialize MultiBoot
50h Display CPU type and speed
51h Initialize EISA board
52h Test keyboard
54h Set key click if enabled
55h Enable USB devices
58h 2-2-3-1 Test for unexpected interrupts
59h Initialize POST display service
5Ah Display prompt "Press F2 to enter SETUP"
5Bh Disable CPU cache
5Ch Test RAM between 512 and 640 kB
60h Test extended memory
62h Test extended memory address lines
64h Jump to UserPatch1
66h Configure advanced cache registers
67h Initialize Multi Processor APIC
68h Enable external and CPU caches
69h Setup System Management Mode (SMM) area
6Ah Display external L2 cache size
6Bh Load custom defaults (optional)
6Ch Display shadow-area message
6Eh Display possible high address for UMB recovery
70h Display error messages
72h Check for configuration errors
76h Check for keyboard errors
7Ch Set up hardware interrupt vectors
7Dh Initialize Intelligent System Monitoring
7Eh Initialize coprocessor if present
80h Disable onboard Super I/O ports and IRQs
81h Late POST device initialization
82h Detect and install external RS232 ports
83h Configure non-MCD IDE controllers
84h Detect and install external parallel ports
85h Initialize PC-compatible PnP ISA devices
86h Re-initialize onboard I/O ports.
87h Configure Motherboard Configurable Devices (optional)
88h Initialize BIOS Data Area
89h Enable Non-Maskable Interrupts (NMIs)
8Ah Initialize Extended BIOS Data Area
8Bh Test and initialize PS/2 mouse
8Ch Initialize floppy controller
8Fh Determine number of ATA drives (optional)
90h Initialize hard-disk controllers
91h Initialize local-bus hard-disk controllers
92h Jump to UserPatch2
93h Build MPTABLE for multi-processor boards
95h Install CD ROM for boot
96h Clear huge ES segment register
98h 1-2 Search for option ROMs. One long, two short beeps on check-sum failure
99h Check for SMART Drive (optional)
9Ah Shadow option ROMs
9Ch Set up Power Management
9Dh Initialize security engine (optional)
9Eh Enable hardware interrupts
9Fh Determine number of ATA and SCSI drives
A0h Set time of day
A2h Check key lock
A4h Initialize typematic rate
A8h Erase F2 prompt
AAh Scan for F2 key stroke
ACh Enter SETUP
AEh Clear Boot flag
B0h Check for errors
B1h Inform RomPilot about the end of POST.
B2h POST done - prepare to boot operating system
B4h 1 One short beep before boot
B5h Terminate QuietBoot (optional)
B6h Check password (optional)
B7h Initialize ACPI BIOS
B9h Prepare Boot
BAh Initialize SMBIOS
BBh Initialize PnP Option ROMs
BCh Clear parity checkers
BDh Display MultiBoot menu
BEh Clear screen (optional)
BFh Check virus and backup reminders
C0h Try to boot with INT 19
C1h Initialize POST Error Manager (PEM)
C2h Initialize error logging
C3h Initialize error display function
C4h Initialize system error handler
C5h PnPnd dual CMOS (optional)
C6h Initialize note dock (optional)
C7h Initialize note dock late
C8h Force check (optional)
C9h Extended checksum (optional)
CAh Redirect Int 15h to enable remote keyboard
CBh Redirect Int 13h to Memory Technologies Devices such as ROM, RAM, PCMCIA, and serial disk
CCh Redirect Int 10h to enable remote serial video
CDh Re-map I/O and memory for PCMCIA
CEh Initialize digitizer and display message
D2h Unknown interrupt

The following are for the boot block in Flash ROM:

Boot Block Flash ROM Terminal POST Errors

Code Description

E0h Initialize the chipset

E1h Initialize the bridge

E2h Initialize the CPU

E3h Initialize system timer

E4h Initialize system I/O

E5h Check force recovery boot

E6h Checksum BIOS ROM

E7h Go to BIOS

E8h Set Huge Segment

E9h Initialize Multi Processor

EAh Initialize OEM special code

EBh Initialize PIC and DMA

ECH Initialize Memory type

EDh Initialize Memory size

EEh Shadow Boot Block

EFh System memory test

F0h Initialize interrupt vectors

F1h Initialize Run Time Clock

F2h Initialize video

F3h Initialize System Management Manager

F4h Output one beep

F5h Clear Huge Segment

F6h Boot to Mini DOS

F7h Boot to Full DOS

If the BIOS detects error 2C, 2E, or 30 (base 512K RAM error), it displays an additional word-bitmap (xxxx) indicating the address line or bits that failed. For example, "2C 0002" means address line 1 (bit one set) has failed. "2E 1020" means data bits 12 and 5 (bits 12 and 5 set) have failed in the lower 16 bits. The BIOS also sends the bitmap to the port-80 LED display. It first displays the checkpoint code, followed by a delay, the high-order byte, another delay, and then the low order byte of the error. It repeats this sequence continuously.

Appendix B

Standardized Warning Statements for AC Systems

About Standardized Warning Statements

The following statements are industry standard warnings, provided to warn the user of situations which have the potential for bodily injury. Should you have questions or experience difficulty, contact Supermicro's Technical Support department for assistance. Only certified technicians should attempt to install or configure components.

Read this appendix in its entirety before installing or configuring components in the Supermicro chassis.

These warnings may also be found on our website at http://www.supermicro.com/about/policies/safety_information.cfm.

Warning Definition

Warning! This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents.

警告の定義

この警告サインは危険を意味します。

Installation Instructions

Warning! Read the installation instructions before connecting the system to the power source.

設置手順書

Warning! This product relies on the building's installation for short-circuit (overcurrent) protection. Ensure that the protective device is rated not greater than: 250 V, 20 A.

サーキット・ブレーカー

Power Disconnection Warning

Warning! The system must be disconnected from all sources of power and the power cord removed from the power supply module(s) before accessing the chassis interior to install or remove system components.

CAUTION: This unit has redundant power sources. Please disconnect all the power cords before servicing.

電源切断の警告

Equipment Installation

Warning! Only trained and qualified personnel should be allowed to install, replace, or service this equipment.

機器の設置

Warning! This unit is intended for installation in restricted access areas. A restricted access area can be accessed only through the use of a special tool, lock and key, or other means of security. (This warning does not apply to workstations).

アクセス制限区域

(.תְבָרִיֹהַעָהָהָה)תְבָרִיֹהָה,הַעָהָה

Warning! There is the danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer's instructions

電池の取り扱い

Redundant Power Supplies

Warning! This unit might have more than one power supply connection. All connections must be removed to de-energize the unit.

冗長電源装置

Warning! Hazardous voltage or energy is present on the backplane when the system is operating. Use caution when servicing.

バックプレーンの電圧

Comply with Local and National Electrical Codes

Warning! Installation of the equipment must comply with local and national electrical codes.

地方および国の電気規格に準拠

Warning! Ultimate disposal of this product should be handled according to all national laws and regulations.

製品の廃棄

Warning! Hazardous moving parts. Keep away from moving fan blades. The fans might still be turning when you remove the fan assembly from the chassis. Keep fingers, screwdrivers, and other objects away from the openings in the fan assembly's housing.

ファン・ホットスワップの警告

Power Cable and AC Adapter

Warning! When installing the product, use the provided or designated connection cables, power cables and AC adaptors. Using any other cables and adaptors could cause a malfunction or a fire. Electrical Appliance and Material Safety Law prohibits the use of UL or CSA-certified cables (that have UL/CSA shown on the cord) for any other electrical devices than products designated by Supermicro only.

電源コードとACアダプター

Specifications and Compliance

Processor

4th Generation Intel Xeon Scalable processor in socket E (LGA4677) with up to 56 cores and a thermal design power (TDP) of up to 350W (depending on model and cooling system)

Chipset

Intel PCH C741

Memory

Supports up to 4TB of DDR5 ECC 3Ds RDIMM/3DS LRDIMM/Registered R/LR/3DSLR memory with speeds of up to 4800 MT/s; DIMM size up to 256GB

BIOS

256Mb SPI Flash EEPROM with AMI BIOS

Motherboard

B13SEE-CPU-25G, 9.358" x 11.408" (23.77cm x 28.98cm)

Chassis

MCP-680-61006-0N; 6U height; 9.75" x 1.75" x 23.5" (24.8 x 4.4 x 59.7 cm); SBI-611E-5T2N width, 3.5" (8.8cm)

Storage Drives

Two hot-swap U.2 2.5" bays: SBI-611E-1C2N, two SAS, SATA or NVMe; plus one M.2 SATA/NVMe SBI-611E-1T2N, two SATA or NVMe; plus one M.2 SATA/NVMe and two M.2 NMVe SBI-611E-5T2N, two SATA or NVMe; plus one M.2 SATA/NVMe and two M.2 NMVe

LAN Connections

Two 25G onboard

Expansion Slots

One PCIe Gen5 x16 slot, one PCIe Gen5 x8 slot, support one FHFL double-width GPU or two single-width PCIe cards

Operating Environment

Operating Temperature: 10° to 35° C (50° to 95° F) Non-operating Temperature: -40° to 70° C (-40° to 158° F) Operating Relative Humidity: 8% to 90% (non-condensing) Non-operating Relative Humidity: 5% to 95% (non-condensing)

Regulatory Compliance

FCC, ICES, CE, VCCI, RCM, UKCA, NRTL, CB

Safety Model

UL or CSA model M628-22

Applied Directives, Standards

EMC/EMI: 2014/30/EU (EMC Directive)

Electromagnetic Compatibility Regulations 2016

FCC Part 15

ICES-003

VCCI-CISPR 32

AS/NZS CISPR 32

BS/EN55032

BS/EN55035

CISPR 32

CISPR 24/CISPR 35

BS/EN 61000-3-2

BS/EN 61000-3-3

BS/EN 61000-4-2

BS/EN 61000-4-3

BS/EN 61000-4-4

BS/EN 61000-4-5

BS/EN 61000-4-6

BS/EN 61000-4-8

BS/EN 61000-4-11

Environment:

2011/65/EU (RoHS Directive)

EC 1907/2006 (REACH)

2012/19/EU (WEEE Directive)

California Proposition 65

Product Safety: 2014/35/EU (LVD Directive)

UL/CSA 62368-1 (USA and Canada)

Electrical Equipment (Safety) Regulations 2016

IEC/BS/EN 62368-1

Perchlorate Warning

California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. "Perchlorate Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate"

General Data Center Environmental Specifications

Particulate contamination specifications

Air filtration: Data centers must be kept clean to Class 8 of ISO 14644-1 (ISO 2015). The air entering the data center should be filtered with a MERV 11 filter or better. The air within the data center should be continuously filtered with a MERV 8 filter or better.

Conductive dust: Air should be free fo conductive dust, zinc whiskers, or other conductive particles.

Corrosive dust: Air should be free of corrosive dust.

Gaseous* contamination specifications

Copper coupon corrosion rate: <300 Å/month per class G1 as defined by ANSI.ISA71.04-2013, reference by ASHRAE TC 9.9

Silver coupon corrosion rate: <200 Å/month per class G1 as defined by ANSI.ISA71.04-2013, reference by ASHRAE TC 9.9

*If testing with silver or copper coupons results in values less than 200 Å/month or 300 Å/month, respectively, then operating up to 70% relative humidity (RH) is acceptable. If the testing shows corrosion levels exceed these limits, then catalyst-type pollutants are probably present and RH should be driven to 50% or lower.