SBS-820H-420P - Server Supermicro - Free user manual and instructions

USER MANUAL SBS-820H-420P Supermicro

Two views of a server rack with green circuit boards and power modules (no visible text or labels)

USER'S MANUAL

Revision 1.0a

The information in this User's Manual has been carefully reviewed and is believed to be accurate. The vendor assumes no responsibility for any inaccuracies that may be contained in this document, and makes no commitment to update or to keep current the information in this manual, or to notify any person or organization of the updates. Please Note: For the most up-to-date version of this manual, please see our website at www.supermicro.com.

Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product described in this manual at any time and without notice. This product, including software and documentation, is the property of Supermicro and/or its licensors, and is supplied only under a license. Any use or reproduction of this product is not allowed, except as expressly permitted by the terms of said license.

IN NO EVENT WILL Super Micro Computer, Inc. BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE USE OR INABILITY TO USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. IN PARTICULAR, SUPER MICRO COMPUTER, INC. SHALL NOT HAVE LIABILITY FOR ANY HARDWARE, SOFTWARE, OR DATA STORED OR USED WITH THE PRODUCT, INCLUDING THE COSTS OF REPAIRING, REPLACING, INTEGRATING, INSTALLING OR RECOVERING SUCH HARDWARE, SOFTWARE, OR DATA.

Any disputes arising between manufacturer and customer shall be governed by the laws of Santa Clara County in the State of California, USA. The State of California, County of Santa Clara shall be the exclusive venue for the resolution of any such disputes. Supermicro's total liability for all claims will not exceed the price paid for the hardware product.

FCC Statement: This equipment has been tested and found to comply with the limits for a Class A or Class B digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in industrial environment for Class A device or in residential environment for Class B device. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the manufacturer's instruction manual, may cause harmful interference with radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case you will be required to correct the interference at your own expense.

California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. "Perchlorate Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate".

WARNING: This product can expose you to chemicals including lead, known to the State of California to cause cancer and birth defects or other reproductive harm. For more information, go to www.P65Warnings.ca.gov.

The products sold by Supermicro are not intended for and will not be used in life support systems, medical equipment, nuclear facilities or systems, aircraft, aircraft devices, aircraft/emergency communication devices or other critical systems whose failure to perform be reasonably expected to result in significant injury or loss of life or catastrophic property damage. Accordingly, Supermicro disclaims any and all liability, and should buyer use or sell such products for use in such ultra-hazardous applications, it does so entirely at its own risk. Furthermore, buyer agrees to fully indemnify, defend and hold Supermicro harmless for and against any and all claims, demands, actions, litigation, and proceedings of any kind arising out of or related to such ultra-hazardous use or sale.

Manual Revision 1.0

Release Date: December 09, 2021 mk

Unless you request and receive written permission from Super Micro Computer, Inc., you may not copy any part of this document. Information in this document is subject to change without notice. Other products and companies referred to herein are trademarks or registered trademarks of their respective companies or mark holders.

Copyright © 2021 by Super Micro Computer, Inc.

All rights reserved.

Printed in the United States of America

Preface

About this Manual

This manual is written for professional system integrators and PC technicians. It provides information for the installation and use of this blade server. Installation and maintenance should be performed by experienced technicians only.

Please refer to the SBI-420P-1(C2/T3)N specifications page on our website for updates on supported memory, processors and operating systems (http://www.supermicro.com).

Notes

For your system to work properly, download all necessary drivers/utilities and the user's manual for your server.

• Supermicro product manuals: http://www.supermicro.com/support/manuals/
- Product drivers and utilities: https://www.supermicro.com/wftp
- Product safety info: http://www.supermicro.com/about/policies/safety_information.cfm

If you have any questions, contact our support team at: [e-mail]

This manual may be periodically updated without notice. Check the Supermicro website for possible updates to the manual revision level.

Secure Data Deletion

A secure data deletion tool designed to fully erase all data from storage devices can be found on our website: https://www.supermicro.com/about/policies/disclaimer.cfm?url=/wftp/utility/Lot9_Secure_Data_Deletion_Utility/

Warnings

Special attention should be given to the following symbols used in this manual.

Warning! Indicates important information given to prevent equipment/property damage or personal injury.

Warning! Indicates high voltage may be encountered when performing a procedure.

Contents

Chapter 1 Introduction

1.1 Overview....7 Eligible Enclosures....7

1.2 System Features 8

1.3 Front View....9 Control Panel....10

1.4 Components ...... 11

Chapter 2 Installation and Setup

2.1 Unpacking the System ....13

2.2 Installing or Removing the Blade Unit....14 Installing a Blade Unit into the Enclosure....14 Removing a Blade Unit from the Enclosure....14 Removing and Replacing the Blade Cover....15

2.3 Powering Up or Down....16 Powering Up a Blade Unit....16 Powering Down a Blade Unit....16

2.4 Processor and Heatsink Installation....17 The Processor Carrier Assembly....18 The Processor Heatsink Module (PHM)....20 Installing the PHM into the CPU Socket....21 Removing the PHM from the CPU Socket....24 Removing the Processor Carrier Assembly from the PHM....25 Removing the Processor from the Carrier Assembly....26

2.5 Memory....27 Memory Support....27 Memory Population Guidelines....30 Guidelines Regarding Mixing DIMMs....30 DIMM Construction....30 Memory Population Sequence....30 Installing Memory....31

2.6 Storage Drives....32 Drive Carrier Indicators....32 Hot-Swap for NVMe Drives....35

Checking the Temperature of an NVMe Drive....36

M.2 Solid State Drives 37

2.7 Installing the Operating System....38
Linux Installation with Two Storage Drives 39
2.8 Configuring and Setting up RAID....39
2.9 Driver Installation....40

Chapter 3 Management and Maintenance

3.1 Management Software 41

IPMI 41

BMC ADMIN User Password....41

3.2 Motherboard Battery....42
3.3 System Cooling 43

Installing the Air Shrouds 43

Checking the Server Air Flow 44

Overheating....44

3.4 CMOS Clear 44
3.5 Where to Get Replacement Components....45
3.6 Returning Merchandise for Service....45

Chapter 4 Motherboard Connections

4.1 Motherboard Layout 46
Chipset Block Diagram....48
4.2 Power Connections 49
4.3 Headers and Connectors .... 49
4.4 Jumpers....51

Explanation of Jumpers....51

4.5 LED Indicators....52

Chapter 5 UEFI BIOS

5.1 Introduction....53
Starting BIOS Setup Utility....53
5.2 Main Setup....53
5.3 Tab Pages ....55

Appendix A BIOS POST Codes

Appendix B Standardized Warning Statements for AC Systems

Appendix C CPU-Based RAID for NVMe

Appendix D Specifications and Compliance Statements

Contacting Supermicro

Headquarters

Address: Super Micro Computer, Inc.

980 Rock Ave.

San Jose, CA 95131 U.S.A.

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Technical Support)

Website: www.supermicro.com

Europe

Address: Super Micro Computer B.V.

's-Hertogenbosch, The Netherlands

Tel: [phone]

Fax: [phone]

Email: [e-mail] (General Information)

[e-mail] (Technical Support)

[e-mail] (Customer Support)

Website: www.supermicro.nl

Asia-Pacific

Address: Super Micro Computer, Inc.

3F, No. 150, Jian 1st Rd.

Zhonghe Dist., New Taipei City 235

Taiwan (R.O.C)

Tel: [phone]

Fax: [phone]

Email: [e-mail]

Website: www.supermicro.com.tw

Chapter 1

Introduction

1.1 Overview

The SBI-420P-1(C2/T3)N blade is a compact self-contained server that connects into a pre-cabled enclosure that provides power, cooling, management and networking functions. One 8U enclosure can hold up to twenty blades. Each blade contains one computing node.

In this manual, “blade” or “blade unit” refers to a single blade, and “blade enclosure” is the chassis that houses the blades, power supplies and other modules. “Blade system” refers to the enclosure, blades units, and various management and networking modules.

This chapter outlines of the functions and features.

Models
System Storage	Type Add-on Module
SBI-420P-1T3N SATA/NVMe hybrid ports AOM-SB1-SATA31
SBI-420P-1C2N SAS/NVMe hybrid ports AOM-B3108-H8-B11

In addition, several important parts are listed below.

Main Parts List
Description Part Number Quantity
Motherboard B12DPT-6 1
Blade chassis MCP-680-41001-0N 1
Add-on card, front control board	AOM-SB410-01	1
Air Shroud	MCP-310-41008-0N 1
License key for OOB BIOS	SFT-OOB-LIC	1
CPU heatsink, passive	SNK-P1044V	2
(optional) Mezzanine add-on card, Omnipath, 100Gbps	AOC-OPH-WFR	1*
(optional) Mezzanine add-on card, M.2 support	AOM-B-4M	1*
(optional) Mezzanine add-on card, Infiniband EDR	AOC-IBH-X4ESO	1*
(optional; SBI-420P-1T3N only) Mezzanine add-on card, two-port 25GbE	AOC-B25G-M2SP	1*

*Mutually exclusive—only one mezzanine card can be installed.

Eligible Enclosures

• SBE-820C-[4/6/8]22
• SBE-820H-[4/6/8]22
• SBE-820J-[4/6/8]22
• SBE-820L-[4/6/8]22

1.2 System Features

The following is an overview of the main features of the SBI-420P-1(C2/T3)N blade server.

System Features

Motherboard

B12DPT-6

CPU

Dual Intel Xeon 3rd Gen Scalable processors (in Socket P4 LGA4189) with up to 38 cores and a thermal design power (TDP) of up to 270W; for the updates, refer to our website at Supermicro.com

Chipset

Intel C621A

Memory

Sixteen slots for up to 2TB of 3DS LRDIMM/LRDIMM/3DS RDIMM/RDIMM DDR4 ECC memory with speeds of 3200/2933/2666MT/s; DIMM size up to 128G at 1.2V

Storage Drives

SBI-420P-1C2N: Two hot-swap 2.5" SAS or NVMeSBI-420P-1T3N: Two hot-swap 2.5"SATA or NVMe, and one hot-swap 2.5" SATAOne M.2 SSD; an optional add-on module can provide four additional M.2 SSDs

BIOS

256Mb SPI Flash EEPROM with AMI BIOS

LAN Connections

Two 25GbE onboard; an optional mezzanine card can provide two additional 25GbE, EDR, HDR, or OPA

Form Factor

4U (WxHxD) 1.75 x 6.5 x 23.5 in. (44.5 x 165 x 597 mm)

1.3 Front View

The illustration below shows the features on the front of the blade unit.

Diagram of a server rack with labeled ports and connectors, highlighting a red annotation on the port 2.

Figure 1-1. Front View, SBI-420P-1T3N

Chassis Front Features
Item Features Description
1 Control	Panel Power button and status indicators
2 Storage	Drives	Two hot-swap 2.5" drive bays for NVME, SAS, or SATA; the drive carriers display status lights
3 Storage	Drive SBI-420P-1T3N only; one hot-swap 2.5" drive bay for SATA
4 KVM Keyboard, video, mouse connector
5	Service/Asset Tag	Pull-out identifier (with BMC ADMIN default password sticker)

Docker Top Cover Docker Top Cover 5 BMC Password Sticker

Figure 1-2. Location of the BMC Password Label

Control Panel

Power switches and status LEDs are located on the control panel on the front of the chassis.

1 2 3 4 5

Figure 1-3. Control Panel

Control Panel Features
Item	Feature State	Description
1 Power button			The main power switch applies or removes primary power from the power supply to the server but maintains standby power.
2 Power LED		Green Power on
		Flashing amber Before the BMC is ready, blinks until every node is ready
3 KVM/UID LED		Steady blue Indicates that KVM has been initialized
		Flashing blue	Unit IDentifier indicator (The UID function is activated with a management program.)
4 NIC	LED	Flashing green	Indicates traffic (Tx and RX data) on the LAN connection to this blade module
		Flashing orange Indicates traffic over the network (when present in the system)
5	System Fail LED	Red	Indicates a fatal error. This may be a memory error or any other error that prevents the operating system from booting.

1.4 Components

SBI-420P-1C2N SBI-420P-1T3N

Figure 1-4. Components Labeled (see next page)

Blade Components
Item Feature Description
1 Drive SBI-420P-1T3N only; 2.5" SATA storage drive
2 CPU Processor and heatsink
3 Memory DIMM slots
4 Air Shrouds Direct air flow
5 Module SBI-420P-1C2N only; enables Broadcom 3108 RAID controller
6 Module SBI-420P-1T3N only; provides SATA for third drive
7 Chipset Motherboard chipset
8 Mezzanine		Add-on card provides one of: EDR, HDR, M.2 storage, Omnipath 100Gbps, or 25GbE network speed; see Section 1.1 for more details; for SBI-420P-1T3N, the mezzanine connector is shown

Chapter 2

Installation and Setup

This chapter provides instructions on installing and replacing user-replaceable components. To prevent compatibility issues, use only components that match the specifications or parts qualified by Supermicro.

Up to twenty blade modules may be installed into a blade enclosure. Blade modules with different operating systems, such as Windows and Linux, may be mixed together in the same blade enclosure.

2.1 Unpacking the System

Inspect the box the system was shipped in and note if it was damaged in any way. If any equipment appears damaged, please file a damage claim with the carrier who delivered it.

Decide on a suitable location for the rack unit that will hold the enclosure. It should be situated in a clean, dust-free area that is well ventilated. Avoid areas where heat, electrical noise and electromagnetic fields are generated. It will also require a grounded AC power outlet nearby. Be sure to read the precautions and considerations noted in Appendix B.

2.2 Installing or Removing the Blade Unit

Installing a Blade Unit into the Enclosure

Make sure the cover of the blade unit is on before insertion.

Installing a Blade Unit into the Enclosure

1. Pull open the blade locking handle and slowly push the blade into its bay.
2. After the blade is seated in the enclosure, push the handle into its locked position, making sure the notch in the handle catches the lip of the enclosure.

Caution: Insert the blade with caution so the rear connectors are not damaged. If mezzanine connectors are installed, be aware that the stand-offs are properly installed before the mezzanine card is installed.

Line drawing of a computer power supply unit with internal components and connectors (no text or symbols)

Figure 2-1. Blade Locking Handle

Removing a Blade Unit from the Enclosure

A blade can be removed from the enclosure while other blades continue to operate.

Removing a Blade Unit from the Enclosure

1. Power down the blade unit.
2. Release the handle completely and use it to pull the blade unit from the enclosure.

Removing and Replacing the Blade Cover

The blade cover can be removed to access the mainboard and install processors, memory modules, the onboard battery, etc.

• To remove the cover, remove the screw as illustrated below. Slide the cover toward the rear and lift it off.
• To replace the cover, fit the six studs on the inside of the cover into the slots of the chassis, then slide the cover toward the front of the blade to lock it into place. Replace the locking screw.

Screw Studs Slots

Figure 2-2. Removing the Cover

2.3 Powering Up or Down

Each blade unit may be powered on and off independently from the rest of the blades in the enclosure.

Powering Up a Blade Unit

A blade unit may be powered up in two ways:

• Press the power button on the blade unit.
• Use IPMIView or the browser based management utility to apply power using either a CMM module, or by the use of the onboard BMC chip in the blade module.

Powering Down a Blade Unit

A blade unit may be powered down in any of the following ways:

• Press the power button on the blade unit.
• Use IPMIView or the browser based management utility to power down; requires Operator or Admin privileges on the CMM.
• Use SMCIPMltool when connected to the CMM to power down; requires Operator or Admin privileges on the CMM.
• Use IPMIview or a browser connected to the onboard BMC chip to power down.
• Use SMCIPMItool Command Line Interface (CLI) to the onboard BMC chip; requires Operator or Admin privileges.

2.4 Processor and Heatsink Installation

The processor (CPU) and processor carrier should be assembled together first to form the processor carrier assembly. This will be attached to the heatsink to form the processor heatsink module (PHM) before being installed onto the CPU socket.

Notes:

• Use ESD protection.
• Remove the blade from the enclosure and the blade chassis cover to expose the motherboard.
• Check that the plastic protective cover is on the blade CPU socket and that none of the socket pins are bent. If they are, contact your retailer.
• When handling the processor, avoid touching or placing direct pressure on the land grid array (gold contacts).
• Improper installation or socket misalignment can cause serious damage to the processor or the socket and may require manufacturer repairs.
  • Thermal grease is pre-applied on new heatsinks. No additional thermal grease is needed.
  • Refer to the Supermicro website for updates on processor support.
• Graphics in this manual are for illustration only. Your components may look different.

The Processor Carrier Assembly

The processor carrier assembly is comprised of the processor and the processor carrier.

1. Hold the processor with the land grid array (LGA, gold contacts) facing down. Locate the gold triangle at the corner of the processor and the corresponding hollowed triangle on the processor carrier as shown below. These triangles indicate the location of pin 1.

Pin 1

Processor

Pin 1

Carrier

1. Turn the processor over (with the gold LGA up). Locate the CPU keys on the processor and the four latches on the carrier as shown below.

Processor (Reverse Side Up) Latch Carrier (Top Side Up) CPU Key Latch Latch Latch CPU Key

1. Locate the lever on the carrier and, if necessary, press it down as shown below.

Lever

1. Align the CPU keys on the processor (A & B) with those on the carrier (a & b) as shown below.

CPU Key (on the processor) CPU Key (on the carrier) Latch 1 B b A 2 a Latch CPU Key (on the processor) CPU Key (on the carrier)

1. Carefully place one end of the processor under latch 1 on the carrier, and then press the other end down until it snaps into latch 2 and is properly seated on the carrier.

Processor Carrier Assembly (Top View) Processor Carrier Assembly (Underside view)

The Processor Heatsink Module (PHM)

After creating the processor carrier assembly, mount the heatsink onto the carrier assembly to form the processor heatsink module (PHM).

Note: If this is a new heatsink, the thermal grease has been pre-applied. Otherwise, apply the proper amount of thermal grease to the underside of the heatsink.

1. Turn the heatsink over with the thermal grease facing up. Note the two triangle cutouts (A, B) located at the diagonal corners of the heatsink as shown in the drawing below.

2. On the processor carrier assembly, find pin 1, as noted by the triangles. Hold the processor carrier assembly over so that the gold LGA is facing up.

3. Align clip "a" (pin 1) on the carrier assembly with the triangular cutout A on the heatsink and b, c, d on the carrier assembly with B, C, D on the heatsink.

4. Push the carrier assembly onto the heatsink, making sure that all four clips on each corner are properly secured.

Processor Carrier Assembly Pin1 Heatsink

Note: These diagrams show an air cooled heatsink.

Installing the PHM into the CPU Socket

1. Remove the plastic protective cover from the CPU socket. Gently squeeze the grip tabs then pull the cover off.

CPU Socket with Plastic Protective Cover Grip Tabs

1. Locate four threaded fasteners (a, b, c, d) on the CPU socket.

CPU Socket Threaded Fastener a b c d (a, b, c, d: Threaded Fasteners) CPU Socket Pin1

1. Locate four PEEK nuts (A, B, C, D) and four rotating wires (1, 2, 3, 4) on the heatsink as shown below.

A, B, C, D: Peek Nut 1, 2, 3, 4: Rotating Wire a, b, c, d: Threaded Fastener Heatsink Rotating Wire 2 B D Rotating Wire 4 Rotating Wire 3 Rotating Wire 1 Peek Nut CPU Socket Rotating Wire Peek Nut (Unlatched) (latched)

1. Check that the rotating wires (1, 2, 3, 4) are in the unlatched position as shown.

Unlatched State Rotating Wire Side View Top View Peek Nut

1. Align nut A (next to the triangles and pin 1) on the heatsink with threaded fastener "a" on the CPU socket. Also align nuts B, C, D on the heatsink with threaded fasteners b, c, d on the CPU socket.

2. Gently place the heatsink on the CPU socket, making sure that each nut is properly aligned with its corresponding threaded fastener.

A, B, C, D: Peek Nut on the Heatsink B D C A b c a, b, c, d: Threaded Fastener on the CPU socket

1. Press all four rotating wires outward to latch the PHM onto the CPU socket.

Rotating Wire Rotating Wire Latched State Top View

1. With a t30-bit screwdriver, tighten all PEEK nuts in the sequence of A, B, C, and D with even pressure not greater than 12 lbf-in.

Technical diagram showing a mechanical assembly before and after transformation, with labeled components A, B, C, D and red arrows indicating motion or movement.

Removing the PHM from the CPU Socket

Be sure the system is shut down and all AC power cords are unplugged.

1. Use a t30-bit screwdriver to loosen the four PEEK nuts on the heatsink in the sequence of A, B, C, and D.

B D C Peek Nut

1. Press the four rotating wires inward to unlatch the PHM as shown below.

Unlatched State Rotating Wire Side View Peek Nut

1. Gently lift the PHM upward to remove it from the CPU socket.

Technical line drawing of a mechanical housing assembly with red directional arrows indicating movement or force (no text or symbols present)

Removing the Processor Carrier Assembly from the PHM

Detach the four plastic clips (a, b, c, d) on the processor carrier assembly from the four corners of the heatsink (A, B, C, D) as shown below, and lift off the processor carrier assembly.

Processor Carrier Assembly Pin 1 Pin 1 a b c D Heatsink A B C

Removing the Processor from the Carrier Assembly

Unlock the lever from its locked position and push it upwards to disengage the processor from the carrier as shown below right. Carefully remove the processor from the carrier.

Processor Carrier Assembly

Isometric technical drawing of a microchip or integrated circuit board (no text or symbols visible)

Lever

Note: Handle the processor with care to avoid damage.

Isometric technical diagram of a computer motherboard with a highlighted component and red arrow indicating upward motion (no text or symbols present)

2.5 Memory

Memory Support

The B12DPT-6 has 16 DIMM slots for up to 2TB of 3DS LRDIMM/LRDIMM/3DS RDIMM/RDIMM DDR4 (288-pin) ECC memory with speeds of 3200/2933/2666MT/s. For validated memory, use our Product Resources page.

DDR4 Memory Support for 83xx/63xx/53xx/43xx Processors
Type	Ranks Per DIMM and Data Width	DIMM Capacity (GB)		Speed (MT/s)
				One Slot per Channel
		DRAM Density		One DIMM per Channel
		4 Gb 8 Gb	1.2 Volts
RDIMM	SRx8 8GB 16GB			3200*
	SRx4 16GB 32GB
	DRx8 16GB 32GB
	DRx4 32GB 64GB
RDIMM 3DS (4R/8R) x4		2H-64GB4H-128GB	2H-128GB4H-256GB
LRDIMM QRx4	64GB 128GB
LRDIMM 3DS (4R/8R) x4 4H-		128GB	2H-128GB4H-256GB

*Only the 83xx and 63xx series support 3200MT/s; for other processors, memory speed as supported by the CPU.

Memory Population Guidelines

• All DIMMs must be DDR4.
- Balance memory. Using unbalanced memory topology, such as populating two DIMMs in one channel while populating one DIMM in another channel, reduces performance. It is not recommended for Supermicro systems.
- In dual-CPU configurations, memory must be installed in the slots associated with the installed CPUs. Also, an odd number of total DIMMs is not recommended.
- For MM, NM/FM ratio is between 1:4 and 1:16. The capacity not used for FM can be used for AD. (NM = Near Memory; FM = Far Memory).

Guidelines Regarding Mixing DIMMs

• Populating slots with a pair of DIMM modules of the same type and capacity results in interleaved memory, which improves memory performance.
• Use memory modules of the same type and speed, as mixing is not allowed.
• x4 and x8 DIMMs can be mixed in the same channel.
• Mixing of LRDIMMs and RDIMMs is not allowed in the same channel, across different channels, and across different sockets.
• Mixing of non-3DS and 3DS LRDIMM is not allowed in the same channel, across different channels, and across different sockets.

DIMM Construction

• RDIMM (non-3DS) Raw Cards: A/B (2Rx4), C (1Rx4), D (1Rx8), E (2Rx8)
  • 3DS RDIMM Raw Cards: A/B (4Rx4)
  • LRDIMM (non-3DS) Raw Cards: D/E (4Rx4)
  • 3DS LRDIMM Raw Cards: A/B (8Rx4)

Memory Population Sequence

Blue slots versus black slots: Install the first DIMM in the blue memory slot, which is the first of a memory channel.

The following memory population sequence table was created based on guidelines provided by Intel to support Supermicro motherboards.

Memory Population for the X12 DP Motherboard, 16 DIMM Slots
CPUs/DIMMs Memory Slots
1 CPU & 1 DIMM*	A1
1 CPU & 2 DIMMs	A1, E1
1 CPU & 4 DIMMs	A1, C1, E1, G1
1 CPU & 6 DIMMs*	A1, B1, C1, E1, F1, G1
1 CPU & 8 DIMMs	A1, B1, C1, D1, E1, F1, G1, H1
2 CPUs & 2 DIMMs	CPU1: A1CPU2: A1
2 CPUs & 4 DIMMs	CPU1: A1, E1CPU2: A1, E1
2 CPUs & 6 DIMMs*	CPU1: A1, C1, E1, G1CPU2: A1, E1
2 CPUs & 8 DIMMs	CPU1: A1, C1, E1, G1CPU2: A1, C1, E1, G1
2 CPUs & 10 DIMMs*	CPU1: A1, B1, C1. E1, F1, G1CPU2: A1, C1, E1, G1
2 CPUs & 12 DIMMs	CPU1: A1, B1, C1. E1, F1, G1CPU2: A1, B1, C1. E1, F1, G1
2 CPUs & 14 DIMMs*	CPU1: A1, B1, C1. D1, E1, F1, G1, H1CPU2: A1, B1, C1. E1, F1, G1
2 CPUs & 16 DIMMs	CPU1: A1, B1, C1. D1, E1, F1, G1, H1CPU2: A1, B1, C1. D1, E1, F1, G1, H1

*Unbalanced, not recommended.

P1-DIMME1 P1-DIMME1 P1-DIMMH1 P1-DIMMG1 CPU1 BAR CODE HD MAC CODE B12DPT-6 REV:1.00 P2-DIMMB1 P2-DIMMA1 P2-DIMMD1 P2-DIMMC1 P1-DIMMC1 P1-DIMMA1 P1-DIMMB1 CPU2 Front P2-DIMMG1 P2-DIMMH1 P2-DIMME1 P2-DIMMF1 A1

Figure 2-3. Memory Slots

Installing Memory

ESD Precautions

Electrostatic Discharge (ESD) can damage electronic components including memory modules. To avoid damaging DIMM modules, it is important to handle them carefully. The following measures are generally sufficient.

• Use a grounded wrist strap designed to prevent static discharge.
• Handle the memory module by its edges only.
• Put the memory modules into the antistatic bags when not in use.

Installing Memory

Begin by removing power from the system as described in Section 3.1. Follow the memory population sequence in the table above.

1. Push the release tabs outwards on both ends of the DIMM slot to unlock it.

Notches Release Tabs

1. Align the key of the DIMM with the receptive notch on the memory slot and with your thumbs on both ends of the module, press it straight down into the slot until the module snaps into place.

Key

1. Press the release tabs to the locked position to secure the DIMM module into the slot.

Caution: Exercise extreme caution when installing or removing memory modules to prevent damage to the DIMMs or slots.

Removing Memory

To remove a DIMM, unlock the release tabs then pull the DIMM from the memory slot.

2.6 Storage Drives

The blade has two or three 2.5" hot-swap storage drive bays. The drives are mounted in drive carriers that simplify their removal from the chassis. These carriers also help promote proper airflow. Even carriers without drives must remain in the chassis for proper airflow.

The blade can be purchased with drives installed or without drives installed.

For VROC configurations, refer to the VROC appendix in this manual.

Note: Enterprise level storage drives are recommended for use in Supermicro systems. For information on recommended drives, visit the Supermicro website.

Drive Carrier Indicators

Each drive carrier has two LED indicators: an activity indicator and a status indicator. For RAID configurations using a controller, the meaning of the status indicator is described in the table below. For OS RAID or non-RAID configurations, some LED indications are not supported, such as hot spare.

Drive Carrier LED Indicators
	Color Blinking Pattern Behavior		for Device
Activity LED	Blue Solid On SAS/NVMe drive installed
	Blue Blinking I/O activity
Status LED	Red Solid On Failure of drive with RSTe support
	Red Blinking at 1 Hz Rebuilding drive with RSTe support
	Red Blinking with two blinks and one stop at 1 Hz		Hot spare for drive with RSTe support (not supported in VMD mode)
	Red On for five seconds, then off		Power on for drive with RSTe support
	Red Blinking at 4 Hz Identify drive with RSTe support
	Green Solid On Safe to remove NVMe device (not supported in VMD mode)
	Amber Blinking at 1 Hz Attention state—do not remove NVMe device (not supported in VMD mode)

Removing a Hot-Swap Drive Carrier from the Chassis

1. Press the release button (shown below as ①) on the drive carrier, which will extend the drive carrier handle.
2. Use the drive carrier handle to pull the drive out of the chassis.

Technical diagram showing assembly steps of a computer drive, with labeled components and directional arrows indicating assembly direction.

Figure 2-4. Removing a Drive Carrier

Installing a Drive

1. Remove the dummy drive, which comes pre-installed in the drive carrier. Pull out the two locking clasps on the left outside of the carrier and lift out the dummy drive.
2. Position the drive above the carrier with the PCB side facing down and the connector end toward the rear of the carrier.

Mountion Clasps Alignment Posts Align Both Mounting Flakes Locking Clasps

Figure 2-5. Removing the Dummy Drive from a Carrier

1. Tilt the drive to insert it onto the two posts on the right inside of the carrier.
2. Pull out the two spring locking clasps and allow the drive to sit fully in the carrier, then close them to secure the drive.
3. Insert the drive carrier into its bay, keeping the release button on the right. When the carrier reaches the rear of the bay, the release handle will retract.
4. Push the handle in until it clicks into its locked position

Hot-Swap for NVMe Drives

Supermicro servers support NVMe surprise hot-swap. For even better data security, NVMe orderly hot-swap is recommended. NVMe drives can be ejected and replaced remotely using IPMI.

Note: If you are using VROC, see the VROC appendix in this manual instead.

Ejecting a Drive

1. IPMI > Server Health > NVMe SSD
2. Select Device, Group and Slot, and click Eject. After ejecting, the drive Status LED indicator turns green.
3. Remove the drive.

Note that Device and Group are categorized by the CPLD design architecture.

Slot is the slot number on which the NVMe drives are mounted.

Host Identification Server: 172.031.048.114 User: ADMIN (Administrator) System Server Health Configuration Remote Control Virtual Media Maintenance Miscellaneous Help Server Health Sensor Readings Health Event Log Power Consumption Power Source NVMe SSD NVMe SSD This page displays NVMe SSD information. Locate / Stop Locate SSD Devices:0 Groucp0 Slot:9 Locate Stop Locate End CPLD / BPN ID: EF Rev: 10 Slot:0 Status Present Temperature 35 degrees C Vendor ID 80.86 Serial Number BTLF7270687N4P0IGN Model Number INTEL SSDPE2X040T7 Slot:1 Status Present Temperature 33 degrees C Vendor ID 80.86 Serial Number PHLF729690684P0IGN Model Number INTEL SSDPE2X040T7 Slot:2 Copyright © 2016 Topo Micro Computer, Inc.

Figure 2-6. IPMI Screenshot

Replacing the Drive

1. Remove the old drive and insert the replacement drive.
2. IPMI > Server Health > NVMe SSD
3. Select Device, Group and slot and click Insert. The drive Status LED indicator flashes red, then turns off. The Activity LED turns blue.

Checking the Temperature of an NVMe Drive

There are two ways to check using IPMI.

Checking a Drive

• IPMI > Server Health > NVMe SSD – Shows the temperatures of all NVMe drives, as in Figure 2-6.
• IPMI > Server Health > Sensor Reading > NVME_SSD – Shows the single highest temperature among all the NVMe drives.

M.2 Solid State Drives

One option for the mezzanine card supports M.2 solid state drives (SSDs) of 80mm in length. There is an hole in the motherboard standoff for a plastic clasp.

Installing an M.2 SSD

Caution: Use industry-standard anti-static equipment, such as gloves or wrist strap, and follow precautions to avoid damage caused by ESD.

1. Insert the SSD into the socket on the motherboard. Then push it flat against the standoff.
2. Secure the SSD by firmly inserting the plastic plug into the standoff.

Plastic Plug M.2 SSD 1 M.2 Socket

Figure 2-7. Installing M.2 SSD

2.7 Installing the Operating System

An operating system (OS) must be installed on each blade module. Blades with Microsoft Windows OS and blades with Linux OS can both occupy and operate within the same blade enclosure. Refer to the SuperMicro web site for a complete list of supported operating systems.

There are several methods of installing an OS to the blade modules.

Installing with an External USB CD-ROM Drive

Note: Installing the OS from an external CD-ROM drive may take several hours to complete.

1. Connect an SUV cable (Serial port/USB port/Video port cable) to the KVM connector on the front of the blade module. You will then need to attach a USB hub to the USB port on this cable to provide multiple USB ports.
2. Connect the external CD-ROM drive, a USB keyboard and a mouse to the USB hub. You will also need to connect a monitor to the video connector on the SUV cable. Turn on the blade module.
3. Insert the CD containing the OS into the CD-ROM drive.
4. Follow the prompts to begin the installation.

Installing by using PXE Boot (common method)

Preboot Execution Environment (PXE) is used to boot a computer over a network. To install the OS via PXE, the following conditions must be met:

• The PXE BOOT option in BIOS must be enabled.
• A PXE server has been configured (this can be another blade in the system).
• The PXE server must be connected over a network to the blade to be booted.
• The blade has only non-partitioned, unformatted hard drives installed and no bootable devices attached to it.

Once these conditions are met, make sure the PXE server is running. Then turn on the blade on which you wish to boot or install the OS. The BIOS in the blade will look at all bootable devices and finding none, will connect to the PXE server to begin the boot/install.

Installing by using Virtual Media (Drive Redirection)

You can install the OS via Virtual Media through either the IPMIview (Java based client utility), SuperBladeTool or the Web-based Management Utility. With this method, the OS is installed from an ISO image that resides on another system/blade.

Refer to the manuals on your SuperBlade CD-ROM for further details on the Virtual Media (CD-ROM or Drive Redirection) sections of these two utility programs.

Linux Installation with Two Storage Drives

When installing Linux with two storage drives on the you may encounter a situation where one drive is recognized as HDA and the other drive is recognized as SDA. This is normal since in this case the connection for SATA HDDs is from two different controllers.

So under Native IDE mode (which is the default), your Linux OS will see one drive as HDA and the other as SDA. If the SATA controller mode operation is changed to AMD_AHCI in the BIOS, then the HDDs will appear as SDA and SDB.

2.8 Configuring and Setting up RAID

For RAID setup, see http://www.supermicro.com/support/manuals/ under RAID Installation Guides.

2.9 Driver Installation

The Supermicro website contains drivers and utilities for your system at https://www.supermicro.com/wftp/driver. Some of these must be installed, such as the chipset driver.

After accessing the website, go into the CDR_Images (in the parent directory of the above link) and locate the ISO file for your motherboard. Download this file to a USB flash drive or a DVD. (You may also use a utility to extract the ISO file if preferred.)

Another option is to go to the Supermicro website at http://www.supermicro.com/products/. Find the product page for your motherboard, and "Download the Latest Drivers and Utilities". Insert the flash drive or disk and the screenshot shown below should appear.

SUPERMICRO B12DPT-6 Motherboard Drivers & Tools (Win2019) Intel C621A Chipset B12DPT-6 SUPERMICRO Computer Inc. Intel Chipset INF files Microsoft .Net Framework 4.8 (Optional) ASPEED Graphics Driver Intel Virtual RAID on CPU 25G Mellanox Ethernet Driver SUPERMICRO SuperDoctor 5 Build driver diskettes and manuals Browse CD Auto Start Up Next Time For more information, please visit SUPERMICRO's web site.

Figure 2-8. Driver & Tool Installation Screen

Note: Click the icons showing a hand writing on paper to view the readme files for each item. Click the computer icons to the right of these items to install each item (from top to the bottom) one at a time. After installing each item, you must re-boot the system before moving on to the next item on the list. The bottom icon with a CD on it allows you to view the entire contents.

Chapter 3

Management and Maintenance

This chapter provides instructions for managing the blade unit and performing maintenance on components.

3.1 Management Software

System management may be performed with any of three software packages: IPMIview, IPMItool or a Web-based Management Utility. These are designed to provide an administrator with a comprehensive set of functions and monitored data to keep tabs on the system and perform management activities.

BMC

The motherboard provides remote access, monitoring and management through the baseboard management controller (BMC) and other management controllers distributed among different system modules. There are several BIOS settings that are related to BMC. For general documentation and information on BMC, visit our website at:

www.supermicro.com/en/solutions/management-software/bmc-resources

BMC ADMIN User Password

For security, each system is assigned a unique default BMC password for the ADMIN user. This can be found on a sticker on the chassis, as noted in Section 1.4, and a sticker on the motherboard. The sticker also displays the BMC MAC address. For more information, refer to: https://www.supermicro.com/en/support/BMC_Unique_Password.

BMC AC1F6BC PWD SUOKJ

Figure 3-1. BMC Password Label

3.2 Motherboard Battery

The motherboard uses non-volatile memory to retain system information when system power is removed. This memory is powered by a lithium battery residing on the motherboard.

Replacing the Battery

Begin by removing power from the system as described in section 3.1.

1. Push aside the small clamp that covers the edge of the battery. When the battery is released, lift it out of the holder.
2. To insert a new battery, slide one edge under the lip of the holder with the positive (+) side facing up. Then push the other side down until the clamp snaps over it.

Note: Handle used batteries carefully. Do not damage the battery in any way; a damaged battery may release hazardous materials into the environment. Do not discard a used battery in the garbage or a public landfill. Please comply with the regulations set up by your local hazardous waste management agency to dispose of your used battery properly.

LITHIUM BATTERY BATTERY HOLDER

Figure 3-2. Installing the Onboard Battery

Warning: There is a danger of explosion if the onboard battery is installed upside down (which reverses its polarities). This battery must be replaced only with the same or an equivalent type recommended by the manufacturer (CR2032).

3.3 System Cooling

Installing the Air Shrouds

Air shrouds concentrate airflow to maximize fan efficiency.

Installing the Standard Air Shrouds

- Position the air shrouds as illustrated in the figure below, sliding them over the components, and secure them with screws. The screws are shared with the motherboard.

Shrouds

Figure 3-3. Installing the Standard Air Shrouds

Checking the Server Air Flow

• Make sure there are no objects to obstruct airflow through the server.
• Do not operate the server without drive carriers in the drive bays.
• Use only Supermicro recommended server parts.

- Make sure no wires or foreign objects obstruct air flow through the chassis. Pull all excess cabling out of the airflow path or use shorter cables.

The control panel LEDs display system heat status. See “Control Panel” in Chapter 1 for details.

Overheating

There are several possible responses if the system overheats.

• Use the LEDs to determine the nature of the overheating condition.
• Confirm that the chassis covers are installed properly.
  • Make sure all fans are present and operating normally.
• Check the routing of the cables.
• Verify that the heatsinks are installed properly.

3.4 CMOS Clear

JBT1 is used to clear CMOS (see motherboard drawing, Chapter 4). Instead of pins, this jumper consists of contact pads to prevent accidentally clearing the contents of CMOS.

To Clear CMOS

1. First power down the system and unplug the power cord(s).
2. Remove the cover of the chassis to access the motherboard.
3. Remove the onboard battery from the motherboard.
4. Short the CMOS pads with a metal object such as a small screwdriver for at least four seconds.
5. Remove the screwdriver (or shorting device).
6. Replace the cover, reconnect the power cord(s) and power on the system.

Notes: Clearing CMOS will also clear all passwords.

Do not use the PW_ON connector to clear CMOS.

JBT1 contact pads

3.5 Where to Get Replacement Components

If you need replacement parts for your system, to ensure the highest level of professional service and technical support, purchase exclusively from our Supermicro Authorized Distributors/System Integrators/Resellers. A list can be found at: http://www.supermicro.com. Click the menu icon > Buy > Where to Buy link.

3.6 Returning Merchandise for Service

A receipt or copy of your invoice marked with the date of purchase is required before any warranty service will be rendered. You can obtain service by calling your vendor for a Returned Merchandise Authorization (RMA) number. When returning to the manufacturer, the RMA number should be prominently displayed on the outside of the shipping carton, and mailed prepaid or hand-carried. Shipping and handling charges will be applied for all orders that must be mailed when service is complete.

For faster service, RMA authorizations may be requested online (http://www.supermicro.com/support/rma/).

Whenever possible, repack the chassis in the original Supermicro carton, using the original packaging material. If these are no longer available, be sure to pack the chassis securely, using packaging material to surround the chassis so that it does not shift within the carton and become damaged during shipping.

This warranty only covers normal consumer use and does not cover damages incurred in shipping or from failure due to the alteration, misuse, abuse or improper maintenance of products.

During the warranty period, contact your distributor first for any product problems.

Chapter 4

Motherboard Connections

This chapter describes the jumpers, connections and LEDs on the motherboard and provides pinout definitions. Some connections might not be used in this system.

4.1 Motherboard Layout

Below is a layout of the B12DPT-6 with descriptions on the following page.

25G Midplane MB PWR VROC(JRK1) JPME1 JTPM1 BIOS I-SATA1 P1-DIMMB1 P1-DIMMA1 P1-DIMMD1 P1-DIMMC1 MP1 MPB VROC VLAN CTRL PCH CPU1 BAR CODE SUPER B12DPT-6 REV:1.00 MAC CODE P2-DIMMF1 P2-DIMME1 P2-DIMMH1 P2-DIMMG1 FAN1 JFP1 M.2 MH (Mounting Hole) HDD0 D4 LED1 HDD1JM2-1 JLED HDD2 PWR JLED B1 BT1 JBT1 MEZZ2 D7 D5 D4 CPLD0 D6 CPLD0 D4 LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM LAM

Figure 4-1. Motherboard Layout

Quick Reference

Jumper Description Default Setting

JBT1 CMOS Clear Open (Normal)

JPME1 ME Manufacturing Recovery Pins 1-2 (Normal)

Connector Description

BT1 Onboard CMOS battery
CPLD0 Complex Programmable Logic Device (CPLD) chip
FAN1 Cooling fan header reserved for liquid cooling
HDD0/HDD1 Front accessible Hard Driver Connector #0/Hard Driver Connector #1
HDD2_PWR Power connector for HDD 2
JBT1 Contacts for CMOS Clear
JFP1 Front Panel Control board with KVM & USB 2.0 support (via Cable CBL-0218L)
JLED HDD2 SATA Activity LED
JM2-1 & M.2 MH M.2 slot (PCIe 4.0 x4) and M.2 Mounting Hole (M.2 MH)
JTPM1	Trusted Platform Module/Port 80 connector
MEZZ1/MEZZ2	PCI Mezzanine Card Connector 1/Mezzanine Card Connector 2
MP1	Onboard 25G midplane
PWR1	Motherboard Power connector
I-SATA 1	Intel PCH SATA 3.0 port (with RAID 0, 1, 5, 10)
VROC (JRK1)	Intel VROC key header for NVMe RAID support

LED	Description	State: Status
DA4 HDD0 Heartbeat LED		Green: HDD0 Normal; Red: HDD0 Failure
DA5 HDD1 Heartbeat LED		Green: HDD1 Normal; Red: HDD1 Failure
DA6 HDD0 Activity LED		Blue: HDD0 Active
DA7 HDD1 Activity LED		Blue: HDD0 Active
LED1	M.2 Activity LED	Green: M.2 Normal
LEDM1 BMC Activity LED		Green: BMC Normal

Motherboard Block Diagram

graph TD
    subgraph Central Components
        A["CPU1-B1"] --> B["DDR4 2666/2933/3200"]
        C["CPU1-A1"] --> D["DDR4 2666/2933/3200"]
        E["CPU1-D1"] --> F["DDR4 2666/2933/3200"]
        G["CPU1-C1"] --> H["DDR4 2666/2933/3200"]
    end

    subgraph External Components
        I["PEIC:30 Socket ID: 0"] --> J["PCIe GEN4 X16"]
        K["PEIC:31 Socket ID: 1"] --> L["PCIe GEN4 X4"]
        M["DMI3"] --> N["PCIe GEN4 X8"]
        O["Mezzanine Connector (For NIC AOC)"] --> P["MEZZ1"]
        Q["Mezzanine Connector (For SAS AOC)"] --> R["PCle X4"]
        S["Mezzanine Connector (For SAS AOC)"] --> T["PCle X8"]
        U["Mezzanine Connector (For SAS AOC)"] --> V["PCle X4"]
        W["Mezzanine Connector (For SAS AOC)"] --> X["PCle X8"]
    end

    subgraph External Components
        Y["PCH"] --> Z["SATA/NVMe SAS (Optional)"]
        AA["BMC AST2600"] --> AB["PCIe GEN2 X1"]
        AB --> AC["USB 2.0, USB2_5, USB2_7, ESPI"]
        AD["CPLD (PFR)"] --> AE["CPLD FLASH"]
        AF["BMC Boot Flash"] --> AG["SPI"]
        AH["TPM HEADER Debug Card"] --> AI["BIOS"]
        AJ["VGA/USBx2/COM1 To KVM connector"] --> AK["JFP1"]
    end

    style Central Components fill:#f9f,stroke:#333
    style External Components fill:#ccf,stroke:#333

Figure 1-7. Motherboard Block Diagram

4.2 Power Connections

Main Power Connector

The proprietary main power header is PWR1.

Storage Drive Power Connectors

The proprietary 4-pin connector, HDD2_PWR, is connected to the AOM-SB1-SATA31 to provide power to the HDD2.

4.3 Headers and Connectors

Fan Header

A 4-pin fan header (FAN1) is located on the motherboard. This fan header is reserved for liquid cooling.

Fan HeaderPin Definitions
Pin#	Definition
1	Ground (Black)
2	+12V (Red)
3	Tachometer
4	PWM Control

TPM Header

The JTPM1 header is used to connect a Trusted Platform Module (TPM)/Port 80, which is available from Supermicro. A TPM/Port 80 connector is a security device that supports encryption and authentication in hard drives. It allows the motherboard to deny access if the TPM associated with the storage drive is not installed in the system.

Trusted Platform Module/Port 80 Header Pin Definitions
Pin# Definition Pin# Definition
1 P3V3	2 SPI_TPM_CS_N
3 PCI-E_RESET_N#	4 SPI_PCH	MISO
5	SPI_PCH_CLK#	6	Ground
7 SPI_PCH_MOSI	8 N/A
9	JTPM1_P3V3A	10	IRQ_TPM_SPIN_N

M.2 Slot

The M2-1 connector is an M.2 slot for SSD storage. It is PCIe 4.0 x4 SSD in the 2280 form factor with support of M-Key 2280.

RAID Key Header

A VROC RAID Key header is located at JRK1. It supports VMD used in creating optional advanced NVMe RAID configurations. See the VROC appendix for details.

RAID Key HeaderPin Definitions
Pin#	Definition
1	Ground
2	3.3V Standby
3	Ground
4	PCH_RAID_KEY

I-SATA 3.0 Port (for SBI-420P-1T3N)

The motherboard has one SATA 3.0 port (I-SATA1), which is supported by the Intel C621A PCH. Connect this port to the AOM-SB1-SATA31 add-on module.

HDD2 SATA Activity LED Connector (for SBI-420P-1T3N)

JLED is a 3-pin connector used to indicate the status of HDD2 SATA Activity. Connect JLED to the AOM-SB1-SATA31 add-on module to show HDD2 SATA activity.

4.4 Jumpers

Explanation of Jumpers

To modify the operation of the motherboard, jumpers are used to choose between optional settings. Jumpers create shorts between two pins to change the function associated with it. Pin 1 is identified with a square solder pad on the printed circuit board. See the motherboard layout page for jumper locations.

Note: On a two-pin jumper, "Closed" means the jumper is on both pins and "Open" indicates the jumper is either on only one pin or has been completely removed.

Connector Pins Jumper Setting 3 2 1 ● ● ■ 3 2 1

ME Recovery

JPME1 is used for ME Firmware Recovery mode, which will limit system resource for essential function use only without putting restrictions on power use. In the single operation mode, online upgrade will be available in Recovery mode.

ME RecoveryJumper Settings
Jumper Setting Definition
Pins 1-2 Normal	(Default)
Pins 2-3 ME Recovery

CMOS Clear Contacts

JBT1 is used to clear CMOS. Instead of pins, this jumper consists of contact pads. See Chapter 3 for more information.

JBT1 contact pads

4.5 LED Indicators

BMC Heartbeat LED

LEDM1 is a BMC Heartbeat indicator. It blinks green when the BMC is working properly.

M.2 Activity LED

LED1 is an M.2 Activity indicator. When it is blinking green, M.2 is active.

Chapter 5

UEFI BIOS

5.1 Introduction

This chapter describes the AMI UEFI BIOS setup utility for the B12DPT-6 and provides the instructions on navigating the setup screens. The BIOS is stored in a Flash EEPROM and can be updated.

Note: Due to periodic changes to the BIOS, some settings may have been added or deleted since this manual was published.

Starting BIOS Setup Utility

To enter the BIOS setup utility screens, press the key while the system is booting up. (There are a few cases when other keys are used, such as , , etc.)

The BIOS screens have three main frames. The large left frame displays options can be configured by the user. These are blue. When an option is selected, it is highlighted in white. Settings printed in Bold are the default values.

In the left frame, a "▶" indicates a submenu. Highlighting such an item and pressing the key opens the list of settings in that submenu.

The upper right frame displays helpful information for the user. The AMI BIOS has default informational messages built in. The manufacturer retains the option to include, omit, or change any of these informational messages.

The lower right frame lists navigational methods. The AMI BIOS setup utility uses a key-based navigation system called hot keys. Most of these hot keys can be used at any time during setup navigation. These keys include , , , , arrow keys, etc.

Some system parameters may be changed.

5.2 Main Setup

When running the BIOS setup utility, it starts with the Main screen. You can always return to it by selecting the Main tab on the top of the screen.

Aptio Setup - American Megatrends International, LLC. Main Advanced Event Logs Server Mgmt Security Boot Save & Exit System Date [Thu 10/08/2020] System Time [17:15:27] Supermicro B12DPT BIOS Version 1.0 Build Date 08/28/2020 CPLD Version F1.04.00 Memory Information Total Memory 524288 MB Memory Speed 2934 MT/s Set the Date. Use Tab to switch between Date elements. +: Select Screen ↑↓: Select Item Enter: Select +/-: Change Opt. F1: General Help F2: Previous Values F3: Optimized Defaults F4: Save & Exit ESC: Exit Ver. 2.21.1277 Copyright (C) 2020 American Megatrends International, LLC.

The Main tab page allows you to set the date and time, and it displays system information.

System Date/System Time

Use this option to change the system date and time. Highlight System Date or System Time using the arrow keys. Enter new values using the keyboard. Press the key or the arrow keys to move between fields. The date must be entered in MM/DD/YYYY format. The time is entered in HH:MM:SS format.

Note: The time is in the 24-hour format. For example, 5:30 P.M. appears as 17:30:00. The date's default value is 01/01/2016 after RTC reset.

Supermicro B12DPT-6 (Motherboard model)

BIOS Version

Build Date (of the BIOS)

CPLD (Complex Programmable Logic Device) Version

Memory Information

Total Memory (for the system)

Memory Speed

5.3 Tab Pages

Advanced

Boot Function

CPU Configuration

Chipset Configuration

SATA Configuration

sSATA Configuration

PCIe/PCI/PnP Configuration

Super IO Configuration

Serial Port Console Redirection

ACPI Settings

Trusted Computing

SIO Common Settings

T1s Auth Configuration

All Cpu Information

RAM Disk Configuration

iSCSI Configuration

IPv4 Network Configuration

VLAN Network Configuration

IPv6 Network Configuration

Event Logs

Change SMBIOS Event Log Settings

ViewSMBOIS Event Log

Server Mgmt

Serial MUX

System Event Log

BMC self test log

BMC network configuration

BMC User Settings

Security

Administrator Password

Password Check

Secure Boot

Boot

Boot Mode Select

Delete Boot Option

Delete Driver Optioni

UEFI Application Boot Priorities

UEFI NETWORK Drive BBS Priorities

Save & Exit

Save Options

Default Options

Boot Override

Appendix A

BIOS POST Codes

A.1 BIOS POST Messages

During the Power-On Self-Test (POST), the BIOS will check for problems. If a problem is found, the BIOS will activate an alarm or display a message. The following is a list of such BIOS messages. Not all options may be supported.

BIOS POST Messages
BIOS Message Description
Failure Fixed Disk	Fixed disk is not working or not configured properly. Check to see if fixed disk is attached properly. Run Setup. Find out if the fixed-disk type is correctly identified.
Stuck key Stuck key on keyboard.
Keyboard error Keyboard not working.
Keyboard Controller Failed Keyboard controller failed test. May require replacing keyboard controller.
Keyboard locked - Unlock key switch	Unlock the system to proceed.
Monitor type does not match CMOS - Run SETUP	Monitor type not correctly identified in Setup
Shadow Ram Failed at offset: nnnn	Shadow RAM failed at offset nnnn of the 64k block at which the error was detected.
System RAM Failed at offset: nnnn	System RAM failed at offset nnnn of in the 64k block at which the error was detected.
Extended RAM Failed at offset: nnnn	Extended memory not working or not configured properly at offset nnnn.
System battery is dead - Replace and run SETUP	The CMOS clock battery indicator shows the battery is dead. Replace the battery and run Setup to reconfigure the system.
System CMOS checksum bad - Default configuration used	System CMOS has been corrupted or modified incorrectly, perhaps by an application program that changes data stored in CMOS. The BIOS installed Default Setup Values. If you do not want these values, enter Setup and enter your own values. If the error persists, check the system battery or contact your dealer.
System timer error The timer test	failed. Requires repair of system board.
Real time clock error Real-Time	Clock fails BIOS hardware test. May require board repair.
Check date and time settings	BIOS found date or time out of range and reset the Real-Time Clock. May require setting legal date (1991-2099).
Previous boot incomplete - Default configuration used	Previous POST did not complete successfully. POST loads default values and offers to run Setup. If the failure was caused by incorrect values and they are not corrected, the next boot will likely fail. On systems with control of wait states, improper Setup settings can also terminate POST and cause this error on the next boot. Run Setup and verify that the waitstate configuration is correct. This error is cleared the next time the system is booted.
Memory Size found by POST differed from CMOS	Memory size found by POST differed from CMOS.
Diskette drive A error
Diskette drive B error	Drive A: or B: is present but fails the BIOS POST diskette tests. Check to see that the drive is defined with the proper diskette type in Setup and that the diskette drive is attached correctly.
Incorrect Drive A type - run SETUP	Type of floppy drive A: not correctly identified in Setup.
Incorrect Drive B type - run SETUP	Type of floppy drive B: not correctly identified in Setup.
System cache error - Cache disabled	RAM cache failed and BIOS disabled the cache. On older boards, check the cache jumpers. You may have to replace the cache. See your dealer. A disabled cache slows system performance considerably.
CPU ID: CPU socket number for	Multi-Processor error.
EISA CMOS not writeable Server	BIOS2 test error: Cannot write to EISA CMOS.
DMA Test Failed	ServerBIOS2 test error: Cannot write to extended DMA (Direct Memory Access) registers.
Software NMI Failed	ServerBIOS2 test error: Cannot generate software NMI (Non-Maskable Interrupt).
Fail-Safe Timer NMI Failed Server	BIOS2 test error: Fail-Safe Timer takes too long.
device Address Conflict Address	conflict for specified device.
Allocation Error for: device	Run ISA or EISA Configuration Utility to resolve resource conflict for the specified device.
CD ROM Drive CD ROM Drive identified.
Entering SETUP... Starting Setup program
Failing Bits: nnnn	The hex number nnnn is a map of the bits at the RAM address which failed the memory test. Each 1 (one) in the map indicates a failed bit. See errors 230, 231, or 232 above for offset address of the failure in System, Extended, or Shadow memory.
Fixed Disk n Fixed disk n (0-3) identified.
Invalid System Configuration Data	Problem with NVRAM (CMOS) data.
I/O device IRQ conflict I/O device	IRQ conflict error.
PS/2 Mouse Boot Summary Screen:	PS/2 Mouse installed.
nnnn kB Extended RAM Passed	Where nnnn is the amount of RAM in kilobytes successfully tested.
nnnn Cache SRAM Passed Where nnnn is the amount of system cache in kilobytes successfully tested.
nnnn kB Shadow RAM Passed Where nnnn is the amount of shadow RAM in kilobytes successfully tested.
nnnn kB System RAM Passed Where nnnn is the amount of system RAM in kilobytes successfully tested.
One or more I2O Block Storage Devices were excluded from the Setup Boot Menu	There was not enough room in the IPL table to display all installed I2O block-storage devices.
Operating system not found	Operating system cannot be located on either drive A: or drive C:. Enter Setup and see if fixed disk and drive A: are properly identified.
Parity Check 1 nnnn	Parity error found in the system bus. BIOS attempts to locate the address and display it on the screen. If it cannot locate the address, it displays ???? . Parity is a method for checking errors in binary data. A parity error indicates that some data has been corrupted.
Parity Check 2 nnnn	Parity error found in the I/O bus. BIOS attempts to locate the address and display it on the screen. If it cannot locate the address, it displays ???? .
Pressto resume,to Setup,for previous	Displayed after any recoverable error message. Pressto start the boot process orter Setup and change the settings. Pressto display the previous screen (usually an initialization error of an Option ROM, i.e., an add-on card). Write down and follow the information shown on the screen.
Pressto enter Setup Optional message displayed during POST. Can be turned off in Setup.
PS/2 Mouse: PS/2 mouse identified.
Run the I2O Configuration Utility	One or more unclaimed block storage devices have the Configuration Request bit set in the LCT. Run an I2O Configuration Utility (e.g. the SAC utility).
System BIOS shadowed System	BIOS copied to shadow RAM.
UMB upper limit segment address: nnnn	Displays the address nnnn of the upper limit of Upper Memory Blocks, indicating released segments of the BIOS which can be reclaimed by a virtual memory manager.
Video BIOS shadowed Video BIOS	successfully copied to shadow RAM.

A.2 BIOS POST Codes

This section lists the POST (Power-On Self-Test) codes for the AMI BIOS. POST codes are divided into two categories: recoverable and terminal.

Recoverable POST Errors

When a recoverable type of error occurs during POST, the BIOS will display an POST code that describes the problem. BIOS may also issue one of the following beep codes:

One long and two short beeps – video configuration error

One repetitive long beep – no memory detected

Terminal POST Errors

If a terminal type of error occurs, BIOS will shut down the system. Before doing so, BIOS will write the error to port 80h, attempt to initialize video and write the error in the top left corner of the screen.

The following is a list of codes that may be written to port 80h.

Terminal POST Errors
Code Description
02h Verify Real Mode
03h Disable Non-Maskable Interrupt (NMI)
04h Get CPU type
06h Initialize system hardware
07h Disable shadow and execute code from the ROM.
08h Initialize chipset with initial POST values
09h Set IN POST flag
0Ah Initialize CPU registers
0Bh Enable CPU cache
0Ch Initialize caches to initial POST values
0Eh Initialize I/O component
0Fh Initialize the local bus IDE
10h Initialize Power Management
11h Load alternate registers with initial POST values
12h Restore CPU control word during warm boot
13h Initialize PCI Bus Mastering devices
14h Initialize keyboard controller
16h 1-2-2-3 BIOS ROM checksum
17h Initialize cache before memory Auto size
18h 8254 timer initialization
1Ah 8237 DMA controller initialization
1Ch Reset Programmable Interrupt Controller
20h 1-3-1-1 Test DRAM refresh
22h 1-3-1-3 Test 8742 Keyboard Controller
28h Auto size DRAM
29h Initialize POST Memory Manager
2Ah Clear 512 kB base RAM
2Ch 1-3-4-1 RAM failure on address line xxxx*
2Eh 1-3-4-3 RAM failure on data bits xxxx* of low byte of memory bus
2Fh Enable cache before system BIOS shadow
32h Test CPU bus-clock frequency
33h Initialize Phoenix Dispatch Manager
36h Warm start shut down
38h Shadow system BIOS ROM
3Ah Auto size cache
3Ch Advanced configuration of chipset registers
3Dh Load alternate registers with CMOS values
41h Initialize extended memory for RomPilot
42h Initialize interrupt vectors
45h POST device initialization
46h 2-1-2-3 Check ROM copyright notice
47h Initialize I20 support
48h Check video configuration against CMOS
49h Initialize PCI bus and devices
4Ah Initialize all video adapters in system
4Bh QuietBoot start (optional)
4Ch Shadow video BIOS ROM
4Eh Display BIOS copyright notice
4Fh Initialize MultiBoot
50h Display CPU type and speed
51h Initialize EISA board
52h Test keyboard
54h Set key click if enabled
55h Enable USB devices
58h 2-2-3-1 Test for unexpected interrupts
59h Initialize POST display service
5Ah Display prompt "Press F2 to enter SETUP"
5Bh Disable CPU cache
5Ch Test RAM between 512 and 640 kB
60h Test extended memory
62h Test extended memory address lines
64h Jump to UserPatch1
66h Configure advanced cache registers
67h Initialize Multi Processor APIC
68h Enable external and CPU caches
69h Setup System Management Mode (SMM) area
6Ah Display external L2 cache size
6Bh Load custom defaults (optional)
6Ch Display shadow-area message
6Eh Display possible high address for UMB recovery
70h Display error messages
72h Check for configuration errors
76h Check for keyboard errors
7Ch Set up hardware interrupt vectors
7Dh Initialize Intelligent System Monitoring
7Eh Initialize coprocessor if present
80h Disable onboard Super I/O ports and IRQs
81h Late POST device initialization
82h Detect and install external RS232 ports
83h Configure non-MCD IDE controllers
84h Detect and install external parallel ports
85h Initialize PC-compatible PnP ISA devices
86h Re-initialize onboard I/O ports.
87h Configure Motherboard Configurable Devices (optional)
88h Initialize BIOS Data Area
89h Enable Non-Maskable Interrupts (NMIs)
8Ah Initialize Extended BIOS Data Area
8Bh Test and initialize PS/2 mouse
8Ch Initialize floppy controller
8Fh Determine number of ATA drives (optional)
90h Initialize hard-disk controllers
91h Initialize local-bus hard-disk controllers
92h Jump to UserPatch2
93h Build MPTABLE for multi-processor boards
95h Install CD ROM for boot
96h Clear huge ES segment register
98h 1-2 Search for option ROMs. One long, two short beeps on check-sum failure
99h Check for SMART Drive (optional)
9Ah Shadow option ROMs
9Ch Set up Power Management
9Dh Initialize security engine (optional)
9Eh Enable hardware interrupts
9Fh Determine number of ATA and SCSI drives
A0h Set time of day
A2h Check key lock
A4h Initialize typematic rate
A8h Erase F2 prompt
AAh Scan for F2 key stroke
ACh Enter SETUP
AEh Clear Boot flag
B0h Check for errors
B1h Inform RomPilot about the end of POST.
B2h POST done - prepare to boot operating system
B4h 1 One short beep before boot
B5h Terminate QuietBoot (optional)
B6h Check password (optional)
B7h Initialize ACPI BIOS
B9h Prepare Boot
BAh Initialize SMBIOS
BBh Initialize PnP Option ROMs
BCh Clear parity checkers
BDh Display MultiBoot menu
BEh Clear screen (optional)
BFh Check virus and backup reminders
C0h Try to boot with INT 19
C1h Initialize POST Error Manager (PEM)
C2h Initialize error logging
C3h Initialize error display function
C4h Initialize system error handler
C5h PnPnd dual CMOS (optional)
C6h Initialize note dock (optional)
C7h Initialize note dock late
C8h Force check (optional)
C9h Extended checksum (optional)
CAh Redirect Int 15h to enable remote keyboard
CBh Redirect Int 13h to Memory Technologies Devices such as ROM, RAM, PCMCIA, and serial disk
CCh Redirect Int 10h to enable remote serial video
CDh Re-map I/O and memory for PCMCIA
CEh Initialize digitizer and display message
D2h Unknown interrupt

The following are for the boot block in Flash ROM:

Boot Block Flash ROM Terminal POST Errors

Code Description

E0h Initialize the chipset

E1h Initialize the bridge

E2h Initialize the CPU

E3h Initialize system timer

E4h Initialize system I/O

E5h Check force recovery boot

E6h Checksum BIOS ROM

E7h Go to BIOS

E8h Set Huge Segment

E9h Initialize Multi Processor

EAh Initialize OEM special code

EBh Initialize PIC and DMA

ECH Initialize Memory type

EDh Initialize Memory size

EEh Shadow Boot Block

EFh System memory test

F0h Initialize interrupt vectors

F1h Initialize Run Time Clock

F2h Initialize video

F3h Initialize System Management Manager

F4h Output one beep

F5h Clear Huge Segment

F6h Boot to Mini DOS

F7h Boot to Full DOS

If the BIOS detects error 2C, 2E, or 30 (base 512K RAM error), it displays an additional word-bitmap (xxxx) indicating the address line or bits that failed. For example, "2C 0002" means address line 1 (bit one set) has failed. "2E 1020" means data bits 12 and 5 (bits 12 and 5 set) have failed in the lower 16 bits. The BIOS also sends the bitmap to the port-80 LED display. It first displays the checkpoint code, followed by a delay, the high-order byte, another delay, and then the low order byte of the error. It repeats this sequence continuously.

Appendix B

Standardized Warning Statements for AC Systems

About Standardized Warning Statements

The following statements are industry standard warnings, provided to warn the user of situations which have the potential for bodily injury. Should you have questions or experience difficulty, contact Supermicro's Technical Support department for assistance. Only certified technicians should attempt to install or configure components.

Read this appendix in its entirety before installing or configuring components in the Supermicro chassis.

These warnings may also be found on our website at http://www.supermicro.com/about/policies/safety_information.cfm.

Warning Definition

Warning! This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents.

警告の定義

この警告サインは危険を意味します。

Installation Instructions

Warning! Read the installation instructions before connecting the system to the power source.

設置手順書

Warning! This product relies on the building's installation for short-circuit (overcurrent) protection. Ensure that the protective device is rated not greater than: 250 V, 20 A.

サーキット・ブレーカー

Power Disconnection Warning

Warning! The system must be disconnected from all sources of power and the power cord removed from the power supply module(s) before accessing the chassis interior to install or remove system components.

CAUTION: This unit has redundant power sources. Please disconnect all the power cords before servicing.

電源切断の警告

Equipment Installation

Warning! Only trained and qualified personnel should be allowed to install, replace, or service this equipment.

機器の設置

Warning! This unit is intended for installation in restricted access areas. A restricted access area can be accessed only through the use of a special tool, lock and key, or other means of security. (This warning does not apply to workstations).

アクセス制限区域

Warning! There is the danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer's instructions

電池の取り扱い

Redundant Power Supplies

Warning! This unit might have more than one power supply connection. All connections must be removed to de-energize the unit.

冗長電源装置

Warning! Hazardous voltage or energy is present on the backplane when the system is operating. Use caution when servicing.

バックプレーンの電圧

Comply with Local and National Electrical Codes

Warning! Installation of the equipment must comply with local and national electrical codes.

地方および国の電気規格に準拠

Warning! Ultimate disposal of this product should be handled according to all national laws and regulations.

製品の廃棄

Warning! Hazardous moving parts. Keep away from moving fan blades. The fans might still be turning when you remove the fan assembly from the chassis. Keep fingers, screwdrivers, and other objects away from the openings in the fan assembly's housing.

ファン・ホットスワップの警告

Power Cable and AC Adapter

Warning! When installing the product, use the provided or designated connection cables, power cables and AC adaptors. Using any other cables and adaptors could cause a malfunction or a fire. Electrical Appliance and Material Safety Law prohibits the use of UL or CSA-certified cables (that have UL/CSA shown on the cord) for any other electrical devices than products designated by Supermicro only.

電源コードとACアダプター

CPU-Based RAID for NVMe

Intel® Virtual RAID on CPU (Intel VROC) is an enterprise RAID solution for NVMe SSDs directly attached to Intel Xeon Scalable processors. Intel Volume Management Device (VMD) is an integrated controller inside the CPU PCIe root complex.

• A single processor supports up to 12 NVMe SSDs and up to 6 RAID arrays.
• A dual processor system supports up to 24 NVMe SSDs and 12 RAID arrays.

Strip sizes are 4K, 8K, 16K, 32K, 64K, 128K.

Requirements and Restrictions

• Intel VROC is only available when the system is configured for UEFI boot mode.
• To enable the mdadm command and support for RSTe, install the patch from

- Linux: https://downloadcenter.intel.com/download/28158/Intel-Virtual-RAID-on-CPU-Intel-VROC-and-Intel-Rapid-Storage-Technology-enterprise-Intel-RSTe-Driver-for-Linux-

- Windows: https://downloadcenter.intel.com/download/28108/Intel-Virtual-RAID-on-CPU-Intel-VROC-and-Intel-Rapid-Storage-Technology-enterprise-Intel-RSTe-Driver-for-Windows-

• To enable Intel VROC, a hardware key must be inserted on the motherboard, and the appropriate processor's Virtual Management Devices must be enabled in the BIOS setup.
• It is possible to enable Intel VROC without a hardware key installed, but only RAID0 will be enabled.
• Intel VROC is not compatible with secure boot. This feature must be disabled.
• When creating bootable OS RAID1 devices, you must have both devices on the same CPU, and a VMD on that CPU.
• Spanning drives when creating RAID devices is not recommended to due to performance issues, even though it is supported.

Supported SSDs and Operating Systems

To see the latest support information: https://www.intel.com/content/www/us/en/support/articles/000030310/memory-and-storage/ssd-software.html

Additional Information

Additional information is available on the product page for the Supermicro add-on card and the linked manuals.

www.supermicro.com/products/accessories/addon/AOC-VROCxxxMOD.cfm

www.supermicro.com/manuals/other/AOC-VROCxxxMOD_Windows.pdf

C.1 Hardware Key

The Intel VROC hardware key is a license key that detects the Intel VROC SKU and activates the function accordingly. The key must be plugged into the Supermicro motherboard (connector JRK1). The key options are:

Intel® VROC Keys
VROC Package Description Part Number Intel MM Number
Standard	RAID 0, 1, 10Supports 3rd party SSDs	AOC-VROCSTNMOD 95	1605
Premium	RAID 0, 1, 5, 10Supports 3rd party SSDs	AOC-VROCPREMOD 95	1606
Intel SSD only	RAID 0, 1, 5, 10Supports Intel SSDs only	AOC-VROCINTMOD 95	822

VROC_HW_KEY PIN 1 GND KEY1 GND KEY2

Figure C-1. Intel® VROC RAID Key and Motherboard Connector JRK1

C.2 Enabling NVMe RAID

RAID for NVMe SSDs must be enabled through the UEFI BIOS.

1. Install the patch as described in the Restrictions and Requirements section on a previous page.
2. Reboot the server and press [DEL] key to enter BIOS.
3. Switch to Advanced > Chipset Configuration > North Bridge > IIO Configuration > Intel® VMD Technology > Intel® VMD for Volume Management Device on Socket for CPU1 & CPU2.
4. Select each VMD on the CPU and enable VMD mode for the NVMe device.

VMD Config for PStack0 Intel® VMD for Volume Management Device [Enable] P1_NVMe0 VMD [Enable] P1_NVMe1 VMD [Enable] Hot Plug Capable [Enable] VMD Config for PStack1 Intel® VMD for Volume Management Device [Disable] VMD Config for PStack2 Intel® VMD for Volume Management Device [Disable]

Figure C-2. BIOS, Enabling VMD Mode

1. Select the desired PStack# to Enable or Disable the corresponding Intel VMD controller

Aptio Setup Utility - Copyright (C) 2017 American Megatrends, Inc. Advanced VMD Config for PStack0 Intel® VMD for Volume Management Device [Disable] VMD Config for PStack1 Intel® VMD for Volume Management Device [Disable] VMD Config for PStack2 Intel® VMD for Volume Management Device [Disable] Intel® VMD for Volume Management Device for PStack0 Disable Enable Enable/Disable Intel® Volume Management Device Technology in this Stack.

Figure C-3. BIOS, Enabling VMD for Pstack0

1. Select the desired PCIe slot to Enable or Disable Intel VMD functionality according to the current hardware configuration being used. Hot Plug Capability can also be Enabled or Disabled

Aptio Setup Utility - Copyright (C) 2017 American Megatrends, Inc. Advanced VMD Config for PStack0 Intel® VMD for Volume Management Device [Enable] CPU1 SLOT1 PCI-E 3.0 X8 VMD [Disable] Hot Plug Capable [Disable] VMD Config for PStack1 Intel® VMD for Volume Management Device [Disable] VMD Config for PStack2 Intel® VMD for Volume Manage CPU1 SLOT1 PCI-E 3.0 X8 VMD Disable Enable Enable/Disable Intel® Volume Management Device Technology on specific root port

Figure C-4. BIOS, Enabling VMD Functionality per Slot

1. Repeat steps 5-6 for each PStack# on each CPU to be enabled or disabled. In this example, we enabled CPU1 Slot1 (Figure C-5) and CPU2 Slot5 (Figure C-6) (our 4x U.2 form factor SSDs), as well as CPU1 M.2 C-1 and CPU1 M2. C-2 (our 2x M.2 form factor SSDs).

Aptio Setup Utility - Copyright (C) 2017 American Megatrends, Inc. Advanced
VMD Config for PStack0 Intel® VMD for Volume Management Device [Enable] CPU1 SLOT1 PCI-E 3.0 X8 VMD [Enable] Hot Plug Capable [Enable]	Enable/Disable Intel® Volume Management Device Technology in this Stack.
VMD Config for PStack1 Intel® VMD for Volume Management Device [Enable] CPU1 M.2 C-2 PCI-E 3.0 X4 VMD [Enable] CPU1 M.2 C-1 PCI-E 3.0 X4 VMD [Enable] CPU1 SLOT3 PCI-E 3.0 X8 VMD [Disable] Hot Plug Capable [Enable]
VMD Config for PStack2 Intel® VMD for Volume Management Device [Disable]	The Select Devices

Figure C-5. BIOS, Enabling CPU1 Example

Aptio Setup Utility - Copyright (C) 2017 American Megatrends, Inc. Advanced
VMD Config for PStack0 Intel® VMD for Volume Management Device [Disable] VMD Config for PStack1 Intel® VMD for Volume Management Device [Enable] CPU2 SLOTS PCI-E 3.0 X16 VMD [Enable] Hot Plug Capable [Enable] VMD Config for PStack2 Intel® VMD for Volume Management Device [Disable]	Enable/Disable Hot Plug for PCIe Root Ports 2A-2D

Figure C-6. BIOS, Enabling CPU2 Example

1. Press [F4] to save the configuration and reboot the system and press [DEL] to enter BIOS.
   Note: Disabling the VMD controller without first deleting the associated existing RAID volume can lead to unexpected behavior. This action is strongly not recommended.
   Note: The effects of physically changing or swapping a CPU on the VMD controller enablement has not yet been thoroughly tested or documented.
2. Switch to Advanced > Intel(R) Virtual RAID on CPU > All Intel VMD Controllers > Create RAID Volume.
3. Set Name.
4. Set RAID Level.
5. If cross-controller RAID is required, select Enable RAID spanned over VMD Controller.

AutoCAD Utility - Copyright (C) 2000 American Markets, Inc. Advanced Create RADS Volume Name: Volued RAD Level: [1] (1) Create RADS volume over VMD Controller: 1.1 Select Block DHTML DESPECKXMMRT3 (DBPFLZDSXMMXMPX1) [1] DHTML DESPECKXMMRT3 (DBPFLZDSXMMXMPX2) [1] DHTML DESPECKXMMRT3 (DBPFLZDSXMMXMPX3) [1] DHTML DESPECKXMMRT3 (DBPFLZDSXMMXMPX4) [1] Style lines: [1] (1) Capacity limit: 3075.43 Create Volume +1 Select Screen +2 Select Time Enter Select +3 Change Opt. +4 General Max +5 Previous Value +6 Detected Defaults +7 Save & Exit +8 Exit Version 2.13.2000, Copyright 107.07.2000 American Markets, Inc.

Figure C-7. Created Volume without enabling RAID spanned over VMD controller

Auto Setup utility - Copyright (C) 2018 American Regatrends, Inc. Advanced Create PID Volume Name: PID Level: Enable PID SPRING SVP AND CERTIFICATE OK Select Block INTL STOP/SPRING/INTT SPAIN/TOX/SPAIN/SPAIN INTL STOP/SPRING/INTT SPAIN/TOX/SPAIN/SPAIN INTL STOP/SPRING/INTT SPAIN/TOX/SPAIN/SPAIN INTL STOP/SPRING/INTT SPAIN/TOX/SPAIN/SPAIN OK Key Flow: Capacity (OK): [1264.00] 14153.68 ▶ Create Volume K - to Select Risk. ++: Select Reserve F1: Select Item Order: Select +/-: Change Dot F1: General Http F2: Previous Values F3: Defined Defaults F4: Dave A Exit F5: Exit Version: 2.13.2000. Copyright: ICJ 2018 American Regatrends, Inc.

Figure C-8. Created Volume with enabling RAID spanned over VMD controller

1. Select specific disks for RAID with an [X].

• RAID0: Select at least two [2 - 24] disks
• RAID1: Select only two disks
• RAID5: Select at least three [3 - 24] disks
• RAID10: Select only four disks

1. Select Strip Size (Default 64KB).
2. Select Create Volume.
3. If another RAID is needed, start again at step 9.
4. Press [F4] to save and reboot.

C.3 Status Indications

An LED indicator on the drive carrier shows the RAID status of the drive.

Drive Carrier Status LED Indicator
Status State (red)
Normal function Off
Locating 4 Hz blink
Fault Solid on
Rebuilding 1 Hz Blink

IBPI SFF 8489 Defined Status LED States

C.4 Hot Swap Drives

Intel VMD enables hot-plug and hot-unplug for NVMe SSDs, whether from Intel or other manufacturers. Under vSphere ESXi, several steps are necessary to avoid potential stability issues. See the information at link [1] below.

Hot-unplug

1. Prevent devices from being re-detected during rescan:

esxcli storage core claiming autoclaim --enabled=false

1. Unmount the VMFS volumes on the device. Check [2] for details.
2. Detach the device. Check [3] for details.
3. Physically remove the device.

Hot-plug

• Physically install the device.

ESXi will automatically discover NVMe SSDs, but a manual scan may be required in some cases.

Related Information Links

[1] https://kb.vmware.com/s/article/2151404
[2] https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.storage.doc/GUID-1B56EF97-F60E-4F21-82A7-8F2A7294604D.html
[3] https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.storage.doc/GUID-F2E75F67-740B-4406-9F0C-A2D99A698F2A.html

Appendix D

Specifications and Compliance

Processors

Intel Xeon 3rd Gen Scalable in a P4 LGA4189 type socket, with up to 38 cores and a thermal design power (TDP) of up to 270W

Chipset

Intel C621A

Memory

Sixteen slots for up to 2TB of 3DS LRDIMM/LRDIMM/3DS RDIMM/RDIMM/ DDR4 ECC memory with speeds of 3200/2933/2666MHz; DIMM size up to 128G at 1.2V

BIOS

256Mb SPI Flash EEPROM with AMI BIOS

Motherboard

B12DPT-6

Chassis

MCP-680-41001-0N; (WxHxD) 1.75 x 6.5 x 23.5 in. (44.5 x 165 x 597 mm)

Storage Drives

SBI-420P-1C2N: Two hot-swap 2.5" SAS or NVMe

SBI-420P-1T3N: Two hot-swap 2.5"SATA or NVMe, and one hot-swap 2.5" SATA

One M.2 SSD; an optional add-on module can provide four additional M.2 SSDs

LAN Connections

Two 25GbE onboard; an optional mezzanine card can provide two additional 25GbE, EDR, HDR, or OPA

Operating Environment

Operating Temperature: 10° to 35° C (50° to 95° F)

Non-operating Temperature: -40° to 70° C (-40° to 158° F)

Operating Relative Humidity: 8% to 90% (non-condensing)

Non-operating Relative Humidity: 5% to 95% (non-condensing)

Regulatory Compliance

FCC, ICES, CE, VCCI, RCM, NRTL, CB

Applied Directives, Standards

EMC/EMI: 2014/30/EU (EMC Directive)

FCC Part 15

ICES-003

VCCI 32-1

AS/NZS CISPR 32

EN55032

EN55035

CISPR 24

EN 61000-3-2

EN 61000-3-3

EN 61000-4-2

EN 61000-4-3

EN 61000-4-4

EN 61000-4-5

EN 61000-4-6

EN 61000-4-8

EN 61000-4-11

Green Environment:

2011/65/EU (RoHS Directive)

EC 1907/2006 (REACH)

2012/19/EU (WEEE Directive)

Product Safety: 2014/35/EU (LVD Directive)

UL/CSA 60950-1, 62368-1 (USA and Canada)

IEC/EN 60950-1, 62368-1

Perchlorate Warning

California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. "Perchlorate Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate"