Cray ClusterStor L300 Manuale utente
ClusterStor™ L300 and L300N Field
Installation Guide
(3.0.0)
H-6163
Contents
1 About the ClusterStor L300 and L300N Field Installation Guide............................................................................5
2 Install a ClusterStor L300 and L300N System........................................................................................................7
3 Prerequisites to Installation.....................................................................................................................................9
3.1 Service Area Clearances.........................................................................................................................10
3.2 Plan for Rack Position.............................................................................................................................11
3.3 Safe Handling and Precautions Check List.............................................................................................12
4 Uncrate the ClusterStor L300 and L300N Rack...................................................................................................15
5 Move the Rack to its Final Position.......................................................................................................................17
6 Install Disk Drives in a 5U84 Enclosure................................................................................................................19
7 Install Disk Drives in a 2U24 Enclosure................................................................................................................25
8 Cable the External Power Connections................................................................................................................27
8.1 Reset the Circuit Breaker........................................................................................................................31
8.2 Reset a Circuit Breaker on a Raritan PDU..............................................................................................31
9 Standardized Rack Configurations.......................................................................................................................34
10 Cable Inter-rack Local Management Network (LMN).........................................................................................36
11 Environment Connections...................................................................................................................................38
11.1 Cable SMU and 36-Port InfiniBand EDR LDN with Dual 24-Port LMN Switches..................................38
11.2 Cable SMU and 36-Port InfiniBand EDR LDN with Dual 48-Port LMN Switches..................................40
11.3 Cable SMU and 36-Port 40GbE LDN and Dual 24-Port LMN Switches................................................41
11.4 Cable SMU and 36-Port 40GbE LDN with Dual 48-Port LMN Switches...............................................43
11.5 Cable SMU and 24-Port Omni-path LDN with Dual 24-Port LMN Switches..........................................45
11.6 Cable SMU and 24-Port Omni-path LDN Dual 48-Port LMN Switches.................................................46
11.7 Cable SMU and Omni-path 48-Port LDN with Dual 24-Port LMN Switches..........................................48
11.8 Cable SMU and Omni-path 48-Port LDN with Dual 48-Port LMN Switches..........................................50
11.9 Cable SMU and 36-Port 100GbE LDN with Dual 24-Port LMN Switches.............................................52
11.10 Cable SMU and 36-Port 100GbE LDN with Dual 48-Port LMN Switches...........................................54
12 Power on the System..........................................................................................................................................56
13 Perform the First-Run Configuration...................................................................................................................65
13.1 Create the Administrator Password.......................................................................................................65
13.2 Configure the Lustre Network Interfaces...............................................................................................68
13.3 Troubleshoot the Lustre Network Screen..............................................................................................69
13.4 Run Lustre Validation Tests...................................................................................................................71
13.4.1 Run Lustre Validation Tests -- Lustre Not Mounted..................................................................73
13.4.2 Troubleshoot the Lustre Validation Tests..................................................................................73
13.5 Monitor Progress Screen.......................................................................................................................76
Contents
2
13.6 Review the Complete Screen................................................................................................................77
13.7 Specify Lustre User Authentication.......................................................................................................78
13.7.1 Specify Lustre User Authentication via the Active Directory Screen........................................79
13.7.2 Specify Lustre User Authentication via the LDAP Screen........................................................80
13.7.3 Specify Lustre User Authentication via the NIS Screen...........................................................84
13.7.4 Specify Lustre User Authentication via the Local Screen.........................................................85
13.7.5 Troubleshoot the Lustre Users Screen.....................................................................................86
13.8 Post-Install Checkout.............................................................................................................................86
13.9 Test Client Mount of Lustre File System................................................................................................87
14 Configure SMTP Relay, RAS Email, Remote Support, and Service Console.....................................................91
15 Configure an Additional MMU to Support DNE...................................................................................................94
16 Reference...........................................................................................................................................................95
16.1 Power Off the ClusterStor L300 and L300N with CSCLI.......................................................................95
16.2 Power Off the ClusterStor L300 and L300N System via CSSM............................................................96
16.3 Internal Rack Cabling Reference...........................................................................................................97
16.3.1 ClusterStor L300 and L300N SMU Physical and Logical Port Numbering...............................99
16.3.2 ClusterStor L300 and L300N MMU Physical and Logical Port Numbering............................100
16.3.3 Cable AMMU SAS 1U Servers...............................................................................................101
16.3.4 ClusterStor L300 and L300N SSU Physical and Logical Port Numbering.............................101
16.3.5 Internal Rack Cabling - Local Management Network.............................................................102
16.3.6 Internal Rack Cabling - Local Data Network..........................................................................107
16.3.7 Cable an Optional Additonal MMU.........................................................................................115
16.3.8 Cable SSU ESU SAS Ports....................................................................................................116
16.3.9 Cable AMMU Networks..........................................................................................................118
16.4 Component Descriptions.....................................................................................................................119
16.4.1 Embedded Application Controller (EAC)................................................................................119
16.4.2 Operator Control Panel - 5U84...............................................................................................121
16.4.3 Operator Control Panel - 2U24...............................................................................................123
16.5 Power Diagrams..................................................................................................................................124
16.5.1 Base Rack Standard Power Distribution for PX2-5965X3-V2................................................124
16.5.2 Base Rack Inverted Power Distribution for PX2-5965X3-V2..................................................124
16.5.3 Storage Rack Optional MMU Standard Power Distribution for PX2-5965X3-V2....................125
16.5.4 Storage Rack Optional MMU Inverted Power Distribution for PX2-5965X3-V2.....................126
16.5.5 Base Rack Standard Power Distribution for PX2-5104X2-V2................................................127
16.5.6 Base Rack Inverted Power Distribution for PX2-5104X2-V2..................................................128
16.5.7 Storage Rack Optional MMU Standard Power Distribution for PX2-5104X2-V2....................129
16.5.8 Storage Rack Optional MMU Inverted Power Distribution for PX2-5104X2-V2.....................130
16.5.9 Base Rack Standard Power Distribution for PX2-5100X2-V2................................................131
Contents
3
16.5.10 Base Rack Inverted Power Distribution for PX2-5100X2-V2................................................132
16.5.11 Storage Rack Optional MMU Power Distribution for PX2-5100X2-V2.................................133
16.5.12 Storage Rack Optional MMU Inverted Power Distribution for PX2-5100X2-V2...................134
16.5.13 Base Rack Standard Power Distribution for PX2-5551-N4V2..............................................135
16.5.14 Base Rack Inverted Power Distribution for PX2-5551-N4V2...............................................136
16.5.15 Storage Rack Optional MMU Standard Power Distribution for PX2-5551-N4V2.................137
16.5.16 Storage Rack Optional MMU Inverted Power Distribution for PX2-5551-N4V2...................138
16.5.17 Base Rack Optional AMMU Standard Power Distribution for PX2-5965X3-V2....................139
16.5.18 Base Rack Optional AMMU Standard Power Distribution for PX2-5104X2-V2....................140
16.5.19 Base Rack Optional AMMU Standard Power Distribution for PX2-5100X2-V2....................140
16.5.20 Base Rack Optional AMMU Standard Power Distribution for PX2-5551-N4V2...................141
16.5.21 Base Rack Optional AMMU Inverted Power Distribution for PX2-5965X3-V2.....................142
16.5.22 Base Rack Optional AMMU Inverted Power Distribution for PX2-5104X2-V2.....................143
16.5.23 Base Rack Optional AMMU Inverted Power Distribution for PX2-5100X2-V2.....................144
16.5.24 Base Rack Optional AMMU Inverted Power Distribution for PX2-5551-N4V2.....................144
Contents
4
1About the ClusterStor L300 and L300N Field
Installation Guide
The ClusterStor™ L300 and L300N Field Installation Guide H-6163 includes instructions for setting up,
configuring, and testing a ClusterStor L300 or L300N storage system.
Release 3.0.0 Information
This version includes procedures to support Cray software release 3.0.0.
Table 1. Record of Revision
Publication Title Date Updates
ClusterStor L300 and L300N Field Installation Guide
(3.0.0) H-6163
January 2018 Initial release of 3.0.0
version.
Related Publications
Additional documentation for ClusterStor L300 and L300N systems is available from pubs.cray.com.
●ClusterStor L300/L300N Cabling Guide
●Lustre® Client Software Build and Installation Guide for Cray® Cluster Connect™ S-2550
●XC™ Series Lustre® Administration Guide S-2648
Scope and Audience
This publication is written for trained Cray technicians and provides instructions to unpack and install a
ClusterStor L300 and L300N system at the customer site. It does not include information about daily operation.
Typographic Conventions
Monospace Indicates program code, reserved words, library functions, command-line prompts,
screen output, file/path names, and other software constructs.
Monospaced Bold Indicates commands that must be entered on a command line or in response to an
interactive prompt.
Oblique or Italics Indicates user-supplied values in commands or syntax definitions.
Proportional Bold Indicates a GUI Window, GUI element, cascading menu (Ctrl→Alt→Delete), or
key strokes (press Enter).
\ (backslash) At the end of a command line, indicates the Linux® shell line continuation character
(lines joined by a backslash are parsed as a single line).
About the ClusterStor L300 and L300N Field Installation Guide
5
Trademarks
The following are trademarks of Cray Inc. and are registered in the United States and other countries: CRAY and
design, SONEXION, URIKA, and YARCDATA. The following are trademarks of Cray Inc.: APPRENTICE2,
CHAPEL, CLUSTER CONNECT, ClusterStor, CRAYDOC, CRAYPAT, CRAYPORT, DATAWARP, ECOPHLEX,
LIBSCI, NODEKARE. The following system family marks, and associated model number marks, are trademarks
of Cray Inc.: CS, CX, XC, XE, XK, XMT, and XT. The registered trademark LINUX is used pursuant to a
sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a worldwide basis. Other
trademarks used in this document are the property of their respective owners.
About the ClusterStor L300 and L300N Field Installation Guide
6
2Install a ClusterStor L300 and L300N System
ClusterStor L300 and L300N systems are scale-out Lustre® storage solutions that support InfiniBand™ (EDR,
FDR), 40/100 GbE, and Intel Omni-Path (OPA) as well as a broad number of popular of Linux distributions
through the Lustre Client by Cray.
Procedures include unloading and unpacking the system at the customer site, cabling the system into the
installation environment, powering on the system, running the CSM configuration wizard, and running connectivity
and performance tests.
Additionally, this document contains a section on common installation problems (including workarounds) and
troubleshooting information.
The installation process consists of the following procedures:
1. Complete the pre-installation checklist
2. Unload the shipping crates
3. Uncrate the cabinet(s)
4. Move each rack into position
5. Install the disk drives
6. Cable the system
7. Power on the system
8. Perform the first-run configuration
9. Perform the advanced MMU (AMMU) installation procedure, If the system being installed includes an AMMU.
The figure shows an each of the ClusterStor L300 and L300N networks:
● Lustre client network (LCN) - Customer Lustre client network:
○ InfiniBand EDR
○ 40 GbE
● External administration network (EAN) - Customer administration network.
● Local management network (LMN) – Dual-bonded GigE local management private network
● Local data network (LDN) - Dual fault-tolerant local data network with external uplinks. EAC 0 in each SMU,
MMU, and SSU is connected to LDN 0 switch, and EAC 1 is connected to LDN 1 switch.
Install a ClusterStor L300 and L300N System
7
Figure 1. ClusterStor L300 and L300N Networks Overview
SMU
Base MMU
SSU
OSS OSS
LMN LDN
LCN
RAS Service Ports
Base Rack
LMN To
Storage
Rack(s)
(x500)
*AD/LDAP/NIS server must be same on all networks
External administration network (EAN)
Local data network (LDN)
EAN
MDS/MGSMDS/MGS
MGMT 0 MGMT 1
(x1)
(x1)
(x1)
Lustre Client
Network (LCN)
Local management network (LMN)
Lustre client network (LCN)
unspecified network connection
Customer client node
Customer server node
Server node
Chassis
AMMU
MDS/MGSMDS/MGS
EAC 1
EAC 0
EAC 0 EAC 1
EAC 0 EAC 1
EAC 0 EAC 1
LMN Switch 0
LMN Switch 1
LDN Switch 1
LDN Switch 0
PDU
A
PDU
B
Lustre Clients
(x 1000s) Lustre Client Network
Core Switch
AD/LDAP/NIS server*
CSM Administrator
Station
AD/LDAP/NIS server*
NTP server
DNS server
SMTP server
Install a ClusterStor L300 and L300N System
8
3Prerequisites to Installation
Verify that these prerequisites have been satisfied before installing the ClusterStor L300 and L300N system.
Acclimate the System Cabinets
When the system is unloaded from the climate controlled truck, the shipping environment
temperature will be close to the computer room. If the system has been stored in an
environment that is significantly colder than the environment in which it will be installed (15
°F [8 °C] or greater disparity), leave the system cabinets in their shipping configuration for at
least 24 hours at room temperature. This prevents damage to the equipment from thermal
shock and condensation.
Inspect the System
Cray has provided the pre-configured rack: built, configured, and tested to the valid
configuration ordered.
Forklift Size Requirements
● 10 ton (9,072 kg) minimum lifting capacity
● Forklift with tine side travel is strongly recommended
Rack Weight and Dimensions
If the system cabinets do not fit through all access doors, you may need to uncrate the
system cabinets as described in Uncrate the ClusterStor L300 and L300N Rack on page
15 in a secure (dry) staging area. Check the flooring on the entire path from the truck or
staging area to the computer room. Be sure the flooring can:
● Support the weight of the system cabinet
● Allow the cabinet to roll, or if aluminum plates are required
● Will not be damaged during the installation
Table 2. Rack Weight and Dimensions
Height: 78.40in (1991 mm)
Width: 23.60in (600 mm)
Depth: 47.25in (1200mm)
Weight: Maximum 2513 lbs. (1140 kg)
Weight: Shipping 1720 lbs. (780 kg)
Aluminum Plates
Aluminum plates may be required to roll the system across uneven flooring, thresholds,
carpeting, or other obstacles. Angle the cabinet when rolling it over the edge of the
aluminum plate so that only 1 wheel makes contact with the plate.
Prerequisites to Installation
9
Infrastructure is Present
Access to corporate switch (Intranet)
KVM or monitor/keyboard to set up ethernet port
Browser access (from a separate system with a 1024 x 768 monitor)
Cables at lengths sufficient to connect the system to the user environment
● Ethernet
● InfiniBand, Omni-Path (OPA), 40GbE, or 100GbE (depending on the configuration)
Tools
13mm socket wrench
T10 and T20 Torx screwdrivers
Pozidriv screwdriver
61mm/24 inch spirit level (optional)
Work gloves, safety shoes
Personnel
2 people are required for the unpacking process
2 people (minimum) are required to move the rack from the pallet and into its final position
1 person is required to configure and provision the system software
System Updates
SU10
System Access Requirements
Most ClusterStor L300 and L300N software procedures and CSCLI commands require
access to the administrative (admin) user account. However, root access is required to
perform the system installation procedure. Contact Cray Support for root access.
3.1 Service Area Clearances
Before unpacking the ClusterStor L300 and L300N system crate and beginning the installation, verify that
recommended service area clearances are available and that personnel are aware of the safe handling
procedures and precautions.
The top of the rack has cutout holes for cables to be passed into the rack. If cables are to be passed into the rack
from underneath the floor, the recommended cutout is 152.4 mm x 152.4 mm (6 inches x 6 inches) in the rear of
the tile on which the rack is positioned.
Verify that the recommended service area clearances are available at the installation location as shown:
● 45.3in (1150 mm) from the front of the rack
● 25.6 inches (650 mm) from the rear of the rack
● 16.5 inches (420 mm) and 23.6 inches (600 mm) on the side of the rack
Prerequisites to Installation
10
Questo manuale è adatto per i seguenti modelli
1
Indice
