Myricom Myrinet-2000 Manuale utente
© 2001, Myricom, Inc. -- 1 -- Revision: 27 August 2001
Guide to Myrinet-2000
Switches and Switch Networks
Myricom, Inc.
Revision: 27 August 2001
Safety, Installation, and Use Precautions
Provide power to the switch enclosures only with a 3-wire, IEC line cord from an outlet that
includes a safety ground. The input power is 100–127 V~ or 200–240 V~, 50Hz or 60Hz. All
switch enclosures are shipped with a power cord appropriate to the unit and to the shipping
destination. The power cord should be rated for 10A except for 100–127V~ power cords for the
M3-E128, which are rated for 15A.
An M3-E128 switch enclosure fully populated with line cards weighs approximately 31 kg (68
pounds). Two people are required to mount this unit in a rack. One person should not attempt to
install an M3-E128 switch unit in a rack. The rack mounting holes conform to the EIA-310
standard for 19-inch racks.
Line cards and the fan tray may be “hot swapped” (inserted or removed with the power on).
However,
•Insert and remove line cards gently, according to the instructions that start on page 4.
•Do not operate a switch for extended periods with missing line cards. Use M3-BLANK
panels to fill in any empty line-card slots. The front of the enclosure needs to be closed
both for the efficiency of the fan cooling and to avoid electromagnetic interference
(EMI).
•Each line card includes thermal sensors. Operating the switch without a fan tray for more
than one minute may result in line cards turning themselves off.
Configurations with optical-fiber Myrinet ports are Class I Laser Products. Optical-fiber
components with this classification pose no threat of biological damage.
The most recent revision of this document can be downloaded from
http://www.myri.com/myrinet/m3switch/guide/ ,
and supplemental information is linked from this same web page.
© 2001, Myricom, Inc. -- 2 -- Revision: 27 August 2001
Safety and EMC Certifications
This device complies with Part 15 of FCC Rules and all Class A
requirements for digital apparatus of the Canadian Interference
Causing Equipment Regulations. Operation is subject to the following
two conditions: (1) this device may not cause harmful interference,
and (2) this device must accept any interference received, including
interference that may cause undesired operation.
Made in U.S.A.
from domestic
and foreign
components
Myrinet M3-E32
A label similar to that above appears on the back of M3-E16, M3-E32, M3-E64, and M3-E128
switch enclosures shipped after the corresponding certifications were received.
Safety. Myricom is proud to employ TUV Rheinland of North America, Inc., to perform the
independent safety certifications for these Myrinet-2000 switch products. The M3-E16, M3-
E32, M3-E64, and M3-E128 were certified in accordance with the IEC System for Conformity
Testing and Certification of Electrical Equipment (IECEE), CB Scheme. In addition, TUV
certified these products to the US and Canadian safety standards.
Electromagnetic Compatibility (EMC). Subject to the limitations listed below, Myrinet-
2000 switch products based on the M3-E16, M3-E32, M3-E64, and M3-E128 are within Class A
limits for emission of and susceptibility to electromagnetic interference (EMI).
•An M3-BLANK filler panel must be used in any line-card slot without a line card.
•The M3-E16 may use any combination of M3-M, M3-SW16-8F, M3-SPINE-8F, M3-
SW16-8S, M3-SPINE-8S, or M3-SW12-4L line cards.
•The M3-E32, M3-E64, and M3-E128 may use any combination of M3-M, M3-SW16-8F,
or M3-SPINE-8F line cards.
EMI certification of additional configurations of line cards is in progress as of the date of this
revision. As additional EMI certifications are completed, they will be added to this document.
© 2001, Myricom, Inc. -- 3 -- Revision: 27 August 2001
Characteristics and Installation of Myrinet Cables
Although all Myrinet links carry packets in the same format at the Data Link level, Myrinet has
five different Physical level (PHY) implementations over four different types of cables. It is
important to understand the characteristics of the types of cables used at your site, and how to
install them. All Myrinet cables may be plugged or unplugged with the component power on.
Fiber is the preferred Myrinet PHY for several reasons:
•The fiber cables are small, light, flexible, and easy to install.
•The fiber cables and connectors exhibit exceptionally high reliability.
•Myrinet-Fiber cables may be up to 200m in length.
•Myrinet-Fiber components and cables operate within Class A limits for the emission of
electromagnetic interference (EMI).
Myrinet-Fiber links carry packet data at a 2+2 Gb/s data rate on industry-standard 50/125
multimode-fiber-pair cables with “LC” connectorization. Myricom ships Fiber components with
a dust plug in each port. It is recommended that the dust plug be left in place in unused switch
ports. The fiber cable is shipped with dust caps on the ends of the fibers. To install the fiber
cable, just remove the dust plug and dust caps, and insert the fiber-end connector.
Press the locking tab on the LC cable-end connector to remove the fiber cable.
Dust Plug
Press tab
to remove cable
© 2001, Myricom, Inc. -- 4 -- Revision: 27 August 2001
Serial links carry Myrinet packet data at a 2+2 Gb/s data rate on industry-standard HSSDC
(High-Speed Serial Data Connector) cables up to 10m in length. These shielded cables are
certified in several small configurations to Class A limits for the emission of EMI1. Myricom
supplies HSSDC cables from manufacturers whose cables we have qualified through extensive
testing. Not all HSSDC cables will function properly as Myrinet Serial cables.
The HSSDC cable-end connector snaps into a Serial port. Please observe the locking tab on the
cable-end connector:
In order to remove a cable, it is very important to depress the locking tab. Otherwise, the port
connector will be damaged.
SAN (System Area Network) links are carried on unshielded “microstrip” ribbon cables up to
3m in length. In fact, a single cable carries two links, designated as
the “A” and “B” links, and the same cables are used for two different
Myrinet PHYs, SAN-1280 (1.28+1.28 Gb/s) and SAN-2000 (2.0+2.0
Gb/s). As you can see from the photo to the left, SAN cable-end
connectors include a locking mechanism. These locking springs
should both be depressed when inserting and when removing a SAN
cable-end connector.
SAN links are for “in-cabinet” applications only. Switch configurations with any SAN line
cards (M3-SW16-8M or M3-SW16-4DM) exceed Class A limits for EMI, and should be used
only within a shielding enclosure. Also, it is best to restrict the use of SAN cables to within an
enclosure because they will withstand only moderate physical abuse.
LAN (Local Area Network) links are a legacy PHY of Myrinet-1280. These multi-conductor,
shielded cables are rugged, and are terminated with DB-37 connectors (similar to SCSI DB-25
connectors, but larger). LAN cables and components are certified for EMI to Class A limits.
The only installation precaution is to be sure that the locking screws are screwed down tightly.
1Tests of large configurations of Myrinet switches show that products such as the M3-E128 with all Serial ports
show very small margins relative to Class A limits in the 3–6 GHz range. These tests show that this EMI is due to
characteristics of the HSSDC connectors. Myricom is continuing engineering efforts to reduce the EMI emissions,
including using a different type of HSSDC connector in the most recent production products.
Locking
Tab
© 2001, Myricom, Inc. -- 5 -- Revision: 27 August 2001
Hot swapping line cards and the fan tray
Each Myrinet-2000 switch line card has a pair of “handles” that lock the card in place, and that
serve as a lever for inserting or removing the line card. Let’s start with a line card that is inserted
in a card slot of an enclosure. The handles should be snapped toward the middle, like this:
You’ll notice from the illuminated LEDs that this switch is powered. That’s OK, and why it’s
called “hot swapping.” To remove the line card, first press the red tabs to unlock its handles.
Then, turn both handles outward together. You should be able to see and feel the lever action as
the front panel of the line card is ejected toward you.
It is normal that the ejection is a little stiff while the line card’s front panel is in contact with the
front panel or bezels above and below. The bottom of each front panel (except for types with
© 2001, Myricom, Inc. -- 6 -- Revision: 27 August 2001
SAN ports) and the bottom of the top bezel have spring gasketing as an EMI seal. Once the
handles reach their outward extreme, the line card should slide easily out of the card cage.
The backplane power connector in the center of the line card protrudes downward below other
components on the bottom of the circuit board. Hence, just before the card is completely free,
you may need to lift the card slightly so that the power connector will clear the top of the front
panel or bezel below the line card you are removing.
Hot insertion of a line card just reverses the steps above. Be sure to start with the handles in
their out position, and guide the line card carefully into the card guides. The signal pins and
ground blades on the high-density connectors on the line card align
themselves by means of alignment/grounding posts on the backplane.
Conical depressions on the line-card connectors center themselves on
the posts to align the connectors with great precision. Nevertheless,
it is best to insert the line cards slowly and gently. When you use the
lever handles to complete the insertion of the line card, you’ll hear
the locks snap into place, and see the red tabs pop out.
When the connectors start to seat, you may see the Status LED momentarily showing yellow
before it becomes green. The yellow is a fault indication, and is due to some of the pins being
not yet connected. As the line card becomes fully inserted, the Status LED should show green
within ~0.5s. The green Status LED indicates that the line card has passed all of its self tests,
and is operating.
© 2001, Myricom, Inc. -- 7 -- Revision: 27 August 2001
The fan tray can be removed by loosening the two locking screws and pulling the fan tray out
with the handle.
When a fan try is removed, it should be replaced within approximately one minute, or else line
cards may power themselves off in response to an over-temperature condition.
© 2001, Myricom, Inc. -- 8 -- Revision: 27 August 2001
Principles of Operation
1. Introduction..................................................................................................................................9
2. The Family of Enclosures ........................................................................................................ 11
3. Other Features of the Enclosures and Line Cards .................................................................... 12
4. Port Line Cards ......................................................................................................................... 14
5. Monitoring Line Card ............................................................................................................... 17
6. Configurations up to 128 Hosts ................................................................................................ 19
7. Topology Concepts................................................................................................................... 21
8. Clos Networks .......................................................................................................................... 22
9. Clos Networks for more than 128 Hosts .................................................................................. 23
© 2001, Myricom, Inc. -- 9 -- Revision: 27 August 2001
Guide to Myrinet-2000 Switches and Switch Networks
Principles of Operation
1. Introduction
The basic building block of this family of switch products is a 16-port Myrinet crossbar switch,
which is implemented on a single chip designated as the XBar16. The XBar16 is pictured in the
block diagrams below as a circle, and the Myrinet links as lines. A Myrinet link is a full-duplex
pair of Myrinet channels.
The structure of these highly modular switches is best understood by starting with the maximal
configuration within a single enclosure, a 128-host Clos network, which includes 24 XBar16s:
Spine of the Clos Network (backplane)
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
8
hosts
Clos
“spreader”
network
Ports to up to 128 hosts (line cards)
The network pictured above provides routes from any host to any other host.
•There is a unique shortest route between hosts connected to the same XBar16.
•The eight minimal routes between hosts connected to different XBar16s traverse three
XBar16 switches.
This topology provides so many paths between hosts that the minimum bisection of this network
– its traffic-handling capacity – is as large as possible. This full-bisection and other properties of
the Clos network topology will be discussed in greater detail starting in section 7.
The upper row of 8 XBar16s is the Clos network spine, which is packaged as an active backplane
(green box) built into an enclosure that also provides power and cooling. The spine is connected
© 2001, Myricom, Inc. -- 10 -- Revision: 27 August 2001
through a Clos spreader network inside of the backplane to the lower row of 16 leaf XBar16s,
which are packaged on port line cards (red boxes). These port line cards have up to 8 Myrinet
ports through their front panel, and 8 Myrinet-SAN ports at connectors to the backplane.
Myricom supplies different port line cards in which the front-panel ports are Myrinet Fiber,
Serial, SAN (switchable between SAN-2000 and SAN-1280), or (legacy) LAN.
An M3-E128 enclosure with no line cards, allowing a view of part of the backplane
A port line card with connectors to the backplane on the back,
and connectors to external links on the front panel
The topology is a full-bisection Clos network with any combination of port line cards inserted or
omitted. For example, you could plug 10 port line cards into the M3-E128 enclosure to support
up to 80 hosts. Ten of the ports on each of the 8 spine XBar16s would be used, and the other 6
ports on each XBar16 would be unused. The 8 10-port spine switches would have exactly the
capacity required to carry traffic between 80 hosts. Even with only two line cards, the network is
a full-bisection Clos topology. The spine would be 8 2-port switches, which can carry all of the
traffic between the two line cards.
Indice
Manuali Hardware di rete popolari di altre marche
Matrix Switch Corporation
Matrix Switch Corporation MSC-HD161DEL Manuale utente
B&B Electronics
B&B Electronics ZXT9-IO-222R2 Manuale utente
Yudor
Yudor YDS-16 Manuale utente
D-Link
D-Link ShareCenter DNS-320L Manuale utente
Samsung
Samsung ES1642dc Istruzioni per l’uso
Honeywell Home
Honeywell Home LTEM-PV Istruzioni per il montaggio
