Cisco GEM318P Manuale utente

Americas Headquarters

Cisco Systems, Inc.

170 West Tasman Drive

San Jose, CA 95134-1706

USA

http://www.cisco.com

Tel: 408 526-4000

800 553-NETS (6387)

Fax: 408 527-0883

Cisco SFS InfiniBand Host Drivers

User Guide for Linux

Release 3.2.0

June 2007

Text Part Number: OL-12309-01

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL

STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT

WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT

SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE

OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH

ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT

LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF

DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING,

WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO

OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the

document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.

Cisco SFS InfiniBand Host Drivers

User Guide for Linux

CVP, the Cisco logo, and the Cisco Square Bridge logo are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn is a service mark of Cisco Systems,

nc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco

Press,

isco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, Follow Me Browsing,

ormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, LightStream, Linksys,

eetingPlace, MGX, Networking Academy, Network Registrar, Pa ck et , PIX, ProConnect, ScriptShare, SMARTnet, StackWise, The Fastest Way to Increase Your Internet

uotient, and TransPath are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

ll other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship

etween Cisco and any other company. (0705R)

iii

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

CONTENTS

Preface vii

Audience vii

Organization vii

Conventions viii

Root and Non-root Conventions in Examples ix

Related Documentation ix

Obtaining Documentation, Obtaining Support, and Security Guidelines ix

CHAPTER

1About Host Drivers 1-1

Introduction 1-1

Architecture 1-2

Supported Protocols 1-3

IPoIB 1-3

SRP 1-3

SDP 1-3

Supported APIs 1-4

MVAPICH MPI 1-4

uDAPL 1-4

Intel MPI 1-4

HP MPI 1-4

HCA Utilities and Diagnostics 1-4

CHAPTER

2Installing Host Drivers 2-1

Introduction 2-1

Contents of ISO Image 2-2

Installing Host Drivers from an ISO Image 2-2

Uninstalling Host Drivers from an ISO Image 2-3

CHAPTER

3IP over IB Protocol 3-1

Introduction 3-1

Manually Configuring IPoIB for Default IB Partition 3-2

Contents

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

Subinterfaces 3-2

Creating a Subinterface Associated with a Specific IB Partition 3-3

Removing a Subinterface Associated with a Specific IB Partition 3-4

Verifying IPoIB Functionality 3-5

IPoIB Performance 3-6

Sample Startup Configuration File 3-8

IPoIB High Availability 3-8

Merging Physical Ports 3-8

Unmerging Physical Ports 3-9

CHAPTER

4SCSI RDMA Protocol 4-1

Introduction 4-1

Configuring SRP 4-1

Configuring ITLs when Using Fibre Channel Gateway 4-2

Configuring ITLs with Element Manager while No Global Policy Restrictions Apply 4-2

Configuring ITLs with Element Manager while Global Policy Restrictions Apply 4-4

Configuring SRP Host 4-6

Verifying SRP 4-7

Verifying SRP Functionality 4-7

Verifying with Element Manager 4-8

CHAPTER

5Sockets Direct Protocol 5-1

Introduction 5-1

Configuring IPoIB Interfaces 5-1

Converting Sockets-Based Application 5-2

Explicit/Source Code Conversion Type 5-2

Automatic Conversion Type 5-2

Log Statement 5-3

Match Statement 5-3

SDP Performance 5-4

Netperf Server with IPoIB and SDP 5-6

CHAPTER

6uDAPL 6-1

Introduction 6-1

uDAPL Test Performance 6-1

uDAPL Throughput Test Performance 6-2

uDAPL Latency Test Performance 6-3

Compiling uDAPL Programs 6-4

Contents

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

CHAPTER

7MVAPICH MPI 7-1

Introduction 7-1

Initial Setup 7-2

Configuring SSH 7-2

Editing Environment Variables 7-5

Setting Environment Variables in System-Wide Startup Files 7-6

Editing Environment Variables in the Users Shell Startup Files 7-6

Editing Environment Variables Manually 7-7

MPI Bandwidth Test Performance 7-7

MPI Latency Test Performance 7-8

Intel MPI Benchmarks (IMB) Test Performance 7-9

Compiling MPI Programs 7-12

CHAPTER

8HCA Utilities and Diagnostics 8-1

Introduction 8-1

hca_self_test Utility 8-1

tvflash Utility 8-3

Viewing Card Type and Firmware Version 8-3

Upgrading Firmware 8-4

Diagnostics 8-5

APPENDIX

AAcronyms and Abbreviations A-1

NDEX

Contents

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

vii

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

Preface

This preface describes who should read the Cisco SFS InfiniBand Host Drivers User Guide for Linux,

how it is organized, and its document conventions. It includes the following sections:

•Audience, page vii

•Organization, page vii

•Conventions, page viii

•Root and Non-root Conventions in Examples, page ix

•Related Documentation, page ix

•Obtaining Documentation, Obtaining Support, and Security Guidelines, page ix

Audience

The intended audience is the administrator responsible for installing, configuring, and managing host

drivers and host card adapters. This administrator should have experience administering similar

networking or storage equipment.

Organization

This publication is organized as follows:

Chapter Title Description

Chapter 1 About Host Drivers Describes the Cisco commercial host driver.

Chapter 2 Installing Host Drivers Describes the installation of host drivers.

Chapter 3 IP over IB Protocol Describes how to configure IPoIB to run IP

traffic over an IB network.

Chapter 4 SCSI RDMA Protocol Describes how to configure SRP.

Chapter 5 Sockets Direct Protocol Describes how to configure and run SDP.

Chapter 6 uDAPL Describes how to build and configure

uDAPL.

Chapter 7 MVAPICH MPI Describes the setup and configuration

information for MVAPICH MPI.

viii

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

Preface

Conventions

This document uses the following conventions:

Chapter 8 HCA Utilities and Diagnostics Describes the fundamental HCA utilities

and diagnostics.

Appendix A Acronyms and Abbreviations Defines the acronyms and abbreviations

that are used in this publication.

Chapter Title Description

Convention Description

boldface font Commands, command options, and keywords are in

boldface. Bold text indicates Chassis Manager elements or

text that you must enter as-is.

italic font Arguments in commands for which you supply values are in

italics. Italics not used in commands indicate emphasis.

Menu1 > Menu2 >

Item…

Series indicate a pop-up menu sequence to open a form or

execute a desired function.

[ ] Elements in square brackets are optional.

{ x | y | z } Alternative keywords are grouped in braces and separated by

vertical bars. Braces can also be used to group keywords

and/or arguments; for example, {interface interface type}.

[ x | y | z ] Optional alternative keywords are grouped in brackets and

separated by vertical bars.

string A nonquoted set of characters. Do not use quotation marks

around the string or the string will include the quotation

marks.

screen font Terminal sessions and information the system displays are in

screen font.

boldface screen

font

Information you must enter is in boldface screen font.

italic screen font Arguments for which you supply values are in italic

screen font.

^ The symbol ^ represents the key labeled Control—for

example, the key combination ^D in a screen display means

hold down the Control key while you press the D key.

< > Nonprinting characters, such as passwords are in angle

brackets.

!, # An exclamation point (!) or a pound sign (#) at the beginning

of a line of code indicates a comment line.

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

Preface

Root and Non-root Conventions in Examples

Notes use the following convention:

Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the

manual.

Cautions use the following convention:

Caution Means reader be careful. In this situation, you might do something that could result in equipment

damage or loss of data.

Root and Non-root Conventions in Examples

This document uses the following conventions to signify root and non-root accounts:

Related Documentation

For additional information related to the Cisco SFS IB host drivers, see the following documents:

•Cisco InfiniBand Host Channel Adapter Hardware Installation Guide

•Release Notes for Linux Host Drivers Release 3.2.0

•Release Notes for Cisco OFED, Release 1.1

•Cisco OpenFabrics Enterprise Distribution InfiniBand Host Drivers User Guide for Linux

•Cisco SFS Product Family Element Manager User Guide

•Cisco SFS InfiniBand Fibre Channel Gateway User Guide

Obtaining Documentation, Obtaining Support, and Security

Guidelines

For information on obtaining documentation, obtaining support, providing documentation feedback,

security guidelines, and also recommended aliases and general Cisco documents, see the monthly

What’s New in Cisco Product Documentation, which also lists all new and revised Cisco technical

documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Convention Description

host1#

host2#

When this prompt appears in an example, it indicates that you

are in a root account.

host1$

host2$

When this prompt appears in an example, it indicates that you

are in a non-root account.

Cisco SFS InfiniBand Host Drivers User Guide for Linux

OL-12309-01

Preface

Obtaining Documentation, Obtaining Support, and Security Guidelines

Questo manuale è adatto per i seguenti modelli

Indice

Altri manuali Cisco Software

Cisco

Cisco 861W - Integrated Services Router Wireless Manuale utente

Cisco

Cisco C8510-CHAS5-RF - Catalyst 8510 Campus Switch Router Modular Expansion... Istruzioni per l'installazione e il funzionamento

Cisco

Cisco Catalyst Series Switch 2940 Manuale utente

Cisco

Cisco Servers Manuale utente

Cisco

Cisco CSACSE-1111-K9 - Secure Access Control Server Solution... Manuale utente

Cisco

Cisco TELEPRESENCE MANAGEMENT SUITE EXTENSION 2.2 - FOR MICROSOFT... Manuale di servizio

Cisco

Cisco TELEPRESENCE CALL DETAIL RECORDS FILE FORMAT... Manuale utente

Cisco

Cisco GO GREEN Manuale utente

Cisco

Cisco 15454-TCC - Network Processor Card Manuale utente

Cisco

Cisco Router IOS XR Manuale utente

Cisco

Cisco TMS SQL DATABASE Manuale utente

Cisco

Cisco IP Communicator Manuale di istruzioni

Cisco

Cisco Voice Interworking Services (VISM) Manuale dell'operatore

Cisco

Cisco Nexus 7000 Series Guida

Cisco

Cisco Unified Contact Center Express Manuale utente

Cisco

Cisco IPS-4240-K9 - Intrusion Protection Sys 4240 Manuale utente

Cisco

Cisco ONS 15327 Manuale utente

Cisco

Cisco MCS-7825-H3-IPC1 Manuale utente

Cisco

Cisco IP Communicator Manuale utente

Cisco

Cisco TELEPRESENCE MOVI 4.1 - ADMINISTRATOR GUIDE... Manuale di servizio

Cisco

Cisco CRS-1 - Carrier Routing System Router Manuale di istruzioni

Cisco

Cisco CiscoWorks 1105 Manuale utente

Cisco

Cisco WS-SUP32-GE-3B - Supervisor Engine 32 Manuale utente

Cisco

Cisco PVDM2 24DM - Digital Modem Module Manuale utente

Manuali Software popolari di altre marche

COMPRO

COMPRO COMPROFM Manuale utente

Muratec

Muratec OFFICEBRIDGE ONLINE Manuale utente

Oracle

Oracle Contact Center Anywhere 8.1 Manuale utente

Adobe

Adobe 65007312 - Photoshop Lightroom Manuale

Avaya

Avaya NULL One-X for RIM Blackberry Manuale utente

HP ProLiant BL420c Manuale utente

PS Audio

PS Audio PowerPlay Manuale del proprietario

Brady

Brady LOCKOUT PRO 3.0 Manuale di servizio

Avaya

Avaya Interaction Center Manuale utente

Texas Instruments

Texas Instruments TI-83 Plus Silver Edition Manuale

Novell

Novell GROUPWISE 8 - INTERNET AGENT Manuale utente

Oracle

Oracle Application 9i Manuale utente

Acer

Acer RDM Manuale utente

Canon

Canon Vixia HF21 Manuale utente

Canon

Canon ZR950 Manuale utente

Samsung

Samsung Auto Backup Manuale utente

Polycom

Polycom Vortex EF2201 Istruzioni per l'installazione e il funzionamento

Brocade Communications Systems

Brocade Communications Systems Brocade 8/12c Manuale utente