Alpha Data ADM-PCIE-7V3 Manuale utente

ADM-PCIE-7V3

Getting Started Guide

Revision: V1.4

7/ 1/15

ADM-PCIE-7V3

Getting Started Guide

publication may be reproduced, in any shape or form, without prior written consent from Alpha

Data Parallel Systems Ltd.

Head Office

Address: 4 West Silvermills Lane,

Edinburgh, EH3 5B , UK

Telephone: +44 131 558 2600

Fax: +44 131 558 2700

email: [email protected]

website: http://www.alpha-data.com

US Office

3507 Ringsby Court Suite 105,

enver, CO 80216

(303) 954 8768

(866) 820 9956 toll free

[email protected]

http://www.alpha-data.com

All trademarks are the property of their respective owners.

ADM-PCIE-7V3
 Getting Started Guide
Table Of Contents
1  Introduction ...................................................................................................................................... 1
2  FPGA Configuration Guide ............................................................................................................. 2
  2.1      Configuration from Flash  ............................................................................................................... 2
  2.2      Configuration using  TAG .............................................................................................................. 3
  2.3      IPROG Reconfiguration from Flash ............................................................................................... 4
  2.4      OpenCL Reconfiguration ............................................................................................................... 4
  2.5      Partial Reconfiguration  .................................................................................................................. 4
3  Xilinx OpenCL  .................................................................................................................................. 5
4  Using the Alpha Data host API and Driver ..................................................................................... 6
  4.1      Installing the driver ......................................................................................................................... 6
  4.2      Building example host applications ................................................................................................ 7
  4.3      Testing host examples with reference bitstreams .......................................................................... 7
5  FPGA Firmware Reference Designs ............................................................................................. 12
  5.1      A Tutorial on building Xilinx IP ...................................................................................................... 12
    5.1.1          Building MIG (Memory Interface Generator) ............................................................................ 12
  5.2      AXI-4 Reference designs demonstrating PCIe and Memory ....................................................... 17
    5.2.1          Basic AXI-4 PCIE Reference Design ....................................................................................... 17
    5.2.2          AXI-4 Reference Design with Memory Interfaces .................................................................... 19
  5.3      SDK FPGA Designs ..................................................................................................................... 25
6  Troubleshooting Guide  ................................................................................................................. 26
  6.1      Cannot find the device ................................................................................................................. 26
  6.2      Access Denied ............................................................................................................................. 28
  6.3      Incorrect Direct Slave data returned ............................................................................................ 28
List of Tables
Table 1  SW1 Switch Functions ....................................................................................................................... 2
Table 2  Memory Map  S_AXI (39 address bits = 512G)  ............................................................................. 21
Table 3  Memory Map  MA0_AXI (39 address bits = 512G)  ........................................................................ 21
List of Figures
Figure 1  A M-PCIE-7V3 Headers and Switches ............................................................................................. 2
Figure 2  Configuration using Vivado Hardware Manager ................................................................................ 3
Figure 3  Top Level Architecture of Basic PCIE Reference  esign  ................................................................ 17
Figure 4  Top Level Architecture of SO IMM Reference  esign .................................................................... 20
Figure 5  LE s ................................................................................................................................................ 27

ADM-PCIE-7V3

Getting Started Guide

1 Introduction

Welcome to the Alpha ata ADM-PCIE-7V3 FPGA accelerator card. This guide aims to describe the board

functionality and bring a new user up to speed on much of the basic functionality of the board. This guide will

take the user through some basic tests, such as configuring the FPGA from flash and over JTAG, and accessing

FPGA resources from host software.

The A M-PCIE-7V3 is designed to provide FPGA based accelerated computing to users in data centre and high

performance computing applications. It has a high performance Gen3x8 PCIE interface for communication with

the host. It comes with two 10 Gigabit Ethernet interfaces to allowing users to process their Ethernet packets in

FPGA fabric with minimum latency. It also supports 2 SATA ports for users who require very large but locally

accessible storage.

The boards primary development language and environment is the Xilinx OpenCL development kit. This is

supplied directly by Xilinx, and full documentation for that comes with the Xilinx tools, however this guide will

explain the basic operations required to get the board to boot from a Xilinx OpenCL bitstream.

The board also supports development using traditional H L design flows. A PCIe endpoint with one to four high

performance MA engines is provided for use with the board. This endpoint is compatible with the Alpha ata

FPGA board driver and software API, to allow high performance data transfer between host and FPGA

applications. This driver and API is available for both Linux and Windows.

In all cases this guide assumes that the user has upgraded their Xilinx software to the latest version, currently

2014.4

Page 1Introduction

ad-ug- 33_v1_4.pdf



ADM-PCIE-7V3

Getting Started Guide

2 FPGA Configuration Guide

2.1 Configuration from Flash

The A M-PCIE-7V3 FPGA board can be configured in a number of different ways. From power-on, the FPGA will

be configured using a bitstream stored in a specific area of flash. IP switches are used to select one of four

possible bitstreams to use. This allows the user to select between bitstreams compatible with the Xilinx OpenCL

API and the Alpha ata API, as well as allowing space for the user application bitstream and a failsafe bitstream

(in case the user application bitstream has problems.)

J1 JTAG

Header

SW1

Figure 1 : ADM-PCIE-7V3 Headers and Switches

Figure 1 shows both sides of the A M-PCIE-7V3 board with the location of the IP switches (SW1) clearly

indicated. Table 1 below indicates the switch settings available. A full description of all these options is available

in the A M-PCIE-7V3 manual.

Switch Factory

efault Function OFF State ON State

SW1-1 OFF Xilinx/A

Mode Configure from Alpha ata region Configure from Xilinx region

SW1-2 ON Flash

Lockdown Flash block Lockdown enabled Flash block Lockdown disabled

Table 1 : SW1 Switch Functions (continued on next page)

Page 2FPGA Configuration Guide

ad-ug- 33_v1_4.pdf

ADM-PCIE-7V3

Getting Started Guide

Switch Factory

efault Function OFF State ON State

SW1-3 ON Failsafe/

efault Configure from failsafe region Configure from default region

SW1-4 OFF User

Switch Pin AB30 = '1' Pin AB30 = '0'

Table 1 : SW1 Switch Functions

Power on configuration is the most reliable way to use the board, however it is also possible to reconfigure the

board when powered on using a number of different methods. Configuring using other methods will disrupt the

PCIe link and may require a soft reset of the host PC to restore this link.

Flash programming can be accomplished in a number of different ways. The recommended way is to boot the

board in A mode and use the flash program to write the bitstream to the specified flash address. Writing the

flash can also be achieved more slowly using JTAG.

2.2 Configuration using JTAG

The JTAG interface is the primary means of reconfiguration, while the board is powered on, for development and

debug purposes. Header J1, shown in Figure 1 indicates where a Xilinx Platform Cable USB II can be

connected to allow access to the boards JTAG chain, and allows configuration and debug using Xilinx Vivado

Hardware Manager. The older Xilinx Impact and Chipscope tools can also be used for this purpose.

Figure 2 : Configuration using Vivado Hardware Manager

Page 3FPGA Configuration Guide

ad-ug- 33_v1_4.pdf

ADM-PCIE-7V3

Getting Started Guide

To configure the board using JTAG, first open Vivado 2014.3, and click on the "Open Hardware Manager" icon.

Select "Open new hardware target" to run the Wizard and detect the Xilinx USB cable. If successful you

should see the xc7vx690t device shown as in Figure 2. Select and right click on it to bring up a menu from which

you should select "Program Device...". Selecting this will allow you to select a bitstream which will then be

used to configure the device. If you have created a bitstream with debug cores, then the probes file can also be

specified here.

After configuration, if you are using PCIe, you may wish to perform a soft reset of the host computer as this will

guarantee the correct set up of the BIOS information. In some systems, if the new bitstream has identical PCIe

configuration, I and BAR sizes then it may be possible to restore PCIe connectivity and the initial BIOS setup

by restarting the driver under Windows, or by saving and restoring the relevant contents of /proc/pci/ for the

device if using Linux, but neither of these methods can be guaranteed to work in all cases.

2.3 IPROG Reconfiguration from Flash

In many situations the connection of a JTAG cable may not be practical, however configuration at times other

than at power on may be required. This is possible using IPROG reconfiguration as detailed in Xilinx UG470 - 7

Series FPGAs Configuration User Guide. In this case, a start address of a bitstream in flash can be written to

the ICAPE2 component within the FPGA, along with a sequence of commands which will result in the FPGA

reconfiguring itself from flash. The new bitstream must be written into flash before this sequence is sent, and it is

recommended that the driver is stopped before the firmware in the FPGA sends this sequence to the ICAPE2.

After reconfiguration, the re-connection of PCIe will have the same restrictions as if the FPGA were reconfigured

using JTAG and therefore a warm reset may be required.

2.4 OpenCL Reconfiguration

Users of the Xilinx OpenCL development environment have the additional option of run-time reconfiguration of

the kernels in their design. This does not fully reconfigure the FPGA and therefore does not affect the PCIe

connection. etails of this reconfiguration option are documented with the Xilinx OpenCL tool.

2.5 Partial Reconfiguration

Users who need run-time reconfiguration of the FPGA for H L designs without the previously noted PCIe

restrictions, will need to purchase an additional Xilinx licence for the dynamic partial reconfiguration flow. When

using this, it is recommended that the user instantiates the PCIe endpoint and their ICAPE2 control logic in a

fixed partition of the design. Currently no reference design showcasing this reconfiguration option is available.

Page 4FPGA Configuration Guide

ad-ug- 33_v1_4.pdf

ADM-PCIE-7V3

Getting Started Guide

3 Xilinx OpenCL

This guide only covers a few basic points on the use of the board with Xilinx OpenCL. To use the board with

Xilinx OpenCL, switch SW1-1 must be set to on to enable Xilinx mode. This will cause the board to boot up with

the Xilinx OpenCL bitstream. This will appear as a different PCIe device from the H L designed bitstreams used

in A mode. To access the card, follow the instructions provided in the Xilinx OpenCL documentation for

installing the OpenCL driver and for configuring the card to run OpenCL kernels. If the default OpenCL bitstream

has not been pre-programmed into the OpenCL area of the flash, you may need perform this step using a JTAG

cable.

Page 5Xilinx OpenCL

ad-ug- 33_v1_4.pdf

ADM-PCIE-7V3

Getting Started Guide

4 Using the Alpha Data host API and Driver

The A M-PCIE-7V3 can be used with a range of Gen3x8 PCIe endpoint backends provided by third party IP

suppliers, or the customer can even choose to instantiate their own PCIe design, however this document

assumes that the Alpha ata PCIe backend IP will be used. This core has the advantage that it includes extra

board specific built in IP for communicating with the on-board flash, Product ata ROM and system monitoring

chip, along with irect Slave and irect Master data transfer options. This core is also required for compatibility

with the Alpha ata driver and software API which is available for both Windows and Linux.

4.1 Installing the driver

The first release of the multi-platform driver to support the A M-PCIE-7V3 is version 1.4.15. The Linux driver

can be downloaded from:

ftp://ftp.alpha-data.com/pub/admxrcg3/linux/adb3-driver-linux-1.4.15.tar.gz

and the Windows driver can be downlaoded from

ftp://ftp.alpha-data.com/pub/admxrcg3/windows/adb3-driver-win-1.4.15.zip

The archive contains installation information. For linux, the driver should be installed as the root user. In the

install directory, ./configure should be run to correctly configure the build options. This should be followed by

make all install to build and install the driver.

The driver should be started using modprobe adb3. Note that if very high MA performance is required, some

additional parameters should also be set when starting the driver e.g. modprobe adb3 PciAddress64Bit=1

which enables the use of 64 bit PCI addressing which can significantly improve throughput on some systems.

Please contact

[email protected]

for a full list of recommended driver parameters to maximize

performance, if that is a priority.

Page 6Using the Alpha Data host API and Driver

ad-ug- 33_v1_4.pdf

ADM-PCIE-7V3

Getting Started Guide

4.2 Building example host applications

Host applications for the A M-PCIE-7V3 are included in Alpah ata A MXRC3 S K release 1.7.0. The Linux

version can be downloaded from :

ftp://ftp.alpha-data.com/pub/admxrcg3/linux/beta/admxrcg3sdk_apps-linux-1.7.0b2.tar.gz

and the Windows version from :

ftp://ftp.alpha-data.com/pub/admxrcg3/windows/admxrcg3-sdk-win32-1.7.0.exe

If using Linux, after installation, to build the applications, cd to the apps/linux directory, run ./configure followed

by make all. Individual applications can be rebuilt from within their own directories simply by running make.

Note that it is recommended that you build the applications as a non-root user, but in general they should be run

with root privileges.

By default the board is configured with the simple reference design bitstream. As the API cannot dynamically

reconfigure the FPGA, only some of the applications will work in this configuration. Example applications

supported in this configuration are: simple, flash, dump, info, monitor, sysmon and vpd. All these applications

except simple, will work with any bitstream built with the Alpha ata bridge IP. Other examples require the FPGA

to be configured with a specific bitstream before running.

4.3 Testing host examples with reference bitstreams

In this section the use of host example code to control the card will be demonstrated, stepping through the

commands required to start the driver and run many of the applications. In Linux commands should be run as

user root and in Windows the user must have Administrator privileges.

1. Use lspci to check if the FPGA has a valid PCIe endpoint and has appeared on the PCIe bus. This will also

show how many PCIe lanes are running and at what speed. Windows users can download a version of

lspci from eternallybored.org.

[root@jura2 ~]# lspci -vvv -d 4144:adb3

01:00.0 Bridge: Alpha Data Device adb3 (rev 01)

Subsystem: Alpha Data Device 0706

Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-

Stepping- SERR- FastB2B- DisINTx-

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <

MAbort- >SERR- <PERR- INTx-

Interrupt: pin A routed to IRQ 11

Region 0: Memory at f7c01000 (32-bit, non-prefetchable) [disabled]

[size=4K]

Region 1: Memory at f7c00000 (32-bit, non-prefetchable) [disabled]

[size=4K]

Region 2: Memory at f7800000 (64-bit, non-prefetchable) [disabled]

[size=4M]

Region 4: Memory at f0000000 (64-bit, prefetchable) [disabled] [size=4M]

...

LnkSta: Speed 8GT/s, Width x8, TrErr- Train- SlotClk- DLActive-

BWMgmt- ABWMgmt-

...

2. Start the driver

[root@jura2 ~]# modprobe adb3

3. Read back information from the board

[root@jura2 linux]# ./info/info

API information

API library version 1.4.15

Page 7Using the Alpha Data host API and Driver

ad-ug- 33_v1_4.pdf

Indice

Altri manuali Alpha Data Hardware per computer

Alpha Data

Alpha Data ADA-VPX3-7V1 Manuale utente

Alpha Data

Alpha Data ADM-VPX3-9V2 Manuale utente

Alpha Data

Alpha Data ADA-R9100 Manuale utente

Alpha Data

Alpha Data ADM-PA101 Manuale utente

Alpha Data

Alpha Data ADM-XRC-9R4 Manuale utente

Alpha Data

Alpha Data FMC-PLUS-QSFP-DD Manuale utente

Alpha Data

Alpha Data ADM-PCIE-KU3 Manuale utente

Alpha Data

Alpha Data ADM-SDEV-CFG1 Manuale utente

Alpha Data

Alpha Data ADM-XRC-7Z1 Manuale utente

Alpha Data

Alpha Data ADM-PA120 Manuale utente

Alpha Data

Alpha Data ADM-XRC-7V1 Manuale utente

Alpha Data

Alpha Data ADM-VPX3-9Z5-RTM Manuale utente

Alpha Data

Alpha Data FMC-CLINK-MINI Manuale utente

Alpha Data

Alpha Data XRM2-CLINK-R-GIGE Manuale utente

Alpha Data

Alpha Data ADM-SDEV-BASE/XCKU060 Manuale utente

Alpha Data

Alpha Data ADM-PCIE-9V5 Manuale utente

Alpha Data

Alpha Data ADM-XRC-7Z2 Manuale utente

Manuali Hardware per computer popolari di altre marche

EMC2

EMC2 VNX Series Manuale del proprietario

Panasonic

Panasonic DV0PM20105 Manuale utente

Mitsubishi Electric

Mitsubishi Electric Q81BD-J61BT11 Manuale utente

Gigabyte

Gigabyte B660M DS3H AX DDR4 Manuale utente

Raidon

Raidon iT2300 Manuale utente

National Instruments

National Instruments PXI-8186 Manuale utente

Hauppauge

Hauppauge WinTV-HVR-2250 Manuale utente

Kontron

Kontron CP6002 Manuale utente

Manhattan

Manhattan 702843 Manuale utente

Aphex

Aphex 230 Manuale utente

AMD

AMD Phenom II Manuale operativo

Intel

Intel NUC 12 Pro Manuale utente

dbx

dbx Zone Pro 1260 Manuale utente

Galaxy

Galaxy GHDX2-2430S-24F4D Manuale utente

Intel

Intel AXXRMFBU4 Manuale utente

Kontron

Kontron DIMM-PC/MD Manuale utente

ST X-NUCLEO-SAFEA1B Istruzioni operative

STEINWAY LYNGDORF

STEINWAY LYNGDORF SP-1 Manuale utente