ECE 492 Manuale utente

ECE 492 Capstone Project

2013W

Virtual Guitar Gloves

Summary: Play a virtual guitar in the air by using two gloves fitted with tactile sensors

and an accelerometer.

Group Members:

Elysia Jong ejong@ualberta.ca

Qingyue Zhou qingyue@ualberta.ca

Abstract

Terasic’s V.1.6 DE2_SD_Card_Audio demonstration was used as a base for this

project. By default, it waited for an SD card to be inserted into the FPGA’s port, then

played all the files within it in sequence through audio out. The files are assumed to be

of .wav format with a 48 kHz sampling rate. It also displayed on the 7-segment and LEDs.

The project had no documentation and the files were written in Verilog.

This demo project was modified to only play one file at a time, depending on

which tactile sensor was pressed and only if the accelerometer detected a strum action

from the user. The LCD was programmed to display whether the system was in note or

chord mode and which octave scale was selected. The tactile sensors, in essence, were

simple on/off switches with hardware debouncing implemented. The digital

accelerometer communicates through the onboard GPIO pins to a second I2C bus,

separated from the I2C bus used for the audio CODEC to avoid any possibility of conflict.

All sounds played are pre-recorded files stored in the SD card. External components are

all wired to the Altera DE2 via the GPIO pins.

Table of Content

FUNCTIONAL REQUIREMENT............................................................................................................3

SPECIFICATION:.........................................................................................................................................3

HOW TO USE:.............................................................................................................................................3

GOALS ACHIEVED: ....................................................................................................................................3

DESIGN AND DESCRIPTION OF OPERATION.................................................................................4

FLOW CHART:............................................................................................................................................5

INPUT TO OUTPUT FUNCTIONALITY:..........................................................................................................5

ALGORITHMS/TASKS:................................................................................................................................5

BILL OF MATERIALS .............................................................................................................................6

REUSABLE DESIGN UNITS ...................................................................................................................7

DATASHEET..............................................................................................................................................8

USER PERSPECTIVE DIAGRAM ..................................................................................................................8

USER I/O SIGNALS: ...................................................................................................................................8

RIBBON HEADER MAPPING.....................................................................................................................10

ACCELEROMETER INTERFACE:................................................................................................................11

SD CARD INTERFACE:.............................................................................................................................11

EXTERNAL CIRCUITRY SCHEMATIC:.......................................................................................................12

POWER:....................................................................................................................................................12

BACKGROUND READING ...................................................................................................................12

SOFTWARE DESIGN .............................................................................................................................15

BLOCK DIAGRAM:..............................................................................................................................16

LIST OF LIBRARIES:.................................................................................................................................17

TEST PLAN ..............................................................................................................................................18

SOFTWARE: .............................................................................................................................................18

HARDWARE:............................................................................................................................................18

RESULTS OF EXPERIMENTS AND CHARACTERIZATION........................................................19

STRUM DETECTION:................................................................................................................................19

CPU CLOCK/PERFORMANCE:..................................................................................................................19

REFERENCES..........................................................................................................................................20

APPENDICES...........................................................................................................................................22

QUICK START MANUAL ..........................................................................................................................22

Hardware:...........................................................................................................................................22

Overview:............................................................................................................................................22

Steps to start from flashed board:.......................................................................................................22

Software:.............................................................................................................................................23

How to play:........................................................................................................................................23

FUTURE WORK......................................................................................................................................23

BACKUP PLANS/OPTIONS:.......................................................................................................................23

EXTENSIONS:...........................................................................................................................................24

HARDWARE DOCUMENTATION.......................................................................................................25

BLOCK DIAGRAM OF INTERACTIONS BETWEEN PROCESSES:...................................................................26

INDEX TO SOURCE CODE: ........................................................................................................................26

Functional Requirement

Specification:

Our virtual guitar consists of 2 gloves. There are 7 tactile sensors on the left hand, which

are used for 7 notes (A to G). There are 4 sensors on the right: 2 used to go up/down octaves, 1 to

switch from playing notes to chords or vice versa, and a reset to the original mode (notes with

default octave). There is also one accelerometer on the right hand to detect a strum from the

player. By combining the glove’s tap sensor and accelerometer signals, we can realize a guitar

playing without an actual guitar.

The player is assumed to be right-handed. Once the player is touching their left thumb to

a left sensor and a strum action from the right hand is detected, the speakers will then play the

certain note/chord the player intended to play. This is meant to reflect the playing of an actual

guitar, where the player would hold the fretboard with their left hand and play the strings with

their right. If the player waves his/her right hand, but no tap sensor is engaged, nothing will

happen.

The 2 gloves have a wired connection to the Altera DE2 FPGA board through the GPIO

pins. Speakers are connected to Line Out for audio output. The guitar sounds (A-G, notes/chords)

are saved in an SD card. Filtering is to be applied to the sound files to obtain the different

octaves.

How to use:

1Playing a note:

auser must press and hold their left thumb to 1 left finger note (“A” - “G”)

ithis is to mimic holding the fretboard on a real guitar

buser moves right hand downward in a strumming action

csimultaneous combination of both a. and b. required

2Changing between single notes and chords:

auser needs to press right thumb to “MS” and release (no hold)

3Changing octaves:

auser must press their right thumb (no hold) to either:

i↓: octave down

ii ↑: octave up

bplay a note normally as in 1.

4Resetting to default octave and note mode:

auser needs to press right thumb to “R” and release (no hold)

Goals achieved:

The tactile sensors, combined with hardware debouncing, had no issues. The left hand

sensors correspond to the correct notes/chords and the right hand’s mode switch (for selecting

notes/chords), octave up, octave down, and reset buttons were functional. The LCD also

correctly displayed whether the user was in note or chord mode and which octave. The threshold

on the accelerometer was set so that it would not recognize motion if the user moved their hand

too slowly. Due to the significant amount of time it took to interface the accelerometer, the

filtering used to achieve the octaves was not done. The original sound files were run through an

audio editor called Audacity 2.0.3, where the frequencies were changed, new sound files

generated and then stored in the SD card. This meant there were 42 files instead of the intended

14 files for guitar sounds.

The loading time required for the SD card is not noticeable. When running from flash

memory, powering up the FPGA and instantly attempting to play a sound will work without any

apparent delay. Originally, there was noise in the output sound. The saved .wav files are from an

acoustic guitar, but the output from the FPGA sounded synthesized and in a minor scale. This

was corrected by changing the Nios II processor clock from 50 MHz to 100 MHz.

Design and Description of Operation

There are 11 tap sensors for the 7 notes (A-G) and 4 modes (R, MS, Octave ↑, Octave ↓)

placed as shown in the image below. Each sensor is located on a finger joint section, so that the

player may bend their fingers. All unused finger joints may be used for future expansion. The

accelerometer will be attached on the back of the right hand near the wrist.

Image 1: Placement of sensors modified from http://media.npr.org/assets/news/2010/01/27/hands-

ef12ac2473025791c19e4478b5ca0078c2fa1942-s6-c10.jpg

Flow chart:

Image 2: Software design flow

Input to output functionality:

1Tap sensor and accelerometer signals detected from GPIO pins

2Based on input, read sound file from SD card

3Output sound file to speakers connected to FPGA

Algorithms/Tasks:

“task_which_note” simply polls for user input on the left glove through an infinite while

loop. It keeps track of the BASE_ADDRESS of the tactile sensors using IORD_PIO_DATA

(BASE_ADDR,0) on the GPIO pins. The tactile sensors, using the internal ~25kohm resistors on

the DE2, are wired to be active high. Therefore, once a 0 value is detected in the

BASE_ADDRESS, it corresponds to a tactile sensor being depressed. The sector that should be

read from the SD card is then updated depending on which sensor it was. While polling is

generally wasteful for CPU cycles, this system has no need to do any other critical computation

or tasks other than to respond to the user’s actions.

Interrupts and queues are used to handle the signals from the right glove, as these are

used to change the modes of play available, versus the actual selection of notes. The thought was

that these buttons will be less often pressed and so there is less need for polling. The current state

of the system is displayed on the LCD by “taskModeDisplay.”

“task_accelerometer” initializes the I2C bus as well as initializing the accelerometer

registers. The values in these registers configure how it will function. For this project, it is set to

detect freefall in the x-axis (1D) from the accelerometer’s point of view. This corresponds to the

axis perpendicular to the earth’s surface.

“task_sd_play” gathers all the information from the other tasks and actually plays

the .wav file from the SD card to the Line Out audio jack. The range of sectors to play,

determined from “task_which_note,” is read one sector at a time and stored in a FIFO buffer that

is fed to audio out.

Bill of Materials

Other than the items listed below, a standard wall outlet (120 V, 60 Hz) is required to

power the Altera DE2 board and external speakers if headphones are not used.

Qty

Part Name

Unit Cost

(CAD)

Total Cost

(CAD)

Altera/Terasic DE2 development board

http://www.altera.com/education/univ/materials/boards/de2

/unv-de2-board.html

$517.72

Ribbon cable (40 & 50-pin)

$10.00

$20.00

Perf board

$7.50

1 roll

Wire wrap

$10.00

Capacitors (0.1 uF)

$0.25

$1.00

1 kit

Resistors (used 4x 180 ohm, 4x 1 kohm)

$7.95

Headers (40 & 50-pin)

$0.20

$0.40

Speakers/headphones

$20.00

Thin wool gloves (pair)

$5.00

2GB SD Card

$10.00

Accelerometer (MMA8452Q)

Digital output via I2C bus. Low voltage to interface (1.6-

3.6V). ±2g/4g/8g 3-axis. Package is about the size of a

quarter. 1.95-3.6V supply. 2 programmable interrupt pins.

Supplier: https://www.sparkfun.com/products/10955

Datasheet:

http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Sensors/Ac

celerometers/MMA8452Q.pdf

$9.95

Tactile sensor (450-1649-ND)

Non-illuminated, top-actuated, 6.00mm x 6.00mm, off-

mom micro switch. Actuator height is 4.30mm with

0.25mm switch travel and through-hole mounting. Max

contact rating at 0.05A @ 12VDC, minimum at 10uA @

1VDC.

Supplier: http://www.digikey.ca/product-

detail/en/FSM2JH/450-1649-ND/1632535

Datasheet:

http://www.te.com/commerce/DocumentDelivery/DDECon

troller?Action=srchrtrv&DocNm=1825910&DocType=Cus

tomer+Drawing&DocLang=English

$0.05

$0.55

TOTAL

$610.07

Reusable Design Units

1. The Altera DE0-Nano board could be used in place of the Altera DE2. There is an

accelerometer already built into the DE0 FPGA and it is only 3x2 inches compared to the 8x6

inches DE2. However, it does not have a audio out jack or an SD card port. A USB-bus can be

attached to the USB port for more connections. The DE0 also only has a 2-pin external header

for power which can be subject to a lot of movement, especially if the board is attached directly

to our glove. The DE2 has many more features and functions that could be additions to our

project, or they may be unused. If the latter, the DE0 may be a more appropriate choice.

2. The Terasic Technologies Inc.V.1.6 “DE2_SD_Card_Audio” demonstration project

contains solutions for both the SD card and audio out. The SD card portion can be substituted

with reusable design unit 3. The audio can be substituted with Altera University’s Audio IP

Core. An application note has been written for it here:

https://www.ualberta.ca/~delliott/local/ece492/appnotes/2013w/audio_altera_university_ip_cores

3. For the SD card, the application note “SD_Card_Interfacing” created by Jason Brown

& Brady Thornton (Group 12) of Winter 2013 can be used as a substitute plan:

https://www.ualberta.ca/~delliott/local/ece492/appnotes/2013w/SD_card_interfacing/

Datasheet

User Perspective Diagram

Image 3: User perspective diagram

User I/O Signals:

All GPIO inputs are driven by +3.3 V and are active low.

Signal

GPIO

Description

Type

Indice

Manuali Proiettore popolari di altre marche

iEmmet

iEmmet MP-312 Manuale utente

Epson

Epson VS400 Manuale utente

Sony

Sony VPL-HS10 Marketing Specifications & Features Manuale utente

Sony

Sony VPL-HW65ES Manuale di servizio

Toshiba

Toshiba TLP-791U Manuale utente

Philips

Philips LC1341 Guida di riferimento

Panasonic

Panasonic PTL735NTU - LCD PROJECTOR-NETWORK IB Manuale utente

NEC

NEC NP600S EDU Manuale utente

BenQ

BenQ SH753P Manuale utente

Eiki

Eiki EK-623U series Manuale utente

Epson

Epson PowerLite Pro Cinema 6020UB Manuale utente

Epson

Epson PowerLite Home Cinema 705HD Manuale utente

Christie

Christie LHD700 Manuale utente

Navitar

Navitar FX Manuale utente

InFocus

InFocus LitePro 560 Manuale utente

Sony

Sony SRX-R815P Manuale utente

Vivitar

Vivitar DP1500 X Manuale utente

Acer

Acer C20 Series Manuale utente