Nurse Call through Voice Recognition

About

For this project, we worked with a client who has multiple sclerosis. She is unable to move any of her muscles below her neck, and has limited control of her chin and facial muscles. Because of her deteriorating condition, she is looking for a better solution to call her nurse. Currently, she taps a switch with her chin in order to trigger a call to the nurse. However, with steadily decreasing motor functionality, she is worried that this is not a permanent solution to her problem. In order to remedy this increasingly important issue, we devised a system that calls her nurse through voice activation. Approach: In order to complete this project, we used an Arduino with an EasyVR for voice recognition. We used the EasyVR Commander software to train the board with our client's voice and perform specific commands based on the recognized keyword. She can activate the two-state system by first saying "start," which triggers a listening state. At this point, our system supports recognition for "help" and "care", which can trigger a call to an emergency nurse and a daily aid for minor issues, respectively. The system at our client’s residence is a Rauland 4000 Responder. To activate a call on this system, we use a pull-down circuit, triggered by one of the pins on our Arduino. The circuit will short the two pins of the nurse call switch. A bi-color LED and an 8-ohm speaker have been interfaced with the EasyVR board in order to give visual and auditory feedback to our client when the call has been initialized. If the listening state was triggered incorrectly, our client may say "stop" in order to go back to the idle state. As it currently stands, the system only supports nurse calls when the listening state is triggered--as was defined in the scope of the project. However, up to 29 new functions can be implemented. Examples of future work on a similar system are turning a television on and off, raising a bed, and turning lights on and off. Parts description: The Arduino is a microcontroller board that can easily interface with many other boards (“shields”), such as the EasyVR. The EasyVR is designed for simple, embedded voice recognition applications. The EasyVR is trained through its Commander application. The EasyVR can be trained on up to five-second-long words or phrases, and training requires just two samples. The EasyVR then returns reasonably accurate and personalized voice recognition results for the user. Once trained, one can modify the existing code in the EasyVR (initially called the Robobasic code) to perform the relevant tasks. The code for our project in specific has been included below. On the hardware side, our client’s nurse call switch previously relied on a simple pull-up/pull-down circuit that was triggered by an application of mechanical force. In order to preserve the existing system in the case of a failure of our new system, we added a pull-down circuit that works in parallel with this existing system. This allows for both systems to operate simultaneously, so our client has two ways to call the nurse.

Step 1: Obtain Hardware

Obtain Arduino and EasyVR. The Arduino can be bought at arduino.cc, and the EasyVR can be bought at www.sparkfun.com.

Step 2: Install Software

Install Arduino software and EasyVR Commander (3.4.10 at time of publication). These can be found at arduino.cc and veear.eu, respectively. At the time of this publication, Arduino 1.0.5 and EasyVR 3.4.10 were the latest software editions.

Step 3: Connect Hardware

Attach EasyVR to Arduino and connect to computer using an A-B USB cable.

Step 4: Familiarization

Familiarize yourself with Arduino code through demos. Once the Arduino software has been installed and the Arduino is connected to the computer, run through some of the built-in demos that the Arduino software features. In Arduino 1.0.5, this can be found by navigating to File > Examples. Once you read and understand how the code works, compile it and upload it to the Arduino. The Arduino website features helpful walkthroughs of how to use the software at http://arduino.cc/en/Tutorial/HomePage.

Step 5: Voice Training

Train EasyVR using Commander Software.
a. Group 0 is the trigger state: if the EasyVR recognizes any word in Group 0, the next state (“listening” in our model) will be entered.
b. Group 1 is the execution state: if the EasyVR recognizes any word in Group 1, it will relay appropriate information back to the Arduino, which will subsequently parse that information and will interact with the hardware by triggering a nurse call.

Step 6: Write Arduino Code

Write or modify Arduino code in order to perform the appropriate actions when the trigger words are uttered.
a. In order to ensure that the attached code works as expected, use Group 0 for the trigger state and Group 1 for the execution state. Furthermore, in order to relay visual cues to the user when there is a state change, use Pin 10 for the positive LED terminal and Pin 11 for the negative terminal. Finally, use Pin 3 for the nurse call relay. These can be changed easily in the code by changing the values of NURSE_CALL_PIN, LED_POS_PIN, and LED_NEG_PIN.

Step 7: Create Audio Out Module

Optionally, install QuickSynthesis 5 and use it to provide audio feedback to the user when the listening state is entered. This requires building a Sound Table for the EasyVR, which is unfortunately somewhat tricky.
a. Download and install the RSC-4x Toolkits CD from here, ftp://ftp4.sensoryinc.com/software/rsc4xtk/rsc4xtoolkit.zip
b. Install the toolkit, and check the option for FluencyChip 3.1.16
c. Run QuickSynthesis with administrator privileges. Right-click the application and select “Run as Administrator” to do this.
d. Create the sound table by importing the audio files you want to play into QuickSynthesis.
e. Go to File  Options, and set the FluentChip for RSC4 Path to where you installed FluencyChip from the RSC-4x Toolkits CD. This is most likely C:SensoryFC3_1_16
f. Build the sound table. Save. Build the sound table again.
g. Set the connection jumper on the EasyVR to UP. Start up commander, and plug in the Arduino.
h. In Commander, go to Update Custom Data and select your saved QuickSynthesis project file. Click Download, and the EasyVR will be loaded with the audio files you just made.
i. Verify that it works by switching the jumper back to PC and viewing the Sound Table entry in Commander.
j. Note: in our design, we had the first sound table entry play back, “Now listening…” and the third entry play, “Calling an aide…” If you change the audio clips but get the order wrong, so they play at the wrong times, you can go into the code and find the lines that say, easyvr.playSound(, EasyVR::VOL_FULL); and change the number to the row of the sound table you want.