KEV3

Kinect integration with the EV3 robot

Installation instructions

There are other integrations of Kinect and robots, but this is done exclusively in Matlab for the new LEGO EV3 robot. It was developed in Matlab 2014a and requires the Image Acquisition Toolbox. It might run on older Matlab versions but probably Kinect support cannot be installed on Matlab versions older than 2013a. For controlling the Kinect, one uses the Kinect for Windows Sensor Support for Matlab Image Acquisition Toolbox, as explained at:

http://www.mathworks.com/help/imaq/installing-the-kinect-for-windows-sensor-support-package.html?searchHighlight=kinect

In addition, prior to plug in the Kinect, one may also need to install the Kinect for Windows Runtime v1.6 (~114 MB), at no cost from: http://www.microsoft.com/en-ca/download/confirmation.aspx?id=34811.

For controlling the EV3 robot one uses the QUT toolkit, originaly from: https://wiki.qut.edu.au/display/raq/QUT+EV3+MATLAB+toolkit. However, that link seems to be broken and you may install the toolkit from ev3_download.zip. Please let me know if it doesn't work, as I didn't test it to make sure is the correct one.

This allows communication over WiFi, Bluetooth (BT), or USB, but here it’s implemented only the BT. The driving motors IDs are B and C, while the arm motor is A.

Alternatively, without the need for Matlab or Image Acquisition Toolbox, one can use the pre-compiled installers, made on a Windows 7 Pro 64 bit computer, available at:

https://1drv.ms/f/s!Aj9Z7_pmMkKijyRhydhQAfNvXjDA

In order to run them, one may be asked to also install the free Matlab Win64 Compiler Runtime (MCR), version 8.3 (in case it’s not already installed). This is a free 475 Mb installation from Mathworks, which will support any executable issued from Matlab compiler, providing all the necessary libraries. KEV3.exe is a standalone executable supposing that the MCR is already installed or you take care of the MCR installation. KEV3_Installer.exe is a Windows installation version which verifies OS compatibility, downloads and installs for you the appropriate MCR.

-----------------------

Driving instructions

A demo on using this program is available on YouTube under: “KEV3: Kinect integration with the EV3 robot”:

http://youtu.be/g4fbeqnG00Q

The application was made for Kinect V1 for Windows and the robot Lego Mindstorms EV3, used mainly as a car, eventually with an arm moving up and down. This arm operates a smart phone as an IPCamera to see where the robot goes. The robot has to be already BT (Bluetooth) paired with the computer. To start, press "Start camera“, wait until that button changes to “Stop Camera” and then press "BT connect". Wait few seconds for that operation to complete (one should hear a beep from EV3 and the "Stop EV3" button becomes active). Position yourself about 1 m away from Kinect and your skeleton should overlay in red your image. For a better flexibility, there is a tilt slider on the left of the camera image to adjust the Kinect viewing angle. Once the skeleton is properly acquired and the dial graph (on top of the logo) shows movements made by your right hand, you're ready to drive. The driving commands sensed by Kinect are now ready for transferring to the sliders and then to be picked up by the robot control. The driving gestures are:

-ssLeft hand starts the robot: move the left hand horizontally to the right, for less than a second, and beyond the neck position. This allows the Kinect to sense your moves, store them on sliders, ready to be picked up by the robot code. Practically this operation starts the robot movements. The same thing is done by pressing GO. One can stop it by pressing HALT, or move again the left hand the same way.

-ssLeft hand drives the robot: upper than elbow drives forward, lower than elbow drives backwards. The motor power is proportional with the moves. It’s shown in the vertical slider, on the left of the lab logo.

-ssRight hand turns left-right the robot - it has to operate like turning a driving wheel. It should follow the dial indication.

In any moment one can stop the robot with the “Stop EV3” button in the EV3 panel. The IPCamera viewing angle can be changed up and down with the Tilt++ or Tilt– buttons. It’s necessary to stop the robot driving during this operation to avoid unexpected robot movements.

----------------------------

References and inspirations

This code was partly inspired by the work or suggestions of other people:

-sshttp://www.mathworks.com/help/imaq/examples/using-the-kinect-r-for-windows-r-from-image-acquisition-toolbox-tm.html?searchHighlight=kinectforwindows?refresh=true

-sshttp://www.mathworks.com/matlabcentral/answers/96242-how-can-i-insert-live-video-into-a-matlab-gui-using-image-acquisition-toolbox

-sshttps://wiki.qut.edu.au/display/cyphy/QUT+EV3+MATLAB+toolkit

-sshttp://stackoverflow.com/questions/19790542/recording-playback-kinect-with-matlab-upside-down-and-in-black-white-rather-th

-sshttp://cogprojects.wordpress.com/2012/07/10/simple-matlab-gui-to-demonstrate-real-time-video-processing-of-depth-perception-data/

-sshttp://lovelandrobotics.weebly.com/windows-integration-for-ev3nxt.html

-ssIsabela Albuquerque, Raymundo Cassani, Tiago Falk from MusaeLab