ROBOTIC EYE
(Object tracking using Sixth Sense Technology)
Now a day's humans are building
advanced robots which are capable of doing any particular task, they are using
different techniques to provide Artificial Intelligence to robots so that they
can sense surrounding environment and can act accordingly. When compared to
humans, intelligence of robots always lags behind. Using sixth sense technology (Visually
Controlled Machine), which makes use of “digital image processing”, we can provide
artificial eye to the robot so that it can sense surroundings and react
accordingly. Hence intelligence of the robots can be enhanced to certain
extent. In this paper we are going to illustrate our project on sixth sense
technology where we are controlling the
movement of robot based on the location of the object, mean to say we are
tracking the target using the concept of sixth sense technology.
Index Terms—Sixth Sense Technology,
MATLAB, Digital Image processing, robot, camera.
Sixth Sense
Technology is a revolutionary way to augment the physical world directly
without using dedicated electronic chips. Sixth Sense is a set of wearable
devices that acts as a gestural interface and aggrandize the physical world
around us with digital information and lets the users to use natural hand
gestures to interact with the digital information through it. This technology
gaining its popularity strength because of the usability, simplicity and
ability to work independently in today’s scenario.
The Sixth Sense Technology makes
use of digital image processing techniques. There are different kinds of digital
images. Ycbcr and RGB are the color images which consumes more memory with good
clarity. The storage and transmission of colored is done in Ycbcr format which
is less prone noise than RGB image. There are binary images which contain only
two logic values either 0 or 1, intensity images which is a black and white
image with varying brightness level ranging from 0 to 255.
Any digital color image
consists of RGB (Red, Green and Blue) components in different proportions. A
digital image of resolution 640x480 means it contain 640x480 pixels in 2
dimensional plane. An each pixel has three planes (sub pixels) i.e. Red plane,
Green plane and Blue plane. Each sub pixels is coded with 8 bit binary values
to represent 256 brightness levels i.e. from 0 to 255. Different brightness
levels (or 8 bit code) of these three sub pixels gives different colors. Each sub
pixel consumes 8 bits or 1 byte memory. In each pixel there are 3 sub pixels, so
each RGB pixel consumes 3 bytes of memory. A RGB image of resolution 640x480
consumes 921600(640x480x3) bytes of memory.
A different colored object contains
different pixel values. When a image is captured the position of required
object is sensed or found by specifying pixel range of the required object. The
colored image is converted into binary image , in which the pixel values whose
values lies in the range of required objects pixel values are converted into binary
logic value 1 (white pixel) and the pixel values colored image which are out of
range of the pixel values of specified object are converted into binary value 0
( black pixel). With the help of this binary converted image it is very easy to
find the location or position of the object.
The main difference between robot and the artificial
intelligent machines is that, robots are not intelligent. They will not be
having their own memory and they are not self- instructed. Using the above
concept of Sixth Sense Technology we can provide eye to robot or machine which
helps robot to see and sense to surroundings and act accordingly.
The following principle will illustrate
you the complete concept and working of our project.
WORKING PRINCIPLE
WORKING PRINCIPLE
We make use of Sixth Sense technology
for our project. In which we capture the image using webcam(wired or wireless)
or any other camera. The more the sensitivity of camera the more resolution we
get and hence we can use it for long distance sensing of objects. The object
captured is processed using some techniques and then its location or position
is found out. And corresponding command is given to robot.
The following block diagram illustrates
the method of working of our project.
 |
| block diagram of working principle |
There are four processes required to carry
out task. And these processes are repeated at regular intervals of time.
i.
Recording Video:
The camera which we need is interfaced to
the computer or any processor which we make use. The camera is switched on
using command "videoinput” in
matlab. And the resolution of the camera and other parameters like picture
format which camera supports is also done by using commands in matlab.
Once the command is given the
camera captures continuous video and gives it to matlab for further processing.
ii.
Converting Video to Images:
The images are extracted by the matlab at
regular intervals of time from the captured video using command "getsnapshot". The time interval or delay between extracting further images
is set by using command "pause".
iii.
Processing Images:
The
captured images are processed by matlab.
First we have to confirm that the extracted images are in "RGB" format, if not we have to convert it into RGB
format. Some camera does not support RGB format for those camera this step is
necessary. We used the camera which support "YUY2"(ycbcr)
format. To convert it into RGB,
we used command "ycbcr2rgb"
 |
| YCBCR image format of extracted image |
 |
RGB format of extracted image
|
Once we get the image in RGB format we have
to get the sample pixel values of the required object. This step is performed
for the first time for a particular object. If object is changed then this step
is performed. We use the command "impixel" to take sample pixel values
of the object.
When we use this command we get a
window which contains extracted image and a (+) mark cursor. By clicking the
cursor on required places we get the pixel values at that position.
The sample pixel values are shown below. The first column show Red pixel values,
second column shows Green pixel values and third columns shows Blue pixel
values.
67
51 161
53
39 139
40
31 104
48
39 111
38
27 110
38
27 106
62
61 164
41
29 131
44
34 106
84
80 175
By taking these pixel values we have to
convert the RGB image to binary image. Pixel values of image which are in this
range specified are converted into logic 1 (white pixel) and the pixel values
which are out of this range are converted into binary value 0 (black pixel).
 |
| binary image of extracted image |
 |
| . Dividing image plane into 9 region |
To find the
location of the object and to give appropriate command to robot for its
movement we divided the plane into 9 regions as show below. And these regions
are labeled from 1 to 9. The labeling of regions depends on the position of
object. The regions are relabeled if object position changes. The region which
contains object is labeled as 1 and other regions are labeled in circular
fashion, which will be explained clearly in next section.
The pixel values of the image are
stored in matlab in matrix form. For example, if the resolution of the image is
640x480, then the pixel values of image are stored in matlab in matrix of size
640 columns and 480 rows. In the above figure Cmax represents the maximum
column number i.e. 640 in this example and Rmax represents maximum row number
i.e. 480 in this example. Here 'c=Cmax/3’, 'r=Rmax/3'.In this way the
plane is divided into 9 regions.
To find the location or position of the
object in the plane we count the number of white pixels (binary value 1) of the
binary image in all regions. The region with highest number of white pixels is
the position of the object in the plane. And then we send appropriate command
to robot. For example, if maximum number
of white pixel is in region 9 then the location of object is region 9 and robot
is given with command to stop .
i.
Computing & sending commands to robot:
Now, we see how to compute the angle through
which the robot should turn clockwise or anticlockwise to follow or track the object. At the beginning the regions from
1 to 9 are labeled from 1 to 9 respectively. First we are going to find the
region of location of object and its label value. Then we calculate the angle
through which the robot should be rotated and what appropriate commands should
be given to robot by taking
label value using following steps.
If label is 9 then the robot is given
command to stop. If label value is greater than 4, then robot is rotated in
anticlockwise direction through the angle, calculated as,
angle=(9-label
value)*45 degrees
After rotating through this angle the
robot is given command to go forward.
If label value is less than or equal
to 4 then robot is rotated in clockwise direction through the angle, which is
calculated as,
angle=(label value -1 )*45 degrees
After rotating through this angle the
robot is given command to go forward.
For example, if object is found to be in
region 4 which is labeled as 4, then robot is rotated in clockwise direction,
since label value is equal to 4. The angle of rotation is,
angle
= (4-1)*45 = 135 degrees
After rotating through 135 degrees the
robot is given command to go straight. After that the regions are relabeled
with the region 4 which is the location of the object labeled as '1'.
EXPERIMENTAL RESULTS
We have simulated this work in
MATLAB 7.12.0 and implemented using microcontroller (Atmega).
If
object found in region 2 which is labeled as 2, then robot instructed to rotate
clockwise through the angle of 450. After rotating through 450 the
robot is instructed to move forward. The simulation results of this step is
shown in figure 9 and 10. After this step the regions are relabeled as shown in
figure 11.
For next interval the object is
made to lie in region 5 which has label value '4'. Then robot is instructed to
rotate in clockwise direction through the angle 1350. After rotation
the robot is instructed to move in forward direction. The simulation results of
this step is shown in figure 12 and 13. After this step the regions are
relabeled as shown in figure 14.
Again for the next interval of time the object is made to
lie in the region 4 which has label value '8'. Then the robot is instructed to
rotate in anti-clockwise direction through the angle 450. After
rotation the robot is instructed to move in forward direction. The simulation
results of this step is shown in figure 15 and 16. After this step the regions
are relabeled.
In this way the process is
continued at regular intervals of time, to track the object and to give appropriate
commands to the robot.
 |
| binary image containing object in region 2 |
 |
Simulation results when object is in region 2 (label 2)
|
 |
Relabeling regions when object is in region 2(label 2)
|
APPLICATIONS
1.
This
project can be implemented in bomb diffusing robots where human presence is
dangerous.
2.
This
project can be implemented in colored object tracking.
3.
This
project can be made use in industries for colored object separation.
4.
This project can be used in tracking the ball
(focusing camera automatically) in sports like tennis, cricket, football etc.
5.
This
project can be implemented in gesture controlled robotics.
6.
This
project can be implemented in security systems like thief detector . This can
be done by installing camera when no one is present at home. The entry of thief
is detected by verifying drastic pixel changes in the captured image.
CONCLUSION
Now a day's sixth sense technology is
new emerging and interesting field having lot of scope. Lot of research and
development work is being carried out in this project. This technology provides
visual sense (eye) to robots which are capable of doing almost human jobs. Now
a day's key boards & scroll balls are getting replaced with touch screen
& touchpad's. Using this technology we can replace touch screen with
virtual mouse and virtual key boards. Lot of research work like background subs
traction and other things are taking place to overcome some of the
disadvantages present in this technology.
No comments:
Post a Comment