Introduction:
In this article i will be explaining , how to make a simple color following robot, which will follow the object of particular color. This article can be seen as first step in getting started with image processing based robotics.
Things you should know:
-Basics of using any micro-controller which has an interface for serial communication.
-Basics of C++
Hardware requirements:
USB camera, Micro-controller board with serial interface, RF modules (for wireless communication) or USB to RS232 cable (for wired communication between PC and micro-controller).
What exactly is this Image Processing based Robotics ??
Generally while developing any robotics application we use different sensors as input and based on their values robot takes necessary decision. Simarly in image processing based robotics we use image from a camera as an input and uses it to take a decision. Now the problem here is image doesn't give a digital value of true or false as most of the sensors do. So we need to do the processing of the image to remove noise,crop unnecessary part, apply different filters to focus only on object of our interest in the image etc.
and that's where the opencv libraries comes extremely helpful.
OpenCV (Open Source Computer Vision Library: http://opencv.org) is an open-source BSD-licensed library that includes several hundreds of computer vision algorithms.It has several modules such as core,imageproc,video,calib3d.features2d,objectdetect,highgui and gpu. Of these modules core,imageproc and highgui module module comes quite handful in image processing based robotics.
1) Setup the overhead camera and connect it to your pc. You may also require to install necessary drivers based on your operating system to get started.
2) Now once the camera is detected , next task is to take a snapshots.
So our main task here is to continuously take snapshots from overhead camera and process them to drive the bot.
3)Image from the overhead camera may not only include the desired arena, it may also show extra part which is not needed and can cause problems in our processing. So next step is to crop only that part of the image which is required.
So now we have only the only the desired portion of image. Next problem is there will be lot of noise in the image because of imperfect lightening conditions. So for smooth processing we have to get rid of this noise.This is achieved using Gaussian blur. It is a most common technique which is used for this purpose.
This completes basic setup of the arena environment in OpenCV. If you want to further reduce noise there are lot of filters available in opencv ,more about which you can find from opencv website.
Now here in this problem statement we have 2 objects - one is the bot itself and 2nd is the circular object which bot will follow. To follow the circular object 1st of all we should know position of bot and object. This can be done by painting the top surface of the bot and object with 2 different colors which are different from the arena color. So now we can use filter to detect that color and use techniques like hough circles or polygon detection to detect the object based on shape its shape.
By default image will be in RGB (Red Green Blue). But under non ideal conditions distinguishing colors on RGB values in not reliable. Distinguishing colors in using hue,saturation and value in grayscale is much reliable option. Hence first we have to convert RGB image in Grayscale format and then decide HSV thresholds using trackbars.
So suppose the object to be followed is circular then we can easily detect it first by filtering that color and then applying hough transform to detect circle and its center.
Similarly assuming that bot top part is rectangular we can apply polygon detection on the bot and get its center. But problem here is we also want to know the orientation of the bot so that accordingly we can give it the directions to move. So we'll need 2 different color objects on top of the bot surface to calculate this angle . Also hough circle transform is much more reliable compared to polygon detection hence its better to paint 2 circles on bot top then going for 2 rectangles or squares.
Now we have to calculate 2 angles:
1) Angles between 2 objects on bot top
2) Angle made by the object with the bot.
This can be done using simple geometry as shown in the code.
Once the 2 angles are calculated, we have to compare 2 angles and determine in which direction bot should move. For example if difference between bot angle and object angle is 30 degree then bot should move right until the angle difference is zero and then then start moving forward until it reaches the object. Commands can sent to bot from OpenCV using serial communication .
Full code can be downloaded from following github repository Github
This repository can be used to find more about image processing techniques and serial communication in OpenCV.
In this article i will be explaining , how to make a simple color following robot, which will follow the object of particular color. This article can be seen as first step in getting started with image processing based robotics.
Things you should know:
-Basics of using any micro-controller which has an interface for serial communication.
-Basics of C++
Hardware requirements:
USB camera, Micro-controller board with serial interface, RF modules (for wireless communication) or USB to RS232 cable (for wired communication between PC and micro-controller).
What exactly is this Image Processing based Robotics ??
Generally while developing any robotics application we use different sensors as input and based on their values robot takes necessary decision. Simarly in image processing based robotics we use image from a camera as an input and uses it to take a decision. Now the problem here is image doesn't give a digital value of true or false as most of the sensors do. So we need to do the processing of the image to remove noise,crop unnecessary part, apply different filters to focus only on object of our interest in the image etc.
and that's where the opencv libraries comes extremely helpful.
OpenCV (Open Source Computer Vision Library: http://opencv.org) is an open-source BSD-licensed library that includes several hundreds of computer vision algorithms.It has several modules such as core,imageproc,video,calib3d.features2d,objectdetect,highgui and gpu. Of these modules core,imageproc and highgui module module comes quite handful in image processing based robotics.
- core - a compact module defining basic data structures, including the dense multi-dimensional array Mat and basic functions used by all other modules.
- imgproc - an image processing module that includes linear and non-linear image filtering, geometrical image transformations (resize, affine and perspective warping, generic table-based remapping), color space conversion, histograms, and so on.
- highgui - an easy-to-use interface to video capturing, image and video codecs, as well as simple UI capabilities.
1) Setup the overhead camera and connect it to your pc. You may also require to install necessary drivers based on your operating system to get started.
2) Now once the camera is detected , next task is to take a snapshots.
So our main task here is to continuously take snapshots from overhead camera and process them to drive the bot.
3)Image from the overhead camera may not only include the desired arena, it may also show extra part which is not needed and can cause problems in our processing. So next step is to crop only that part of the image which is required.
So now we have only the only the desired portion of image. Next problem is there will be lot of noise in the image because of imperfect lightening conditions. So for smooth processing we have to get rid of this noise.This is achieved using Gaussian blur. It is a most common technique which is used for this purpose.
This completes basic setup of the arena environment in OpenCV. If you want to further reduce noise there are lot of filters available in opencv ,more about which you can find from opencv website.
Now here in this problem statement we have 2 objects - one is the bot itself and 2nd is the circular object which bot will follow. To follow the circular object 1st of all we should know position of bot and object. This can be done by painting the top surface of the bot and object with 2 different colors which are different from the arena color. So now we can use filter to detect that color and use techniques like hough circles or polygon detection to detect the object based on shape its shape.
By default image will be in RGB (Red Green Blue). But under non ideal conditions distinguishing colors on RGB values in not reliable. Distinguishing colors in using hue,saturation and value in grayscale is much reliable option. Hence first we have to convert RGB image in Grayscale format and then decide HSV thresholds using trackbars.
So suppose the object to be followed is circular then we can easily detect it first by filtering that color and then applying hough transform to detect circle and its center.
Similarly assuming that bot top part is rectangular we can apply polygon detection on the bot and get its center. But problem here is we also want to know the orientation of the bot so that accordingly we can give it the directions to move. So we'll need 2 different color objects on top of the bot surface to calculate this angle . Also hough circle transform is much more reliable compared to polygon detection hence its better to paint 2 circles on bot top then going for 2 rectangles or squares.
1) Angles between 2 objects on bot top
2) Angle made by the object with the bot.
This can be done using simple geometry as shown in the code.
Once the 2 angles are calculated, we have to compare 2 angles and determine in which direction bot should move. For example if difference between bot angle and object angle is 30 degree then bot should move right until the angle difference is zero and then then start moving forward until it reaches the object. Commands can sent to bot from OpenCV using serial communication .
Full code can be downloaded from following github repository Github
This repository can be used to find more about image processing techniques and serial communication in OpenCV.
No comments:
Post a Comment