Color Tracking in a loaded Video

Hello,

i want to track objects by color. But not with live webcam. I want to track objects by color in a loaded video, but it don'tz work:

   if true      

VideoReader.getFileFormats()   % see full list 
vidObj = VideoReader('C:\VIDEO.MP4'); % open file
get(vidObj)   
nFrames=get(vidObj, 'NumberOfFrames');  
width = vidObj.Width;    % get image width
height =vidObj.Height;   % get image height
for iFrame=1:nFrames
    I = read(vidObj, iFrame);  %get one RGB image
    diff_im = imsubtract(I(:,:,1),1), rgb2gray(I);
    diff_im = medfilt2(diff_im, [3 3]);
    diff_im = im2bw(diff_im,0.18);      
    diff_im = bwareaopen(diff_im,300);      
    bw = bwlabel(diff_im, 8);      
    stats = regionprops(bw, 'BoundingBox', 'Centroid');         
    imshow(I,[]);
   for object = 1:length(stats)
       bb = stats(object).BoundingBox;
       bc = stats(object).Centroid;
       rectangle('Position',bb,'EdgeColor','r','LineWidth',2)
       plot(bc(1),bc(2), '-m+')
       a=text(bc(1)+15,bc(2), strcat('X: ', num2str(round(bc(1))), ' Y: ', num2str(round(bc(2)))));
       set(a, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'black');
   end  
end
   end

Can anybody help me? And my second question: If i have a snapshot from live video and i want to compare this with a second snapshot for example i one objects is at other position, how can I do this?

Answer 1

[Raoul Can]

I loaded my own video but I didnt see the bounding box. But I think it is what I need, thanks. Can i fill the rbounding box with the respective color for better color-detection?

I try to use a similar code for live tracking to control my robot arm. How i can implement this in my own code?

if true
 .....
 .....
  n=length(Daten)
      for i = 1: length(Daten)
          drive_x = RobotArm_Koordinate_x(i)/1000
          drive_y = RobotArm_Koordinate_y(i)/1000
          drive_z = RobotArm_Koordinate_z(i)/1000
         % NULL Position
          R_DriveCoordinate(handles, x, y, z, theta);
          % Drive to ...
          R_DriveCoordinate(handles, drive_x, drive_y, drive_z, drive_theta(i));
     ....
end

At the Moment I can select a color and make a snapshot. After that I can click on "OK" button and drive to the coordinates.

Answer 2

[Image Analyst]

See my attached demo where I track a green Sharpie marker.

[Image Analyst]

Raoul's "Answer" moved here because it's not an answer to his original question at the top, but a comment to me:

Thanks. But I am searching for a simple detection like my example with bounding boxes

[Image Analyst]

Raoul, you obviously didn't even run my demo, or if you did, you didn't notice the middle left box where it's tracking the green regions with bounding boxes and coordinates displayed.

Please at least give it a try.

[Image Analyst]

From your "Answer": "I loaded my own video but I didnt see the bounding box." Did you change the thresholds to find your color instead of the green Sharpie? Probably not. So if you didn't do that, it won't find your color and there will be no boxes. You need to specify your color, not the color I used.

[Image Analyst]

You should put your Comments as comments, not Answers. Regarding your "Answer" below, it's likely it's not getting segmented because the thresholds are wrong. Find out the thresholds by saving the frame where one of the colors is not segmented, then use the Color Thresholder app on the Apps tab of the tool ribbon. Put those thresholds into the code.

[Image Analyst]

AGAIN You should put your Comments as comments, not Answers. Can you at least try that?

You can call imcrop() and then call this code:

set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0 1 1]);
% Get rid of tool bar and pulldown menus that are along top of figure.
set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
set(gcf, 'Name', 'Demo by Raoul', 'NumberTitle', 'Off')

[Raoul Can]

If i select a color I get this here:

if true
  % Define thresholds for channel 1 based on histogram settings
channel1Min = 0.942;
channel1Max = 0.028;
% Define thresholds for channel 2 based on histogram settings
channel2Min = 0.144;
channel2Max = 0.912;
% Define thresholds for channel 3 based on histogram settings
channel3Min = 0.255;
channel3Max = 0.663;
% Create mask based on chosen histogram thresholds
BW = ( (I(:,:,1) >= channel1Min) | (I(:,:,1) <= channel1Max) ) & ...
  (I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
  (I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
end

But I dont know how to use. Like this:

hThresholds = [0.942, 0.028]; 
sThresholds = [0.144, 0.912]; 
vThresholds = [0.255, 0.663];

What is wrong? And is it possible to modificate the code for all colors. Only detecting by movement?

[Raoul Can]

Hi,

I don't understand it. In this example we extract red from grayscale image to extract the red components in the image.

    diff_im = imsubtract(data(:,:,1), rgb2gray(data));:

And in your example we don't have a variable for difference so I have to select the difference (not segmentated) directly. I this correctly?

[Image Analyst]

Yes, those thresholds should be correct. My demo segments the RGB image to get a binary image where the color is. It doesn't get a difference and I don't know why you're using imsubtract() and rgb2gray() at all. What do you want? Do you want a masked image (black everywhere except the colored regions)? Do you want a cropped image? A difference just doesn't seem like it would mean anything at all so I don't know why you'd want that.

[Raoul Can]

I saw imsubtract in a other demo. I want to draw a bounding box for a specific color.

And if I use your example I have to transform V threshold from 0 to 255 but only the value and I don't know why.

[john berndt]

Hi Image Analyst. I am working on a project where i have modified this code to look at points and calculate strain. Would you happen to know of a way that i could graph the x and y positioning of each point of interest for each frame? I can get the code to find and show the x and y position however, the code overwrites this variable and only shows it for the last analyzed frame.

[Image Analyst]

John, store the x and y locations as a function of frame number, then use plot(). Either use "hold on" in the loop, or (better) just wait until after the loop has finished to call plot().

[john berndt]

Would it be possible for you to show an example of this? I am really struggling on this and I cannot get it to work.

[Image Analyst]

Either

hold on;
for k = 1 : numFrames
    % Compute x and y somehow, then:
    plot(x, y, 'b*', 'MarkerSize', 15);  % Plot inside the loop
end
hold off;
grid on;

OR

for k = 1 : numFrames
    % Compute x and y as arrays somehow, then:
end
hold off
plot(x, y, 'b*', 'MarkerSize', 15); % Plot after the loop is done.
grid on;

[Filbert Halim Santosa]

Hi Image Analayst. Could you give the example of the code to store all X and Y data values from each frame based on your green sharpie marker example?

[Judy George]

Hi Image Anlayst, do you know what the threshold values correspond to in the notation you had? How to do the following thresholds make up green?

hThresholds = [0.24, 0.44];

sThresholds = [0.8, 1.0];

vThresholds = [20, 125];

[Image Analyst]

@Judy George run the color thresholder app on the apps tab of the tool ribbon. Read in your image and export the code and you'll see the thresholds that work for your particular image.

Those values I used are what works for my images.

[Iran Neto]

Hi @Image Analyst when I use the thresholder app, it gives me a range from 0 to 1 for the 3 values (h,s and v) and your vThresholds is between 20 and 125, I didn't understand that, what's the range for the v of hsv?

[Image Analyst]

@Iran Neto It depends on whether you're casting to double or not. If you use a using a uint8 image, then the Color Thresholder app and rgb2hsv() will give you the value image in the range 0-1. If you're using a double version, like double(rgbImage), then rgb2hsv() will give yout he value image in the range 0-255

[Iran Neto]

thank you so much!!

Answer 3

[Raoul Can]

Oh I have a problem with my video. I cannot see the one with the boxes, why? And how I can change the size of the images?

I want to see only the image with the bounding boxes as full screen , how i can change this?

[Image Analyst]

If you don't see the boxes with your video, then you don't have anything in your video that matches the green range that I defined for my video. You'll have to adjust your thresholds.

You can maximize the window whenever you want with this code

g = gcf;
g.WindowState = 'maximized';

Answer 4

[Raoul Can]

Hi,

I don't understand it. In this example we extract red from grayscale image to extract the red components in the image.

    diff_im = imsubtract(data(:,:,1), rgb2gray(data));

And in your example we don't have a variable for difference so I have to select the difference (not segmentated) directly. I this correctly? Or why I hav to select the not segmentated part. I don't understand

[Image Analyst]

That code you showed is a different way for detecting red. I don't think it's as robust as doing a true color segmentation like I did when I converted the image to HSV color space. Doing it in RGB space like you're showing probably won't work for all shades of red. I don't get a difference image from subtracting a gray scale version of the image from the red cahnnel because I'm not doing it like that. I get the red mask by thresholding the Hue, Saturation, and Value channels. Again, this is better and more robust.