How do I detect different colors in an image to identify different pieces of fruit?

I have this image:

I need to identify various types of fruit, including pears, apples, oranges and bananas.

Initially I started by recognizing them through the form, with the help of this code:

Im = imread('frame147.jpg'); figure,imshow(Im);title('Imagem Original');

I1 = rgb2gray(Im);

level=graythresh(I1); bw=im2bw(I1,level);

[I2, Num]=bwlabel(bw);

objetos=regionprops(I2); figure,imshow(bw);title('Binario da Imagem com box');

for n=1:size(objetos,1) rectangle('Position',objetos(n).BoundingBox,'EdgeColor','y','LineWidth',1); end

for n=1:size(find([objetos.Area]<100),2) s=find([objetos.Area]<100); d=round(objetos(s(n)).BoundingBox); bw(d(2):d(2)+d(4),d(1):d(1)+d(3))=0; end figure,imshow(bw);title('Objetos Definidos');

[I2, Ne]=bwlabel(bw);

objetos2=regionprops(I2,'ALL');

for n=1:size(objetos2,1) X=rectangle('Position',objetos2(n).BoundingBox,'EdgeColor','y','LineWidth',1); end

for n=1:size(find([objetos2.Area]>100),2) X1=objetos2(n).Centroid(1); X2=objetos2(n).Centroid(2); text('Position',[X1,X2],'String',n,'color','b') end fprintf('\n\nForam encontradas %d objetos:\n\n',size(objetos2,1))

tol1=0.9; % tolerancia circulos - oranges and aples tol2=0.2; % tolerancia triangulos - pear tol3=0.9; % tolerancia retangulos - banana

   for n=1:size(objetos2,1)

    for m=1:15
        dif(m)=(objetos2(n).Extrema(m)/objetos2(n).Extrema(m+1));

    end
                 % Circulos: oranges or apples
        if ((4*pi*(objetos2(n).Area))/((objetos2(n).Perimeter)^2)>tol1);
         fprintf('Peça nº %d: Círculo.\n',n);
         Area=(objetos2(n).Area)

                 % Triângulos: pears
        elseif ((sqrt((objetos2(n).Area/(sqrt(3)/4)))>=(objetos2(n).Perimeter/3)*(1-tol2))&&((sqrt((objetos2(n).Area/(sqrt(3)/4))))<=(objetos2(n).Perimeter/3)*(1+tol2))) 
        fprintf('Peça nº %d: Pêra.\n',n);
        Area=(objetos2(n).Area)

                %Retangulos: banana
        elseif (dif(3)==1 && dif(7)==1 && dif(9)==1 && dif(13)==1)   

            if  (objetos2(n).MinorAxisLength/objetos2(n).MajorAxisLength>=(tol3))
                 fprintf('Peça nº %d: banana.\n',n);
                 Area=(objetos2(n).Area)
                  % Not a fruit
             else fprintf('Peça nº %d: Não é uma peça de fruta.\n',n);
                  Area=(objetos2(n).Area)

            end
        end
   end

bananas have not yet been able to adapt a formula to recognize it

but in the case of apples and oranges the only way to distinguish them would be through color recognition, but I can not find a method of doing so.

Can someone help me?

Answer 1

[Image Analyst]

Try this demo:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;

%===============================================================================
% Get the name of the image the user wants to use.
baseFileName = 'frame147.jpg';
folder = pwd
fullFileName = fullfile(folder, baseFileName);

% Check if file exists.
if ~exist(fullFileName, 'file')
  % The file doesn't exist -- didn't find it there in that folder.
  % Check the entire search path (other folders) for the file by stripping off the folder.
  fullFileNameOnSearchPath = baseFileName; % No path this time.
  if ~exist(fullFileNameOnSearchPath, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end

%=======================================================================================
% Read in demo image.
rgbImage = imread(fullFileName);
% Get the dimensions of the image.
[rows, columns, numberOfColorChannels] = size(rgbImage)

% Display image.
subplot(2, 2, 1);
imshow(rgbImage, []);
axis on;
caption = sprintf('Original Color Image\n%s', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Set up figure properties:
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0.05 1 0.95]);
% Get rid of tool bar and pulldown menus that are along top of figure.
% set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
set(gcf, 'Name', 'Demo by ImageAnalyst', 'NumberTitle', 'Off')
drawnow;

% Do color segmentation:
[BW, maskedRGBImage] = createMask(rgbImage);

% Display the image.
subplot(2, 2, 2);
imshow(BW);
title('Color Segmentation', 'FontSize', fontSize, 'Interpreter', 'None');
axis on;
drawnow;

% Clean up by filling holes and taking the larger blobs, bigger than 1000 pixels.
fruitMask = imfill(BW, 'holes');
fruitMask = bwareaopen(fruitMask, 1000);

% Display the image.
subplot(2, 2, 3);
imshow(fruitMask);
title('Final Mask', 'FontSize', fontSize, 'Interpreter', 'None');
axis on;
drawnow;

% Mask the image using bsxfun() function to multiply the mask by each channel individually.
maskedRGBImage = bsxfun(@times, rgbImage, cast(fruitMask, 'like', rgbImage));
% Display the image.
subplot(2, 2, 4);
imshow(maskedRGBImage);
title('Final Masked Image', 'FontSize', fontSize, 'Interpreter', 'None');
axis on;
drawnow;

% Extract the individual red, green, and blue color channels.
redChannel = rgbImage(:, :, 1);
greenChannel = rgbImage(:, :, 2);
blueChannel = rgbImage(:, :, 3);

% Get the mean color for each blob.
propsR = regionprops(fruitMask, redChannel, 'MeanIntensity', 'Centroid');
propsG = regionprops(fruitMask, greenChannel, 'MeanIntensity');
propsB = regionprops(fruitMask, blueChannel, 'MeanIntensity');
redMeans = [propsR.MeanIntensity]
greenMeans = [propsG.MeanIntensity]
blueMeans = [propsB.MeanIntensity]

% Define reference color
orangeColor = [234, 150, 60];
pearColor = [180, 194, 62];
% Loop over blobs finding out which color is the shortest distance
hold on;
for k = 1 : length(propsR)
  thisColor = [redMeans(k), greenMeans(k), blueMeans(k)];
  distanceToOrange = sqrt((thisColor(1) - orangeColor(1)) .^ 2 + ...
    (thisColor(2) - orangeColor(2)) .^ 2 + ...
    (thisColor(3) - orangeColor(3)) .^ 2);
  distanceToPear = sqrt((thisColor(1) - pearColor(1)) .^ 2 + ...
    (thisColor(2) - pearColor(2)) .^ 2 + ...
    (thisColor(3) - pearColor(3)) .^ 2);
  fprintf('For blob #%d,\n    Distance to Orange = %f, distance to pear = %f.\n', distanceToOrange, distanceToPear);
  xCenter = propsR(k).Centroid(1);
  yCenter = propsR(k).Centroid(2);
  if distanceToOrange < distanceToPear
    text(xCenter, yCenter, 'Orange', 'Color', 'r', 'FontSize', fontSize, 'FontWeight', 'bold');
  else
    text(xCenter, yCenter, 'Pear', 'Color', 'r', 'FontSize', fontSize, 'FontWeight', 'bold');
  end
end

function [BW,maskedRGBImage] = createMask(RGB)
%createMask  Threshold RGB image using auto-generated code from colorThresholder app.
%  [BW,MASKEDRGBIMAGE] = createMask(RGB) thresholds image RGB using
%  auto-generated code from the colorThresholder app. The colorspace and
%  range for each channel of the colorspace were set within the app. The
%  segmentation mask is returned in BW, and a composite of the mask and
%  original RGB images is returned in maskedRGBImage.

% Auto-generated by colorThresholder app on 08-Jun-2018
%------------------------------------------------------

% Convert RGB image to chosen color space
I = rgb2hsv(RGB);

% Define thresholds for channel 1 based on histogram settings
channel1Min = 0.000;
channel1Max = 1.000;

% Define thresholds for channel 2 based on histogram settings
channel2Min = 0.338;
channel2Max = 1.000;

% Define thresholds for channel 3 based on histogram settings
channel3Min = 0.000;
channel3Max = 1.000;

% Create mask based on chosen histogram thresholds
sliderBW = (I(:,:,1) >= channel1Min ) & (I(:,:,1) <= channel1Max) & ...
  (I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
  (I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
BW = sliderBW;

% Initialize output masked image based on input image.
maskedRGBImage = RGB;

% Set background pixels where BW is false to zero.
maskedRGBImage(repmat(~BW,[1 1 3])) = 0;

end

[Sónia Cova]

Thank you! It's works for this image but for other don't because this: Do you know why the values distance to pear don't calculate or don't show in command window?

[Image Analyst]

There would be lots of reasons. Colors are too close, like apples and pears can be the same color green. You chose the wrong reference colors, etc. I have no idea because you didn't attach the image it failed on.

[Sónia Cova]

in your code also does not appear the distance to pear in the command window.

but now I'm trying this image.

I created a mix of codes, to try to recognize the color and shape, which is this, I also think it does not work very well, because this sentences said that all fruits are pears!

[Image Analyst]

"in your code also does not appear." What does that mean? My code does appear. It's right there. I can see it.

And please read this link and fix your formatting or else attach the m-file with the paper clip icon.

[Sónia Cova]

in the comment from above I made the changes and put the .m file

[Image Analyst]

First of all, use higher resolution and don't use jpeg format.

Then you can use regionprops to get the major axis length and minor axis length to determine if the blob is an apple or a pear.

[Sónia Cova]

Should I take the images in tiff or png?

[Image Analyst]

I use PNG. I think most of the imaging community uses that format. But either will work.

[Sónia Cova]

can help with formulas to define circles and triangles using the major and minor axis?

[Image Analyst]

Something like (untested):

propsR = regionprops(fruitMask, redChannel, 'MeanIntensity', 'Centroid', 'MajorAxisLength', 'MinorAxisLength');
for k = 1 : length(propsR)
    eccentricity(k) = props(k).MajorAxisLength / props(k).MinorAxisLength;
    if eccentricity(k) > 1.2 % or whatever....
        uiwait(helpdlg('It is a pear'));
    else
        uiwait(helpdlg('It is an apple'));
    end
end

[yonatan shewazemed]

how can i make orange recognition using image segmentation?

[Image Analyst]

@yonatan shewazemed Try the Color Threshold app on the Apps tab of the tool ribbon.