How to classify images based on the amount of colors in the image?

I have a problem where all images have the same object; however, these objects can either have number_of_colors<=3 OR number_of_colors>3 and images are not labeled.

My attempt starts by converting RGB to LAB and Consider only the A & B to find the color coverage of that image. I was thinking of it as an area on the AB space. So for every image, I found the range of A and B (i.e max(A)-min(A), max(B)-min(B)) and multiplied them together to get the area, assuming it's a rectangle. Then I threshold using this feature.

Please let me know if intuition is correct and why it isn't working. Here is the confusion matrix:

• TP: 0.41935483871, FN: 0.0645161290323
• FP: 0.0967741935484, TN: 0.41935483871

Here is the basic routine That I have used:

    LAB = rgb_to_lab(data_rgb[:,:,0],data_rgb[:,:,1],data_rgb[:,:,2])
    A = LAB[1]
    B = LAB[2]

    minA,maxA = min(A),max(A)
    minB,maxB = min(B),max(B)

    diff_A = maxA - minA
    diff_B = maxB - minB

    area = diff_A * diff_B 
    % p is the normalized area
    p = area/(200*200.)
    % Found 0.53 to be the best possible threshold
    if p >0.53:
        print 'class 1 - colors > 3'
    else:
        print 'class 2 - colors <= 3'

Here is an image to show how the threshold separates positive and negative images. It only considers A and B values with luminance between 16 and 70 which seems to increase the area of separation reducing the FP by 1.

Please let me know if my intuition is correct and why it isn't working. I have no experience in image processing. I would love to know what is the standard way to do this. I could manually find the range of each color in Hue after converting to HSV but it seems too specific which might not handle all the colors in the test set.

I have attached 2 images from both classes for worst and best interms of their position in the graph (area value).

[Image Analyst]

I can help if you attach the original RGB image. Attach at least one image of each type.

[nawaf]

@Image Analyst: thanks! I have attached 2 images from both classes for worst and best interms of their position in the graph (area value).