Weird output image after applying mask

Hello.

So I'm doing some edge detection and want to apply it only to a specific region of interest (ROI). At first everything worked and you could draw a ROI with the mouse and a mask applied a canny filter on the selected bit, detect circles and cut out the rest of the picture.

For some reason the image with the mask applied started showing glitch like artifacts as seen in the picture below.

On the right is the picture with a Gauss-filter and on the left you can see the cut out area in the middle, which is the freehand-ROI, and on the left of it you can see the weird artifacts. The code also detects circles in the wrong spots, on some pictures it detects circles where there is nothing.

Thank you for any answers!

My code:

k2=imread("b3.jpg");

t1 =0.26; %sliders for thresholds and standard deviation for Gaussfiltering
t2 =0.09; 
s = 4.98; 
method1 = "canny"; %drop down menu of filters including sobel, prewitt etc.
% method2 = "canny";
k=rgb2gray(k2); 
k=imgaussfilt(k,s); 
% k=imsharpen(k);
% % imboxfilt(k);
% % imshow(k)
% % BW=edge(k, method1); 
% BW=edge(k,"canny",[t2 t1]);
imshow(k)
hax = drawfreehand(gca);
mask = createMask(hax);
% f = @(x)edge(k,method2);
g = @(x)edge(k, method1, [t2 t1]);
% res = roifilt2(k, mask, f);
res = roifilt2(k, mask, g);


figure;
imshowpair(res,k,"Montage")



n=1;
[centers, radii, metric] = imfindcircles(schritt2,[10 1000]); 
coords=centers(1:n,:);
rad=radii(1:n);
met=metric(1:n);
viscircles(coords, rad, "EdgeColor",'b');

Answer 1

[DGM]

The output of edge() is logical. When roifilt2() goes to composite the filtered and original images, this causes the non-filtered area to be converted to logical as well. The often-problematic behavior of casting images to logical is that logical() considers all non-zero values to be 1.

x = linspace(0,1,10)

x = 1×10

0 0.1111 0.2222 0.3333 0.4444 0.5556 0.6667 0.7778 0.8889 1.0000

a = logical(x)

a = 1×10 logical array
   0   1   1   1   1   1   1   1   1   1

It's not really an image processing tool, and so it's unaware of image data range conventions or where a more useful threshold might be. You'll get a thresholded image, but it will often look completely unrelated to the original image, enough to be confusing.

There were only a few locations in the black region which were actually zero, so that's all that's left of the image. You can fix this by making sure that the output class of your filter function is the same as the image being processed.

g = @(x) im2uint8(edge(k, method1, [t2 t1]));

[Leonard Krist]

Thank you very much for explaining and giving me a fix!