You can use imclearborder to suppresses structures that are lighter than their surroundings and that are connected to the image border.
Also, you can specifiy the connectivity parameter in imclearborder according to your requirements.
conn = 4;
The above code uses 4-connected neighborhood and does not consider pixel at (5,2) to be connected with the border pixel at (4,1), so it will not be cleared whereas the below code uses 8-connected neighborhood and does consider pixel at (5,2) to be connected with the border pixel at (4,1).
conn = 4;
Otherwise, if you want to remove a region around the point (x,y).
idx = img(y, x);
img(img == ind) = 0;
And if you want to exclude objects that touch the a particular border (top/left/bottom/right) then consider using imclearborder after padding the particular edge of the image first so that they are not touching the border.
For further references, please look into this link.
Hope it helps.