Mean Squared error is very large

I am calculating mean squared error between previous and current frame for the factor of 0.5 and 0.25. However the error for 0.25 is very big compared to 0.5 Tne only difference between both is that when using factor of 0.5 I am calucltaing the motion compensated position using below for loop.

blocksize = 4;
factor = 0.5;
max_motion = 8;
rows_scaled_max = 1437;
cols_scaled_max = 2557;
ny = 1;
for ulhc_y = 1 : blocksize : rows % ulhc_y is the upper left hand corner in y direction
     nh = 1;
      for ulhc_x = 1 : blocksize : cols % ulhc_x is the upper left hand corner in x direction
         x = ulhc_x : ulhc_x + blocksize - 1;
         y = ulhc_y : ulhc_y + blocksize - 1;
         for x_vec = -max_motion : factor : max_motion
             for y_vec = -max_motion : factor : max_motion
                 xx = (x + x_vec) / factor - 1;% xx is the motion compensated position for x vector
                 yy = (y + y_vec) / factor - 1;% xx is the motion compensated position for y vector
                  if min(min(xx)) < 1 || max(max(xx)) > cols_scaled_max || ...
                           min(min(yy)) < 1 || max(max(yy)) > rows_scaled_max
                  end
                  continue;
             end
         end
      end
end

I suspect the problem is in below part of code for factor 0.25

blocksize = 4;
factor = 0.25;
rows_scaled_max = 1437;
cols_scaled_max = 2557;
ny = 1;
for ulhc_y = 1 : blocksize : rows % ulhc_y is the upper left hand corner in y direction
    nh = 1;  
    for ulhc_x = 1 : blocksize : cols % ulhc_x is the upper left hand corner in x direction
        x = ulhc_x : ulhc_x + blocksize - 1;
        y = ulhc_y : ulhc_y + blocksize - 1;
        xx = ((x + mv_x - 1) / factor) + 1;% xx is the motion compensated position% mv_x is the motion vector obtained using codec
        yy = ((y + mv_y - 1) / factor) + 1;% yy is the motion compensated position% mv_y is the motion vector obtained using codec
        if isnan(mv_x) || isnan(mv_y)|| ...
                min(min(xx)) < 1 || max(max(xx)) > cols_scaled_max || ...
                min(min(yy)) < 1 || max(max(yy)) > rows_scaled_max
                xx = ((x - 1) / factor) + 1;
                yy = ((y - 1) / factor) + 1;
        end
                
        end
    end
end

Any suggestion please.