When you have a ga parameter that is not an integer data-type, then ga will automatically try continuous parameter values between the upper and lower bound for the parameter. There is no way to tell ga to only try the parameter by 0.1 increments.
If you need a parameter to be a multiple of a fraction f (such as 1/10) then use integer parameter there, but divide the upper and lower bound for that position by f. Then inside of your objective function, multiply the received value by f.
For example
f = 0.1;
ga(@(x) (x*f - pi)^2, 1, [], [], [], [], -1/f, 5/f, [], 1) * f
ga stopped because the average change in the penalty function value is less than options.FunctionTolerance and the constraint violation is less than options.ConstraintTolerance.
ans =
3.1
The objective here was to find the multiple of 0.1 that is closest to pi