# How to replace outliers with NaN

37 views (last 30 days)
012786534 on 22 Aug 2019
Answered: Steven Lord on 22 Aug 2019
Hello,
I am trying to replace values above the 99th percentile (outliers) by NaN for each group (for both group A and group B) in a table t.
group = repelem(['A' 'B'], 1000)';
val = repelem(1:1000, 2)';
t = table(group, val);
unique_gr = unique(t.group);
for g = 1:length(unique_gr)
sub = t(strcmp(t.group, unique_gr(g, 1)), :);
f = filloutliers(sub.val, 'NaN', 'percentiles', [0 99])
end
Ideas ? Please note that I do not have any toolboxes.

Walter Roberson on 22 Aug 2019
Use unique with three outputs and iterate through the group numbers,
[unique_gr, ~, groupnum] = unique(t.group);
for g = 1 : size(unique_gr,1)
end
012786534 on 22 Aug 2019
Thank you Walter, work like a charm

Steven Lord on 22 Aug 2019
You can use grouptransform with an anonymous function that calls filloutliers. Let's use your sample data.
group = repelem(['A' 'B'], 1000)';
val = repelem(1:1000, 2)';
t = table(group, val);
This grouptransform call uses the variable group from the table t as the grouping variable. The anonymous function is the same as what you used and Walter each used in your for loops, though I chose to replace it with the double NaN rather than the text 'NaN' like Walter did.
t2 = grouptransform(t, 'group', ...
@(x) filloutliers(x, NaN, 'percentiles', [0 99]));
Let's see what values of val in t were replaced by NaN in t2.
t(isnan(t2.val), :)
By the way you built t, those do look like the top 1% of values for each group.