Smooth noisy data whilst keep absolute minimum and maximum values

I have a reasonably noisy signal (almost sinusoidal) that I am smoothing using sgolayfilt.

I am using a polynomial of 2 with a frame length of 901.

I am happy with the smoothing using this frame length.

However, there are several parts of the signal that need to have (correctly) a minimum value of zero.

When I smooth the signal the value of the minima are no longer zero. The lowest value of the smoothed curve at the minima takes a non-zero value, that might be positive or negative.

If I "shift" the curve then only one of the minima is zero.

Is there a method to smooth curves whilst keeping absolute minimum (and maximum if required) values? Or is it always a trade-off (I could trace the signal by eye very well, using a drawing package, keeping the minima, but I want to automate the process with matlab)

regards

Answer 1

[Image Analyst]

In spectroscopy there are a number of baseline correction algorithms that do what you're asking. Google it. Or search this forum for "baseline correction".

If you have any more questions, then attach your data and code to read it in with the paperclip icon after you read this:

TUTORIAL: How to ask a question (on Answers) and get a fast answer

Answer 2

[Star Strider]

One option is to use the islocalmin function (probably with the 'MinProminence' name-value pair), fit a polynomial (likely low-order) to those points, evaluate that polynoimial with the entire independent variable vector and then subtract that result from the entire dependent variable vector. This also allows you to selectively add the beginning and end points (and others as necessary) to the initial islocalmmin logical vector. I have used this in the past with good results, however I don’t know what your signal is. You’ll probably have to experiment with this a bit to get the result you want.

EDIT — (26 Sep 2024 at 11:22)

Illustrative example of this approach —

Fs = 250;
L = 10;
t = linspace(0, Fs*L, Fs*L+1).'/Fs;                                 % Time Vector
s = sin(2*pi*t*0.75)+randn(size(t))*0.1 + cos(2*pi*0.25*t)*0.5;     % Noisy Signal With Varying BaSeline

figure
plot(t,s)
grid
xlabel('Time')
ylabel('Voltage')
title('Original Signal')

Lv = islocalmin(s, 'MinProminence',0.75);                           % Identify Minima
Lv([1 end]) = true;                                                 % Include Endpoints
NrPts = nnz(Lv)                                                     % Information, Also Used To Create Polynomial (Order)

NrPts = 9

figure
plot(t, s, 'DisplayName','Signal')
hold on
plot(t(Lv), s(Lv), 'sr', 'DisplayName','Identified Minima')
hold off
grid
xlabel('Time')
ylabel('Voltage')
title('Signal With Identified Minima')
legend('Location','best')

p = polyfit(t(Lv), s(Lv), NrPts-1);                                 % Fit Polynomial
vc = polyval(p, t);                                                 % Evaluate Polynomial
sc = s - vc;                                                        % Correct Signal Baselinee

figure
plot(t, s, 'DisplayName','Signal')
hold on
plot(t, vc, '-g', 'DisplayName','Baseline Polynomial')
hold off
grid
xlabel('Time')
ylabel('Voltage')
title('Signal With Baseline Polynomial')
legend('Location','best')

figure
plot(t, sc)
grid
xlabel('Time')
ylabel('Voltage')
title('Baseline-Corrected Signal')

If your signal has less noise than in this example, you should get even better results.

.

Answer 3

[Umar]

Hi @Bill White ,

You have to understand the challenging part when you apply smoothing techniques like sgolayfilt, the algorithm works by fitting a polynomial to a specified window of data points. This process can lead to a shift in the local minima, as the polynomial fit may not respect the original data's constraints, particularly if those constraints are not explicitly defined in the smoothing process. So, here are some possible solutions to help you out.

Post-Smoothing Adjustment: After applying the smoothing function, you can adjust the smoothed signal to ensure that the local minima are set to zero. This can be done by identifying the local minima and then shifting the entire signal downwards by the minimum value.

Custom Smoothing Function: Instead of using sgolayfilt, you could create a custom smoothing function that incorporates constraints for the minima. This would involve more complex programming but could yield better results tailored to your specific needs.

Using islocalmin: As this function is suggested by @Star Struder, it can be employed to identify the local minima in your smoothed signal. Once identified, you can adjust the signal accordingly.

Below is an example code snippet that demonstrates how to smooth a noisy sinusoidal signal while preserving the local minima using the sgolayfilt function and the islocalmin function.

% Generate a noisy sinusoidal signal
t = linspace(0, 10, 1000); % Time vector
signal = sin(t) + 0.1 * randn(size(t)); % Noisy signal

% Apply Savitzky-Golay filter
frameLength = 901; % Frame length
polynomialOrder = 2; % Polynomial order
smoothedSignal = sgolayfilt(signal, polynomialOrder, frameLength);

% Identify local minima in the smoothed signal
TF = islocalmin(smoothedSignal); % Logical array of local minima
localMinima = smoothedSignal(TF); % Values of local minima

% Adjust the smoothed signal to ensure minima are zero
minValue = min(localMinima); % Find the minimum value
if minValue < 0
  adjustedSignal = smoothedSignal - minValue; % Shift up to make minima zero
else
  adjustedSignal = smoothedSignal; % No adjustment needed
end

% Plotting the results
figure;
plot(t, signal, 'b', 'DisplayName', 'Noisy Signal'); hold on;
plot(t, smoothedSignal, 'r', 'DisplayName', 'Smoothed Signal');
plot(t, adjustedSignal, 'g', 'DisplayName', 'Adjusted Signal');
legend show;
xlabel('Time');
ylabel('Amplitude');
title('Smoothing a Noisy Signal While Preserving Local Minima');
grid on;

So, as you can see in above code snippet, a noisy sinusoidal signal is created using sin and random noise. Then, sgolayfilt function is applied to smooth the signal. Afterwards, islocalmin function identifies the local minima in the smoothed signal. If the minimum value of the smoothed signal is less than zero, the entire signal is shifted upwards to ensure that the local minima are zero. Finally, original noisy signal, the smoothed signal, and the adjusted signal are plotted for comparison.

Please see attached.

Hope this helps. Please let us know if you have any further questions.