How to smooth discontinuous data with a moving average filter

I would like to perform a moving average filter or equivalent to data of the form shown below in yellow, in order to draw comparison to the blue data.

The blue data is uniformly sampled. However, the yellow data discontinuously sampled with multiple values at certain times.

Can this be achieved using the matlab 'filter' command to perform a moving average filter, using weightings to account for the intermittent nature of the data? Otherwise, how might this data be smoothed effectively to give the same time resolution as the blue data?

Thanks.

I have managed to use the 'smooth' command to account for the gaps in the data by averaging the duplicate values and specifying the corresponding x locations for the smoothing operation:

yy = smooth(x,y)

Is this the best approach?

Answer 1

[Star Strider]

I discourage the use of moving-average filters. They don’t give you adequate control over the frequency content of your signal. It’s better to design a filter specific for your signal.

I would take the fft of your data to see what frequencies constitute the signal, and what frequencies constitute the noise. Then for your data, design a low-pass filter to eliminate the noise and use the filtfilt function to do the filtering. There are several ways to design filters in MATLAB, my procedure is here: How to design a lowpass filter for ocean wave data in Matlab?

[Mark McAllister]

Hi thanks for you reply. I've chosen to use a time domain filter for various reasons in this instance. The data are independent, and noise is not the result of particular frequency band. I'm looking for a trend in the time domain which is why I have chosen this method.

[Star Strider]

I would experiment with a low-pass filter as a first approximation. It can give you information about your signal.

The next step after the low-pass filter, or if a low-pass filter doesn’t do what you want, would be the Savtizky-Golay filter (the sgolayfilt function). All signal processing requires experimentation, so with a bit of tweaking, the Savitzky-Golay filter should do what you want.

Answer 2

[Image Analyst]

A simple linear moving average can be done with conv(), which is the same as sgolayfilt() if you pass sgolayfilt an order of 1. Lowess, in the Curve Fitting Toolbox I believe (which I don't have) is a version of a moving average that is more robust to outliers, which seem to affect your data a fair amount.

Answer 3

[Chad Greene]

I'd use interp1 to get the yellow data to even spacing, then use Aslak Grinsted's moving function to get the moving average.

[Elsa Quicazan]

Do you know if Aslak Grinsted's "moving" function is a "simple moving average" or a "weighted moving average"? Thanks.

[Image Analyst]

Why not simply use conv() like I suggested? You can either have "simple" if all the weights are the same, or weighted if the kernel values are different.

Answer 4

[Mark McAllister]

It would appear that using the smooth command, allows for the implementation lowess and Savitzky-Golay filters alongside a simple moving average, which can be used for data of non uniform spacing. This seems like the best method in this case.