How to interpolate between two adjacent points of time series data, if the difference between any two consecutive points is greater than 25 by using for loop?

I have a time series data like this;

x = [555 554 100 10 36 20 17 5 51 70 101 40 13];

and independent variables as follow

L = numel(x);

fs = 4;

t=0:1/fs:(L-1)/fs;

What I want is to use spline interpolation between two consecutive point of 'x' means (xi and xi+1) if the difference between two point is greater than 25. for example from the above time series abs(x4-x3)= 10-100= 90 therefore I want to interpolate between x3 and x4 with t interval of 0.05 and return all the new time series data by using a loop. The loop have to continue until the difference between every consecutive data in “x” becomes less than 25.

Thanks for your contribution in advance

[J. Alex Lee]

Is there any reason you cannot just live with interpolating the whole curve with the new time interval?

[Yared Daniel]

I want only the gap which is greater 25 to be interpolated, the other gap which is less than 25 has to be as it is.

But if the gap is interpolated at time interval 0.05 and still the gap between the interpolted points is greater than 25 you can chnage the point of interpolation to more finer time interval.

Thanks for asking

Answer 1

[Jan]

x  = [555 554 100 10 36 20 17 5 51 70 101 40 13];
L  = numel(x);
fs = 4;
t  = 0:1/fs:(L-1)/fs;

ready = false;
dt    = 0.05;
xx    = x;
tt    = t;
while ~ready
   miss = find(abs(diff(xx)) > 25);
   if isempty(miss)
      ready = true;
   else
      % Insert new time values for gaps:
      for k = numel(miss):-1:1
         tt = [tt(1:miss(k)-1), ...
               tt(miss(k)):dt:tt(miss(k)+1), ...
               tt(miss(k)+2:end)];
      end
      
      xx = interp1(t, x, tt, 'spline');  % Using original x, not xx!
      dt = dt / 10;  % Decrease time interval for next iteration
   end
end

figure;
axes;
plot(t,  x,  'ro');
hold('on');
plot(tt, xx, 'b.');

[J. Alex Lee]

nice.

if you are addressing 1 gap at a time, and you can further assume that the curve is monotonic in the gap (i guess this would depend on properties of splines using the information outside of the interval of interest), then you can exactly control the spacing in x by interpolating t on a fixed x-grid

[Yared Daniel]

Dear Jan,Thank you so much for such a wonderful contribution.

[Yared Daniel]

One more question Mr Jan, Does it have effect if I decrease the rate by which time interval decreases?

[Jan]

Yes, of course: If you decrease the rate, it is smaller. ;-) Choose the factor of decreasing such, that it satisfies your needs. my dt=dt/10 was a pure guess only.

[Yared Daniel]

Ok, Thank you I will adjust the rate based on my interest.

What about the computation speed?

I tried for longer array (1 by1800) it took almost 2 minute for 1 array, I have thousands of such types of array which I need to compute the same computation. Is there any means to speed up the computation speed?

[Jan]

interp1 is slow. A griddedInterpolant will be faster.

If the input is large, the iterative growing of tt is a bad idea.

Maybe this solution is a bad strategy. So please explain, which problem you want to solve actually. If all want want to achive is a list of numbers with gaps < 25, there are better methods.

[Yared Daniel]

Yes please forward that

Answer 2

[J. Alex Lee]

Here's [another approximate] way to do it that will only call spline once (could as well do it with gridded interpolant)

The method I suggested above won't work with spline because the interval between t = 0.5 and t = 0.75 is not monotonic after the global spline fit.

So this method is still only approximate to satisfy dx_min.

x  = [555 554 100 10 36 20 17 5 51 70 101 40 13];
L  = numel(x);
fs = 4;
t  = 0:1/fs:(L-1)/fs;

% do the spline fit to the whole set of points once
pp = spline(t,x)

pp = struct with fields:
      form: 'pp'
    breaks: [0 0.2500 0.5000 0.7500 1 1.2500 1.5000 1.7500 2 2.2500 2.5000 2.7500 3]
     coefs: [12×4 double]
    pieces: 12
     order: 4
       dim: 1

% define the minimum gap in x you want
dxmin = 25

dxmin = 25

x_agaps = abs(diff(x))

x_agaps = 1×12

1 454 90 26 16 3 12 46 19 31 61 27

t_agaps = abs(diff(t))

t_agaps = 1×12

0.2500 0.2500 0.2500 0.2500 0.2500 0.2500 0.2500 0.2500 0.2500 0.2500 0.2500 0.2500

dx_mult = 1.2; % extra refinement tweak

% find the intervals where your gap is violated
intvls = find(x_agaps > 25);

% for each interval, generate 
tc = cell(numel(intvls),1);
xc = cell(numel(intvls),1);

for i = 1:numel(intvls)
% for i = 2
	
	idx = intvls(i);
	
	% split the interval into n points such that
	% ROUGHLY minimum gap is satisfied (assuming linear)
	
	n = ceil(x_agaps(idx)/dxmin * dx_mult);
	tt = linspace(t(idx),t(idx+1),2+n);
	
	xx = ppval(pp,tt);
	
	tc{i} = tt(2:end-1);
	xc{i} = xx(2:end-1);
	
end

tI = horzcat(t,tc{:});
xI = horzcat(x,xc{:});

[tI,srtIdx] = sort(tI);
xI = xI(srtIdx);

% analyze violations there are...
abs(diff(xI))

ans = 1×51

1.0000 18.3053 19.0709 19.7458 20.3298 20.8232 21.2258 21.5376 21.7587 21.8891 21.9286 21.8775 21.7356 21.5029 21.1796 20.7654 20.2605 19.6649 18.9785 18.2013 17.3335 16.3748 15.3254 14.1853 43.5096 28.2867 15.9281 6.4338 0.1962 3.9620

idxViol = find(abs(diff(xI))>dxmin)

idxViol = 1×2

25 26

figure(1); cla; hold on;
plot(t,x,'.','MarkerSize',12)
plot(tI,xI,'o')
fplot(@(t)(ppval(pp,t)),t([1,end]),'-')
plot(tI(idxViol),xI(idxViol),'x','MarkerSize',9,'LineWidth',2)

[Jan]

Of course, this is an efficient idea: My code computes the spline interpolation for the same intervals repeatedly, but computing te spline onces and insert only additional interpolation steps is leaner and faster.

Answer 3

[J. Alex Lee]

If you just want the points to be "appear" roughly evenly distributed, you could consider spacing in the arc length (although you'd have to re-scale one of the dimensions so that it looks "even" in the plot)

x = [555 554 100 10 36 20 17 5 51 70 101 40 13];
L = numel(x);
fs = 4;
t=0:1/fs:(L-1)/fs;

% interpolate more than you ultimately need
tI = linspace(t(1),t(end),2000)';
xI = interp1(t,x,tI,'spline');

% scale by
% scl = 200;
scl = max(xI)/max(tI)

scl = 227.0709

% parameterize the ultra-refined curve w.r.t. its own arclength s
sI = [0;cumsum(sqrt(scl^2*diff(tI).^2+diff(xI).^2))];

% then interpolate both x and t w.r.t. s
% in the steepest part of the curve, dx ~ 25 corresponds to
se = linspace(sI(1),sI(end),50)';
te = interp1(sI,tI,se);
xe = interp1(sI,xI,se);

figure(1); cla; hold on;
plot(t,x,'x')
plot(te,xe,'o')
plot(tI,xI,'-')

[Yared Daniel]

Thank you so much for the detailed exlplanation, Let me check it with with my orginal data