How to include a maximum error constraint of e.g. 10% when using polyfit?

Hi guys,

I am interested to include a pre-defined max error of let's say 10% when using polyfit in my data. More specifically, I am interested to start from the left corner of my X and Y data, and find the earliest sample point (closer to the left corner) that satisfies a constrain of <10% error between my polyfit and my data.

I guess there may be ways to assess the fit on a step by step mode, and then isolate which part of the curve first satisfies this constraint, but I believe there will be a more efficient and concise way to do so.

I am attaching some of my X and Y data for your convenience.

Any ideas?

Thank you,

George

[Catalytic]

How big do your X,Y vectors get? Certainly with the example provided (720 points) there is no reason to pursue better efficiency than a naive step-by-step loop.

[Walter Roberson]

The 10% has to apply over all of the data, right? So all of the data needs to be read and calculated with before you can figure out the boundary.

[GioPapas81]

Yes, thank you both. A question here is, how do you calculate the % error for polyfit?

If I will do:

[p,S]=polyfit(X_data,Y_data,1);

and then:

[f, delta]=polyval(p,X_data,S);

The delta does not give me this % error, but the "standard error in predicting a future observation for each x point", according to Mathworks. So, if I understand well here, I need to use some other way to get these % errors for each polyfit fitting?

[dpb]

" how do you calculate the % error for polyfit?"

For what definition of "error", pray tell? It certainly isn't clear to me what you're driving at here...

[GioPapas81]

If 0% means a perfect fit between polyfit and my X and Y data, then a maximum of 10% error means that I am allowing only the fits that are less than that, if that makes sense.

Is there any way I could get the residuals from the polyfit?

[John D'Errico]

I might try to answer this question, but I have no idea what you intend here.

You can compute the error of a particular point. That is just

err(i) = y(i) - ypred(i)

percent error? Also trivial.

perc_err(i) = 100*(y(i) - ypred(i))/y(i);

So percent error is a relative thing. Note that polyfit is NOT designed to produce a relative error fit.

But again, each of those things are an error for a particular point. To talk about the percent error for the polynomial itself seems to have no meaning that I can think of. You may know what you want, but making up some jargon that makes sense only to you will not get a useful answer. You need to CLEARLY explain what you are trying to do. Again, remember that polyfit is NOT designed to work in terms of a relative error metric. So the fit that polyfit does produce will probably not be targeted at your goal.

You then ask about wanting to find a fit that maintains some goal % error over some region of the curve. Again, polyfit is not designed to essentially truncate the data until some goal on the fit is reached. Before you even try to do such a fit though, you need to define what this percent error should mean.

[Walter Roberson]

I think it could make sense to talk about max(abs(perc_err)) being 10 (%).

I am wondering if the question is along the lines of:

polydegree = 6;   %for example
numpoint = length(X);
found = false;
for idx = polydegree : numpoint
    [p, s] = polyfit(X(1:idx), Y(1:idx));
    ypred = polyval(p, X, s);
    rel_err = abs((Y - ypred)./Y);
    if max(rel_err) <= 0.10
        found = true;
        break
    end
end
if ~found
    fprintf('Your polynomial cannot be fit to within 10% using degree %d\n', polydegree);
else
    fprintf('Success at point #%d\n', idx);
end

... but I have the sneaking suspicion that the they want to keep increasing the degree...

Answer 1

[GioPapas81]

Thank you Walter and John. Combining your answers, I think it is poosible to perform a step by step assessment (as John said), set some tolerance (as Walter said), and accept only the part of the curve that has <10% errors. Still, the quantification differences that I am getting between including and not including this tolerance is very small.

Thank you again,

George