How to interpret criterion value of sequential forward feature selection algorithm?

I am using the sequential forward feature selection algorithm. In the output, it gives the feature number that it has added and also its respective criterion value. My question is, will the criterion value keep decreasing throughout as the less important a feature is, less will be the criterion value? The reason why i am asking this question is that I plotted the criterion value according the order in which they were added to the final feature list, and I found that initially the criterion value decreased but then again after some values it started to increase. I got U shaped curve. Why did the criterion value started to increase all of a sudden?