Constrain Least Mean Square Algorithm

constrain least mean square with L1 and L2 constrains for regression problem

In this code, a linear equation is used to generate sample data using a slope and bias. Later a Gaussian noise is added to the desired output. The noisy output and original input is used to determine the slope and bias of the linear equation using constrain-LMS algorithm. This implementation of constrain-LMS is based on batch update rule of gradient decent algorithm in which we use the sum of error instead of sample error. You can modify this code to create sample based update rule easily. There are three options of constrain I implemented in this code 'None', 'L1', and 'L2'. You can also change input/noise signal distributions as well to see which constrain work best for which type of signal/noise.