My optimal theta vector stays null when using fminunc
Show older comments
I've recently enrolled in the Coursera machine learning, and am working my way through making my own classifier for the Iris dataset problem. I'm training a classifier for each species (the one-vs-all method). The code runs smoothly without any errors, but the results are not quite what I was expecting. Here's my code:
clear, clc
% loading the data
data = csvread('data.csv');
% extracting the features
X = data(:, (1:end-3));
X = [ones(size(X, 1), 1) X];
% computing the number of training examples (m) and features (n)
[m, n] = size(X);
% extracting the labels
y_setosa = data(:, end-2);
y_versicolor = data(:, end-1);
y_virginica = data(:, end);
% initializing the options for optimization algorithm
options = optimset('GradObj', 'on', 'MaxIter', 10);
% creating the initial theta vector
iniTheta = zeros(n, 1);
% tarining a classifier for each label
[theta_setosa, min_setosa] = fminunc(@(theta) costFunction(theta, m, X, y_setosa), iniTheta, options);
[theta_versicolor, min_versicolor] = fminunc(@(theta) costFunction(theta, m, X, y_versicolor), iniTheta, options);
[theta_virginica, min_virginica] = fminunc(@(theta) costFunction(theta, m, X, y_virginica), iniTheta, options);
Here's the code for my costFunction:
function [jValue, gradient] = costFunction(theta, m, X, Y)
V = X*theta;
jValue = (-1/m)*sum(Y.*log(sigmoid(V)) + (1-Y).*log(sigmoid(1-V)));
gradient = (-1/m)*X'*(sigmoid(V)-Y);
end
And finally my sigmoid function:
function sigX = sigmoid(X)
sigX = arrayfun(@(n) 1/(1+exp(-n)), X);
end
The resulting theta_setosa and theta_versicolor vectors are all zeros, and the costFunction has the same minumum for the both of them. the virginica classifier seems to be working fine however. Although for each fminunc I get the following message displayed
Local minimum possible.
fminunc stopped because it cannot decrease the objective function
along the current search direction.
<stopping criteria details>
Any idea what I may be doing wrong?
Answers (0)
Categories
Find more on Statistics and Machine Learning Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!