How can I plot mixed data types of a csv file ?

I'm working on a .csv file that contains one column of mixed data types , the coulmn looks something like:
,"Admit","Gender","Dept","Freq"
1,"Admitted","Male","A",512
2,"Rejected","Male","A",313
How can I read the data from the file and plot them in a reasonable way to determine appropriate features for a classification problem ?
I've tried to use txt2str , csvread , and textscan functions beside some file format conversion functions as well (xlsread ,[num,txt,raw] ) and non of them worked with my data!!
Any good suggestions would be appreciated .

1 Comment

If you have the Statistics Toolbox, create a nominal variable array from the non-numeric fields
doc nominal % and friends
If you don't have the toolbox, convert manually. unique could be of interest, perhaps to create the number of categories for each.
After the above, then you can do something with those values.

Sign in to comment.

 Accepted Answer

You should use the new readtable() function, available in R2013b and later. It can handle different data types. It goes like
t = readtable(fullFileName);
If you want to separate the values, you can do that:
Admit = t.Admit;
Gender = t.Gender;
and so on.

3 Comments

SIDEBAR
IA, what do you think of R2013b and later?
I've resisted updating as R2012b practically brought my old machine to its knees and I figured anything else would be totally unusable. I did add memory to the limit (4G is all this dude will hold) and that helped some.
Also, I've heard complaints over the newer interface...I'm not keen on the R2012b look at all already and if it's more of a departure from traditional as I've not updated (and have no intentions of doing so since retired from active technical consulting) the OS or any other apps I don't much care to have something wildly different...I still have some old DOS apps I use regularly that won't run under anything past XP and am not giving them up and don't want to try to mess w/ dual-boot or the like, either, really...
I know, I'm a closet Luddite... :)
I have no problem with the new interface and I like the new version R2014a. I guess I'd gotten used to the ribbon toolbar from a year or so of using Microsoft Office first. I think the new version has several things that are very nice and worth upgrading for. It has the new table data type, which is just fantastic. Plus it has a better way for selecting editor windows when you have lots of them open, like I usually do, and there's too many to show along the top. The command line intellisense is also improved to popup a list of prior commands right there in the command window if you type a letter and hit the up arrow key, so you can go right to the one you want instead of hitting up arrow a dozen times. The main negative is that the report tools (like dependency report) are hidden away on the Current Folder panel under a tiny triangle rather than being on the main tool ribbon. Lots of other nice features but I don't remember which are different from an old version. Can't say much about the memory since I have 8 GB and soon to have 32 GB RAM and 1.5 TB of solid state drive (hopefully within the month).
Although I'm gonna have to upgrade to the newest version to try what you suggested , but I do believe it would work smoothly .
IA , thank you for your suggestion :)

Sign in to comment.

More Answers (1)

Presuming do have the Statistics Toolboxen...
>> [a,g,d,f]=textread('bara.csv','%*d%s%s%s%d','headerlines',1,'delimiter',',')
a =
'"Admitted"'
'"Rejected"'
g =
'"Male"'
'"Male"'
d =
'"A"'
'"A"'
f =
512
313
>> an= nominal(a)
an =
"Admitted"
"Rejected"
>> cnt=hist(an)
cnt =
1
1
>>
etc., etc., etc., ...

15 Comments

Unfortunately ; I don't have the statistics toolBox !!
I don't know if what you suggested works without statistics toolBox , but I've tried to do so and it gives me this error :
*??? Error using ==> dataread Trouble reading integer from file (row 1, field 1) ==> "1","Admitted","Male","A",512\n
Error in ==> textread at 176 [varargout{1:nlhs}]=dataread('file',varargin{:});*
Is there any way to fix it ??
BTW ; this is just a sample I'm working on and there are some other csv files that needs to be plotted ... I thought that working out the problem in one of them would be a good solution to the others since they're all have alphanumeric data types .
If the file is like the one you posted, the textread format worked as demonstrated here. Need the actual line of code as used and anything that's different in the file itself to tell why it failed.
NB: the same error occurred here if I tried to remove the quote marks so since they're just categorical data I just left them in.
On no Stat Toolbox, then we'll convert the data to nominal w/o the class; not quite as clever but it'll work just fine. Let's figure out where the problem in the data file is first off...
I've tried the following despite of knowing it's not the smartest way to solve the issue , but it seems to work fine :
fid = fopen('sample.csv', 'r');
tline = fgetl(fid);
% Split header
A(1,:) = regexp(tline, '\,', 'split');
% Parse and read rest of file
ctr = 1;
while(~feof(fid))
if ischar(tline)
ctr = ctr + 1;
tline = fgetl(fid);
A(ctr,:) = regexp(tline, '\,', 'split');
else
break;
end
end
fclose(fid);
n=A(1:25,1)
data1=strrep(n,'"','')
... ...
I think I have overcome the data read thing , Now I can't plot the textual attributes against other numerical ones !!
Any suggestions will be always appreciated .
I'd still like to see the actual file lines that failed and the textread and/or textscan that failed...surely can't be that tough.
As far as the categorical data, as above once it's read, unique is your friend. I duplicated the two rows you gave in obverse order to have
>> a
a =
'Admitted'
'Rejected'
'Rejected'
'Admitted'
>>
Then,
>> [ua,~,ic]=unique(a)
ua =
'Admitted'
'Rejected'
ic =
1
2
2
1
gives the two category names and their locations in the index vector. Use the latter for the analysis variables and associate w/ the appropriate string for output. Piece o' cake... :)
Bara'a
Bara'a on 21 Apr 2014
Edited: Bara'a on 21 Apr 2014
Here I tried to convert the file into xlsx file format , though it didn't provide me with the access to individual elements in the resulted array so I can plot them :
Here's another try that throws an error :
fid=fopen('UCBAdmissions.csv','r');
c=textscan(fid,repmat(%s,1,10),'delimiter',',','CollectOutput',true);
Also , I've tried textread as well :
[a,g,d,f]=textread('UCBAdmissions.csv','%*d%s%s%s%d','headerlines',1,'delimiter',',')
celldisp(f)
Where is the problem in those cod pieces ? and how can I plot all the attributes in the file for a classification problem decision ?
Thanks in advance :)
Bara'a
Bara'a on 21 Apr 2014
Edited: Bara'a on 21 Apr 2014
dpb ...
I know it's one of the basics in MatLab to do such tasks , but I'm completely new to this whole thing !!
I will try the unique function you suggested , I hope it could work things out .
Thanks a lot for the enormous help you provided :)
your effort is always appreciated .
fid=fopen('UCBAdmissions.csv','r');
c=textscan(fid,repmat(%s,1,10),'delimiter',',','CollectOutput',true);
Also , I've tried textread as well :
[a,g,d,f]=textread('UCBAdmissions.csv','%*d%s%s%s%d','headerlines',1,'delimiter',',')
Again, need an actual sample data file record or three and the resultant error in context.
What I note in particular is you've got 10 string fields in one format and the numeric and three strings and numeric in the other for the same file.
ADDENDUM:
I DAGS and was unable to find a simple way to simply download the file--it's in an R dataset but I don't have R and don't want to go to the trouble to deal with it...figured somewhere there would be a link to raw data file, but it didn't show up. If you've a convenient spot or attach or post it on a sharing site I'd look at the file itself and see what's the issue.
ADDENDUM 2:
Another search found a shortened version of 24 records--it looks just like yours excepting the first column in also a quoted string instead of numeric. I downloaded it and the following worked just fine (changing the first '%*d' to '%*s', of course)...
>> type ucbadmissions.csv
"","Admit","Gender","Dept","Freq"
"1","Admitted","Male","A",512
"2","Rejected","Male","A",313
"3","Admitted","Female","A",89
"4","Rejected","Female","A",19
"5","Admitted","Male","B",353
...
"20","Rejected","Female","E",299
"21","Admitted","Male","F",22
"22","Rejected","Male","F",351
"23","Admitted","Female","F",24
"24","Rejected","Female","F",317
>> fmt=['%*s' repmat('%s',1,3) '%3d'];
[a,g,d,f]=textread('ucbadmissions.csv',fmt,'headerlines',1,'delimiter',',');
>> a=strrep(a,'"','')
a =
'Admitted'
'Rejected'
'Admitted'
'Rejected'
'Admitted'
...
'Rejected'
'Admitted'
'Rejected'
>>
Bara'a
Bara'a on 21 Apr 2014
Edited: Bara'a on 21 Apr 2014
Here's the repository from where I obtained my sample data file :
Attached is the sample file it self .
Again , this is just an experiment to perform the task , and there are other data files .
dpb ... would you please write your latter reply as an answer to this question so I can accept it :)
I just ran this code and it ran fine:
folder = 'C:\Users\Bara\Documents';
baseFileName = 'UCBAdmissions.csv';
fullFileName = fullfile(folder, baseFileName)
t = readtable(fullFileName)
All the data are in variable t:
t =
Var1 Admit Gender Dept Freq
____ __________ ________ ____ ____
'1' 'Admitted' 'Male' 'A' 512
'2' 'Rejected' 'Male' 'A' 313
'3' 'Admitted' 'Female' 'A' 89
'4' 'Rejected' 'Female' 'A' 19
'5' 'Admitted' 'Male' 'B' 353
'6' 'Rejected' 'Male' 'B' 207
'7' 'Admitted' 'Female' 'B' 17
'8' 'Rejected' 'Female' 'B' 8
etc....
Great !! that worked perfectly fine , thanks a lot ... but still losing the point of the question !!
I haven't achieved the task completely , as the original question was about plotting the mixed data types of a csv file , though you have done your best so far and I'm so grateful for your precious time .
So far , we have gained the access to the data ... how can I plot them ?
I don't know what you want to plot. How about a bar chart showing the number of admitted and rejected people broken down by Gender? Specify what column you want plotted vs. another column. Or something else, like a histogram or something. I just don't know what to plot. Did you mention that yet? If so, I must have missed it, so please tell me again.
It's the same (identical) answer as the first... :)
On plotting, and analyzing these kinds of data is where (imo) Matlab falls somewhat short of SAS, R and other statistical analysis systems in not having builtin categorizing keywords. You can always "get there from here" but it may take some additional effort/data rearrangement. Perhaps the new table structure that IA is enamored with does have the added ability; it has some features that undoubtedly will make such simpler.
Actually, that makes me realize I'm forgetting about the (relatively) new dataset class...let's see what mayhaps can do therein--I've not fiddled with it much yet meself...
>> UCB=dataset({nominal(a),'Admit'},{nominal(g),'Gender'},{nominal(d),'Department'},{f,'Freq'})
UCB =
Admit Gender Department Freq
Admitted Male "A" 512
Rejected Male "A" 313
Admitted Female "A" 89
Rejected Female "A" 19
Admitted Male "B" 353
Rejected Male "B" 207
Admitted Female "B" 17
Rejected Female "B" 8
Admitted Male "C" 120
Rejected Male "C" 205
Admitted Female "C" 202
Rejected Female "C" 391
Admitted Male "D" 138
Rejected Male "D" 279
Admitted Female "D" 131
Rejected Female "D" 244
Admitted Male "E" 53
Rejected Male "E" 138
Admitted Female "E" 94
Rejected Female "E" 299
Admitted Male "F" 22
Rejected Male "F" 351
Admitted Female "F" 24
Rejected Female "F" 317
>>
Ooohh!!!! :) I don't yet know what can do with it, but it's kewl, nonetheless. Unfortunately, I've got another commitment at the moment but maybe you can figure out something from here.
ADDENDUM:
I see I hadn't yet removed the double quotes from Department but that's easy enough to handle.
Thank you for the hint ... I will try to figure out the rest of the way by my self !! Don't want to be demanding :)
Well, I've not had time to actually try to learn much about the dataset methods other than just a (very) quick perusal thereof. Oh, bummer!! I went back and realized you said you don't have the Stat Toolbox so unless it's also in some other tooboxen you do have you're stuck with the basic Matlab cell/structure data types it would seem.
That makes going back to the original idea I showed of making your own nominal/ordinal variables by associating the string value with a level via unique then working with those numeric values.
The real pita in Matlab is that, unlike SAS, BMDP, R, the other really "statistics-aware" packages, there is no FREQUENCY variable, even in the Statistics Toolbox. Hence, you'll have to either
a) duplicate the data from the summary table form given to replicate each record type the frequency number of times to use builtin Matlab functions over those (a real waste of memory, of course), or
b) compute the proportions numerically from the frequencies themselves.
Again, it just isn't that convenient to do these kinds of analyses in Matlab unless there are some major tricks that aren't being revealed in the Stats Toolbox doc's...

Sign in to comment.

Categories

Asked:

on 17 Apr 2014

Commented:

dpb
on 24 Apr 2014

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!