use xlread to read csv file with mixed type data is very slow
2 views (last 30 days)
Show older comments
Hi
I have a csv file with mixed data type like below, first column is string, 2nd column is numerical, 3rd column tothe end columns is date string. Total row number is 25000.
I used xlread() shown below to read the two csv files. it tooks me 10 minutes to read two csv file. The first csv file has only 10 columns while 2nd file has 400 columns. I am wondering if it is possible to speed it up using different approaches? Thanks
[num_steamdate,txt_steamdate,raw_steamdate] =xlsread(filename_WellDate);
[num_fracdate,txt_fracdate,raw_fracdate] =xlsread(filename_WellFracDate);
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/352613/image.png)
0 Comments
Answers (1)
Cris LaPierre
on 27 Aug 2020
I would suggest using readtable instead. It creates a table, which supports mutiple data types. If you need help, consider sharing your data files. I'm not feeling motivated enough to transcribe your screenshot for testing.
16 Comments
Cris LaPierre
on 29 Aug 2020
Ok, it appears you cannot use implicit expansion on datetime arrays. You must make sure both arrays have the same number of rows AND columns. You need to repeat the column of wellfracdate_array to match the width of wellsteamdate_array (391 columns). You can do this with repmat.
filter_bfFrac_c1=and(wellsteamdate_array >= repmat(wellfracdate_array(:,1),[1,size(wellsteamdate_array,2)]),...
wellsteamdate_array < repmat(wellfracdate_array(:,2),[1,size(wellsteamdate_array,2)]));
filter_bfFrac_c2=and(wellsteamdate_array >= repmat(wellfracdate_array(:,3),[1,size(wellsteamdate_array,2)]),...
wellsteamdate_array < repmat(wellfracdate_array(:,4),[1,size(wellsteamdate_array,2)]));
filter_bfFrac_c3=and(wellsteamdate_array >= repmat(wellfracdate_array(:,5),[1,size(wellsteamdate_array,2)]),...
wellsteamdate_array < repmat(wellfracdate_array(:,6),[1,size(wellsteamdate_array,2)]));
filter_bfFrac_c4=and(wellsteamdate_array >= repmat(wellfracdate_array(:,7),[1,size(wellsteamdate_array,2)]),...
wellsteamdate_array < repmat(wellfracdate_array(:,8),[1,size(wellsteamdate_array,2)]));
filter_bfFrac_c5=and(wellsteamdate_array >= repmat(wellfracdate_array(:,9),[1,size(wellsteamdate_array,2)]),...
wellsteamdate_array < repmat(wellfracdate_array(:,10),[1,size(wellsteamdate_array,2)]));
filter_onFrac_c1=(wellsteamdate_array == repmat(wellfracdate_array(:,2),[1,size(wellsteamdate_array,2)]));
filter_onFrac_c2=(wellsteamdate_array == repmat(wellfracdate_array(:,4),[1,size(wellsteamdate_array,2)]));
filter_onFrac_c3=(wellsteamdate_array == repmat(wellfracdate_array(:,6),[1,size(wellsteamdate_array,2)]));
filter_onFrac_c4=(wellsteamdate_array == repmat(wellfracdate_array(:,8),[1,size(wellsteamdate_array,2)]));
filter_onFrac_c5=(wellsteamdate_array == repmat(wellfracdate_array(:,10),[1,size(wellsteamdate_array,2)]));
See Also
Categories
Find more on Time Series Objects in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!