Parfor loop slicing question.

Can someone tell me how to slice the array "data" that is within the parfor loop in the code below?:

          parfor i=1:32 %fprintf('Now on year %d\n',i);
            %             data((1+(366*(i-1)):366*i),1)=1979+(i-1);
            %             data((1+(366*(i-1)):366*i),2)=1:366;
            data((1+(366*(i-1)):366*i),3)=squeeze(motemp.TMIN(loni,lati,:,i));
            data((1+(366*(i-1)):366*i),4)=squeeze(motemp.TMAX(loni,lati,:,i));
            data((1+(366*(i-1)):366*i),5)=squeeze(mowind.WIND(loni,lati,:,i));
            data((1+(366*(i-1)):366*i),6)=squeeze(moshum.DATA(loni,lati,:,i));
            data((1+(366*(i-1)):366*i),7)=squeeze(morad.RADIATION(loni,lati,:,i));
        end

FYI The array 'data' is 11712x15.

If you want to see the full code go here: http://pastebin.com/H3HEwAwR

Thanks and much appreciated.

I'm willing to paypal someone a little cash if they can help me out here.

[Michael]

Maybe I'm missing something and it's hard to work without being sure of the dimensions of the input, but I think this is possible without the need for a loop since it's just reshaping and concatenating data.

eg. rather than squeezing each input matrix (motemp.TMIN for example) down to a single dimension, squeeze it to a matrix for that particular loni and lati, which I presume will give you a 32-by-366 matrix, then reshape it to get an entire column of "data" without needing to loop over i.

Maybe this is not what you want, I'm not an expert but I have a feeling you're using lots memory bandwidth when you send each worker the entire input (motemp,mowind,moshun,morad) and ask it to return a section of it.

[andrew joros]

You are correct Michael. Don't know why I didnt think of that. LOL. I need to do this for 224x464 lat lon points (103936 points total) though. How would I go about doing this in a parallel process to speed up the time? Ideas?

[Michael]

Ah, that is tricky. The result would be 103936 .mat files? I would suggest running profiler to find where most time is spent (perhaps for 5-10 lat/long points) because I have a bad feeling the majority of it will be spent writing the large files to storage which would be considerably slower than any matrix reshaping done by the CPU (and would also probably not benefit from parallel architecture within the CPU).

[andrew joros]

Yes I will be making ~104k .mat files. I am thinking parfor wont be any fast than a regular for-loop. Right?

Answer 1

[Edric Ellis]

Looking at your complete code, I would strongly recommend that you try to apply PARFOR to the outermost loop that gives you sufficient parallelism (i.e. has at least as many iterations as you have workers). This is generally the most efficient. It looks to me like either of the 2 outermost loops would be reasonable candidates.