Mex file still too slow

I have created a mex file for my matlab functions as follow:

% Create a MEX configuration object
cfg = coder.config('mex');
% Turn on dynamic memory allocation
cfg.DynamicMemoryAllocation = 'AllVariableSizeArrays';
% Generate MEX function
codegen -config cfg MYfunction -args {Data,Mask}
tic
result = MYfunction_mex(Data,Mask);
toc

But it is still too slow for what I want to do:

    telapsed = 2262.62 sec  % normal Matlab function
    telapsed =  670.64 sec % mex function

It is an improvement but I was expecting much higher speed up as this is running not the complete for loop I would need. The 3rd for loop is running from 1:50 instead 8760, so it would take more than 1 day to run the entire code! I know that there are certainly many ways to speed it up but I can't see them.

As "MYfunction" is about my research I cannot share the code but the basic things are:

for l = 1:4                      

      for n = 1:24

          for i = 1:50     % i shoudl go till 8760  

              INDEXvector = DataSelectionINDEX(inputs);    % function that create an array of indices to select specific data of MyData.mat file

              X = MyData(INDEXvector(:),:);

              % Covariance matrix
              [R,P] = CalcCOVARIANCEmatrix(X);   % function that calculates the covariance matrix

              %  do other minor calculations  %

              Result = "from previous calculations";
          end;
      end;
  end;

I know that the function that is more time consuming is the one in which the cross-correlations of my data are computed. But I don't understand why it is still so slow and how can I further speed it up. Thank you!

[dpb]

So how big is X and how much memory do you have?

Sometimes things are just time consuming and that's the way it is...

I don't see how anybody could possibly be expected to tell you anything specific about speeding up this code with no more information than this.

The one thing regarding optimization is that almost always the way to get significant speedup is not optimizing a given set of coding operations but in finding alternative algorithms or, the largest savings of all, finding things that may be being computed that are invariant to the looping construct and hence, not calculating those at all (or more than once, more precisely).

[Alice Malvaldi]

Sorry I forgot to specify that X is 52590x23 and I have 8GB of RAM (if this was your question).

Anyway, I know that some problems are just time consuming and nothing can be done but I think that in my case it's probably my poor skills at programming that make it even slower than it could be.

I also know that I shared very few things but, as I said, I cannot share much more about my code and I thing that the key things are there.

Thanks.

[dpb]

"...I cannot share much more about my code and I thing that the key things are there."

Well, I guess that's research... :) Good luck.

[Ryan Livingston]

It is hard to tell from the example code snippet but if your code lends itself to parallelism, you could consider using PARFOR if the loop iterations are independent.

Answer 1

[Rick Rosson]

Please try:

   cfg.DynamicMemoryAllocation = 'off';

Also:

   cfg.IntegrityChecks = false;
   cfg.ResponsivenessChecks = false;

[Alice Malvaldi]

Thank you very much!

I tried it and it's better, even though I had to use

cfg.DynamicMemoryAllocation = 'Threshold';

instead of

cfg.DynamicMemoryAllocation = 'off';

because otherwise I had this error: "Computed maximum size exceeds maximum allowed number of elements etc...".

Many thanks!

[Seyedmohammadhadi Sadati]

Thanks.