How to extract sub-matrices from a big matrix?

Hello everyone,

I have two matrices, kindly find the attached file

soho ---> size = 23×12 , each row represents an event.

omni ---> size = 8760 × 17 , each row represent an hour (timeseries data).

The first three columns are, in order, "year", "month", "day".

I need to do the following operation:

if soho_day = omni_day

take that omni_day and the following 120 rows (hours) and put them as a separate matrix with a prefix (i.e., omniSUB_1).

and so on

I have these code blocks

% to find the matched date and store the that day along with the following 5 days (120 hours) in another matrix. 
for n = 1:length(soho) 
    for m = 1:length(omni) 
        if datetime(soho(n,1),soho(n,2),soho(n,3),...
                'Format','dd/MM/yyyy') == ...
                datetime(omni(m,1),omni(m,2),omni(m,3),...
                Format','dd/MM/yyyy') 
            for k = m:120 
                omniSUB{n,:} = omni(k,:); 
            end 
        end
    end 
end 

and

% to create sub-matrices. 
Prefix = 'omniSUB_'; 
for i = 1:Month_length 
    var_name = strcat(Prefix, num2str(i)); 
    data_child = genvarname(var_name); 
    eval([data_child ' = omniSUB{i}']); 
end 

but I don't know how to put them together to do that operation. Please correct me.

I appreciate your help.

Thanks in advance

[Stephen Cobeldick]

"...put them as a separate matrix with a prefix (i.e., omniSUB_1)."

Ugh, why do you need to do such an ugly thing like that?

Having numbered variables is a sign that you are doing something wrong.

Accessing variable names is a sign that you are doing something wrong.

Your approach will force to writing slow, complex, inefficient, buggy, and difficult to debug code. Is there a particular reason why you cannot just use the simple and efficient indexing that you are already using to access your data?

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

[Mohamed Nedal]

soho contains data about storms and omni contains time-series data of the variables affected by such storms.

So basically what I want to do is finding how these variables change over a time period of 5 days starting from the storm's starting time and separate each storm's data in a matrix to proceed further analysis.

That's why I need to separate each 5-day period in a separate matrix because I have another function that will loop over these matrices and do further analysis for each one.

For me, it's okay if the code is slow or complex as long as it gets the job done :)

[Stephen Cobeldick]

"That's why I need to separate each 5-day period in a separate matrix because I have another function that will loop over these matrices and do further analysis for each one."

Exactly as I wrote in my earlier comment, you just need to use indexing.

You can easily store the separated data in one array (e.g. a cell array) using basic MATLAB indexing, which will be much simpler and more efficient than what you are doing now, and then it is trivial to "loop over these matrices and do further analysis for each one" as you wrote. So far nothing you have shown or described requires your complex and inefficeint approach of accessing variable names dynamically.

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

[Mohamed Nedal]

I'll try the indexing approach. Thanks!

Answer 1

[Ridwan Alam]

% to find the matched date and store the that day along with the following 5 days (120 hours) in another matrix. 
for n = 1:size(soho,1) 
    omniRowInd = find(omni(:,1)==soho(n,1) & omni(:,2)==soho(n,2) & omni(:,3)==soho(n,3),1,'first');
    if ~isempty(omniRowInd)
        tempTable = omni(omniRowInd:min(omniRowInd+119,size(omni,1)),:);
        eval(['omniSUB_' num2str(n) '=' 'tempTable;']);
    end
end

Hope this helps!

[Stephen Cobeldick]

"Hope this helps"

What really helps is showing other MATLAB users how they can write simpler, neater, less buggy, and much more efficient code, e.g. by avoiding slow and complex constructs such as eval used to access variable names.

[Mohamed Nedal]

Perfect! that's what I need.

Thank you very much

[Mohamed Nedal]

Hey Stephen, Could you suggest a simpler, neater version?

[Stephen Cobeldick]

"Perfect! that's what I need."

Most likely you have not considered the inherent problems with your approach. For example, how do you detect which of the omniSUB_XXX arrays have been magically created or not (depending on that if condition)? You wrote in your comment that you want to "do further analysis for each one", but how will you even know which of those matrices exist in order to process them? How will you ensure that you have not missed one?

"Could you suggest a simpler, neater version? "

Just use indexing.

N = size(soho,1);
C = cell(1,N);
for k = 1:N
    omniRowInd = ...
    if ~isempty(omniRowInd)
        C{k} = ...
    end
end

And then it is also trivial to know which of those submatrices have been created or not:

cellfun(@isempty,C)

And of course looping over those matrices for your "further analysis" is trivial:

for k = 1:numel(C)
    M = C{K};
    ... do whatever analysis with M
end

[Mohamed Nedal]

Thank you Stephen!

Excuse me could you tell what does "a" here refer to?

C = cell(a,N); 

[Stephen Cobeldick]

"Excuse me could you tell what does "a" here refer to?"

My mistake, it should be 1.

I fixed my earlier comment to avoid confusion.

[Mohamed Nedal]

Thank you very much!

[Mohamed Nedal]

Hi Stephen, I tried your code with much larger dataset, kindly find the attached file.

N = size(soho,1); 
C = cell(1,N); 
for k = 1:N 
    omniRowInd = find(omni(:,1)==soho(k,1) & omni(:,2)==soho(k,2) & omni(:,3)==soho(k,3),1,'first'); 
    if ~isempty(omniRowInd) 
        C{k} = omni(omniRowInd:min(omniRowInd+119,size(omni,1)),:); 
    end 
end 

but the resulted C{k} is empty. Could you determine the problem here plz?

[Stephen Cobeldick]

"Could you determine the problem here plz? "

Are you checking all of the cells of the cell array, not just the last one?:

find(~cellfun(@isempty,C))

It is quite possible that the if condition is never true. You can check this by printing it in the loop, or collect it into a vector and checking it after the loop:

N = size(soho,1); 
C = cell(1,N);
L = false(1,N); % logical vector
for k = 1:N 
    omniRowInd = find(omni(:,1)==soho(k,1) & omni(:,2)==soho(k,2) & omni(:,3)==soho(k,3),1,'first'); 
    if ~isempty(omniRowInd)
        L(k) = true; % if the condition is true
        C{k} = omni(omniRowInd:min(omniRowInd+119,end),:); 
    end 
end
% Check on which iteration/s the conditions was true:
any(L)
find(L)

Note how easy it is using indexing to access the new variable L. Trying to use eval to do something similar with lots of numbered variables would be much more complex.

[Mohamed Nedal]

Yes, you're right.

I found where's the error, it was a glitch in defining the date-time pairs and I corrected it.

Thank you!