Finding the optimal minimum of two vectors

I have two vectors (97x1) where each element is a standard deviation of a range of data from a larger vector (145x1). I want to find where the optimal solution is to minimize both standard deviations (so not the absolute minimum of each vector individually). I tried looking at some of the optimization tools but couldn't figure out how to apply them to this problem... Any help would be appreciated but please be specific since I am not an experienced programmer.

[Alan Weiss]

I'm not sure that I understand you. Is the question to choose the best 97 elements out of 145 elements? I mean, you have a way to figure out a "goodness" for a given set of 97 elements, and you want MATLAB to pick the best 97 out of 145? Or is there something else that you are trying to do?

Alan Weiss

MATLAB mathematical toolbox documentation

[Aaron Morrison]

Sorry, it's a little hard to explain. I want to find where two vectors are both minimized. Each vector will have its own minimum value but the location (row#) will be different for both vectors. Instead, I am trying to find where both vectors are optimally minimized at the same time. So that should be the same location (row#) in each vector. This location will not be the minimum value for either vector individually but instead, where the values are both relatively low. If that makes sense...

[Torsten]

So you try to minimize 1/2*(x1(i)+x2(i)) where x1,x2 are the two (97x1) vectors ?

Or which other "shared form of a minimum" do you have in mind ?

Best wishes

Torsten.

[John D'Errico]

This is not making any sense at all. Sorry, but still it is not. Almost always when that happens, it is because the person is trying to solve a problem they have not thought out themselves, or do not themselves understand. So it is not your fault, but a question of communication. And if we cannot understand what you are asking, then we cannot show how to write code for something that is not clearly understood.

A good way to resolve the problem is to give an example. Start with two SMALL vectors. Random numbers may be sufficient.

u = rand(1,5)
v = rand(1,6)

u =
   0.6004  0.56389  0.55587  0.58886  0.44195
v =
     0.13  0.44605   0.3695  0.35616  0.86208  0.6484

Now, what would you have us compute on those two vectors? Give an example! Without that, it becomes impossible to guess your intentions.

[Aaron Morrison]

Ok, I clearly can't explain this well enough. I'll try to show what I mean:

u = randi(10,[10,1]) %First vector
v = randi(10,[10,1]) %Second vector of same length
  u =       v = 
      10        9
       7        8
       8        7
       9        10
       6        3
       1        4  %min(u) is in this row
       5        5  
       3        2  %optimal minimum where both vectors are as low as possible when considered together
       4        1  %min(v) is in this row
       2        6
% I need some function or code that will take the input of u and v and output where that optimal minimum exists

Perhaps it would make more sense to concatenate the two vectors into a 10x2 matrix?

Answer 1

[John D'Errico]

Ok, I NEVER thought this was where you were going with your original question. That is why an example helps. Perhaps you have given enough information to make a stab at an answer.

But how do you define the optimum? [3 2] makes some sense, I suppose, but neither of 2 or 3 are the minimum values for either vectors. If it is the sum of the two numbers, then either of the rows with [1 4] or [4 1] are equally good as [3 2]. All three rows sum to 5.

So really, you need to define what it is that makes [3 2] "better" than the others. You may have some idea in the back of your mind, but I think you have not stated it firmly. I don't even think you know what rule you want to use.

I'll hazard a guess that perhaps the minimum you might want to see is the row that has a minimum sum of squares of the elements. At least that rule would allow you to select [3 2] over the others.

So, this will do most of what you want, all in one simple call. I've used the same vectors u and v that you showed.

[objmin,objind] = min(u.^2 + v.^2)
objmin =
    13
objind =
     8

So the minimum is 13, which occurs from the 8th pairing of elements. Here are all of those sums of squares:

u.^2 + v.^2
ans =
   181
   113
   113
   181
    45
    17
    50
    13
    17
    40

Now, the problem is, I have NO IDEA if this is what you really want. As well, there will be some vectors where the minimum is still not unique. But it does succeed in selecting the pair that you want selected.

I'm still a little worried, because you originally claimed the vectors were not the same lengths, and I have no idea how what you described in your comment reflects on your original question. This is how consulting often goes in the real world though. A client comes in, asking a question that they don't have formulated well. Then you need to tease their real needs out. It can be an iterative process that takes you all over the map.

[Aaron Morrison]

Thank you very much, this does appear to give me the results I was originally looking for. I thought that leaving out much of the larger context of my goal would make it easier to explain but clearly not. I will give you a bit more detail/background that might help things.

I am a geologist looking at large data sets from some instrument. The data I am dealing with is a .txt file (attached) that has columns of time, temperature, torque, RPM, and segment number (therefore each row corresponds to one data point). Time is the elapsed time in minutes at each data point, temp and torque are recorded for each time, RPM is the rotation rate at each data point (should be constant for each segment), and the segment number defines when the RPM changes (i.e. segment 6 = RPM 50, segment 7 = RPM 30, etc) or...

while segment number = 6
  RPM = 50 
end
while segment number = 7
  RPM = 30
end

What the end-goal is, is to find the ~10 min section (which is ~50 rows) where the temp AND torque are most stable (so where the standard deviations are the lowest). Herein lies the impetus for the original question. I had coded up to this point and got stuck on how to find the best (lowest standard deviation) ~10 min section of data that I wanted. Note: each row is a data point so I can't take the lowest std section of temp and the lowest std section of torque if the rows are not the same.

There are ~145 data points (rows) for each segment number which covers ~60 minutes. I then created a vector of standard deviations for 10 minute segments using the movestd function.

    winsz = I-t; %Size of the moving window (~50 rows, ~10 min)
    SEG_TEMP2 = Data_Ind(t3:I,2); %Temp section over last 30 minutes (from t3 to I) 
    window_T = movstd(SEG_TEMP2,winsz,'Endpoints','discard'); %vector containing stds from the ~10 min range of the moving window
    SEG_TORQUE2 = Data_Ind(t3:I,3);
    window_t = movstd(SEG_TORQUE2,winsz,'Endpoints','discard');

The movestd function creates the two 97x1 vectors I originally referred to. So I thought if I could figure out this "optimal minimum" we have been discussing (the row where the std(temp) and std(torque) are as low as possible), I can then determine the best ~10 minute section, since the rows represent a section of data from the movestd command.

Below is my code thus far if it helps clear anything up. The optimization section is what I am working on. The index of "k" refers to a larger for loop that will run the code for the number of files the user selects. I have only been using matlab for 3 months so forgive me if the code is sloppy and inefficient...it's all I know how to do at this point.

for m = (mode(SEGMENT):max(SEGMENT))
  %Load the data from individual segments
  load (['Data_Ind' num2str(k) '.txt']) %Load the individual segment file
  Data_Ind = eval(['Data_Ind' num2str(k)]);
  [I,J] = size(Data_Ind);

    %Define key times
    t1 = Data_Ind(I,1); %Define time at end of segment
    t2 = Data_Ind(:,1); %Define some other time during the segment
    tol = 0.5;
    time10 = find((t1-t2-10)<tol); %Find where t1-t2 is within the tolerance of 0.5
    t = min(time10); %Define the smallest time from the find command

    time30 = find((t1-t2-30)<tol); %Find where t1-t2 is within the tolerance of 0.5
    t3 = min(time30); %Define the smallest time from the find command

    SEG_TIME = t1 - Data_Ind(t,1);  %Define the time interval
    SEG_TEMP = Data_Ind(t:I,2);     %Temp over time interval
    SEG_TORQUE = Data_Ind(t:I,3);   %Torque over time interval
    SEG_SPEED = Data_Ind(t:I,4);    %Speed over time interval
    SEG_NUM = Data_Ind(t:I,5);      %Segment number over time interval

    %Optimization of STDEV
    winsz = I-t; %Size of the window (~50 rows, ~10 min)
    SEG_TEMP2 = Data_Ind(t3:I,2);  
    window_T = movstd(SEG_TEMP2,winsz,'Endpoints','discard');
    SEG_TORQUE2 = Data_Ind(t3:I,3);
    window_t = movstd(SEG_TORQUE2,winsz,'Endpoints','discard');

    [winmin,winrow] = min(window_T.^2 + window_t.^2); %Finds row of optimal minimum

    AVG_TEMP = mean(SEG_TEMP);      %Avg Temp
    STD_TEMP = std(SEG_TEMP);       %Stdev Temp
    AVG_TORQUE = mean(SEG_TORQUE);  %Avg Torque
    STD_TORQUE = std(SEG_TORQUE);   %Stdev Torque
    AVG_SPEED = mean(SEG_SPEED);    %Avg Speed (should be constant)
    AVG_NUM = mean(SEG_NUM);        %Avg Segment number

    %Output averages: seg#,time interval,avg temp,temp stdv,avg torque,stdv
    %torque,speed
    AVG_OUT = [AVG_NUM,SEG_TIME,AVG_TEMP,STD_TEMP,AVG_TORQUE,STD_TORQUE,AVG_SPEED];

    %Output the segment averages to new text file and append
    FID4 = fopen(['Data_Avg' num2str(k) '.txt'],'a+');
    fprintf(FID4,'%4.0f %8.2f %8.2f %7.3f %7.2f %7.3f %7.1f\n',AVG_OUT);
    fclose(FID4);
  end