mapSolution broken in R2019b?

In R2019a, the following code works as intended,

           %% Define the variables
           x=optimvar('x',{'x1','x2'},'LowerBound',[-100,-200]);
           y=optimvar('y',1,3,'LowerBound',[0 0 0],'UpperBound',[+300,+400,+500]);
  
           %% Define some constraints
           C.inequ=2*sum(y)+sum(x)<=1000;
           C.equ=5*sum(x)==100;
            
           
           prob=optimproblem('Constraints',C,'Objective',sum(x)+sum(y));
           
           idx=mapSolution(prob,1:5)
           

resulting in the output,

    idx = 
    
      struct with fields:
    
        x: [1 2]
        y: [1 2 3]

Something must have been changed in R2019b/R2020a, however, because I instead get an error:

    Not enough input arguments.
    
    Error in optim.internal.problemdef.ProblemImpl/mapSolution
    
    Error in test (line 12)
               idx=mapSolution(prob,1:5)

Were there any intentional changes in the behavior of the mapSolution function, and if so how do I get the code above to work in current Matlab?

[Walter Roberson]

https://www.mathworks.com/help/optim/ug/optim.problemdef.optimizationproblem.mapsolution.html

shows a number of mandatory inputs

[Matt J]

There is a qualifying Note there, however, showing that certain subsets of the input argument list are allowable... Why would the two-input combination no longer be allowable based on what's written there, and considering also that it clearly did work in R2019a?

[Walter Roberson]

Darn, it is a .p file so we can't read it.

Answer 1

[Paul Kerr-Delworth]

Hi Matt and Walter,

Firstly Matt, many apologies that you've run into this unintended behavior. Also, many thanks to you both for looking into this and flagging it up.

What has happened here? The short answer is that mapSolution was made an internal function in R2019a and replaced with varindex. Once we made mapSolution internal, we repurposed it in R2019b leading to the internal error you encountered. We will tidy up the error and add a note in the documentation pointing you to varindex

Matt's original code works with varindex

%% Define the variables
x=optimvar('x',{'x1','x2'},'LowerBound',[-100,-200]);
y=optimvar('y',1,3,'LowerBound',[0 0 0],'UpperBound',[+300,+400,+500]);
  
%% Define some constraints
C.inequ=2*sum(y)+sum(x)<=1000;
C.equ=5*sum(x)==100;
            
           
prob=optimproblem('Constraints',C,'Objective',sum(x)+sum(y));

idx = varindex(prob);

Resulting in the output

idx = 

  struct with fields:

    x: [1 2]
    y: [3 4 5]

Also, varindex allows you to get the variable index by name, e.g.

idx = varindex(prob, 'y')

Resulting a double as the output

idx =

     3     4     5

Matt, may I ask if you were aware of varindex? Also, is there anything in mapSolution that you are relying on that varindex doesn't provide?

Thanks again for flagging this and hope varindex helps!

Cheers,

Paul

Documentation for varindex:

https://uk.mathworks.com/help/optim/ug/optim.problemdef.optimizationproblem.varindex.html

[Matt J]

Thanks, Paul. I was not aware of varindex, but regardless, I need a solution which is back compatible with pre-R2019 Matlab versions.

[Paul Kerr-Delworth]

Hi Matt,

Thanks for accepting the answer and for your comment - much appreciated!

Just to clarify, I guess ideally you'd like mapSolution to work as it did in R2019a and earlier. Would you be happy if there were another function that mapped the solution output from the Solver-Based interface to a solution structure that is returned from the Problem-Based interface? Or, do you really just want mapSolution back?

In the meantime, would this solution work for you? You'd need to add this function to your path and replace all your mapSolution calls to mattMapSolution, but this should reproduce mapSolution.

function solstruct = mattMapSolution(prob, solvec)
idx = varindex(prob);
varNames = fieldnames(idx);
for i = 1:length(varNames)
    solstruct.(varNames{i}) = solvec(idx.(varNames{i}));
end

Cheers,

Paul

[Matt J]

Hi Paul,

What I am really looking for is a way to implement varindex for people with older versions of Matlab (R2018b and earlier), but which will also continue to work in R2019a and later. The solution I mentioned in my answer below,

solstruct = mapSolution(prob,solvec,[],[],[],[],[])

appears to be a sufficient workaround, although it is a bit unattractive that I need to supply solvec at all. For me, solvec will always be 1:N, where N is the total number of unknowns in the problem, so it would be nice to have a way, for preR2019 Matlab versions, to get N directly from prob.

Thanks again for your help!

[Paul Kerr-Delworth]

Hi Matt,

Many thanks for the clarification, that's really helpful for us.

For a backwards compatible solution that can work, you can add your own version of varindex to the path as a separate function

function solstruct = varindex(prob)

if ~isa(prob, 'optim.problemdef.OptimizationProblem')
    error('First input to VARINDEX must be an OptimizationProblem');
end

if verLessThan('matlab', '9.6')
    error('This function is intended for R2017b - R2018b. Make this a separate function on the path.')
end

N = sum(structfun(@numel, prob.Variables));
solstruct = mapSolution(prob, 1:N);

end

Now for R2017b-R2018b, this function will be called as varindex doesn't exist. From 19a onwards the shipping OptimizationProblem method will be called as it is higher in the precedence order than your varindex.

https://uk.mathworks.com/help/matlab/matlab_prog/function-precedence-order.html

Would you be happy to move to varindex with this solution?

Cheers.

Paul

[Matt J]

Thanks,Paul. That looks great. And yes, varindex is what I would use.

Answer 2

[Matt J]

This seems to be a simple work-around,

idx=mapSolution(prob,1:5,[],[],[],[],[])

but I can't help wondering if it's really supposed to be required. It is very uncharacteristic of TMW-supplied commands to require trailing [] input arguments.

[Walter Roberson]

Internal code does not do detailed argument checking for performance reasons.