TimeLimit option in Patternsearch being ignored?

Hi everyone,

I have a problem which I am solving via the patternsearch function. Essentially, I have an objective function (func) which includes a parfor loop, using the parallel computation toolbox and a (large) parpool (several hundred cores). I have an issue in that I need to ensure that this runs within a certain time frame, which is proving difficult and I'm not sure why.

When I set the time-limit parameter in the options, patternsearch does not adhere to it. It will eventually time out, but the period is approx 2-4 times longer than the option set. The same set up in a different solver (ga) adheres exactly, without an issue.

Does anyone know how or why this is occurring? Any suggestions?

My intuition is that it has to do with the communication between the parpool agents, similar to a documented issue with ga when used in parallel, but since this is not a parallel computation I don't think this can be the same issue.

For example, here are the same problem being run with the same time-limit in patternsearch and ga:

PS:
Time limit reached: increase options.TimeLimit.
Elapsed time is 84952.876117 seconds.
GA: 
Optimization terminated: time limit exceeded.
Elapsed time is 36458.878759 seconds.

[jgg]

As an aside, it's not very efficient to try and benchmark a single objective evaluation, then set a "time limit" by using the number of evals instead for a couple of reasons:

• The method used involves simulation, so the function evaluation time is not stable
• This is being run on a large cluster, and due to some idiosyncracies in the way it is being scheduled the number of cores requested does not always precises match the number of cores in the parpool cluster. This is a problem on our end, but this means that function evaluation time is also unstable in this way as well.
• The cluster is shared, so it's very important that we ask for just exactly what we need. Over-requesting time leads to inefficient server use, while under-requesting will cause the scheduler to kill our process before completion, which would be very bad.

A possibility could be to build in some kind of way to output the solution when the scheduler kills the process, but I'm not clear how to do that, and it's probably not desirable behaviour in any case.

Answer 1

[Alan Weiss]

I am very sorry to say that you got bit by a bug. We will fix it as soon as we can.

Here is the bug. patternsearch uses the cputime function to measure the amount of time that has passed. However, when run in parallel, this measurement is done for the host machine, not the parallel workers. So the host machine sends work to the workers and does not incur cputime while waiting for the results. As a result of this behavior, as you found, patternsearch does not believe that much time has passed, because, when run in parallel, it ignores the time that the workers take. So it does not respond to the time limit you set, a real bug.

Again, I apologize for the bug. The fix will be to use tic and toc to measure time. If you are brave, you can try to edit patternsearch to take a more sensible view of time using tic/toc.

Alan Weiss

MATLAB mathematical toolbox documentation

[jgg]

Hi Alan

Thanks for the feedback; I figured it was something like that.

I will give your suggestion a try, and wait for the update in the meantime.