Compiled MATLAB with TORQUE

7 views (last 30 days)
Danny
Danny on 3 Jan 2013
I need to process a lot of data in parallel, and then plot the result of that processing. I have access to a cluster (~100 cores), and currently run the parallel data processing using compiled matlab code (mcc -mv), and then using qsub. What would be optimal, however, is to be able to call qsub from within a matlab script, have it submit the jobs to the cluster, wait until they are all completed, and then continue plotting, processing, in serial, in matlab. Is such a thing possible? If so, any help would be great.
Thanks!

Accepted Answer

Jason Ross
Jason Ross on 3 Jan 2013
If you have Parallel Computing Toolbox and MATLAB Distributed Computing Engine installed/configured on the cluster, you can set up a direct integration using the Parallel menu (this assumes a number of things -- you have qsub on the path, and a common file system). If you don't have those things, there are examples in matlabroot/toolbox/distcomp/examples that detail how to set up the integration between MATLAB and your cluster for a variety of scenarios.
Once the integration is set up, you can submit the jobs to the cluster using various commands and techniques (batch, job/task, spmd, parfor) from within MATLAB that are detailed in the Parallel Computing Toolbox documentation.
  1 Comment
Danny
Danny on 8 Jul 2013
My school does not provide neither the parallel computing tool box or the distributed computing toolbox to clusters. Thank you for the help, though.

Sign in to comment.

More Answers (1)

Muthu Annamalai
Muthu Annamalai on 8 Jul 2013
You may alternatively consider writing a "monitor" thread or app using the MATLAB timer() functionality.

Categories

Find more on Startup and Shutdown in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!