Running anonymous functions in parallel avoiding comunication overhead

Question

0 votes

Hi,

I want to run a set of anonymous function with different input data.
My code strucutre looks like this:
    hugeData = loadData(); 
    nJobsW = 10;
    obj =Composite();
    for w = 1:numel(obj)
        objW = cell(nJobsW,1);
        for i = 1:nJobsW
            funData = importFunData(w,i);
            objW{i} = @(x) fun(x,workerData,hugeData);
        end
        obj{w} = objW;
    end
    result = Composite();
    condition = true;
    while condition
        spmd
            x = getX(result);
            result = cell(nJobsW,1);
            for i = 1:nJobsW
                result{k} = obj{i}(x);
            end
        end
        condition = testCondition(result);
    end

So as you can see, there is a large constant data, and data per worker-job. This code is consuming a lot of memory, much more than if executed sequentially.

I suspect that hugedata is being copied many times.

My questions are: If all workers are local (same machine), is hugeData being copied? Can I avoid copying hugeData? How do I check what is consuming memory? How do I minimize the overall data overhead?

Thanks