Error using parallel.F​uture/fetc​hOutputs in Denoise Speech Using Deep Learning Networks Example

Hello everyone,

I am trying to run the Matlab example to Denoise Speech Using Deep Learning Networks but I keep receiving an error at this line:

[targets,predictors] = gather(targets,predictors);

This line is expected to Use gather to evaluate the targets and predictors.

Although the tall expression is 100% being evaluated using the Parallel Pool, it still spits out some errors after evalutation.

Here's the full output when running the above line:

Evaluating tall expression using the Parallel Pool 'local':
- Pass 1 of 1: Completed in 4 min 31 sec
Evaluation 100% complete

Error Output:

Error using parallel.Future/fetchOutputs (line 346)
Internal problem while evaluating tall expression. The problem was:

One or more futures resulted in an error.

Error in parallel.internal.bigdata.ParallelPoolBroadcastMap>iRunOnAll (line 150)
    [varargout{1:nargout}] = fetchOutputs(f);

Error in parallel.internal.bigdata.ParallelPoolBroadcastMap/synchronizeWorkerBroadcasts (line 112)
            [newWorkerKeys, newWorkerValues] = iRunOnAll(@obj.getBroadcasts, keys);

Error in ...

Error in tall/gather (line 50)
[varargout{:}, readFailureSummary] = iGather(varargin{:});

I am using Matlab R2019a Update 9.

Does anyone know what I am possibly doing wrong?

[Kojiro Saito]

It seems the same problem as this. You might get out of memory error. What is RAM size of your computer?

[Daniel Graham]

it's 8GB RAM

Answer 1

[Kojiro Saito]

It might be out of memory error.

tall array can handle big data, but as tall/gather document says,

"MATLAB® can run out of memory if the result of the gather calculation is too large."

In this example, when I placed a break point after gather command, 7.4GB memory is occupied by MATLAB. 6.6 GB is used for MATLAB desktop process and other memories are used by Parallel workers.

So, 8GB RAM is not enough for executing this example.

I think there are two solutions.

• Try it on another machine which has more RAM.
• Reduce the audio files which were unzipped in tempdir\commonvoice (in Windows, it would be C:\Users\USERNAME\AppData\Local\Temp\commonvoice).

[Daniel Graham]

Thanks a lot for your response Kojiro.

I just tried reducing the dataset to contain just 500 voices in the commonvoice folder.

But it still spits the very error.

[Kojiro Saito]

@Daniel Graham

You might still have out of memory error.

Could you open Windows Task Manager or top command in Linux to monitor memory usage?

[Daniel Graham]

@Kojiro Saito, I did that.

And just as you earlier said, it consumes up to 90%, even though, I stopped so many other processes.

But I'm currently trying to migrate to Azure, do you think, using 4 vcpus and a 16 GB RAM could do the job?

[Kojiro Saito]

@Daniel Graham

Possibly yes.

I tried this example with 4 CPU cores (8 logical cores) and 32GB RAM. In tall/gather executioin, up to 22GB was used. So, I can assure wiht 32 GB RAM.

[Daniel Graham]

Ok, @Kojiro Saito, I'm so greatful for your kind assistance on this matter.

I'll get back to you then when I eventually succeed.

[studentmatlaber]

@Kojiro Saito I have 16gb of RAM. So I won't be able to run this code?

[Daniel Graham]

I don't think so @studentmatlaber.

But just try

Answer 2

[amina]

svp comment vous avez fait pour reduire la taille, j'ai le meme probleme et j'essai de trouver une methode pour reduire les fichiers audio . aidez moi svp !