parallel thread needs to wait for notification from FileSystemWatcher

I want ot run at least three threads in parallel. Thread 1 is the FileSystemWatcher, with the rest of the workers doing somethine else very specific, as determined by the action from the FileSystemWatcher.

How would I set this up so that threads 2 and 3 is just sitting and wait to get the spmdSend from thread 1. My code belows throw the error:

Error using testScript

Error detected on worker 2.

Caused by:

Error using testScript

A communication mismatch was encountered during spmdReceive. Worker 1 reached the end of the SPMD block without

completing the communication.ntered during spmdReceive. Worker 1 reached the end of the SPMD block without completing the communication.

spmd
    if spmdIndex == 1
        try
            fswObj = System.IO.FileSystemWatcher('C:\Waves2');
            fswObj.Filter = '*.*';
            fswObj.NotifyFilter = System.IO.NotifyFilters.FileName;
            fswObj.EnableRaisingEvents = true;
            addlistener(fswObj,'Renamed',@notifyThread);
            addlistener(fswObj,'Deleted',@notifyThread);
            addlistener(fswObj,'Created',@notifyThread);
        catch ME
            fid = fopen('error.txt','w');
            fprintf(fid,ME.message);
            fprintf(fid,[num2str(ME.stack(1).line), ' : ', ME.stack(1).name, ' : ',  ME.stack(1).file]);
            fclose(fid);
        end
    elseif spmdIndex == 2
        while true
            [data, source] = spmdReceive(1);
            disp(['Worker 2 received: ', data, ' from worker ', num2str(source)]);
        end
    else
        while true
            [data, source] = spmdReceive(1);
            disp(['Worker 3 received: ', data, ' from worker ', num2str(source)]);
        end
    end
end
cleanup = onCleanup(@()myCleanupFun(fswObj));
function myCleanupFun(fswObj)
    fswObj.EnableRaisingEvents = false;
end
function notifyThread(src,event)
    filename = char(event.Name);
    eventTypeFromNotify = char(event.ChangeType.ToString);
    if strcmp(eventTypeFromNotify, 'Renamed')
        spmdSend(['Sending to 2 for rename: ' filename],2)
    end
    if strcmp(eventTypeFromNotify, 'Deleted')
        spmdSend(['Sending to 2 for delete: ' filename],2)
    end
    if strcmp(eventTypeFromNotify, 'Created')
        spmdSend(['Sending to 3 for analysis: ' filename],3)
    end
end

[Handy]

I don't know the parallel computing toolbox well enough to say, but I don't think you can do this as the error points out that worker 1 has already completed its block of code and there is nothing to send, while woker 2 and 3 is just sitting there waiting for the notification from 1. I don't think you can use FileSystemWatcher in this way. I am not sure if this is limitation of Matlab's code or something else...

I have implemented a fileSystemWatcher before, but I used a database to store the notifications from the FileSystemWather insteafd of sending it directly to the other threads. I put the FileSystemWatcher's notification in a queue utilizing mongoDB, and then have the other threads check and pull from that databse using a timer.

[Quy]

Thanks for the comment. I was able to get something to work, but the message displayed is delayed by one event of each core. Anyway around this?

function testScript()
    spmd
        if spmdIndex == 1
            try
                fswObj = System.IO.FileSystemWatcher('C:\Waves2');
                fswObj.Filter = '*.*';
                fswObj.NotifyFilter = System.IO.NotifyFilters.FileName;
                fswObj.EnableRaisingEvents = true;
                addlistener(fswObj,'Renamed',@notifyThread);
                addlistener(fswObj,'Deleted',@notifyThread);
                addlistener(fswObj,'Created',@notifyThread);
                ii = 1;
                while true
                    pause(.5); ii = ii+1;
                    if mod(ii,50) == 0
                        disp('loop')
                    end
                end
            catch ME
                fid = fopen('error.txt','w');
                fprintf(fid,ME.message);
                fprintf(fid,[num2str(ME.stack(1).line), ' : ', ME.stack(1).name, ' : ',  ME.stack(1).file]);
                fclose(fid);
            end
        elseif spmdIndex == 2
            while true
                [data, source] = spmdReceive(1);
                disp(['Worker 2 received: ', data, ' from worker ', num2str(source)]);
            end
        else
            while true
                [data, source] = spmdReceive(1);
                disp(['Worker 3 received: ', data, ' from worker ', num2str(source)]);
            end
        end
    end
end
function notifyThread(src,event)
    filename = char(event.Name);
    eventTypeFromNotify = char(event.ChangeType.ToString);
    if strcmp(eventTypeFromNotify, 'Renamed')
        spmdSend(['Rename: ' filename],2)
    end
    if strcmp(eventTypeFromNotify, 'Deleted')
        spmdSend(['Delete: ' filename],2)
    end
    if strcmp(eventTypeFromNotify, 'Created')
        spmdSend(['Analysis: ' filename],3)
    end
end

[Handy]

Ahh, I see that you put in the while loop in thread 1 as well to prevent it from completing. I did not think of that.

Answer 1

[Quy]

I put in the while loop in the spmdIndex == 1 block of code to prevent the code running to completion. See my comment above.

[Quy]

But now I have a delay in the threads receiving the data. Please see this question for more info:

https://www.mathworks.com/matlabcentral/answers/2014591-delay-in-displaying-messages-in-parallel-threads-with-filesystemwatcher?s_tid=srchtitle