2014b-64bit python from within matlab, problem with threading

Having issues running a python script with threading from within matlab. It appears that matlab stops the python interpreter much of the time. At the end of the message I have a python script that starts a thread and looks at its iterations, without and with a small amount of sleep within the script.

My python version is 2.7.9 64bit. No other versions of python have been installed (clean machine built 2 weeks ago)

Lots of iterations when run from the DOS command line :

>>python process_test.py

run for 3.0 seconds:
2485 585859 1126612 1669999 2208775 2754896 3226523
iterations no sleep :  3226523

1 501 1000 1500 2000 2497 2997
iterations with sleep :  2998

Or even from within matlab when using system:

system('python process_test.py')

But, using a matlab script:

insert(py.sys.path,int32(0),'.');

p = py.process_test.process_test(0.001,3);
ts = now + 5/86400.;
while now < ts 
    disp(p.cnt);
    pause(0.5);
end
disp(sprintf('Iterations w sleep %d', p.cnt));

There are zero iterations (with or without python sleep)

However, starting from the matlab command line and can poll and get some cnts. p = py.process_test.process_test(0.001,3); wait a bit then type p.cnt or p to see all the members

My final application is real-time plotting with python reading data from a TCPIP connection in a thread. In this case, the thread does run, but not enought or consistently. I suspect that calling drawnow expose many many times per sec was giving the python interpreter a chance to run.

The python script "process_test.py" follows:

import threading
import time, sys

class process_test(threading.Thread) :

      def __init__(self, sleep_tm=0.001, watchdog_timeout=5) :
          self.cnt = 0;

          self.sleep_tm = sleep_tm;
          self.terminate = False
          self.time_of_last_watchdog = time.clock();
          self.watchdog_timeout = watchdog_timeout;
          threading.Thread.__init__(self)
          self.start()

      def __del__(self) :
          self.terminate = True

      def run(self) :
          self.time_of_last_loop = time.clock()

          while not self.terminate :
              self.cnt += 1
              t = time.clock()
              dt = t - self.time_of_last_watchdog;
              if dt > self.watchdog_timeout:
                  self.terminate = True

              dt = t - self.time_of_last_loop
              self.time_of_last_loop = t;

              if self.sleep_tm > 1.0e-4 :  #
                  time.sleep(self.sleep_tm);

if __name__ == "__main__" :

      tmout = 3;
      print "run for %.1f seconds:" % tmout

      p = process_test(0,tmout)
      ts = time.clock() + tmout+0.1;
      while time.clock() < ts :
          print p.cnt,
          time.sleep(0.5)
      print
      print "iterations no sleep : ", p.cnt
      print
      print

      p = process_test(0.001, tmout)
      ts = time.clock() + tmout + 0.1;
      while time.clock() < ts:
          print p.cnt,
          time.sleep(0.5)
      print
      print "iterations with sleep : ", p.cnt

      sys.exit()

Answer 1

[Chris Barnhart]

From MATWORKs support:

I would also like to notify that the issue is not reproducible in R2015a. You may download the Pre-release of R2015a and confirm the same. I could see the counter values getting incremented in R2015a, just like it would be in a Python environment.

In R2014b, there seems to be an inconsistency in calling "run" function of the thread and hence the counter was not getting incremented. To confirm this, I further simplified the code to increment the counter only once in Python and this does not seem to work. .... Our development is now aware of this issue.

Answer 2

[Robert Snoeberger]

MATLAB is holding Python's thread lock (GIL) and not allowing your thread to run.

[Chris Barnhart]

Thank you.

Under what conditions would you believe this (GIL holding) to be true? (Or how can I get around it?)

One oddity is that in a more complex situations I do get activity in the thread. Not sure if it was during the drawnow refresh, or if the "main" python code was using more cycles.