parfeval with background pool fails when using Java classes

17 views (last 30 days)
I'm trying to either use an existing database connection, based on com.mysql.cj.jdbc.Driver or to get that created in a parfeval call using the background worker.
Minimal examples using all the workarounds with and without function handles, parallel.pool.Constant(), external functions, etc.
db = com.mysql.cj.jdbc.Driver
fun1 = @()db.acceptsURL("asdf")
fun2 = @(db)db.acceptsURL("asdf")
fun3 = @(c)c.Value.acceptsURL("asdf")
C1 = parallel.pool.Constant(db);
getDriver = @()com.mysql.cj.jdbc.Driver;
C2 = parallel.pool.Constant(@getDriver);
C3 = parallel.pool.Constant(@getDriverExt);
fun1() % no error, direct call, outputs false
fun2(db) % no error, direct call, outputs false
fetchOutputs(parfeval(backgroundPool, fun1, 1)) % error serializing
fetchOutputs(parfeval(backgroundPool, fun2, 1, db)) % error serializing
fetchOutputs(parfeval(backgroundPool, fun3, 1, C1)) % error serializing, The 'Value' field cannot be retrieved because the parallel.pool.Constant is invalid.
fetchOutputs(parfeval(backgroundPool, fun3, 1, C2)) % Unrecognized function or variable 'getDriver'.
fetchOutputs(parfeval(backgroundPool, fun3, 1, C3)) % Unable to resolve the name 'com.mysql.cj.jdbc.Driver'.
getDriverExt.m
function conn = getDriverExt()
conn = com.mysql.cj.jdbc.Driver;
end
Here are the results of all runs:
>> fun1() % no error, direct call, outputs false
ans =
logical
0
>> fun2(db) % no error, direct call, outputs false
ans =
logical
0
>> fetchOutputs(parfeval(backgroundPool, fun1, 1)) % error serializing
Warning: com.mysql.cj.jdbc.Driver@3662c887 is not serializable
Error using parallel.BackgroundPool/parfeval
Internal error occurred when passing the input argument of type "function_handle" to the thread-based worker:
MATLAB:Serialize
Error during serialization
>> fetchOutputs(parfeval(backgroundPool, fun2, 1, db)) % error serializing
Warning: com.mysql.cj.jdbc.Driver@3662c887 is not serializable
Error using parallel.BackgroundPool/parfeval
Internal error occurred when passing the input argument of type "com.mysql.cj.jdbc.Driver" to the thread-based worker:
MATLAB:Serialize
Error during serialization
>> fetchOutputs(parfeval(backgroundPool, fun3, 1, C1)) % error serializing, The 'Value' field cannot be retrieved because the parallel.pool.Constant is invalid.
Error using parallel.Future/fetchOutputs
One or more futures resulted in an error.
Caused by:
Error using parallel.pool.Constant/get.Value (line 191)
The 'Value' field cannot be retrieved because the parallel.pool.Constant is invalid.
>> fetchOutputs(parfeval(backgroundPool, fun3, 1, C2)) % Unrecognized function or variable 'getDriver'.
Error using parallel.Future/fetchOutputs
One or more futures resulted in an error.
Caused by:
Error using parallel.pool.Constant/get.Value (line 196)
Error occurred while creating parallel.pool.Constant on the workers.
Unrecognized function or variable 'getDriver'.
>> fetchOutputs(parfeval(backgroundPool, fun3, 1, C3)) % Unable to resolve the name 'com.mysql.cj.jdbc.Driver'.
Error using parallel.Future/fetchOutputs
One or more futures resulted in an error.
Caused by:
Error using parallel.pool.Constant/get.Value (line 196)
Error occurred while creating parallel.pool.Constant on the workers.
Unable to resolve the name 'com.mysql.cj.jdbc.Driver'.
Is it impossible to use Java classes in the workers? If no, how to do it properly?
Thanks!

Answers (1)

Harald
Harald on 22 May 2024
Hi Jan,
for functionality to work in a background pool, it needs to be explicitly supported. For more information, please see here:
Have you tried the desired functionality in a process-based parallel pool, and would that be an alternative for your application?
p = parpool("Processes");
Best wishes,
Harald
  3 Comments
Harald
Harald on 22 May 2024
Hi,
Have you tried the desired functionality in a process-based parallel pool to rule out other issues? For example, Unrecognized function or variable 'getDriver' does not sound like a problem specific to thread-based pools.
I suspect the problematic part is External Language Interfaces, which includes Java.
While I understand that thread-based pools would be preferable, I do not yet understand why process-based pools are not a suitable alternative. You'd admittedly have to live with the initial time it takes to open the pool.
It may be a long shot, but could timers be an alternative?
Best wishes,
Harald
Jan Kappen
Jan Kappen on 23 May 2024
Hi Harald,
> While I understand that thread-based pools would be preferable, I do not yet understand why process-based pools are not a suitable alternative. You'd admittedly have to live with the initial time it takes to open the pool.
Not all users have parallel computing toolbox. backgroundPool was introduced for these reasons if I understand correctly.
> I suspect the problematic part is External Language Interfaces, which includes Java.
yes... but there's an example that uses python (an external language) which works fine via constant:
(edit: this seems to use processes, not threads :()
Two cross links:
> It may be a long shot, but could timers be an alternative?
yes, maybe.
Is a Matlab AppDesigner GUI running on a different thread per default? I just realized, that GUI elements remain responsive (or at least animations still work). How does that work?

Sign in to comment.

Categories

Find more on Startup and Shutdown in Help Center and File Exchange

Products


Release

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!