How to properly call system on Windows

I'm trying to make a project cross platform, it already works on Linux systems.

The main problem I am having is that I need to call an external executable qafm / qafm.exe

On Linux I just put the executable on in ~/bin, which is on the PATH for my user, so running

system('qafm --rest-of-arguments')

works nicely.

For instance

system('qafm')

returns 1 and prints usage to the command window, as I'd expect.

However on windows I am having trouble.

First, I don't know how to set a user specific path, so I just copied the executable qafm.exe to a MATLAB folder.

This isn't some broken rpath or anything though, if I run .\qafm or .\qafm.exe in powershell or cmd in that folder I get the expected usage output.

However, if I run in the same folder in MATLAB R2019b

system('.\qafm')

it returns -1.0737e+09 (Not a valid error code for the program I promise) and prints nothing to the command window.

Any idea what the problem might be?

[dpb]

1. Build a fully-qualified path name to the installed executable on the system and include that in the system() call...if the executable is properly installed so it knows where it's needed stuff is -- copying just an executable to another directory may not get everything it needs, Or
2. Build a .bat file that contains commands to set whatever environment the application needs to run -- then call the .bat file from MATLAB instead.

[Kylie MacFarquharson]

The executable runs fine in it's new location from cmd, so I don't think that's the issue.

I'm wondering now if maybe the problem is environment though. MATLAB system() calls seem to try run with MATLAB's version of the c++ standard library. I have no idea what version of this it has, and the exec is compiled with c++20. I'd have assumed it would print an error here though rather than silently failing.

[dpb]

system just spawns the default command processor -- normally on Windows that's CMD, but it starts a new session with the default location and environment; it doesn't inherit anything from the MATLAB session that spawned it.

The .exe will try to run with whatever set of DLLs are in the path at the time -- if you have moved it from its default installed location, then it may well pick up a set from the given installation and not those that were packaged with the application itself.

I'll still bet odds are favorable that if you run the system() command with a fully-qualified filename to the original installed application it will work as desired.

Try just

!cmd

then set at the resulting prompt inside MATLAB and then compare that with the environment from the CMD shell from START. Also try executing the executable from each.

[Kylie MacFarquharson]

Fully qualified path has the same problem.

The exe only links against dlls that are on the system path as it was built on the computer I'm trying to run it on.

!cmd and then trying to run the exec in that also doesn't work.

However, did notice that the cmd called from matlab has C:\Program Files\MATLAB\R2019b\bin\win64 as the first entry on the path. This is full of dlls. I don't know what the Windows version of ldd is so I don't know which dll specifically is the problem.

If I remove this from the path by hand in the cmd the program works so it's defintely the problem.

The 'easy' fix is to statically link the standard lib but I'd prefer not to because it's huge.

[Yongjian Feng]

Set your PATH env var to point to that folder?

[Kylie MacFarquharson]

Well there's a few path locations that need to be set, and they'll be machine specific. I just noticed though that getenv PATH, called from MATLAB, doesn't prepend the extra folder. So I can use that I think.

[dpb]

Hmmm....I just noticed that MATLAB does that same dirty here, too...I almost never use CMD.EXE as I use the JPSoft TakeCommand command processor instead so hadn't noticed MATLAB inserting that when it spawns the shell--that's just rude!

So, use the .BAT file route to circumvent MATLAB doing something not wanted--in the .BAT file you can set the environment as needed and working directory wherever you wish it to be before invoking the app.

YECCCCHHH! I just did a

|getenv('path')

inside MATLAB and discover that the ML install has left the every previously-installed version in the current path...

ans = 
...;C:\ML_R2020b\runtime\win64;C:\ML_R2020b\bin;C:\ML_R2019b\runtime\win64; ...

going all the way back to R2014b (fortunately, I only install 'b' releases).

Anyways, I think the solution to your problem is to put your code in its own folder and use a batch file to dispatch it -- then you can control the search path and any other environment issues you need to.

[Kylie MacFarquharson]

I think the reason those runtime directories are in the path (in the middle) is because those are actually on the system path (they're added when installed and I guess not removed when uninstalled). Try set PATH from a cmd, they'll be there.

However it seems that when you run system() it also prepends and extra folder to the path, ahead of C:\Windows\System32, so it overwrites the default system dlls like libstdc++ stuff.

To fix, getenv('path') seems to return the actual system path, and I just

system(sprintf('set PATH=%s & qafm.exe'), getenv('path'))

It feels a little hacky but it works so I'm calling it a win.

Should I submit this as an answer for googlers?

[dpb]

That'll work but it seems to me a better way would still be to have the executable on its own rather than adding into a ML directory...but it's your code so your choice on that organization structure.

May as well put it in as an Answer -- that'll indicate to others that watch but may not be around today that there is a solution so that if they're busy don't spend time on Q? that do have at least one acceptable Answer.

As for the other old paths; I grok that; I just would have hoped the older releases would have been cleaned up with the new install -- although I suppose they're needed if one wanted to be able to run older releases as well. I haven't figured out how to keep the license active on them all so while the older installs are still on the system, they're not activated. I suppose if I were to uninstall them, then it would clean things up. But, who knows--I might at some point in the future need one of the earlier releases and have to go figure out the license thingie then... :) As long as "out of sight, out of mind!" I don't guess it hurts too much unless the path reaches whatever its internal length limit is...

Answer 1

[Kylie MacFarquharson]

Answering myself as we worked it out in the comments:

When MATLab makes system call with system() or !, it prepends to the PATH environmental variable "C:\Program Files\MATLAB\R2019b\bin\win64". Because the PATH is read left to right, any .dlls in this folder will be found first by the OS as it tries to execute whatever program you are running. This folder includes (somewhere) a .dll which defines libstdc and libstdc++, and these defininitions will be used over the OS versions in C:\Windows\system32. If your executable was compiled with a recent version of c++, like c++20, it won't won't be able to run (assuming it wasn't staticly linked). Apparently on Windows this fails silently, hence the lack of useful error messages.

The solution to this is to change the PATH variable before running the executable.

One option is to write a batch script that sets the PATH and then runs the exec (versetile, but adds another file to keep track of)

Another option is to simply set the path immeditely before executing the exec. Thankfully getenv('PATH') returns the actual system PATH.

system(sprintf('set PATH=%s & qafm.exe'), getenv('PATH'))

[dpb]

"Thankfully genenv('PATH') returns the actual system PATH."

One could always save the PATH and edit it if absolutely needed -- I looked at doing that in the batch file but is a real pain with the limited toolset built into CMD.

[Kylie MacFarquharson]

It would probably be easier to edit the PATH in matlab and then set it before running the batch file.

[dpb]

That's what I was suggesting, yes...it's too much grief in CMD. TCC (the JPSoft CMD replacement) would make it trivial exercise, however... :)