persistent mxArray untill matlab exits wtih Dll

I exported a double* (Status) from a Dll and then I do the following in a mex file;

        plhs[0] = mxCreateDoubleMatrix(10, 1, mxREAL);
        Status = mxGetPr(plhs[0]);

This works very nice, untill I call this function the second time. Then my computer crashes (not immediately however) with a blue screen;

As an alternative I changed the code to the following;

//allocate persistent memory for status pointer 
    if (Status == NULL){ //if status is empty
        plhs[0] = mxCreateDoubleMatrix(10, 1, mxREAL);
        Status = mxGetPr(plhs[0]);
    } else {
        mxSetPr(plhs[0], Status);//if status is not empty
    }

Now I can call this function over and over.

This indicates that the mxArray I allocated with mxCreateDoubleMatrix is persistent as long as matlab runs.

I'm surprised because because I would have thought that matlab would clear the old memory when the pointer (Status) points to a newly allocated array. (maybe it tries en then everything go's haywire?).

Is my interpretation correct that as long as the exported pointer points to a matlab mxArry it will be persistent, and stay that way untill I exit matlab?

[James Tursa]

It is very likely you still have a bug in your code based on what you have posted so far, but we can't tell for sure. Please post a more complete example of what you are doing.

[Chris van der Togt]

Well there is not a bug in the second snippet of code because this works fine. The question is why?

[James Tursa]

Again I would ask that you please post more code. How is Status declared? Is it automatic, global, or what? Is plhs[0] always created as shown in the first code snippet above each time the mex routine is called, or not? If not then where does it come from? Why do you think the memory is persistent? Etc. Etc. I cannot possibly answer questions as to why something does or doesn't work unless I see the actual code used. Code snippets are *not* good enough. I can comment all day about why it might be doing this or might be doing that, but most of it will be speculation based on a very incomplete picture of what you are actually doing.

[Chris van der Togt]

I've been experimenting with code and I see now that the problem does not lie in the code snippets I sent you.

This shows how I made the Dll and the mex files interfacing it.

First the code in the Dll, followed by code for three example mex files to initialize and update an array.

******Dll

////////////////////pers.cpp

#include "stdafx.h"

#include "pers.h"

// This is an example of an exported variable

double* npers;

// This is an example of an exported function.

PERS_API int fnpers(void)

{

for(int i = 0; i < 10; i++){

npers[i] += 1;

}

return 1;

}

///////////Pers.h

#ifdef PERS_EXPORTS

#define PERS_API __declspec(dllexport)

#else

#define PERS_API __declspec(dllimport)

#endif

extern "C" PERS_API double* npers;

extern "C" PERS_API int fnpers(void);

******mex files

/////setpers.cpp

#include "mex.h"

#include "pers.h"

void mexFunction(

int nlhs, mxArray *plhs[],

int nrhs, const mxArray *prhs[])

{

plhs[0] = mxCreateDoubleMatrix(10,1,mxREAL);

npers = mxGetPr(plhs[0]);

}

/////resetpers.cpp

#include "mex.h"

#include "pers.h"

void mexFunction(

int nlhs, mxArray *plhs[],

int nrhs, const mxArray *prhs[])

{

if(npers == NULL){

plhs[0] = mxCreateDoubleMatrix(10,1,mxREAL);

npers = mxGetPr(plhs[0]);

}else {

mxSetPr(plhs[0], npers);

}

}

/// runpers.cpp

#include "mex.h"

#include "pers.h"

void mexFunction(

int nlhs, mxArray *plhs[],

int nrhs, const mxArray *prhs[])

{

fnpers();

}

I see that it works exactly as you and Kaustubha probably expect. It crashes when I call resetpers instead of setpers.

So here matlab does delete the array as it should.

However, my actual Dll, which contains too much code to print here interfaces a National Instruments IO card on which a callback routine is initialized that updates a value on the array every ms. Could this prevent Matlab from destroying the array?

If so how can I properly check for this?

[Chris van der Togt]

A small update on the previous.....

In addition , I have found that I can call the initializing routine in my DLL over and over without any problem. Matlab basically handles everything fine when I just use:

plhs[0] = mxCreateDoubleMatrix(10, 1, mxREAL);

Status = mxGetPr(plhs[0]);

So this means that the callback routine has no influence on the persistence of the mxArray.

I understand also that since a pointer to this mxArray is returned to matlab space, this memory is persistent as long as that variable remains in workspace and is not cleared.

Now here I think something goes wrong in my case. I initialize the variable in my Dll from a GUI interface, and save the mxArray in a global variable.

When I quit from the Gui interface, it seems that everything is cleared. However, even if i manually clear the global variable this is not enough. Because when I open the Gui again and initialize the variables in the DLL I get a segmentation error.

This does not happen if I first call "clear all"

[Kaustubha Govind]

I believe you need "clear global <varname>" to clear a global variable.

Also, I think that persistent variables (http://www.mathworks.com/help/techdoc/matlab_external/f25255.html#f26136%29) are a better option that global variables, because they don't pollute the global workspace, and their lifetime is controlled by the lifetime of the MEX-function (which is cleared from memory either when you call "clear mex" or "clear all", or exit MATLAB).

[Chris van der Togt]

Yes, but this does not explain why I get a segmentation error when I don't call "clear all" before I allocate and set the mxArray pointer to Status the second time.

[Kaustubha Govind]

I'm not entirely clear on how you assign the global variable from a call to the MEX-file - could you clarify on that? The symptom seems to suggest a memory leak however.

Answer 1

[Kaustubha Govind]

I would expect your original code to work (unless your external library somehow re-allocates the memory pointed to by Status) - I'm assuming that by "I exported a double* (Status) from a Dll", you mean that you pass Status into the library - please correct me if I'm wrong. If so, like James said, there might be some other bug in your code causing the crash. A good place to start would be to step through the MEX-function and find the cause for the issue: http://www.mathworks.com/support/tech-notes/1600/1605.html#gen_debugging

In general, you need mexMakeArrayPersistent to declare persistent mxArrays (see example here: http://www.mathworks.com/help/techdoc/matlab_external/f25255.html#f26136).

Regarding your use of (Status == NULL) - even if MATLAB clears the memory that Status previous pointed to - the onus of setting Status to NULL is on you. Therefore, my explanation is that you are pointing to stale memory.

I would recommend using the previously cited examples to use persistent data. Note that you cannot make plhs persistent - declare a static mxArray pointer instead (again, as in the example).

[Chris van der Togt]

I'm sorry but I do not agree with you. Here I'm not putting Status to NULL, I'm checking whether it is NULL or not. If Not I'm reusing this memory. That's precisely the point: Matlab has not cleared the old memory, I can reuse it without matlab complaining or leading to a segmentation error.

Moreover if I don't reuse that memory I get a blue screen, as in the original code.

My library is not reallocating memory, and I do not pass status to the library, it is exported from the library as a double pointer and simply set to point to the allocated mxArray.

Actually it is very simple to see my point. JUst compile a Dll that exports a double pointer,

Make a mex file that calls;

either of the two pieces of code I used. There is really nothing more to it.

I'm using 32bits Matlab2009b, and VC++ 2008

[Kaustubha Govind]

It is likely that MATLAB is reusing the same memory address again, or, in the circumstance that the memory has indeed been freed, you happen to be getting away with writing to a stale pointers (in the past, I have found that Linux is typically more robust with detecting SegVs as compared to Windows) - in either case, you probably don't want to take a chance.

As a simple exercise you could try something like:

#include "mex.h"

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])

{

mxArray *temp = mxCreateDoubleMatrix(1, 1, mxREAL);

double *ptr = mxGetPr(temp);

mxDestroyArray(temp);

mexPrintf("ptr==NULL: %d", ptr==NULL);

}

Compile and run:

>> testmex

ptr==NULL: 0

You will see that even though temp has been destroyed, ptr is not NULL.

[Kaustubha Govind]

Also to reiterate my suggestion about your original code: that is indeed the recommended way to do things (again, the memory pointed to by Status is owned by MATLAB and must not be freed/re-allocated). Could something else in your code be causing the blue screen?

[Chris van der Togt]

Your example demonstrates that it may not be sufficient to test if the pointer is NULL or not.

However,

if the pointer had been destroyed I suppose my Dll would not be able to write data to this memory without SegVs occurring. After mxSetPr(plhs[0], Status) is set the second time this function is called I can still read out values in the matlab workspace without matlab complaining. I've been able to call this function over and over with out any problem, so I think matlab has lost control over this piece of memory, once a persistent pointer in DLL memory points to it and fails to destroy it.

It is difficult to say whether there was something else in my code that could have caused the blue screen. Reusing the pointer solved this issue, but clearly you disagree with this solution.

[James Tursa]

Technically, you can't test the pointer *at all* once the memory it points to is freed. Doing so is non-conforming. The only thing you can legally do is assign it a new valid value.

Answer 2

[James Tursa]

Thanks for posting more of your code. Are you statically linking to a lib file? Otherwise I don't see in your posted code how your "dll" is getting npers and vice-versa. Maybe you didn't post that part?

Here are some specific comments about your posted code:

/////setpers.cpp

As soon as setpers returns to MATLAB you are at the mercy of when the returned variable gets destroyed or modified in the calling workspace. As soon as that happens your npers pointer will no longer be valid, and any further use could crash MATLAB. This memory is NOT persistent ... it is simply hanging around as long as the calling routine does not clear it. This is not good to rely on unless you are very careful about not clearing or modifying it in the calling workspace. That is, you must not do anything that would cause MATLAB to clear the memory associated with npers.

/////resetpers.cpp

As near as I can tell the if(npers == NULL) test is pretty much useless. Since you don't set npers to NULL anywhere in your posted code (is this done somewhere else?) this test will either test against an uninitialized value (non-conforming, unpredictable results), an invalid value if the memory has been cleared already (non-conforming, unpredictable results), or it will pass. This is not good. The value of npers doesn't tell you anything about the validity of the memory it points to.

And then this line:

mxSetPr(plhs[0], npers);

will probably crash MATLAB since you don't create plhs[0] in this mex routine before you try to set its pr pointer. And a side note that even if you did you would have a temporary memory leak since mxSetPr doesn't free the old memory first.

/// runpers.cpp

Not at all clear how this mex routine (and the others) are actually getting npers from the dll.

------------------------

It appears you are attempting to simply create some persistent memory for your dll to work with via several mex routines. In that case I would advise that you simply create it using one of the MATLAB API routines, make it persistent with mexMakeArrayPersistent or mexMakeMemoryPersistent as appropriate (and remember the pointers in some global variable in the mex routine), and then lock your mex routine with mexLock. Then register a mexAtExit function so that you can clean it up when desired. That will ensure that the memory is valid throughout your processing and you have direct control over when it gets cleared. When the mex function gets cleared your registered mexAtExit function can destroy/free the memory that was made persistent. I find it simpler to put all of this functionality inside one mex routine and then call it with various options.

Do you need to have this memory/variable available in the caller workspace? I don't really have a clear picture of what your needs are. If so, then you may want to consider returning a shared data copy of the variable back to the MATLAB workspace. That way you can maintain the validity of the memory that the dll is accessing and nothing bad will happen if the caller modifies the returned variable in any way.

[Chris van der Togt]

Yes I'm statically linking to a lib file. I didn't post that part because I thought it was obvious.

I agree that the method in resetpers is not correct. And the method in setpers is better if you are careful in managing that variable in workspace. This I do of course, but that is not the problem i'm facing. Its not that I'm losing the variable, its as if I cannot get rid of it. I need to call "clear all" before I can call setpers() a second time, otherwise matlab crashes.

Actually I think this is not directly associated with the example code shown here, but how I use it in other circumstances, like initializing from a GUI made with guid or something.

"It appears you are attempting to simply create some persistent memory for your dll to work with via several mex routines."

Yes this is exactly what i'm trying to do and I don't see how a global variable in one mex routine can be a global variable in another. Since the Dll can export a global variable it will be global for any mex function interfacing with it. It just seemed to me to be the most parsimonious solution.

The suggestions you make are fairly complicated and to be honest I'm not sure how to implement them. More important they all are directed at safekeeping the memory. Whereas I think it is a clearing up problem in my case.

Under what circumstances do I need to call "clear all" when I call setpers() a second time.

[James Tursa]

> Yes I'm statically linking to a lib file. I didn't post that part because I thought it was obvious.

Not to me, since you frequently use the term DLL also (dynamic linking). Your use of the DLL terminology is confusing me.

> I don't see how a global variable in one mex routine can be a global variable in another.

Neither do I, and I wasn't trying to imply that it was. My advice was to make a single file with all the functionality.

> Since the Dll can export a global variable it will be global for any mex function interfacing with it.

Correct me if I am wrong, but it seems to me that you are attempting to do the equivalent of the following:

// foolib.c

int x;

// foo1.c

#include "mex.h"

extern int x;

void mexFunction(int nlhs, mxArray *plhs[],int nrhs, const mxArray *prhs[])

{

x = 1;

}

// foo2.c

#include "mex.h"

extern int x;

void mexFunction(int nlhs, mxArray *plhs[],int nrhs, const mxArray *prhs[])

{

mexPrintf("x = %d\n",x);

}

>> mex -c foolib.c

>> mex foo1.c foolib.obj

>> mex foo2.c foolib.obj

>> foo1

>> foo2

x = 0

So I have statically linked each foo1 and foo2 to foolib.obj, and then demonstrated that the "x" global variable in each is different. They each get their own copy. It seems to me that you are doing essentially the same thing with npers and somehow expecting that it is the same global variable in each of your mex functions even though as I understand it you are statically linking each one to the lib file, so they all get their own copy of npers.

What am I missing? How are you actually compiling/linking your mex functions?

[Chris van der Togt]

Ah I see you have missed the point. I am not statically linking. There are two types of lib files. One for statically linking and another for dynamic linking. For dynamic linking the lib file tells the linker from which Dll the fuctions should be exported. When a mex file uses one of these functions the Dll is loaded in the matlab process memory and has its own memoroy space which is shared across all functions accessing the dll.

This means that a global pointer in the Dll is global for all mex files accessing the Dll.

And it evidently works.

This is how I compile;

After making the Dll and the lib file.

mex -LC:\librarydir\pers -lpers setpers.cpp

and

mex -LC:\librarydir\pers -lpers runpers.cpp

[James Tursa]

OK, thanks for clearing that up. I think I finally understand how you are building and linking all of this together. You are doing a DLL build (not a static LIB build) and then linking to the produced .lib file from the DLL build. Now back to your code ...

I still don't see where npers ever gets set to NULL, so how does your code ever know when the pointer is invalid? e.g., if you clear the variable that npers is associated with in the workspace and then call resetpers it looks like your code will bomb.

The resetpers.cpp as posted will bomb if npers is not NULL. You call mxSetPr(plhs[0],npers) but you haven't created plhs[0] in that branch. What is the resetpers function supposed to be doing? i.e., under what conditions is this function supposed to be called? Are you calling resetpers after clearing the original variable from the MATLAB workspace, or what?