Problem with large arrays (c++ & variable editor) in R2012a win64

I've experienced problem when working with large array that is included in a cell array, ( z{1,1}(1,1) is the first element). The cell array z contains only the array, array dimensions are 2 times 18622656. I can access the array from .m scripts and functions without any problems, but

1) I can open cell array z in variable explorer and I can see that there is 2x18622656 double array in z{1,1}, but any attempt to open the array in will makes Matlab not responding ...

2) I can't access the array in C++ (VS2008), below is the source code

int main (int argc, char *argv[]) {

    // get a name of a file
    if (argc != 2) {
      cout << "wrong parameters";
      return(RETURN_ERROR);
    }

    string file = argv[1]; 

    // open a matlab file7
    MATFile *pmat = matOpen(file.c_str(), "r");
    if (pmat == NULL) {
      cout << "Error opening file " << file << endl;
      return(RETURN_ERROR);
    }
    else cout << "Opening " << file << " ..." << endl;

    // go through the content
    int ndir;
    const char **dir;
    dir = (const char **)matGetDir(pmat, &ndir);
    if (dir == NULL) {
      cout << "Error reading directory of file " << file << endl;
      return(RETURN_ERROR);
    }
    else {
      cout << endl << "Directory of " << file << endl;
      for (int i=0; i < ndir; i++) cout << dir[i] << endl;
    }
    cout << endl;

    // more about z
    mxArray *z = matGetVariableInfo(pmat, "z");
    const mwSize *dims = mxGetDimensions(z);

    cout << "z is " << mxGetClassName(z) << " with dimensions ";
    for (mwSize c = 0; c < mxGetNumberOfDimensions(z); c++) cout << ((c == 0)? "":" x ") << dims[c];
    cout << endl << "total number of elements : " << mxGetNumberOfElements(z) << endl;

    // go for elements
    for (mwIndex index = 0; index < mxGetNumberOfElements(z); index++) {
      const mxArray *cell = mxGetCell(z, index);
      if (cell != NULL) {
        cout << "   element " << index + 1 << " : " << mxGetClassName(cell);
        const mwSize *cell_dims = mxGetDimensions(cell);
        for (mwSize c = 0; c < mxGetNumberOfDimensions(cell); c++) cout << ((c == 0)? " ":" x ") << cell_dims[c];

        cout << endl << "mxIsStruct  :" << mxIsStruct(cell);
        cout << endl << "mxIsDouble  :" << mxIsDouble(cell);
        cout << endl << "mxIsComplex :" << mxIsComplex(cell);
        cout << endl << "mxIsCell    :" << mxIsCell(cell);

        cout << endl << "rows :" << mxGetM(cell);
        cout << endl << "columns :" << mxGetN(cell);

        double *data = (double *) mxGetData(cell);

        // get an item
        int x = 2;
        int y = 73456;
        mwIndex *xy_index = (mwIndex *)mxCalloc(2, sizeof(mwSize));
        xy_index[0] = x;
        xy_index[1] = y;
        mwIndex index = mxCalcSingleSubscript(cell, mxGetNumberOfDimensions(cell), xy_index);
        mxFree(xy_index);

        if (data == NULL) cout << endl << "can't get data" << endl;
        else cout << endl << "   [" << x << ", " << y << "] = " ; 

        // destroy xy_index
        mxFree(xy_index);
      }
    }

    mxDestroyArray(z);
    mxFree(dir);

    // close file
    if (matClose(pmat) != 0) {
        cout << "Error closing file " << file << endl;
        return(RETURN_ERROR);
    }  
    cout << endl << "=== DONE ===" << endl;
    return(RETURN_OK);
  }

The code is compiled and linked without any problems, the output is following:

Opening test.mat ...

Directory of test.mat
z
r
path
file_in
cfg
c

z is cell with dimensions 1 x 1
total number of elements : 1
   element 1 : double 2 x 18622656
mxIsStruct  :0
mxIsDouble  :1
mxIsComplex :0
mxIsCell    :0
rows :2
columns :18622656
can't get data

I've tested mxGetPr as well, but the result is the same : NULL

Does anybody know how to access the data saved within such a structure in C++?

Answer 1

[Friedrich]

Hi,

this is expected behavior. matGetVariableInfo loads the Array header information only, so the actual data is not loaded. Try to use matGetVariable to actually load the data and then use mxGetPr.

Why would one like to open 2x18622656 values (approx. 37million numbers) in MATLAB interactively? This seems pretty strange. I guess due all the graphic in ML is JAVA, opening this will fill up all the JAVA Heap space which finally leads to a crash.

[marek]

Hi Friedrich,

thank you for your comment.

Badly written Java code that pump out heap could be the reason why case 1 happens.

I suppose that when there is possibility to use such a data structure directly in .m files, also an access usign different means (Variable Editor, C++) would work. I'm solving a task where the input format is given (.mat file & its structure) and these data should be processed by different tool through C++ interface (case 2). But to view the data on particular possitions might help to debug the application a lot (case 1).

Definitely, I can live without Variable Editor (.m reads the data correctly), but still looking for a way how to load data from .mat file and work with them in C++.

[Friedrich]

Hi,

try replacing matGetVariableInfo with matGetVariable in your code and you should be free to go.

[marek]

Hi,

It works now - thanks.