mxCreateNumericMatrix and sparse matrix in C/Matlab hybrid programming

Hi,

I was doing C/Matlab hybrid programming. Basically I need to load into Matlab a sparse matrix with entries in M and indeices in MI and MJ.

Below is what I have, and the error message I received is:

Undefined function 'sparse' for input arguments of type 'int16'

I am not sure whether I create the matrices MI and MJ wrongly, or Matlab does not support sparse operation in this manner.

Thanks!

Paul

------------------

M_matlab = mxCreateDoubleMatrix(1, nzero, mxREAL);

MI_matlab = mxCreateNumericMatrix(1, nzero, mxINT16_CLASS, mxREAL);

MJ_matlab = mxCreateNumericMatrix(1, nzero, mxINT16_CLASS, mxREAL);

memcpy((void *)mxGetPr(M_matlab), (void *)M, sizeof(M));

memcpy((void *)mxGetPr(MI_matlab), (void *)MI, sizeof(MI));

memcpy((void *)mxGetPr(MJ_matlab), (void *)MJ, sizeof(MJ));

engPutVariable(ep, "M", M_matlab);

engPutVariable(ep, "MI", MI_matlab);

engPutVariable(ep, "MJ", MJ_matlab);

engEvalString(ep, "MS=sparse(MI,MJ,M);");

Answer 1

[Bruno Luong]

I am not sure whether I create the matrices MI and MJ wrongly, or Matlab does not support sparse operation in this manner.

Thet later one.

Matlab mxArray and internal storage cannot be mixed with C-array and MATLAB array. You can not mix those three things.

Sorry but you have to go to the nasty road of building the sparse matrix with the internal storage by using function such as mxSetJc, mxSetIr, mxSetPr etc... That requires a full understand of how data are stored in sparse matrix internally.

Answer 2

[James Tursa]

Convert the index arrays to double either inside the C code or on the Engine side. E.g.,

engEvalString(ep, "MS=sparse(double(MI),double(MJ),M);");

P.S. Your memcpy stuff will only work if M and MI and MJ are arrays. It won't work if M and MI and MJ are pointers because the sizeof(etc) will not give you the number of bytes to copy ... they will only give you the size of the pointers. You would need an expression involving nzero instead if M and MI and MJ are pointers. E.g.,

	memcpy(mxGetData(M_matlab),  M,  nzero*sizeof(*M));
	memcpy(mxGetData(MI_matlab), MI, nzero*sizeof(*MI));
	memcpy(mxGetData(MJ_matlab), MJ, nzero*sizeof(*MJ));

or

        mwSize i;
        double *mi, *mj;
	MI_matlab = mxCreateDoubleMatrix(1, nzero, mxREAL);
	MJ_matlab = mxCreateDoubleMatrix(1, nzero, mxREAL);
        mi = (double *) mxGetData(MI_matlab);
        mj = (double *) mxGetData(MJ_matlab);
        for( i=0; i<nzero; i++ ) {
            mi[i] = MI[i];
            mj[i] = MJ[i];
        }
        

SIDE NOTE: Note that your method has four deep copies of the data floating around in memory at the same time:

1) The original data inside your C code

2) A copy of the data in your C code in your mxArray variables

3) A copy of the data when you put the mxArray variables into the Engine via engPutVariable

4) A copy of the data when you called the sparse( ) function in the Engine

What are you doing with that sparse matrix downstream in your code? Maybe some of this data duplication can be eliminated by changing things.

[Paul Zhou]

Thank you James!

I would want to use the powerful linear solve of MATLAB to solve this sparse linear system, basically. I can see four full copies of this large matrix in my current implementation. But I did not find ways to eliminate some of the duplication, unless I do `sparsing' manually. I would much appreciate it if you would elbaorate your ideas of such elimiation.

Thanks again!

[James Tursa]

Any suggestions would depend on where this sparse data is originally coming from. Generated by your C/C++ code, or read from a file, or ...?

Once you solve the linear system, do you pull the solution back into your C/C++ code?

Are you open to using a mex routine instead of an Engine application?

[Paul Zhou]

The matrix and vector are generated in C and will be directly fed to MATLAB. The solution of this sparse linear system will be fed back to C. I do not want to write the matrix/vector to hard disk and read them to MATLAB. That will take much time, as I have to solve such sparse linear system many times.

I am open to mex routine or Engine applicationm, and prefere whichever is simpler and more efficient. Indeed I am not quite familar with this sort of hybrid programming as there are only very limited reference available. The current implmentation is modeling the given examples coming with MATLAB.

I understand that M_matlab, MI_matlab, and MJ_matlab in my implmentation are sort of redundant. They only serve as an interface between my C matrices and MATLAB matrices. There must be other more efficient implmentations.

Thanks!

[Bruno Luong]

Migh I suggest - if not already done- that you break the mex in two parts

• generating the sparse matrix
• Processing of solution of sparse matrix

in between MATLAB will get the sparse matrix as LHS of the first mex and call "\" or whatever operator to compute the solution the feed the solution as RHS of the second mex.

It seems more natural and more clealy architectured like that than making a big C-mex code with usage of engEvalString, mxCallMatlab to perform the MATLAB task in betwenn as you seem to intend to carry out the task.

[Rylan]

Hi, @Paul Zhou, how about creating the sparse matrix in C and processing of the solution in C, i.e. we don't transfer the sparse matrix back to matlab. If so, maybe this process can be much faster. Is this possible?