Indices after accumarray in order to do N-dimensional histograms

Hello everyone,

I am using an accumarray function in my script in order to computer N-dimensional histograms. This is the line:

ht  = accumarray(sub, 1, nBins+1 .* ones(1, nFeat));

where nBins stands for the number of Bins in my histogram and nFeat the number of features (so the dimension of my histogram).

If I use for example this set of data:

[1 2 ; 2 3 ; 1 2 ; 1 3 ; 1 1 ; 3 2]

My 2-dimensional histogram will be:

H = [1 2 1;
     0 0 1;
     0 1 0]

The first value for example says that there is 1 occurrence of 1 1, the second value 2 occurrences of 1 2...

However I want to keep the indices. In this example, I would like to have something like that:

Hind = {5}, {[1 3]}, {4};
       { }, {     }, {2};
       { }, {  6  }, { }

I don't really care about it being in cells or something else.

The number is the cell are just the linear indices of the values that contributed to the histogram.

Basically, I want the indices that could allow me to generate my histogram for example by doing:

H = cell2mat(cellfun(@numel, Hind, 'Un', false));

I hope this was clear.

Thanks a lot,

Quentin

[Steven Lord]

How large is N and how many bins do you want in each dimension? If either of those numbers are large (or if both of them are medium sized) that could require a lot of memory.

[Quentin Garçon]

I think that the number of dimension (N) will never exceed 10.

The number of point depends on the application. I am working on clinical data so there can be lots of types of data.

However, since all my data are transformed into scalars in each feature. I don't need to have the entire data in my output but only the indices.

So the output would be of size nBins^nFeat cells. And each cell would contain some integers.

But I don't really care about the size of the variable since the algorithm will be running on supercomputer for Big Data application. And then the code will be written in C++.

For now, I am more or less trying to do the proof of concept of the method in Matlab so I can use small dataset to make sure that it works (and that my computer will not explode if possible...).

[Steven Lord]

So nFeat could be up to 10. What is an upper bound on nBins? If it's too large, even storing "only the indices" could require more memory than is available on the planet or in the known universe.

[Quentin Garçon]

nBins follows a dimensionality curse... So the goof thing is that I have to decrease it as soon as nFeat is increasing. So for now nBins = 16 is already big.

Answer 1

[Walter Roberson]

ht  = accumarray(sub, (1:size(sub,1)).', nBins+1 .* ones(1, nFeat), @(v) {v});

[Quentin Garçon]

Thank you!

This is exactly what I wanted. This is perfect. I had already done another method thinking about what Steven Lord told me. However reducing the memory cost was possible but it has a big cost in computation time.

I will try this solution!

Answer 2

[Quentin Garçon]

Hi!

I can't find any other solution than using a lot of if loop in order to find back the indices. No one has a better syntax with the accumarray?