Segmenting a large data set into a subset then sequencing the subset

Dear Matlab Answers!

I have a segmentation problem within Matlab.

I’ve imported using swallow_csv (which is amazing) a large sequenced data set. The dataset is imported as a variable called numbers which is a 44320x102 double.

Now I can already produce a new variable called rowsGrasp which is a subset of numbers which puts the whole lines into the new variable when a grasp is detected (column 11. Either 0 or 1).

This is achieved with : rowsGrasp = numbers(numbers( :,11) == 1, :);

What I would like to do is from numbers produce a new variable (graspReleaseSequence) which takes the first row instance of grasp ( so the 11th column value is 1) in the numbers variable all the way down to the last row instance of grasp where the value is 1.

With this new variable what I would like to do is to add another column which sequences from 1 onwards when the grasp value changes. So if the first 6 rows of graspReleaseSequence’s grasp value = 1 then the new column (called sequenceNumber is 1) when the row with grasp type = 0 starts the sequenceNumber value would be 2 (say for 8 rows) until the row with grasp = 1 then it switches to sequenceNumber 3. And so on until the bottom of the variable.

What would be the best way of doing this?

Any pointers would be gladly appreciated!

Thanks,

Peter

[Peter Snow]

I'm thinking this might be easier to do in python then import it into Matlab!

[Stephen23]

"I'm thinking this might be easier to do in python then import it into Matlab!"

Really? Why would that be easier?

What would help us most to help you is when you provide a complete set of example input and output data (this does not need to be the real data, just something with the same properties): then we can test our answer code and show that it does what you want.

Answer 1

[Stephen23]

Some ideas for you to try:

• diff and cumsum.
• unique's third output.
• accumarray.
• etc, etc

But without any example input and output data it is impossible to test your requirements.

[Peter Snow]

Thanks I'll have a look at those methods.

As for the sample data a (cut down version) looks like this..

0.786203 83.164 461.123 193.000 -7.407838 0

Where the first value is a timestamp, the next 3 values are the XYZ coordinates and the final value is a boolean. There is a lot more columns but basically I would like to preserve the columns but add one more at the end which increments after the first 1 value of the boolean and then changes when the boolean changes and so forth until the end of the file.

So lets say the input looks like this:

What I would like is something like this in the same dataset:

So basically adding the incremental column and starting from the first 1 Boolean value and then going down the whole dataset until the bottom just adding the column next to the Boolean value.

[Stephen23]

@Peter Snow: Screen-shots are not useful. Please do either of these:

• upload your data in either a .csv file or a .mat file (add .txt to the extension) by clicking the paperclip button
• show a small matrix as text in a comment.

PS: diff and cumsum will do what you want:

>> V = [0;0;0;1;1;1;0;0;1;0;0;1;1;1]; % the column of data
>> cumsum([V(1)>0;diff(V)~=0])
ans =

     0
     0
     0
     1
     1
     1
     2
     2
     3
     4
     4
     5
     5
     5

[Peter Snow]

Thanks, I'll give cumsum ago.

Here (attached) is the txt version of one of the datasets. Yeah a massive file.

The Boolean for grasping is column 95 with the timestamp being column 1 and the 3 coordinates being 2,3,4 respectively. There is 96 columns so what I will try to do is get the cumsum result to be the 97th column.

[Peter Snow]

Thanks,

I've managed to fit everything into my script!

Much much simpler than I originally though!