Best way to analyze two waveform signals

I am trying to find the best way to analyze two waveform that were collected for an experiment. The reference waveform was collected from a motion device and the direction of movement it had moved in pitch and roll. Subjects were supposed to input their perception of pitch and roll with a joystick. Data is very messy and RMS error, correlation values, and FFT are not great measures. Any suggestions would be helpful!

[Jonas]

so if i understand correctly, you have a ground truth of position (for pitch and jaw) and a measured position and you want to compare the subjects performance? what do you want to know? were the joystick values cut, or is this the natural border of the maximum joystick motion?

[Austin Bollinger]

Your understanding is correct. The natural border of the joystick data is the maximum of the joystick, the joystick can read +/- 18 degrees. I am just trying to evaluate how subjects performed by matching the wave motion they felt with the joystick and give them a score of such in a way to compare multiple trials they did. Correlation score are like 0.2 or below so I am trying to find another way to analyze this data.

[Jonas]

i would suggest the follwing things: use a crosscorrelation to look for a constant (but reasonable) delay between ground truth and measured vector.

you can also try to look for inconsistent delays (varying reaction time) using dynamic time warping (matlab dtw() )

i would also think about excluding areas in which the joystick values capped since the subjects may wanted to move further but couldnt due to the physical restriction of the joystick

if you want further tips, please supply a data file

[Austin Bollinger]

Thank you for your support. I will give those options a try and see what I end up with.

I attach some data files if you want to try it out. Thank you!

[Jonas]

looks interesting. hiw old were the subjects, did they get a specific instruction before the task and in which (acoustic) environment was the data recorded?

the upper figure is a bit confusing in the way that it looks like the recorded data is ahead of the ground truth data movement

[Austin Bollinger]

Subjects were between 25-50 years of age. Subjects were instructed to keep the joystick earth vertical so in a sense they would need to move the joystick in the oposite direction that they were being tilted on a 6 degree of freedom motion device. I flipped the polarity of the data to make it easier to look at since they should of moved in the opposite direction. Very complex task. Subjects were in a capsule that had no outside views of their surroundings so just had to rely on their perception.

Answer 1

[Adam Danz]

Hello @Austin Bollinger, looks like @Jonas has made some good recommendations. I'll add another perspective.

My grad school research was similar but not exactly the same as what you describe. Monkeys used a joystick (JS) to steer through virtual environents. The JS controlled angular velocity and they had to stay within a narrow path boundary but the catch was, the path dissappeard after 0.5 seconds so they had to maintain an internal prepresentation of the path and update their estimate of their location relative to the path boundaries by integrating optic flow and their joystick commands across time. No other visual features were present.

Here are three bits of advice gained from my experience with this.

Joystick gain

It sounds like you are studying vestibular (or visual, maybe visual?) signals and, based on the length of time (x-axis), it looks like the trials are long and the JS estimates are done in real-time which means you would expect the JS estimates to be similar to the real motion but with a temporal lag to account for perecptual and motor processing. But, that's often not the case. Instead, we see periods of flat signals where the JS is maxed out in the pos or neg directions.

Bummer, but not unusual. I had the same problem at first. To fix that, I inceased the joystick gain so that the max/min was much greater than what was actually needed to do the task. This had 2 effects (keep in mind, this was with monkeys so I can't just tell them to not do that).

1. Much less frequent max-outs
2. Much more sensitive JS control. This means that smaller errors had larger consequences and finer control of the JS was required (my monkeys were trained to steer for 1-2 years each).

If I'm reading your plots correctly, the max target signal is +/-15 so your max JS gains should be lager than +/- 18, especially if your task is open-loop (ie, subjects perceive something without the ability to predict the next move so JS control is completely reactionary).

Bang-bang control

Even if you work out the JS saturation problem, you still may see sudden jolts of the JS in the pos/neg directions within the velocity signal. Both of my monkeys independently adopted somewhat of a "bang-bang" control of the JS (which controlled angular velocity) meaning that to take a smooth turn, instead of smoothly controlling the JS they would often assert several jolts of the JS in the direction of the curve. Critically, though this is a nuisance to the data analyst, 1) it was a natural behavior (monkey can't talk or understand verbal instructions) and 2) it had no effect on the task and resulted in what appeared to be smooth turns. If you're analyzeing orientation such as pitch/roll (ie, units = degrees) rather than angular velocity (units = deg/sec) and if the subjects are controlling angular velocity, any bang-bang control will integerate-out and the signal should be smoother.

The take-home message is that what looks sub-optimal in the data may be completely a completely natural control system developed organically within the brain (or whatever is controlling your JSs).

This leads us to the final, most critical point.

Lots of data

Though I haven't seen your data, I can make a bet: there is a LOT of trial-to-trial variability. If the same subjects does the same condition 10 times, there will not be any near-repeats of behavior over the entire 6 minute trials. This finding is frequently mentioned in human steering literature. Again, a nuisance for data analysis but the result of a natural phenomenon.

To overcome this, you need lots of data -- lots and lots of it. I think I had something like 5000 repetitions of each condtion but if you're using humans, that will obviously be much smaller. With lots and lots of data, I was able to fit the responses from all repetitions from the same condtion to estimate the underlying timeseries but you would need to know the expected function that the behavior follows which doesn't seem to be the case from your description. Nevertheless, with lots and lots of data, you can eliminate the noise and access the underlying system.

Example: the gray lines below are single trials (100s) showing the displacement of the monkey from the center of a curved path. Variance grows with time because the monkey had no visual feedback of the path. But if I bin the data and measure the median within each bin, now I see the underlying trend and the distribution is normal within each bin which tells me the monkey was generally folloiwing the same strategy across time (months).

Take-home message: Law of Large Numbers: fitting, averaging, computing something from lots and lots of data from the same condition will eliminate noise (but trial-to-trial variability which is often the interesting part and should not be ignored).

To answer your question...

Sorry, I got carried away there. There is no "best way" to analyze the wave forms. This is where the art of data analytics comes in. Before we can recommend a good metric, we need the precise question being asked and sometimes that question needs to be adapted to the data.

[Austin Bollinger]

"I think" the big answer I am trying to answer is how well subjects kept the joystick earth vertical compared to gravity while riding a motion device moving in pitch and roll. I reveresed the polarity of the subject data to look at a comparrison to the motion device data because in theory if they did the task 100% accurate the signals would be alligned if reveresed. Thank you for your feedback it put a new perspective in my head.

FYI most subjects did not max out the joystick like this subject did. The joystick has a physical range of +/- 18 degrees in pitch and in roll so this subject was just going from max to max a lot on the physical range of the joystick.

[Adam Danz]

What value along the y-axis is earth-verticle? 0?

> if they did the task 100% accurate the signals would be alligned if reveresed

That's only true if the subjects could predict the motion of the device and therefore compensate for response delays. Maybe they can predict it, for example, if the motion is cyclical or predictable. But if they can't predict the device motion, then a perfect performance (ie a perfect replication of the device orientation signals) would still have a delay caused by perceptual-motor processing. In other words, before the subject moves the JS, the subjects have to

1. perceive the angular changes (vestibular afferents, optic flow processing, whatever the sensory is)
2. Estimate the angular changes based on the incoming sensory data
3. compute the error between their current JS position and the JS position needed to match the device
4. The motor system has to develop a plan to move the JS (See "inverse model" of motor control).
5. And then the motor system has to exectue the JS movement.

By the time this all happens, the device is still changing position. If you can measure the general shift between the device movement and the JS compensation, you can estimate the sensory-processing delay, but keep in mind that this delay also includes external factors such as limitations in the mechanics and control of the device.

To get back to your research question, "how well subjects kept the joystick earth vertical ", the word "well" is not defined. This is pedantic but research question design should be pedantic. How is "well" measured?

Also, this question is not falsifiable. What's the null hypothesis?

This is one of the hardest parts of research: to ask the right question. The guestion will guide decisions such as how the measure the responses. I'd be glad to help more.

[Austin Bollinger]

On the basis of what this tasked was hoped to be used for. This was a motion sickness study where we created a wave motion on a motion platform that would make subjects become motion sick to the point of almost vommitting. I was going to see how the subjects accuracy of perceieving earth vertical as their motion sickness symptoms increased over time. The subjects road the wave motion for a total of 45 minutes if possible and completed this task 3 times over the course of the 45 minute section.

Y-axis is measured in degrees. I attached another image of another subjects data with it not reversed so it is soley the motion platform position and the subjects input into the joystick. If they were completely alligned with earth vertical the summation of the motion platform position plus the joystick input would be zero since they would need to move the joystick in the opposite position of the motion platfrom movement. I highlighted an area that looks like the summation would be close to zero.

Yes, lots of steps before they make an input into the joystick. I tried looking at some delays between the signals with a cross correlation and found on average a subject had a delay of about 0.5-0.8 seconds.

I really do appreciate all your feedback. Been looking at this data for 3 weeks wondering what I should do with it.

[Adam Danz]

I wonder what motion platform you use (just curious). The lab I was in used a MOOG to simuluate the vestibular system.

Anyway, I can see a delay nicely in some areas of the data such as below but it looks more like 1-2 seconds. Perhaps the delay was much closer to what you reported when looking at the full dataset.

Another issue might be that the delay changes as motion sickness becomes worse and as the brain deals with bigger issues (ie, not throwing up). You could break up the data into something like 5-minute epochs and apply the delay analysis to each epoch if you are interested in that. But that also reduces the amount of data for each epoch-analysis.

Also, the amont of effort probably decreases as the subject becomes more concerned with keeping their stomach content in place. 45 minutes is a really long time.

If there weren't a delay and if we could assume that levels of effort, concentration were uniform across the experiment, then, I suppose, a simple subtraction of the two time series would server as an indicator of perceived-earth-vertical in an ideal observer if I'm understanding correctly. But the problems are

1. humans/animals are not ideal observers and therefore insert noise (which is often valuable to analyze)
2. the delay is unlikely constant
3. the amount of attention/effort/concentration is unlikely constant (even if they were merely sitting in an empty room for 45 minutes).

I wonder if the 3 repetiations of the 45-minute session had idental motion platform movement and, if so, how similar the responses are from the same subject. My guess is that inter-trial-variability will greatly increase and quickly saturate with time meaning that responses would be more similar toward the start of the session/trial than at the end (or even after several minutes).

If the behavioral data (JS commands) are mostly similar between sessions within each participant, you should definitely combine the data for your analysis. Otherwise, I guess the plan is to analyze single trials.

Oh, and there's another level of interest I just thought of: perhaps the amount of error (incorrect JS angle) varies with the angle of the motion platform. For example, when the motion platform is at 0 deg, the subject may perform better than when the motion platform is at 15 deg.

This gives me an idea (you're saying finally!).

What if you sort the motion platform data into bins of ~1 deg and then look at the JS data within those bins. So, for example, every time the motion platform is within [10-11) deg, you take those segments of JS data and combine them. Instead of treating the entire trial as a single timecourse data point, this break up the data into lots of data points which increases statistical power. Then, average the JS data within each bin. Now you'll have a series of motion platform angles and a corresponding series of average JS angles at those motion platform angles. Averaging within bins will eliminate some of the noise and you can see how the binned avg JS data correlates with the motion platform data. Note that amount of data within each bin will vary because some motion platform angles are less frequently represented which may affect which statistics you use, but generally this isn't a problem.

In addition to this, I would definitely perform this analysis separately for each ~12 min segment to determine if the results change as the participant becomes more nauseous.

[Austin Bollinger]

We have a MOOG and a CKAS Motion Platform. This test was performed on the CKAS unit. With pitch and roll movements we also had a lot of heave motion to make subjects sick. Frequency of motion was centered around 0.2hz.

I tried your bin seperating idea and it looks promising. Thanks! I attached some files where I took the average of the subjects input during those bin sections. 1st column is average of the ckas bin and 2nd column is subject input average by bin. It might be a good idea for me to shift their position offset and by the subjects average delay to make the bin averages a little clener.

Thank you for all the support! I really do appreciate it.

[Jonas]

I am stilm irritated by the phase of the data: the snippet shown in @Adam Danz comment shows a nice delay, bit shouldn't it be exactly reverse? the yellow line should be ahead of the blue line?

[Adam Danz]

Good points, Jonas. If the polarity of the blue line (JS data) was reversed, I'm a bit confused too. The segment I isolated in the screen shot looks like what I would expect if the JS data (blue line) was not reversed or if the yellow and blue lines are labeled incorrectly.

[Jonas]

flipped or not flipped, in both cases the blue line should be delayed compared to the ground truth yellow line? it does not matter if we depict the blue line without flipping (the actual raw data), or with flipping to be able to compare the angles directly to each other.

the whole thing makes only sence to me if the line labels are wrong or if we have another source like strong lag of the motion platform end position. is this motion platform moving fast enough or is there an unknown source of data recording lag?

[Austin Bollinger]

The yellow line is the recording of the ground truth motion platform data and the blue line is the subject input data. When I first started collecting this data I thought the same thing on there being a source of data recording lag but I ran a lot of tests and found that there wasn't any significant difference. It is a very hard task to keep the joystick in an upright position so I think it is just a signifcant lag and the subjects guessing most of the time.

[Adam Danz]

If the JS response (blue line) is not flipped and if the data from the yellow and blue lines are in the same coordinates system, and if the task is to move the JS to keep it vertical to the earth, then this segment makes sense to me. For example, if the motion platform rotates the observer in the negative direction (event A in the image below) then the observer would have to move the JS in the positive direction to maintain vertical (event B). But while that is happening, the platform starts to rotate in the positive direction (event C) which causes the observer to react by moving the JS in the negative direction (event D) and so on. In this scenario, the JS commands (blue) lag the motion platform movemement (yellow). But this lag is longer than what has been reported by OP.

[Adam Danz]

>The yellow line is the recording of the ground truth motion platform data and the blue line is the subject input data

Ah, that makes sense if you've flipped the polarity of the subject input. In that case, in event B, the device moves in the positive direction followed by a positive deflection of the JS (event C) etc.

[Austin Bollinger]

For your sanity. I used the allignsignal function to shift the joystick data(red line) to allign with the ckas motion platform data(blue line) this data had about 1.04 second delay before it was shifted. I also shifted the joystick data vertically by subjecting the mean average as this subject kept the joystick in a postive roll offset mannor.