How to plot a useful graph with a lot of data

I am trying to plot graphs for different variables against crash risk however each variable has nearly 62000 columns down so my graphs are coming out as shown above with no analysis really being able to be done at all. Just wondering if anyone at any tips

[Mathieu NOE]

hello

so you want to reduce the amount of data ?

this can be achieved many ways , but before deciding on the methods , it would be worth to know why you have so much data - is it due to acquisition sampling rate ? can we decimate / resample the data , etc ...

the more we know from your end goal , the better we can answer you

Decimation — decrease sample rate by integer factor - MATLAB decimate - MathWorks France

Resample uniform or nonuniform data to new fixed rate - MATLAB resample - MathWorks France

or simply pick one every k samples (may not be very smart if you could filter your data according to relevance)

[Mathieu NOE]

FYI

Plot (Big) - File Exchange - MATLAB Central (mathworks.com)

[Danny]

Hello thanks for your reply

The reason for having that much data is because it is about crash risk of a number of different firms in different years (spread over 16 years) so therefore there is a lot to analyse

I tried swapping plot with reduce plot but it didn't seem to work

[Mathieu NOE]

you may have a lot of data but analysing them may not require to plot it all , or ?

if you need to do some statistics then the size of your data may not be an issue anymore (?)

[Mario Malic]

Maybe you should try to analyze your data w.r.t. car brands, years, categories of accidents?

[Sam Chak]

@Danny, the Excel sheet does not contain plottable data values. I think the correct data file is 'Zscore.txt'.

opts = detectImportOptions('Help code and graph.xlsx');
preview('Help code and graph.xlsx',opts)

ans = 8×1 table
                 fprintf__PlottingData____n__              
    _______________________________________________________

    {'data = load('Zscore.txt');'                         }
    {'X = data(:, 1); y = data(:, 2);'                    }
    {'m = length(y); % number of training examples'       }
    {0×0 char                                             }
    {'% Plot Data'                                        }
    {'% Note: You have to complete the code in plotData.m'}
    {'plotData(X, y);'                                    }
    {0×0 char                                             }

Answer 1

[SAI SRUJAN]

Hi Danny,

I understand that you are trying to plot graphs for a very large dataset, with each variable having around 62,000 data points.

This is a common challenge when working with large datasets, as traditional plotting methods can become ineffective due to overplotting, where too many data points overlap and make the plot unreadable. In the context of MATLAB, there are several functions and approaches that can help you manage and analyze your large dataset effectively:

Follow the given code snippet to proceed further,

% To sample your dataset, 
% you can use the datasample function or simply use indexing to select a subset of your data.
sample_data = datasample(data, 1000, 'Replace', false); 

% To decimate your data, 
% MATLAB provides the decimate function, which reduces the sample rate of a signal by an integer factor.
decimated_data = decimate(data, factor);

%If you need to resample your data to a new fixed rate, you can use the resample function.
[resampled_data, new_time] = resample(data, original_time, new_rate);

For a comprehensive understanding of the mentioned functions in MATLAB, please refer to the following documentation.

https://www.mathworks.com/help/stats/datasample.html

https://www.mathworks.com/help/signal/ref/decimate.html

https://www.mathworks.com/help/signal/ref/resample.html

I hope this helps.