Serial reading and splitting data needs to be faster

The serial has to be read and the data has to be converged to a easy readable double matrix like ...x1 double. I need to read the buffer and creates this matrix quickly so other code can run after. Reading the buffer is quick (0.5s), but I would like a speed improvement for creating the matrix (11s). Any ideas?

% stm = serial("COM5", 'Baudrate', 2000000);
% stm.InputBufferSize = 100000000; %100MB buffer, max is 2GB
% fopen(stm);
amount = 200;
data = cell(amount,1);
NotFullData = fscanf(stm); %make sure adc is on before sending and discard this
tic;
for i=1:1:amount
    data{i} = fscanf(stm);
end
toc;
% fclose(stm);

%This part is slower
num_chunks = 8192;
len = length(data);
tic;
chunks = zeros(len*num_chunks, 1);

for a = 1:length(data)
    data_buffer = cell2mat(data(a));
    for i = 1:num_chunks
        start_index = (i - 1) * 4 + 1;
        end_index = start_index + 3;
        chunks((a-1)*num_chunks+i,1) = str2double(data_buffer(start_index:end_index));
    end
end
toc;

fscanf(stm); gives 1x32769 char

e.g. 3036303630343035303630363036303230333035303530353034303430363035303330333033303530353034303530353034303530313032303830353035303530343036303230343034303430363036303530343032303330343033303430353034303530353035303230333033303730343034303230323034303330283032

This code is even slower

% stm = serial("COM5", 'Baudrate', 2000000);
% stm.InputBufferSize = 200000000; %200MB buffer, max is 2GB
% fopen(stm);
amount = 200;
data = zeros(amount*8192,1);
NotFullData = fscanf(stm); %make sure adc is on before sending and discard this
tic;
for i=1:1:amount
    read_data = fscanf(stm);
    data_temp = str2double(cellstr(reshape(read_data(1:end-1),4,[])'));
    for a=1:1:8192
        data((i-1)*8192+a) = data_temp(a);
    end
end
toc;
% fclose(stm);

Answer 1

[UDAYA PEDDIRAJU]

Hi Rick,

To speed up converting serial data to a double matrix, focus on the loop that processes each data cell. Here's how:

1. Use vectorized operations instead of loops to assign values directly. This is much faster.
2. Pre-allocate the "chunks" matrix to avoid resizing within the loop.
3. Split strings in the "data" cell array into numerical arrays outside the loop for faster processing.

% stm = serial("COM5", 'Baudrate', 2000000);
% stm.InputBufferSize = 100000000; %100MB buffer, max is 2GB
% fopen(stm);

amount = 200;
num_chunks = 8192;
len = amount;  % Assuming all cells in 'data' have data

% Pre-allocate for speed
chunks = zeros(len * num_chunks, 1);

% Read data efficiently (outside loop for speed)
NotFullData = fscanf(stm); % Discard initial data
data = cell(amount, 1);
for i = 1:1:amount
    data{i} = fscanf(stm, '%c', 4*num_chunks);  % Read fixed 4*num_chunks chars
end

% Vectorized conversion (no loops!)
data_numeric = cellfun(@str2num, data, 'UniformOutput', false);  % Convert each cell to a number array
data_combined = [data_numeric{:}];  % Combine all numeric arrays into a single matrix

% Fill 'chunks' directly (avoid nested loops)
chunks(:) = data_combined;

% fclose(stm);  % Close the connection as usual