How can count number of memory operations in N*N MLP?

Hi ;

kindly , i need count approximately the number of memory operation required for MLP with 4*4 input , 8 hiden nerons and one output nuerons

so the MLP toplogy : 4 - 8 -1

is there any equation can be used to count the number of memory operations please ?

Answer 1

[Gaurav Garg]

Hi Furat,

You can calculate the number of operations manually.

Number of operations = O(Batch_size * Σ (weights+bias) * sizeofdatatype)

Number of memory operations is a function of batch_size (the number of inputs given to the MLP), weights and bias (Each neuron connection is given a weight, depending on the number of layers and neurons in each layer, you can calculate weights and bias), and sizeofdatatype (depending on the input, the data type for each weight and bias is also defined).

However, If memory limitations are a problem, the amount of temporary storage needed can be reduced by a factor of N. You can refer to the link here.

[Furat Alobaidy]

Thanks, Garg for your explanation, it's very useful for me; i appreciate that.

just one notes :

if i need to apply that equation to my model :

Number of operations = O(Batch_size * Σ (weights+bias) * sizeofdatatype)

Batach size : 4*8

weighted + bias: 4*9+8*1

sizeofdatatype ? such as 32-bits or 64-bits?

I confused about the last term, please

Thanks a lots