Recursive Dynamic Weight-Flow Networks

dynamic_recursive_vs_mlp_ver_01.m

Deep neural networks commonly allocate a separate affine transformation to each hidden layer, so increasing depth often increases storage, optimizer state, and memory bandwidth requirements. This paper proposes a recursive dynamic weight-flow architecture for supervised classification. The model stores an initial hidden transformation, shared low-rank factors, compact generator maps, and an output head. During the forward pass, the effective hidden transformations are produced by a residual recurrence coupled to the activation trajectory. The resulting architecture is a feedforward classifier, but its depth is represented as a trajectory in an activation-parameter state space rather than as a list of independent layer matrices. The paper gives a formal definition of the architecture, specifies the equations that determine its hidden propagation and dynamic parameter updates, and presents a device-resident unrolled training procedure based on automatic differentiation. Sufficient boundedness and contraction conditions are derived through a Lyapunov comparison argument for the augmented recurrence. Numerical experiments on a nonlinear multiclass classification task compare the proposed model with a depth-matched multilayer perceptron using the same hidden activation. The results show competitive predictive performance with substantially lower stored parameter requirements, indicating that recursive weight generation is a promising mechanism for memory-constrained neural modeling.