NNFullyConnectedLay​er

Non-negative fully connected layer

The NNFullyConnectedLayer in MATLAB is a custom layer designed to perform non-negative matrix multiplication, making it highly suitable for extracting non-negative matrix representations. This is especially beneficial in applications like image processing or text analysis, where data features are inherently non-negative.

As a subclass of nnet.layer.Layer, this layer integrates seamlessly with MATLAB's neural network toolbox. Its learnable weight matrix W can be initialized with a user-specified matrix initialW, offering flexibility in setting the initial layer weights. If initialW is not specified, the weights default to a matrix of ones, suitable for ensuring non-negativity.

A key feature of the NNFullyConnectedLayer is its application of the leaky rectified linear unit (leakyReLU) function to the weights during forward propagation. This approach maintains near non-negativity while allowing for a small gradient when the unit is inactive, thus aiding in the training process. For stricter non-negativity requirements, the activation function can be switched to the standard rectified linear unit (ReLU).

This layer is ideal for neural network models where maintaining the non-negative nature of the data is crucial, and it offers the flexibility to adjust the degree of non-negativity as per the specific requirements of the task.