How to Invert a Neural Network

I have trained a model with states as inputs and an output of the cumulative distribution function (CDF) of any specific state, which is designed to mitiage any confusion in the network if multiple points have the same probability. I would like to flip the model instead of training an entirely new model because of the computational requirements and the probability that they won't agree with each other. Does anyone know how to do this with Matlab's Deep Learning Toolbox?

Trained: CDF = NN(X)

Invert: X = NN(CDF)

Answer 1

[Taimoor Tariq]

Maybe this will help.

paper: https://www.robots.ox.ac.uk/~vedaldi/assets/pubs/mahendran15understanding.pdf

code: https://github.com/aravindhm/deep-goggle

[Blake Van Winkle]

To Taimoor:

I appreciate the answer. Have you tried to integrate the Deep Learning toolbox into this process?

Thanks,

Blake

[Taimoor Tariq]

The code is compatible with Image input CNNs defined using matconvnet. Now if you have a Image input model trained on the Deep Learning Toolbox, you could probably export it to matconvnet using ONNX and then use the code. However, I think in your case In dont think you are using images as input, you would probably have to tweak the code a lot, or maybe write your own.

Answer 2

[Srivardhan Gadila]

If your network is a fully connected, has no non-linearities(like activations)/non-linear layers and has invertible wieght matrices for all your layers, then only you can invert your trained network by using the inverted weight matrices & bias vectors.

[Blake Van Winkle]

To Srivardhan:

Maybe invert is the wrong statement. I am looking to pass the outputs in as inputs and recieve results on the other end. I am assuming that this is possible since folks use numerical gradient methods to try and tease out why a specific outcome was generated.

http://dataskunkworks.com/2018/06/12/deepdreaming-neural-networks-with-tensorflow/

[John D'Errico]

Note that any such inverse as you desire need not be unique, or even terribly well posed, just as would be true for any inverse of a general nonlinear relationship.

[Blake Van Winkle]

To John:

I would not have a problem with that. Do you know how to do that with the Deep Learning toolbox?

-Blake