Hi, I have worked with standard fitting and classification networks as well as convolutional networks so far. But now I am stuck with a problem and maybe you can steer me towards the right direction.
In general I want to predict text. As input I have a very long text available. So 26 letters + spaces + some symbols. Lets say 30 parameters. My idea was the following, using standard feed forward networks:
- Hot-One-Encode the letters to create 30 classes (lets call them C)
- Cut the text into N pieces of length L (5 for example) with a sliding window
- use each piece as one sample (of size L x C)
- and then use the following letter as an output (of size 1 x C)
- for example: The text says "alphabet" - this gives input "alpha" + output "b", input "lphab" + output "e", and so on
- finally the network could be used step by step to predict letters
For this strategy one problem I encountered was, that the size of the input is L x C. And squeezing it into an array did not yield proper results.
I assume for this task, a RNN ( layrecnet ) or Nonlinear autoregressive neural network ( narnet ) might be a lot easier to use, as for the latter one - the whole text would be the input vector and no output vector is needed. As I am not very experienced with time-series networks ... maybe some of you can give me a hint what architecture to use - or how I could modify the decribed network to perform this task. Thanks.
