Why sequential data can not be used in traditional neural network?
Answers
Homepage
Data Driven Investor
Get started
HOMEFINTECHA.I.BLOCKCHAINFINANCEABOUTCOLLABORATESUBSCRIBEDDI
Follow
Recurrent Neural Network-RNN
Go to the profile of Renu Khandelwal
Renu Khandelwal
Oct 26
If you are interested to know how does Google voice search or Amazon’s Alexa or Apple’s Siri works. You need to have a basic understanding of Neural networks and CNN’s then read on to know the technology.
For a basic understanding of neural networks
For a basic understanding of CNN
we will start with
What is missing with Artificial Neural Networks and Convolution Neural Network that Recurrent Neural Network helps solve.
Where can we use RNN?
What is RNN and how it works?
Challenges with vanilla RNN like vanishing and exploding gradients
How LSTM(Long Short Term Memory)and GRU (Gated recurrent unit) solves these challenges
Fasten your seat belts and get ready for an exciting journey on RNN
Let’s say we are writing a message “Let’s meet for___“ and we need to predict what would be the next word. The next word could be lunch, or dinner or breakfast or coffee. It is easier for us to make inferences based on the context. Let’s say if we knew that we were meeting in the afternoon and that information persisted in our memory then we can easily make prediction that we are possibly meeting for lunch.
When we need to work on sequential data that needs to be persisted over several time steps then we use Recurrent Neural network(RNN)
Traditional neural network and CNN’s need a fixed input vector, apply activation function on fixed set of layers to produce a fix sized output.
For example we take an input image of 128 by 128 sized vector to predict images of dogs or cats or cars. we cannot take a variable sized image to make the prediction
Now what if we need to operate over sequential data that is dependent on previous input state like our message or when sequential data can be in input or output or both, and that is exactly where we use RNNs.
In RNN, we share the weights and feed the output back into the inputs recursively.This recurrent formulation helps process sequential data.
RNN’s make use of sequential data to make inferences like who is talking, what is being spoken and what might be the next word etc.
RNN’s are neural networks with loops to persist information. RNN are called a recurrent as they perform the same task for every element in the sequence and output elements are dependent on previous elements or states.