Question

Dynamic programming how to find recurrence relation

Answer 1

In the field of artificial intelligence, automatic speech recognition is an extremely important area of research. In attempt to transition from systems using hybrid models that exploit HMMs for data segmentation into a pure end-to-end deep learning system, we needed to introduce a method for automatically segmenting input data and post-processing output data.

In 2006, Alex Graves introduced a method that allowed modern deep recurrent architectures to represent a probability distribution over all possible label sequences. His idea, at its core, automated the requirement of pre-segmenting training data and post-processing outputs into labeled sequences (and it did so extremely efficiently). This method is called connectionist temporal classification[1].

Using the magic of dynamic programming, CTC computes a forward pass that denotes the total probability of a particular label prefix at time-step t and a backwards pass that denotes the total probability of a particular label suffix at time t. Both forward and backward computations are built, time-step by time-step, by applying the concept of dynamic programming.