Question

Advantage of convolutional neural networks

Answer 1

Answer:

What is the benefit of convolutional neural network?

Convolutional Neural Networks take advantage of local spatial coherence in the input (often images), which allow them to have fewer weights as some parameters are shared. This process, taking the form of convolutions, makes them especially well suited to extract relevant information at a low computational cost.

Explanation:

What are the advantages of a convolutional neural network (CNN) compared to a simple neural network from the theoretical and practical perspective?

Here’s what I know ,

• The usage of CNNs are motivated by the fact that they can capture / are able to learn relevant features from an image /video (sorry I dont know about speech / audio) at different levels similar to a human brain. This is feature learning ! Conventional neural networks cannot do this.

• Another main feature of CNNs is weight sharing. Lets take an example to explain this. Say you have a one layered CNN with 10 filters of size 5x5. Now you can simply calculate parameters of such a CNN, it would be 5*5*10 weights and 10 biases i.e 5* 5*10 + 10 = 260 parameters. Now lets take a simple one layered NN with 250 neurons, here the number of weight parameters depending on the size of images is ‘250 x K’ where size of the image is P X M and K = (P *M). Additionally, you need ‘M’ biases. For the MNIST data as input to such a NN we will have (250*784+1 = 19601) parameters. Clearly, CNN is more efficient in terms of memory and complexity. Imagine NNs and CNNs with billions of neurons, then CNNs would be less complex and saves memory compared to the NN.

• In terms of performance, CNNs outperform NNs on conventional image recognition tasks and many other tasks. Look at the Inception model, Resnet50 and many others for instance.

• For a completely new task / problem CNNs are very good feature extractors. This means that you can extract useful attributes from an already trained CNN with its trained weights by feeding your data on each level and tune the CNN a bit for the specific task. Eg : Add a classifier after the last layer with labels specific to the task. This is also called pre-training and CNNs are very efficient in such tasks compared to NNs. Another advantage of this pre-training is we avoid training of CNN and save memory, time. The only thing you have to train is the classifier at the end for your labels.

From practical perspective, both are easy to implement in fact, the code for CNN can be a bit longer than a NN due to additional max-pooling and the last layers consisting of fully-connected layers in case of classification.