Modelling Neural Network
Consider the following neural network:
Though the weights between the layers are not shown for clarity, you must consider that every neuron of the layer K is connected to every neuron of the layer K + 1. For the architecture given above, answer the following questions:
• Write the dimensions of the input vector, the weight matrix, the bias matrix, the preactivation matrix, the activation matrix, and the output vector of every layer
• Suppose that the initial values of the inputs are [1,2,3,4,5] and the weight and bias matrices are initialized to 0.5, show the calculation of every neuron (both the preactivations and activations) for one forward pass. The activation function is sigmoid
• Draw the computation graph of the given architecture for the calculation of derivatives of the loss function with respect to the weights and using sigmoid as an activation function.
• Suppose that you have two training examples x1 = [1,2,3,4,5] and x2 = [5,4,3,2,1] and y1 = [1] and y2 = [0]. Also suppose that you have only the 2nd hidden layer present in the architecture given above i.e. there is no hidden layer 1. Show two iterations of the backpropagation algorithms with all the equations and results
Answers
Answer:
An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons. An artificial neuron that receives a signal then processes it and can signal neurons connected to it. The "signal" at a connection is a real number, and the output of each neuron is computed by some non-linear function of the sum of its inputs. The connections are called edges. Neurons and edges typically have a weight that adjusts as learning proceeds. The weight increases or decreases the strength of the signal at a connection. Neurons may have a threshold such that a signal is sent only if the aggregate signal crosses that threshold. Typically, neurons are aggregated into layers. Different layers may perform different transformations on their inputs. Signals travel from the first layer (the input layer), to the last layer (the output layer), possibly after traversing the layers multiple times.
