How to calculate energy of a pattern in hopfield network?
Answers
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Answer....
Hopfield Networks
(with some illustrations borrowed from Kevin Gurney's notes, and some descriptions borrowed from "Neural networks and physical systems with emergent collective computational abilities" by John Hopfield)
The purpose of a Hopfield net is to store 1 or more patterns and to recall the full patterns based on partial input. For example, consider the problem of optical character recognition. The task is to scan an input text and extract the characters out and put them in a text file in ASCII form. Okay, so what happens if you spilled coffee on the text that you want to scan? Now some of the characters are not quite as well defined, though they're mostly closer to the original characters than any other character:
So here's the way a Hopfield network would work. You map it out so that each pixel is one node in the network. You train it (or just assign the weights) to recognize each of the 26 characters of the alphabet, in both upper and lower case (that's 52 patterns). Now if your scan gives you a pattern like something on the right of the above illustration, you input it to the Hopfield network, and it chugs away for a few iterations, and eventually reproduces the pattern on the left, a perfect "T".
Note that this could work with higher-level chunks; for example, it could have an array of pixels to represent the whole word. It could also be used for something more complex like sound or facial images. The problem is, the more complex the things being recalled, the more pixels you need, and as you will see, if you have N pixels, you'll be dealing with N2 weights, so the problem is very computationally expensive (and thus slow).
All the nodes in a Hopfield network are both inputs and outputs, and they are fully interconnected. That is, each node is an input to every other node in the network. You can think of the links from each node to itself as being a link with a weight of 0. Here's a picture of a 3-node Hopfield network:
In the previous neural models you've seen, the processing (ignoring training) only goes in one direction: you start from the input nodes, do a sum & threshold of those values to get the outputs of the first layer, and possibly pass those values to a second layer of summing & thresholding, but nothing gets passed back from layer 2 to layer 1 or even passed between the nodes in layer 1:
In a Hopfield network, all the nodes are inputs to each other, and they're also outputs. As I stated above, how it works in computation is that you put a distorted pattern onto the nodes of the network, iterate a bunch of times, and eventually it arrives at one of the patterns we trained it to know and stays there. So, what you need to know to make it work are:
How to "train" the network
How to update a node in the network
How the overall sequencing of node updates is accomplised, and
How can you tell if you're at one of the trained patterns
So, let's address these issues one at a time. I should point out that since this is an introductory class, I won't be going into the full gory mathematical details of this stuff, just showing you the basics of how Hopfield networks work. If you want a better understanding of the dynamics of the networks or how they relate to certain systems in physics, I'll be happy to point you to further references.