The Artificial Neuron
History
Comparison
Architecture
Applications
Future
Sources
Neural Network Header
Top-down learning
With the advent of neural networks, there are several tradeoffs between the von Neumann architecture and the architecture used in neural networks. One of the fundamental differences between the two is that traditional computing methods work well for problems that have a definite algorithm, problems that can be solved by a set of rules. Problems such as creating graphs of equations, keeping track of orders on amazon.com, and even to some extent algorithms of simple games, are not difficult for a conventional computer.

In order to examine more deeply the benefits and tradeoffs of conventional computing and think about how a computer learns, we need to introduce the notion of top-down versus bottom-up learning.

The notion of top-down learning is best described in programming lingo as a set of sequential, cascading if/else statements which are not self-modifying. In other words, the computer is programmed to perform certain actions at pre-defined decision points. Let's say you want to write an algorithm to decide whether a human should go to sleep. The program may be simple:

if (IsTired()) {

GoToSleep();
} else {
StayAwake();
}

But in the real world, it is not that simple. If you are in class, you do not want to sleep since you would be missing valuable information. Or if you are working on an important programming assignment, you may be determined to finish it before you go to sleep. A revised version may thus look like:

if (IsTired()) {

if (!IsInClass() && !WorkingOnProject()) {
GoToSleep();
} else {
if (IsInClass()) {
Stay Awake();
} else {
if (WorkingOnProject()) {
if (AssignmentIsDueTomorrow()) {
if (AssignmentIsCompleted()) {
GoToSleep();
} else {
StayAwake();
}
} else {
GoToSleep();
}
} else {
StayAwake();
}
}
} else {
StayAwake();
}

This "simple" program only looks at a few possible things which may affect the decision--whether you're tired, you're in class, or you have an assignment due tomorrow.

This is a decision that human beings make with ease (although college students don't always make the correct decision), yet to program it into a computer takes much more code than the lines presented above. There are so many different aspects which might affect the ultimate decision that an algorithm programmed into the computer using this top-down method would be impossible.

This is the fundamental limitation of the top-down method: in order to program a complex task, the programmer may have to spend years developing a correct top-down approach. This doesn't even begin to include the subtle possibilities which the programmer may not think of. Complex tasks may never be programmed using the top-down approach.

Proceed to bottom-up learning

Back to Comparison