We consider real and computational models of the neuron and networks of neurons. Such networks are suitable for computing a wide variety of functions.

1. Eukaryotic cells.
   1. Has a nucleus that contains the DNA.
   2. Power for the cell is gleaned from ATP generated by the mitochondria.
   3. Cell membranes contain several different features including pumps, channels, receptors, etc.
   4. Cells divide and die.
2. Neurons:
   1. Share many of the features of most of the other somatic cells: nucleus, mitochondria, cell membrane structures.
   2. Have features that are identifiable even at macroscopic level:
      1. Dendrites: a tree-shaped network of "appendages" that deliver incoming signals from adjacent neurons.
      2. Axons: a trunk-like appendage that delivers outgoing signals to adjacent neurons.
      3. Synapses: contact points between neurons that communicate using neurotransmitters.
   3. Deliver signals.
      1. Capable of developing a localized potential difference that can travel along the axon.
      2. There is a threshold that must be reached before a signal is transmitted.
      3. There is a notion of inhibition and excitation.
      4. Basis for this potential difference is a gradient of concentrations of potassium (K+) and sodium (Na+) established across the cell wall. Pumping of atoms across the cell wall causes a change in potential (two K+ atoms are taken in, three NA+ atoms are forced out). Potential of cell interior is about 70milivolts less than outside.
      5. Two types of channels: electrically activated, and chemically activated. Ion channels responsible for signal propagation are turned on for a short time by electrical differences. First, the propagated signal opens sodium channels, bringing Na+ ions inward (positively charging the interior). This is followed by the opening of the potassium channel, which allows the escape of K+ ions, bringing the potential lower than normal. The result is a traveling potential difference; a signal.
      6. Synapses make use of chemically gated channels. Arriving signals cause the release of neurotransmitters (e.g. acetylcholine) that migrate across a synaptic cleft. These activate channels that allow the (potentially selective) migration of ions across the post-synaptic membrane. Acetylcholine activates a non-selective channel, bringing the local cell potential near zero. Other transmitters allow Na+ ions to enter, in an excitatory mode, others allow the K+ ions to escape (at a faster rate than normal): these are the basis for inhibitory synapses. The latter are most frequently found in the brain.
   4. Develop in a more complex manner.
      1. Neurons do not experience normal cell division. Drink carefully.
      2. When neurons first develop, a "cone" is responsible for guiding the growth of a cell's axon to the appropriate contact point. This cone extends and retracts feelers that help to guide the axon. It is clear that this growth is determined, at least to some degree, by the relative expression of genes within each neuron. Cell adhesion molecules (CAD1, CAD2, etc.) help to guide the way for growing axons. Experiments demonstrate neurons will find their way to moving targets.
3. The McCulloch-Pitts neuron model.
   1. The excitatory inputs to the neuron are either 0 or 1.
   2. The inhibitory inputs to the neuron are either 0 or -1.
   3. The output of the neuron is 1 if and only if the combined inputs meet or exceed a threshold.
4. The Perceptron Model. A system for weighing evidence about the state of the environment
   1. Inputs are either 0 or 1.
   2. Inputs have weights associated with them.
   3. Output is 1 if and only if the combined inputs exceed a threshold.
   4. Minsky and others demonstrated that there certain types of general computation that perceptrons were not able to perform. (See readings about the connectedness problem.)
5. The neural network model, a classifier system.
   1. Outputs can be used to identify various inputs that are members of a class.
   2. Networks are trained on inputs with known classifications. Weights associated with neural connections are then adjusted given known errors in the classification of the training set. Networks must often consider training data multiple times.
   3. Common method for adjusting weights can be thought of as a gradient descent approach to adjusting the weights to minimize error.
      1. Large error is never completely eliminated in a single step: unstable.
      2. The learning rate can be thought of as controlling the percent of the error eliminated; may not actually occur, because the weight-error space has local extrema.
   4. Trained networks can be used to classify inputs with unknown classifications.
   5. Batch training: Errors from all training instances are grouped together before adjusting the weights.
   6. On-line training: Individual training instances are used to make adjustments to the weights. Advantages include:
      1. On-line training allows fine control over the adjustments of weights.
      2. On-line training can be be an on-going process that may track moving targets. Batch training is, in some sense, monotonic, or focused on a static identification of the problem.
6. Applications:
   1. Handwriting recognition in PDA's (Newton was one of the first).
   2. Voice recognition (Octel's word-recognition system)

Resources needed for this class:

• The Physics of Baseball, on batter reaction time.
• von Neumann's The Computer and the Brain, 1958. His final lecture. Please read at least pages 39-60.
• An article on The Neuron from Scientific American's The Brain, 1979.

Resources needed for the next class:

• The general artificial neuron applet.
• The McCullough-Pitts neuron applet.
• An article about perceptrons by Dewdney.

These items require Acrobat Reader.