Skip to content

Instantly share code, notes, and snippets.

@5shekel

5shekel/perceptron.md

Created December 15, 2024 15:37
Show Gist options
  • Save 5shekel/925b6e1cf130b2aa630537771037dae5 to your computer and use it in GitHub Desktop.
Save 5shekel/925b6e1cf130b2aa630537771037dae5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
perceptron.md

Perceptron Pegboard Circuit Guide

You:

infromation on building a recreation of the orignal precpetron pegbaord ciruit

ChatGPT:

Building a recreation of Frank Rosenblatt’s original Perceptron pegboard circuit can be a fascinating blend of historical exploration and hands-on electronics. While many modern implementations of perceptrons are purely software-based, recreating the physical hardware can give you unique insight into how early neural networks were first conceived and tested. Below is a comprehensive overview of the original design, the theory behind it, and practical considerations for building your own pegboard Perceptron.

1. Historical Context

  • Frank Rosenblatt’s Perceptron (late 1950s–early 1960s): Rosenblatt was a psychologist and computer scientist who developed one of the earliest neural network models. His “Mark I Perceptron” was famously housed at the Cornell Aeronautical Laboratory. It used an array of photoelectric sensors, adaptive weights realized via adjustable wiring/resistors, and an output unit that made simple classifications (e.g., shape recognition).
  • Pegboard Circuit: Early perceptrons often employed a “patchboard” or “pegboard” approach to represent connections (weights) between the input “retina” (photo sensors or inputs) and the output neuron(s). Wires or pegs could be physically rearranged to alter weights or connectivity.

2. Core Perceptron Theory

Single-Layer Perceptron Model

A single-layer perceptron performs a weighted sum of its inputs and then applies a threshold function: $ y = \begin{cases} 1 & \text{if } \sum (w_i \cdot x_i) \ge \theta \ 0 & \text{otherwise} \end{cases} $

  • $ x_i $ are the inputs (binary or analog).
  • $ w_i $ are the weights (positive, negative, or zero).
  • $ \theta $ is the threshold (bias term).

Learning Rule (Rosenblatt’s Rule)

When the perceptron misclassifies a pattern, it updates weights using the rule: $ w_i \leftarrow w_i + \eta \cdot (t - y) \cdot x_i $

  • $ t $ is the target label (0 or 1).
  • $ y $ is the perceptron’s output (0 or 1).
  • $\eta$ is the learning rate (a small constant).
  • $ (t - y) $ is the error signal.

In hardware terms, “updating the weights” was done by adjusting rheostats (variable resistors), flipping polarity, or physically moving pegs and wires to change connectivity or gain.

3. Original Pegboard Design: Conceptual Overview

Rosenblatt’s early hardware perceptron was large and cumbersome compared to modern standards, but the principle was straightforward:

  1. Input “Retina”: A grid of photoelectric sensors that turned ambient light patterns into electrical signals.
  2. Randomizer or Connector Board (Pegboard): A board with rows representing sensor inputs and columns representing intermediate “association” units (sometimes called “A-units”). Each intersection had a peg or connector, which could be physically placed or removed to define whether a sensor was connected to a given A-unit.
  3. Summation and Threshold Circuits: The outputs of the A-units fed into a summation amplifier or comparator that decided whether the perceptron output was 1 or 0.
  4. Adjustable Weights: Realized by adjustable resistors or by wiring multiple pegs from the same sensor to the same A-unit to increase weight magnitude. Sign (positive/negative) could be introduced through different polarities of the summation circuit.

Because this was an experimental device, the exact design varied from machine to machine. Rosenblatt’s Mark I was quite large and used stepping motors to automatically adjust connections during training.

4. Modern Recreation: Step-by-Step

To recreate a simplified version of the original pegboard perceptron, you’ll need:

  1. A Pegboard or Patch Panel

    • You can use a commercial electronics prototyping pegboard (patch panel or breadboard) or build a custom wooden board with metallic contact points.
    • The board should have labeled rows for inputs and columns for the perceptron’s intermediate nodes or final output node.

  2. Sensors (Input Layer)

    • Photoresistors (CDS cells) or photodiodes to mimic the original’s “retina.”
    • Alternatively, you can skip the photo-sensing layer and feed the perceptron digital signals for demonstration purposes.

  3. Weights and Connections

    • Variable Resistors / Potentiometers: Use these to represent adjustable weights. Each input line can have a small potentiometer dial that adjusts the contribution (weight) from 0 to some maximum.
    • Patching Cords or Pegs: Each input -> weight -> summation node connection can be realized by plugging wires/pegs into a patchboard matrix.
    • Summation Node: You can have a dedicated summation amplifier (e.g., an op-amp adder circuit).

  4. Threshold / Comparator

    • A simple comparator circuit (e.g., op-amp comparator) can replicate the threshold function. Once the summed input exceeds a reference voltage ($\theta$), the output goes “high” (1). Otherwise, it’s “low” (0).
    • The reference voltage (threshold) can be set using another potentiometer or a fixed reference.

  5. Output Indicator

    • A small LED or digital readout to show the perceptron’s classification output (0/1).
    • If you want multiple output units (multiclass scenario), replicate the final comparator stage for each output.

  6. Learning / Weight Update Mechanism

    • Manual Updates: For a fully historical approach, you manually update the weights (pot settings or peg placements) after each classification error. This replicates the original perceptron’s concept of physically rerouting wires or adjusting resistors.
    • Semi-Automated: Use digitally controlled potentiometers (e.g., digital pots via Arduino or other microcontroller) that can automatically adjust weights following the perceptron learning rule. Although not historically accurate, it saves time.

5. Practical Circuit Design Example

Below is a high-level schematic approach for a single perceptron unit with a small input dimension (4 inputs for demonstration). You can expand it to more inputs or more hidden/association nodes:

  1. Inputs $x_1, x_2, x_3, x_4$:

    • Either light sensors (photoresistors) or digital signals.
    • Each input is connected through a variable resistor (pot) which represents weight $w_i$.

  2. Summation Node:

    • Feed each weighted signal into an op-amp configured as a summing amplifier: $ V_{\text{sum}} = -R_f \left(\frac{x_1}{R_{w1}} + \frac{x_2}{R_{w2}} + \dots \right) $ Where $R_{w1}$ is the weight-setting resistor/pot, and $R_f$ is the feedback resistor on the op-amp.

  3. Bias / Threshold:

    • Provide a reference offset to the summation node or a separate comparator stage. The “bias” or “threshold” can be a pot that sets a voltage $\theta$.

  4. Output Comparator:

    • Compare $V_{\text{sum}}$ against the threshold voltage $\theta$.
    • If $V_{\text{sum}} \ge \theta$, output is HIGH (LED on, logic 1). Otherwise LOW (LED off, logic 0).

  5. Learning Rule (Manual):

    • Present an input pattern (e.g., place an object over certain photoresistors).
    • Observe output vs. desired target.
    • If misclassified, adjust the relevant pot(s) to correct the error. For example, if the perceptron output is 0 but target is 1, increase the relevant weights by turning their pots up slightly.

(While this is not an exact blueprint of Rosenblatt’s 1950s apparatus, it follows the same principle and can be realized on a pegboard-like system.)

6. Tips and Considerations

  • Scaling Up: If you want to recreate the “feel” of the original machine, build a larger board with more inputs (like an 8x8 array of photoresistors). This can quickly become bulky but is true to the historical concept.
  • Noise and Stability: Real analog circuits can be noisy. You may need op-amp buffering for each photoresistor and stable power supplies to keep your signals consistent.
  • Mixed-Signal Systems: If you prefer partial automation of the weight update, you can incorporate a microcontroller (Arduino, Raspberry Pi Pico, etc.) to read the perceptron’s output, compare it to the desired target, and automatically step digital potentiometers.
  • Historical Accuracy vs. Practicality: The earliest perceptrons sometimes used vacuum tubes, mechanical relays, or large rotating mechanisms for weight update. Replicating that exactly is a bigger—and more expensive—project. A modern recreation might use solid-state electronics but preserve the pegboard concept.
  • Documentation: Original references include Rosenblatt’s 1958 paper (“The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain,” Psychological Review) and subsequent technical reports from the Cornell Aeronautical Laboratory. While not always containing blueprint-level detail, they provide theoretical background.

7. References & Further Reading

  • Rosenblatt, F. (1958). The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. Psychological Review, 65(6), 386–408.
  • Rosenblatt, F. (1962). Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books.
  • Minsky, M., & Papert, S. (1969). Perceptrons. MIT Press (discusses theoretical limits of single-layer perceptrons).
  • Modern Comparisons: Look at historical reconstructions from computing museums or academic demonstrations. Some universities have student projects replicating the pegboard perceptron.

Concluding Notes

Reconstructing the original Perceptron pegboard is an educational project that blends electronics, computer science, and the history of AI. Your final build might look like a patch panel or breadboard festooned with wires and potentiometers—deliberately so! That tactile “weight adjustment” is the hallmark of Rosenblatt’s earliest models. Although the original Mark I was more elaborate (with automatic motors and such), even a small manual version provides a wonderful demonstration of the hardware origins of neural networks.

Good luck with your build, and enjoy exploring this living piece of AI history!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment