71. Perceptrons and Neurons - Mathematics of Thought
In the middle of the twentieth century, a profound question echoed through science and philosophy alike: could a machine ever think? For centuries, intelligence had been seen as the domain of souls, minds, and metaphysics - the spark that separated human thought from mechanical motion. Yet as mathematics deepened and computation matured, a new possibility emerged. Perhaps thought itself could be described, even recreated, as a pattern of interaction - a symphony of signals obeying rules rather than wills.
At the heart of this new vision stood the neuron. Once a biological curiosity, it became an abstraction - a unit of decision, a vessel of computation. From the intricate dance of excitation and inhibition in the brain, scientists distilled a simple truth: intelligence might not require consciousness, only structure. Thus began a century-long dialogue between biology and mathematics, between brain and machine, culminating in the perceptron - the first model to learn from experience.
To follow this story is to trace the unfolding of an idea: that knowledge can arise from connection, that adaptation can be formalized, and that intelligence - whether organic or artificial - emerges not from commands, but from interactions repeated through time.
71.1 The Neuron Doctrine - Thought as Network In the late nineteenth century, the Spanish anatomist Santiago Ramón y Cajal peered into the stained tissues of the brain and saw something no one had imagined before: not a continuous web, but discrete entities - neurons - each a self-contained cell reaching out through tendrils to communicate with others. This discovery overturned the reigning “reticular theory,” which viewed the brain as a seamless mesh. Cajal’s revelation - later called the neuron doctrine - changed not only neuroscience, but the philosophy of mind. The brain, he argued, was a network: intelligence was not a single flame but a constellation of sparks. Each neuron received signals from thousands of others, integrated them, and, upon surpassing a threshold, sent its own impulse forward. In this interplay of signals lay sensation, movement, and memory - all the riches of mental life. For mathematics, this was a revelation. It suggested that cognition could be understood in terms of structure and relation rather than mystery - that understanding thought meant mapping connections, not essences. A neuron was not intelligent; but a network of them, communicating through signals and thresholds, might be. The mind could thus be seen not as a singular entity, but as a process distributed in space and time, where meaning arises from motion and interaction.
71.2 McCulloch–Pitts Model - Logic in Flesh A half-century later, in 1943, Warren McCulloch, a neurophysiologist, and Walter Pitts, a logician, sought to capture the essence of the neuron in mathematics. They proposed a deceptively simple model: each neuron sums its weighted inputs, and if the total exceeds a certain threshold, it “fires” - outputting a 1; otherwise, it stays silent - outputting a 0. This abstraction transformed biology into algebra. Each neuron could be seen as a logical gate - an “AND,” “OR,” or “NOT” - depending on how its inputs were configured. Networks of such units, they proved, could compute any Boolean function. The McCulloch–Pitts neuron was thus not only a model of biological behavior but a demonstration of computational universality - the ability to simulate any reasoning process expressible in logic. Though their model ignored many biological subtleties - timing, inhibition, feedback loops - its conceptual power was immense. It showed that thought could be mechanized: that reasoning, long held as the province of philosophers, might emerge from the combinatorics of simple elements. The neuron became a symbolic machine, and the brain, a vast circuit of logic gates. In this moment, two ancient disciplines - physiology and logic - fused. The nervous system became an algorithm, and the laws of inference found new embodiment in the tissue of the skull.
71.3 Rosenblatt’s Perceptron - Learning from Error If McCulloch and Pitts had shown that neurons could compute, Frank Rosenblatt sought to show that they could learn. In 1958, he introduced the perceptron, a model that could adjust its internal parameters - its weights - in response to mistakes. No longer was intelligence a fixed program; it was an evolving process. The perceptron received inputs, multiplied them by adjustable weights, summed the result, and applied a threshold function to decide whether to fire. After each trial, if its prediction was wrong, it altered its weights slightly in the direction that would have produced the correct answer. Mathematically, this was expressed as: wᵢ ← wᵢ + η (t − y) xᵢ, where wᵢ are the weights, η is the learning rate, t the target output, y the perceptron’s prediction, and xᵢ the inputs. This formula encoded something profound: experience. For the first time, a machine could modify itself in light of error. It could begin ignorant and improve through iteration - echoing the way creatures learn through feedback from the world. Rosenblatt’s perceptron, built both in theory and in hardware, was hailed as the dawn of machine intelligence. Newspapers declared the birth of a “thinking machine.” Yet enthusiasm dimmed when Marvin Minsky and Seymour Papert demonstrated that single-layer perceptrons could not solve certain non-linear problems, such as the XOR function. Still, the seed had been planted. The perceptron proved that learning could be algorithmic, not mystical - a sequence of adjustments, not acts of genius. Its limitations would later be transcended by deeper architectures, but its principle - learning through correction - remains at the core of every neural network.
71.4 Hebbian Plasticity - Memory in Motion Long before Rosenblatt, a parallel idea had taken root in biology. In 1949, psychologist Donald Hebb proposed that learning in the brain occurred not in neurons themselves, but in the connections between them. His rule, elegantly simple, read: “When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place… such that A’s efficiency, as one of the cells firing B, is increased.” In simpler words: cells that fire together, wire together. This principle of Hebbian plasticity captured the biological essence of learning. Repeated co-activation strengthened synapses, forging durable pathways that embodied experience. A melody rehearsed, a word recalled, a face recognized - all became patterns etched in the shifting geometry of synaptic strength. Hebb’s insight reverberated through artificial intelligence. The weight update in perceptrons, though grounded in error correction, mirrored Hebb’s idea of associative reinforcement. Both embodied a deeper law: learning as structural change, the rewriting of connections by use. In the mathematics of adaptation, the brain and the perceptron met halfway. One evolved its weights through biology, the other through algebra; both remembered by becoming.
71.5 Activation Functions - Nonlinearity and Life A network of neurons that only add and scale their inputs can never transcend linearity; it would remain a mirror of straight lines in a curved world. To capture complexity - edges, boundaries, hierarchies - networks needed nonlinearity, a way to bend space, to carve categories into continuum. The simplest approach was the step function: once a threshold was crossed, output 1; otherwise, 0. This mimicked the all-or-none nature of biological firing. Yet such abrupt transitions made learning difficult - the perceptron could not gradually refine its decisions. Thus emerged smooth activations: Sigmoid: soft threshold, mapping inputs to values between 0 and 1;
Tanh: centering outputs around zero, aiding convergence;
... continue reading