Although machine learning has only become mainstream in the last decade, there are many essential contributors to the field dating back as far as the 1940s. In order to understand the infinite possibilities presented today in the fields of AI, deep learning and more, it is important to understand the original intentions of the scientists responsible for blazing the trail to where we presently. The history of neural networks and AI is a fascinating look at how machine learning models started as theory based mechanical contrivances made of anything from matchboxes to aeroplane scraps, before being implemented in complex, fast, modern day software.
Understanding the History of Neural Networks and AI
The story starts in 1943 when Warren Sturgis McCulloch, an American neurophysiologist and cybernetician published “A Logical Calculus of the Ideas Immanent in Nervous Activity” with Walter Pitts. The paper laid the foundations for artificial neural networks, and attempted to demonstrate that a Turing machine program could be implemented in a finite network of formal neurons.
The idea that a simple program or set of rules could be represented in a less explicit format – a web of neurons – is the fundamental principle of artificial intelligence today. These theories were not implemented until 1951, when the first neural network machine was designed by Marvin Minsky and one of Minsky’s graduate students, Dean Edmonds. This first neural network machine was known as the SNARC (Stochastic Neural Analog Reinforcement Calculator) which was a neuro-computer containing 40 neurons which imitated a rat finding its way around a maze.
Source: cyberneticzoo.com
Minsky was actually inspired by McCulloch’s 1943 paper to build out SNARC which was considered one of the first electronic learning machines. Using components from a B-24 bomber and a few thousand dollars from the Office of Naval Research, Minsky took one of the first initial steps towards building out machine learning capabilities.
Minsky’s machine relied on being informed of basic success, while containing a basic memory to comprehend past errors. With this structure, the machine could adjust itself with a few motors and vacuum tubes. It was also wired randomly, like the weights on a modern day neural network model. This meant it could break in certain places, but still work because of its fail-safe nature. Again, like a neural network today, if you remove one neuron, the model would still effectively work.
Only six years later, Frank Rosenblatt invented the Perceptron, which was an actual electronic device that followed neurological principals. Today, neural networks are made up of perceptrons, which are entirely based on Rosenblatt’s device. The perceptrons were first tested out on an IBM 704 computer at Cornell Aeronautical Laboratory. Testing revealed that the 704 computer was able to recognise the basic imaging of a triangle. This was one of the most important creations in AI history, because it hinted at how helpful AI had the potential to be. It received huge international recognition, including an article in The New York Times titled “New Navy Device Learns By Doing.”
1958 New York Times article on Perceptron’s capabilities. Source: parvez-ahammad.org
The excitement among the community was huge, and there was perhaps too much, too quickly. In 1969, Minsky wrote a book titled Perceptrons, which highlighted the many limitations the Perceptron really had. For example, a single perceptron was unable to learn the XOR function, which was easily solved by creating a network of them.
The intentions to truly “discover” and build out artificial intelligence have long been in researchers and scientists’ minds but it was the essential development of early neural network models that formulated the true opportunity for AI’s creation. Ventures ranging from McCulloch’s foundational paper to Rosenblatt’s Perceptron in the 40s, 50s and 60s marked the fundamental years in which AI transformed from a mere idea to a reality.
Feature Image by Akritasa
