History of AI – from Winter to Winter

AI – Artificial Intelligence.

IF something like this exists, it is currently just a bucket of disciplines that can be optimized for one use case. We struggle to connect the dots.

A definition of AI was provided in 1966 by Marvin Minsky, who is considered one of the founding fathers of AI.

“Artificial intelligence is when machines do things that humans are assumed to be intelligent to do.”

This is, of course, quite vague, and consequently, the classifications get pretty mixed up. 

At the very beginning, there were expert systems. Followed by some kind of self-organizing systems. Next step were complex logic systems. Now neural networks. It went back and forth like that since 70 years.

What happens here is that a technology is completely hyped, being the coolest thing of all, we solve all the problems on earth and then we realize… We can’t. That we can’t solve all the problems with it and then the hype crashes. This is called AI winter. We have such a hypetheme right now, a lot of people equate AI with Machine Learning, or even still with a certain variety of the Machine Learning, the DeepLearning. That’s also a cool technology, invented in 1972, so we understand bleeding edge in IT.

Machine learning is considered one of the oldest areas of AI and is based on 70-year-old algorithms that are currently experiencing a revival due to the massive computing power at our disposal. The algorithms can process orders of magnitude more combinations than was the case 20 years ago. It should enable IT systems to recognize patterns and regularities and correlations (linear or non-linear) in data sets based on algorithms and to develop solutions. In this way, artificial knowledge can be created from experience. The knowledge gained is usually of a general nature and can therefore be used for new fields of application or for the analysis of previously unknown information.

This does not make them capable of thinking. 

DeepLearning is based on theories from the 1960s about the human brain. So it’s not the big thing either. A human brain has several orders of magnitude more synapses than the largest AI models to date can simulate. 

While a large neural network has one million nodes, an average human brain has 84bln nodes. The neural network needs half an atomic power plant, the human being runs with an apple. A brain needs 20Watt which is a huge difference. 

Accordingly, brain research has little overview of the processes that take place in a human head. We assume that something decisively more awesome happens there than in an animal brain or a simulated neuronal network. But actually, we don’t know if anything structurally different is happening at all, nor what exactly constitutes this crucial capability. Neural networks “only” do fully automatic statistics and are already more often right than humans in some applications. 

Neuromorphic Computing

A very interesting area where Hardware and Software seem to merge. Neuromorphic chips mimic neurological functions through computational “neurons” that communicate with one another. However, initial findings indicate that the brain-like chips have also inherited a human weakness: if they are in use for too long, they need a break.

Overtired neurochips are no longer particularly accurate in their work.

What next?

Let’s assume Moore’s Law still holds a little and we could expect even more computing power. Let’s say from 2030 to simulate the brain completely on the electrical level.

BUT with that, we might have recreated just the ELECTRICAL system of the brain.

It took us 70 years to do that. But the brain not only has the electrical system, it also has a CHEMICAL system. This chemical system is a little sow, because it is completely analog and therefore can cause an infinite number of states. This chemical system can reconfigure the electrical system with every state. Love, fear, alcohol, drugs… all of these chemical triggers are reconfiguring the electrical system.

Since a few years research shows us that there is probably another quantum mechanical system. So the brain is more like a quantum computer than a bunch of graphics cards.

This means there are two systems that we haven’t even started to replicate, let alone understand.

What if… there are more level?

More that we can imagine right now?

At the same moment we experience inadequacies due to misuse of the “AI inside” marketing. That AI is being used as a selling point and no real AI is present. Or that the manufacturers do not even know how their system comes to the results.

Data strategies are missing.

Data describes the past, never the future. Thus, our algorhytms process our past biases. Each of us collects data in a different way. Coherent from his point of view, but closely related to individual expectations of data.

Thus, all data sets are biased. Incompatible, data sets, questions with hidden prejudices, measurement errors, low quality data and more can often result in bias of machine learning applications.

Heatmaps reliably show what the system is oriented to because in supervised learning, humans teach computers which answer is correct and which is incorrect.
But only the AI “knows” which features are used to do this. And they are usually completely different from those of humans.

Basically, a machine is deadly serious. It does not interpret.
It does not understand context. Therefore the AI can react unexpectedly when it encounters the real world. Snow, rain or even a pixel error in the camera can generate this noise.
Suddenly, the network can’t match the input.
You can’t guarantee that a system that worked yesterday will work tomorrow for all the special cases.

Vulnerabilities through perturbations

Machine vision is vulnerable to Adversarial Effects, patterns that are not visible to humans but are visible to pattern recognition ML algorithms. You could call them optical malware.
It will be absolutely necessary to test algorithms for such vulnerabilities or to implement secure procedures to prevent undesired results.

Machine learning models are often trained on data from potentially untrustworthy sources, including crowd-sourced information, social media data, and user-generated data such as customer satisfaction ratings, purchasing history, or web traffic. Recent work has shown that adversaries can introduce backdoors or “trojans” in machine learning models by poisoning training sets with malicious samples. The resulting models perform as expected on normal training and testing data, but behave badly on specific attacker-chosen inputs.

For example, an attacker could introduce a backdoor in a deep neural network (DNN) trained to recognize traffic signs so that it achieves high accuracy on standard inputs but misclassifies a stop sign as a speed limit sign if a yellow sticky note is attached to it. Unlike adversarial samples that require specific, complex noise to be added to an image, backdoor triggers can be quite simple and easily applicable to images or even objects in the real world. This poses a real threat to the deployment of machine learning models in security-critical applications.

Waves of progress, but increase of complexity

So on the one hand the technology becomes more mature with every iteration. More flexibility of the algorithms, merging different domains of AI are increasing the complexity and thus the vulnerability of the system. Without a clear strategy, data hygiene and trustworthy sources, the development could slow down. More glitches and disappointment would take over.

On the other hand we have to make sure WHEN we want to use it. Marketing blurb to sell because there’s AI… not a good choice.