I have a friend of mine who began his exploration of machine learning in Python a few months back. Excited to help him out and curious as to where he is in his familiarity in statistics and algorithms (See mental model: the Forest), we hopped on a call and discussed a few things. As we started to discuss certain topics, I realized that there were some mathematical ideas that were difficult to grasp.
This blog series will be designed as an ELI5 (Explain Like I’m Five), intended for people who have zero clue to beginner level ideas about machine learning.
This first blog post tackles the first item, intuition, in the list of fundamental concepts in machine learning and artificial intelligence we’ll be covering:
- Intuition: Modeled from a human’s brain, neural networks
- Data (dimensions, quality, characteristics, quantity)
- Mathematical functions: simple to complex (linear, quadratic, etc)
- Data collection
Intuition: A human’s brain
In designing how we can teach a machine how to learn, we modeled (imitated) it based on how human brains work. A surface level description of how that works is that our brains have neurons and synapses which send messages to each other, depending on the results (of our behavior, our environment, our perceptions, etc).
We learn from our parents by mimicking what they say and do. In like manner, we teach machines how to think by mimicking what our brain does.
At a human-to-human level, neurons and synapses work something like this:
Narrative example
Alice, Bob, and Carlos are classmates. Whenever they have upcoming exams, Alice approaches both boys to ask for help to study for the exams. The day prior to the exam, Alice studied with Bob and she also studied with Carlos.
After receiving the results of the test, she found out that majority of what Bob taught her was correct, while a lot of the things Carlos covered was incorrect. She tells herself, “alright, Bob is pretty reliable. I can come to him again later to ask for help. Maybe Carlos only misunderstood a few things. I’ll study with him again next time just to check.”
On the next exam however, the results were the same. Bob was reliably a better study partner, while practices with Carlos only confused her.
She tells herself, “Okay- I’m quite confident now that I should rely on Bob when it comes to my studies, and that I shouldn’t study with Carlos because I only get confused during the test.”
Example numbers
We could come up with a simple mathematical example below, with made up figures for the numbers just so that we could see how Alice learns who to choose to study with.
The given values are:
- Alice has 0.50 confidence level for both study partners at first.
- Alice ‘reacts’ or adjusts her confidence levels by a factor of 0.20.
- Alice’s criteria/threshold for whether she should study with someone is 0.25, which translates to: if I feel at least 25% confident about a study partner, I’ll study with them again (activation function, threshold)
| Alice’s study partner | Alice’s confidence level before first exam | Alice’s confidence level after first exam | Alice’s confidence level after second exam |
| Bob | 0.50 | 0.70 | 0.90 |
| Carlos | 0.50 | 0.30 | 0.10 |
As seen above, Alice doesn’t discriminate at first, giving both boys the same confidence level (both 0.50) prior to the exam. However, after the results of the exam, Alice readjusted her confidence levels (weights/synapses) accordingly (Bob increases from 0.50 to 0.70, while Carlos reduces from 0.50 to 0.30).
Both boys still pass the activation function (at least 0.25) and so she studies with them both again.
After the second exam, her confidence level in Carlos decreases further (from 0.30 to 0.10) which means it no longer meets Alice’ required confidence level (or activation function threshold) because 0.10 is less than 0.25. This is why she stopped studying with Carlos.
Terminologies, analogies
I like the above analogy, because both tasks (studying for an exam and machine learning) are both about learning.
So let’s take a look at learning at three difference scenarios:
- the first would be the human-to-human learning experience described in the narrative above,
- the second would be how a single brain encodes learning through its network of neurons,
- the third would be how we designed machines to learn similar to the brain
Let’s check it out:
| Analogy | How the brain works | How machine learning works |
| Exam for human learning | The task of the brain to learn | The task for the machine to learn |
| Having a perfect score on the exam | The expected result of the brain | The expected result of the machine |
| Having an actual score that varies from 0 to 100 | The actual result from the world/experience | A mathematical score based on statistics |
| Alice, Bob, and Carlos as human students | The neurons; in an organic neural network (the brain) | Digitally coded neurons, part of a neural network |
| Alice’ minimum level of confidence before choosing to study with a student | The activation level of a synapse | An activation level of a synapse, represented as a threshold value (e.g. > 0.50) |
| Alice studying with another student | Activating or firing (organic) neurons | Activating or firing (digital) neurons |
| Alice pondering about the performance of both Bob and Carlos, as factors for considering future studying sessions | Back propagation; Based on results, specific synapses are strengthened or weakened | Back propagation; the weights or multipliers (you can think of it as 0 to 1 values) which represent the weakness (closer to 0) or strength (closer to 1) of a synapse are adjusted based on the results |
| Alice becoming more inclined to rely on Bob because of his past performance | Strengthening a synapse between one neuron to another (making it easier to activate a specific neuron); | Increasing the weights of a neuron closer to 1, making it more likely to activate. |
| Alice giving Carlos another shot, but no more after the second exam | Although the synapse was weakened after the first exam, the neuron still passed its activation level. So Alice pushed forward with the study session. However, after its synapse was further weakened on the second exam, the activation level was no longer reached. | Reducing the weight of a neuron closer to 0, but still passing the threshold. Reducing it again a second time, but no longer passing the threshold. |
The above table shows the analogies or similarities of the same process of learning, but across different scales (inter-personal vs. intra-personal) and situations (organic, natural vs. digital, machine):
- the analogy at a human-to-human scale (left column)
- how one person’s brain or neural network functions (center column), and
- how we perform machine learning based on how human brains work (right column)
We learn from our parents by mimicking what they say and do. In like manner, we teach machines how to think by mimicking what our brain does.
Mathematically speaking, we encode these neurons and synapses as nodes (neurons) in a graph and weighted vertices (synapses) respectively. This where the machine learning terminology neural network comes from: a network of nodes with weights on the directed vertices.
But what is a neural network?
If you’re more of a visual and auditory learner, I recommend watching 3 Blue 1 Brown’s video on what a neural network is.
Closing
A lot of the math may get overwhelming and confusing especially because of the very technical terminologies, but it really boils down to imitating what real life already does (since it is proven to already work). Making these biological and technological discoveries accessible to more people is important to me, which sparked this blog series on machine learning.
I’ll be writing the next sections in another blog post, so stay tuned. You can follow me on Twitter, where I regularly post my updates on my blog.
How did you like the article? Thank you very much for taking the time to read this, and I hope you enjoyed it.
If you had any questions or you want to send me a message, feel free to tweet me.
I just read an ELI5 on machine learning! @darrensapalo
Tweet