Understanding Machine Learning: An Executive Overview

Machine learning has grown in prominence over the past two decades. Businesses now use it to automate many tasks.

This post gives an overview of the main parts of machine learning. By the end, you should know the main landmarks of the field and its foundations. The overview also gives you a structure for studying the field in more depth without getting overwhelmed.

What is Machine Learning?

Machine learning is a set of computational tools and mathematical techniques that learn from data. A model learns how one variable (the outcome) relates to other variables in a dataset. It then uses what it learned to predict the outcome for new cases.

Medicine gives a simple example. Suppose we want to know whether a patient has diabetes. We can train a model on features such as the patient's age, weight, blood pressure, and geographic and environmental factors. If the model predicts a category, such as "diabetes" or "no diabetes", or "type 1" or "type 2", the task is called classification. If the model predicts a number, such as the patient's blood glucose level, the task is called regression.

In a nutshell, supervised machine learning deals with two kinds of prediction: regression and classification.

What is Learning?

How do we know when learning has occurred?

Let's use a scenario that helps answer this question.

A teacher teaches a physics class for three months. At the end of the lectures, the teacher gives the student a test to find out whether the student has learned anything.

Consider two versions of this story:

1. The teacher tests the student with the exact questions that served as examples in class.
2. The teacher tests the student with a new set of problems that use the principles taught in class.

Only one of these versions lets the teacher be confident that learning has occurred.

Norms of Learning

1. Memorization. In the first version, the test proves little, because the student has seen and memorized the examples. Memorization is the act of storing a pattern so that we can recall it later. For the same reason, we cannot use the training examples to evaluate whether a model has learned.
2. Generalization. In the second version, the test is a fair check of whether the student learned anything during the course. The student's ability to apply the principles from class to new problems is called generalization. Learning is the ability to generalize to new cases.

Large language models make this point more pressing. These models are trained on very large collections of text from the web, which may contain the questions and answers of public tests. A good evaluation uses examples the model has not seen during training.

Core Components of Machine Learning

The three main types of machine learning are:

1. Supervised learning
2. Unsupervised learning
3. Reinforcement learning

Let's look briefly at each one.

1. Supervised learning. In supervised learning, we teach the computer with labeled data. The computer learns the patterns in the data that lead to the desired output. In the diabetes case, we build a predictive model from a dataset of people labeled as having diabetes or not having diabetes. After training, we give the model examples it has not seen to evaluate how well it learned.
2. Unsupervised learning. In unsupervised learning, the examples have no labels. The computer looks for structure in the data on its own, for example by grouping similar examples into clusters.
3. Reinforcement learning. Reward and penalty are central to reinforcement learning. The learner, called an agent, takes actions in an environment and receives a reward or penalty after each action. The agent learns to choose actions that earn the most reward over time. The designer of the system defines the reward, and the reward defines what counts as a good action.

Self-supervised learning has become important in recent years. In self-supervised learning, the labels come from the data itself. A language model, for example, learns by predicting the next word in large amounts of text. Large language models and other foundation models are pre-trained in this way and then adapted to specific tasks, often with supervised learning and reinforcement learning.

Foundations of ML

The foundational disciplines of machine learning are:

• Statistics
• Mathematics
• Computation

The diagram below shows how these fields overlap.

Biology and psychology, in particular mathematical psychology, have also influenced the field.

Unless you work in machine learning research, you do not need a deep understanding of these foundations to start using machine learning. As we see shortly, software packages let businesses apply machine learning algorithms to their problems, provided they have the right data.

Machine Learning Algorithms

Three broad groups of machine learning algorithms are:

1. Linear methods
2. Nonlinear methods
3. Ensemble methods

Note: Some algorithms appear in more than one group.

1. Linear methods assume that the outcome is a weighted sum of the features. In other words, they assume a linear relationship between the features and the outcome. Examples of linear algorithms are:
   • Linear regression
   • Logistic regression
   • Linear support vector machines
2. Nonlinear methods make no assumption about the shape of the relationship. They use the examples to learn it. Examples of nonlinear algorithms are:
   • K-nearest neighbors
   • Classification and regression trees
   • Support vector machines with nonlinear kernels
   • Neural networks
3. Ensemble methods combine the outputs of several models into one model that generalizes better to unseen examples. Two major classes of ensemble methods are:
   • Bagging, as in random forests
   • Boosting, as in gradient boosted trees

Machine Learning Packages

Over the past two decades, machine learning has moved from mathematics into software engineering. One sign of this shift is the number of machine learning packages now available as reusable software. Packages hide the mathematical details from the user.

Python is the main language for machine learning. GitHub's Octoverse 2025 report found that Python remains the clear leader in AI repositories on GitHub. Popular packages include:

Package	Language	Typical use
scikit-learn	Python	Classical machine learning
XGBoost and LightGBM	Python, R and others	Gradient boosted trees
PyTorch	Python	Deep learning
TensorFlow and Keras	Python	Deep learning
JAX	Python	Deep learning and numerical computing
tidymodels and caret	R	Classical machine learning
Weka	Java	Classical machine learning
dlib	C++	Machine learning and computer vision

The rest of machine learning builds on the points in this post. We hope this overview helps you take part in discussions about machine learning and, by extension, artificial intelligence.

Thank you for reading!