Machine Learning on the Cloud: Notes for the Layman

What is Machine Learning?

Machine learning is a set of tools and techniques that learn patterns from data in order to predict outcomes and classify events. The data describes each example through variables, which are also called features or attributes. Software libraries now wrap the mathematics of machine learning in application programming interfaces (APIs). This has put the technology within reach of far more people.

Open-source libraries such as scikit-learn, PyTorch, TensorFlow and JAX moved machine learning from mathematics into everyday software engineering. A wave of software and analytics products that use machine learning and deep learning followed.

A Word on Deep Learning

Deep learning is the branch of machine learning that uses neural networks with many layers. Its main applications include computer vision (for example, object recognition and image captioning), natural language processing, speech recognition and machine translation. The main families of neural networks in deep learning are:

1. Multilayer perceptrons (MLP)
2. Convolutional neural networks (CNN)
3. Recurrent neural networks (RNN)
4. Transformers

The transformer architecture was introduced in 2017 in the paper Attention Is All You Need. Transformers now power large language models and many other foundation models. A foundation model is a large model trained on broad data that can be adapted to a wide range of tasks.

Deep learning is also a core technology in self-driving cars.

Enter Big Data

What counts as big data keeps changing. A few gigabytes of data once seemed large. Today, organizations work with terabytes and petabytes. The growth in available data drove much of the success of machine learning and deep learning. Large language models, for example, are trained on text collections that contain trillions of tokens. Put simply, there is no learning without data.

Computational Challenges of Learning

Computing power is a major challenge in training computers to learn with machine learning and deep learning (ML/DL). On a single desktop processor, one experiment can take hours, days or even weeks before the algorithm converges and produces a result.

This delay hurts because a good result takes many experiments. We tune the hyperparameters of the algorithm and try different forms of feature engineering until we find a model that generalizes well to new examples.

High-end machines with modern graphics processing units (GPUs) are expensive to buy and maintain. That cost may be out of reach for an aspiring practitioner, researcher or enthusiast. Building and running a cluster of machines also takes operations skills. For someone who only wants to explore ML/DL, that work is a distraction.

The Cloud to the Rescue

The cloud is a name for large sets of computers networked together in buildings called data centers. Cloud providers run data centers in many parts of the world. They rent computing power, storage and networking to the public over the internet.

Large companies such as Google, Amazon, Microsoft and IBM offer these resources to businesses and individuals, who pay for what they use.

You can rent machines with many processor cores and modern accelerators from Google Cloud, Amazon Web Services (AWS) or Microsoft Azure. The accelerators include NVIDIA GPUs and, on Google Cloud, Google's own Tensor Processing Units (TPUs). Each provider also runs a managed platform for building, training and deploying models:

Provider	Machine learning platform
Google Cloud	Gemini Enterprise Agent Platform, which now provides the services formerly offered as Vertex AI
AWS	Amazon SageMaker AI
Microsoft Azure	Azure Machine Learning

A grid of experiments that takes many hours on a laptop can run in parallel on many cloud machines and finish in a fraction of the time. We can use thousands of cores for a short period and release them when the job is done.

Cost is another advantage of the cloud for ML/DL. A high-end machine that you buy sits idle whenever you are not using it. An on-premises cluster also needs constant operations work, and you can end up spending more time on operations than on analysis. On the cloud, you rent machines for as long as a job needs them. Cloud bills grow with use, so it is good practice to set a budget alert and to shut down machines you are not using.

The principle is simple. Use the resources the job needs, and release them when it is done.