What Is Deep Learning?

1. The Basics

Deep learning is a type of machine learning, which is a branch of artificial intelligence (AI). It uses computer systems called neural networks, whose design is loosely based on the human brain. Today, deep learning powers most of the advanced AI you hear about, including chatbots, image generators, self-driving cars, and robots.

In traditional machine learning, people often have to decide which details in the data the computer should pay attention to. Deep learning is different. A deep learning model studies huge numbers of examples and figures out for itself which patterns matter.

Here is one way to picture the difference. Traditional machine learning tries to describe the data with a single line or curve. Deep learning builds its answer out of many small, adjustable pieces. Put together, those pieces can form almost any shape the data needs. That flexibility is what makes deep learning so powerful.

2. Inside a Neural Network

A neural network is made of layers of artificial neurons, also called nodes. Each neuron does one small math calculation and passes its result to the neurons in the next layer. There are three kinds of layers:

• Input layer: receives the data. Each input neuron handles one piece of it. For example, a model that looks at a tiny black-and-white image that is 10 pixels wide and 10 pixels tall would have 100 input neurons, one for each pixel.
• Hidden layers: the layers in the middle. This is where most of the learning happens. Early hidden layers notice broad patterns, and later ones notice finer details.
• Output layer: gives the final answer, called the prediction.

Having many hidden layers is what makes a network “deep.” Deep learning usually means a model with at least four layers, and modern models often have far more.

Every connection between two neurons has a weight. A weight is a number that controls how much influence that connection has. A larger weight makes the signal stronger, and a smaller weight makes it weaker. Each neuron also has a bias, an extra number that nudges its result up or down. Together, the weights and biases are called the model's parameters. When you hear that an AI model has “8 billion parameters,” it means the model has 8 billion of these adjustable numbers.

When a finished model makes a prediction, the data flows forward from the input layer, through the hidden layers, to the output layer. This trip through the network is called a forward pass.

3. How a Network Learns

A brand-new network starts out with random weights and biases, so its first guesses are mostly wrong. Training is the process of fixing that. It works as a loop:

1. Guess. The model makes predictions on a group of training examples.
2. Measure. A loss function measures how far the predictions were from the correct answers. The bigger the loss, the worse the model did.
3. Trace the blame. Backpropagation (short for “backward propagation of error”) works backward from the output layer to the input layer. It figures out how much each weight and bias added to the error.
4. Adjust. Gradient descent uses that information to nudge every weight and bias a small step in the direction that lowers the error.

This loop repeats thousands or even millions of times, until the error is low enough. Each small adjustment is one step in which the model “learns” from its training data. You can think of gradient descent like walking down a hill in thick fog: you cannot see the bottom, but you can feel which way the ground slopes, so you keep stepping downhill.

Most deep learning uses supervised learning. In supervised learning, the training data is labeled with the correct answers, like photos tagged “cat” or “dog.” But large networks need massive amounts of data, and labeling millions of examples by hand is slow and expensive. So many large models use self-supervised learning, which lets a model learn from data that has no labels. Self-supervised learning is the foundation of many of today's generative AI tools.

4. Why Deep Learning Took Off

Neural networks have existed since the early days of machine learning, but they did not become truly useful until the late 2000s and early 2010s. The biggest reason was powerful graphics processing units (GPUs). A GPU can do a huge number of calculations at the same time, which is called parallel processing. That is exactly what training a neural network requires.

Deep learning still needs much more data and computing power than traditional machine learning. That is why many companies rent computing power over the internet through cloud computing instead of buying all the machines themselves.

5. The “Black Box” Problem

Deep learning's power comes with a downside. A model's answer comes from billions of tiny calculations, so it is very hard for people to see why it gave that answer. For this reason, deep learning models are often called “black boxes.” Traditional machine learning models, where people picked the important details by hand, are usually easier to explain.

6. Types of Deep Learning Models

A basic neural network cannot handle every job well, so researchers have designed special types of models for different kinds of data.

Convolutional neural networks (CNNs)

CNNs are mainly used for computer vision, such as recognizing images and finding objects in photos. They use small grids of numbers called filters that slide across an image looking for features like edges or corners. Early layers find simple features, and deeper layers combine them into more complex ones. Because the same filter is reused across the whole image, a CNN needs far fewer parameters than a regular network.

Recurrent neural networks (RNNs)

RNNs are built for sequential data, where order matters, such as sentences, speech, or temperatures over time. An RNN feeds the result of each step into the next step. This gives it a hidden state, a kind of memory of what came before. Standard RNNs struggle with long sequences, but improved versions, such as long short-term memory (LSTM) networks, can remember information for longer.

Transformers

Transformers were introduced in a famous 2017 research paper from Google called “Attention Is All You Need,” and they started today's generative AI boom. They use self-attention, which lets the model work out which parts of the input are most related to each other and focus on the most important parts. Transformers power large language models (LLMs) and chatbots. They can also work with audio and even images.

Generative adversarial networks (GANs)

A GAN uses two networks that compete against each other. The generator creates new, fake data, such as images. The discriminator looks at both real images and fake ones and tries to tell which is which. Each round, the generator learns to make more convincing fakes. Training continues until the discriminator can no longer tell the difference. GANs can make very realistic results, but they are tricky and unstable to train.

Diffusion models

Diffusion models are one of the most popular ways to generate images, and they can also create text, video, and audio. During training, the model adds random noise (like TV static) to an image, step by step, until the picture is unrecognizable. Then it learns to reverse the process. To create a brand-new image, it starts with pure noise and removes it one step at a time.

7. Building Deep Learning Models

Programmers build deep learning models with free, open-source tools such as PyTorch, TensorFlow, and the Hugging Face Transformers library. They can build a model from scratch, or they can fine-tune a pretrained model, which means taking a model that has already been trained and giving it extra training for a specific task. If you want to start building models yourself, IBM recommends learning the Python programming language first.

Adapted for grades 9–10 from Dave Bergmann, “What Is Deep Learning?” IBM Think, 2025.