Deep learning is the subset of machine learning built on deep — many-layered — neural networks. Each layer builds a more abstract representation: in images from edges to shapes to objects; in text from characters to words to meaning.
The real leap came after 2012: with powerful GPUs and big data, deep networks overtook hand-crafted methods in vision and speech, and became the platform on which transformers and today’s language models were built.
The price tag: an appetite for data and compute. Deep models need millions of examples and days of GPU-cluster training — a cost that later fueled distillation, quantization and the small-model movement.
