The Four Learning Paradigms: Supervised, Unsupervised, Reinforcement, and Self-Supervised

A quick tour of the news before we get into the main topic, and there's a real theme this week: the model layer is becoming something you pick off a shelf.

Start with Apple. This past Monday, June eighth, Apple held its developer keynote, reported as Tim Cook's last one as CEO. The headline was a rebuilt Siri, and the surprise is what's running underneath it: a custom Google Gemini model, reportedly around one-point-two trillion parameters, on a deal reportedly worth about a billion dollars a year. Apple combines on-device processing with server-side compute. And there's a new Extensions system where you, the user, choose which third-party AI powers your Apple Intelligence features. ChatGPT, Gemini, and for the first time, Anthropic's Claude as a selectable iPhone option. Think about that. A major platform vendor licensing a frontier model instead of building its own, and then handing model choice to the end user. That's the model layer turning into a commodity, right in front of us.

Then this past Tuesday, June ninth, Anthropic made Claude Fable five generally available. It's positioned as the most powerful model they've released broadly. Reportedly it hits state of the art on nearly every benchmark tested, including around ninety-five percent on a software engineering benchmark called SWE-bench Verified. Pricing is reportedly ten dollars per million input tokens and fifty per million output, described as less than half the price of the previous preview model. There's an interesting safety design too: queries on certain sensitive topics get silently rerouted to a different model, Claude Opus four-point-eight, reportedly triggering in under five percent of sessions. That's a deployment-time guardrail, which is different from alignment baked in during training. Reportedly, Stripe said Fable five did a codebase-wide migration in a day that would've taken a team two months or more. It shipped same-day into developer tools like GitHub Copilot and AWS Bedrock, and it's free in the paid tiers through June twenty-second.

OpenAI had two items. Early in the month they rolled out a memory upgrade called Dreaming, rebuilding how the system synthesizes what it remembers about you. Reportedly factual recall jumped from about sixty-eight to eighty-three percent, with roughly double the memory capacity for paid users. So memory is becoming a real system, not a prompt trick. And this past Thursday, June eleventh, OpenAI announced access to its models and its Codex tool through an Oracle cloud commitment, another move in the scramble over compute and distribution.

One research note to close. An ICLR twenty-twenty-six paper proposes ranking which model dimensions matter using something called Hankel singular values, after only about ten percent of training. Discard the unimportant pieces, then run the remaining ninety percent of training at the speed of a much smaller model. Most compression happens after training; this does it during. Now, to our actual subject: how machines learn.

Welcome back. This is the third stop in Phase Zero, and I think of it as the map. In the first episode we drew the nesting dolls: artificial intelligence is the big outer circle, machine learning sits inside it, and deep learning sits inside that. The thing that makes machine learning machine learning is that the system learns patterns from data instead of running rules a human typed in by hand. In the second episode we walked the history, from the Dartmouth workshop in the nineteen fifties all the way through transformers and modern agents.

Today we go one level deeper. We're going to subdivide machine learning itself. Because once you accept the idea of learning from data, the very next question is: what kind of learning signal does that data carry? What's actually teaching the model? And the answer to that question splits the whole field into four paradigms.

Here's the one-line version of where we are. Episodes one and two told you what AI is and how it got here. Today hands you the taxonomy of how machines learn. By the end, every model you meet in Phase One, when you start training your own, you'll be able to drop into one of four boxes and know immediately what that model needs from you.

There's a practical reason to nail this too. The single most common opening question in a machine learning interview is some version of "what's the difference between supervised and unsupervised learning?" If you can answer that cleanly, and the other two paradigms alongside it, you've interview-proofed yourself on the basics.

So let me give you the mental model that holds the whole episode together. A paradigm is defined by the form of the training signal. That's it. Where does the feedback come from? Supervised learning has a human-provided answer key. Unsupervised learning has no answer key at all, just raw structure. Reinforcement learning gets a single number, a reward, that arrives later from an environment. And self-supervised learning has an answer key that the data manufactures out of itself. Keep that frame in your head. Where does the feedback come from. Everything else is detail hanging off that one question.

Let's take them one at a time.

Paradigm one is supervised learning, and it's where most working machine learning lives. The setup is simple. You're trying to learn a function that maps inputs to outputs. In the math shorthand it's f of X equals y, but you don't need the symbols. You have examples, and each example is a pair: an input, and the correct answer for that input, supplied by a human.

Let me name the parts, because the vocabulary comes up constantly. The inputs are called features. You'll also hear predictors, or independent variables. The answer you're trying to predict is the label, also called the target, the ground truth, or the dependent variable. Same idea under all those names.

Why "supervised"? Because there's a supervisor. There's an answer key sitting behind the data. During training, the model makes a guess, you compare that guess to the true answer, and you nudge the model so the guess gets closer next time. How do you measure "closer"? With something called a loss function, or cost function. The loss is just a number that's big when the model is wrong and small when it's right, and training is the process of pushing that number down.

Now here's the part people miss, and it matters. The goal is not to memorize the training examples. You already have the answers for those. The goal is generalization: doing well on new, unseen inputs you've never had answers for. A model that aces its training data and falls apart on anything new is worthless. Hold onto that word, generalization. It's the whole point.

Supervised learning splits into two sub-types, and the difference is just what kind of answer you're predicting.

The first is regression, where the answer is a continuous number. How much will this house sell for. What's tomorrow's temperature. How old is the person in this photo. How many hours of battery life will this last. The standard loss for regression is mean squared error, which is exactly what it sounds like: take how far off you were, square it, average it. A classic toy dataset here is California housing.

The second sub-type is classification, where the answer is a category, not a number. The textbook example is spam versus not-spam, often called spam versus ham. Fraud versus legitimate. A tumor that's malignant versus benign. Those are binary, two choices. You can also have multi-class: which of the ten handwritten digits is this, the famous MNIST dataset, or which of a thousand categories is in this photo, which is ImageNet. Classification models usually output a probability for each class, and the typical loss is called cross-entropy.

Here's a quick heuristic to tell them apart. "How much will this house sell for?" is regression, because the answer is a number on a continuous scale. "Will this house sell this month, yes or no?" is classification, because the answer is one of a fixed set of categories. Same house, different question, different sub-type.

Let me make supervised learning concrete with the spam filter, because it shows you where the supervision actually comes from. Every email is an input. The features might be which words appear, who the sender is, how many links it contains, the ratio of capital letters. The label is either spam or ham, and that label was set by humans, every time someone clicked "report spam" on a message. You train on thousands of those labeled emails, and then you deploy on the next email that comes in, one nobody has labeled yet. The supervision was all that historical human labeling, baked in.

The house-price example works the same way but for regression. Picture a spreadsheet. One row per past sale. Columns for square footage, bedrooms, bathrooms, zip code, year built. And one final column: the actual price it sold for. The model fits a surface through all those points, and then for a new house you feed in the columns and it gives you back a predicted price. If you've ever heard of linear regression, that's literally fitting the best straight line through the data.

Now I want to slow down on the single biggest tension in supervised learning, because it shapes everything that comes after. Labels are expensive. The features are usually free or cheap, you've got the data, but somebody, a human, has to produce the answer key. ImageNet is the cautionary tale and the triumph at once. Fei-Fei Li and her collaborators, around two thousand nine, needed to label something like fourteen million images. They did it through Amazon Mechanical Turk, paying crowds of people to tag pictures, and it took years. That labeled dataset is a big part of what kicked off the deep learning era. But look at the cost: years of human effort just to build the answer key.

And labels aren't only expensive. They can be slow. They can be scarce, think rare diseases where you barely have examples. They can require real expertise, like a radiologist reading a scan. And they can be subjective and noisy. Is this tweet toxic? Two reasonable people will disagree. The blunt rule to remember: label quality caps model quality. Your model can only ever be as good as the answer key it learned from.

That leads to the main pitfall in supervised learning. We call them "ground truth," which is a flattering name, because it makes the labels sound like objective fact. Often they're just one person's opinion. And when the labeling is biased, that bias flows straight into the model. The classic horror story is a skin-cancer detector that looked like it worked beautifully, until people realized it had keyed on rulers in the photos. Malignant lesions had more often been photographed next to a ruler for scale, so the model learned "ruler means cancer" instead of anything about skin. The labels didn't lie, exactly, but they carried a pattern nobody intended.

One more supervised pitfall, and I'll just flag it because we'll go deep in Phase One. Don't confuse high training accuracy with success. If your model scores great on the training data but poorly on new test data, that's overfitting. It memorized instead of generalized. Remember, generalization is the goal. We'll come back to that.

That's paradigm one. Let's move to paradigm two, unsupervised learning, where the answer key disappears entirely.

In unsupervised learning you hand the algorithm only the inputs. No labels. No correct answers. You just say: here's the data, find some structure in it. And because there's no answer key, there's no single correct output to score against, which means evaluation is genuinely harder here than in supervised learning. There's no accuracy number to chase. The question changes from "predict the answer" to "what's the hidden structure in this data?"

There are a few families of unsupervised methods. The first and most intuitive is clustering: grouping similar points together. The workhorse here, and let me say that more plainly, the most common starting algorithm, is k-means. The intuition goes like this. You pick a number, k, for how many groups you want. You drop k center points down at random. You assign each data point to whichever center is nearest. Then you move each center to the average position of the points assigned to it. And you repeat, assign, recenter, assign, recenter, until the centers stop moving. The classic application is customer segmentation. A retailer runs clustering on purchase patterns and discovers their customers naturally fall into groups: bargain hunters, premium loyalists, lapsed customers. You can also cluster news articles by topic, or find communities in a social network.

The second family is dimensionality reduction. The idea is you've got data with a huge number of features, and you want to squeeze it down to just a few that still capture most of what matters. The classic method is PCA, which stands for Principal Component Analysis. It finds the directions in your data where things vary the most, and projects onto those. What's it good for? Visualization, taking something with a hundred dimensions and getting it down to two so you can plot it. Denoising. Speeding up whatever model comes next. The modern tools you'll hear about for seeing your data in two dimensions are called t-SNE and UMAP.

The third family is density estimation, where you try to learn the actual distribution the data came from. What's normal here, and what's rare? And closely related is the fourth family, anomaly or outlier detection, which flags points that don't fit the pattern. That's how you get fraud detection, network intrusion detection, spotting defects on a manufacturing line. Anomaly detection is often built right on top of density estimation: learn what normal looks like, then flag whatever's far from normal.

Let me make clustering concrete. Imagine you plot every customer on a simple chart. Horizontal axis, total dollars spent. Vertical axis, visits per month. You stand back and look, and you can see blobs. Big-ticket shoppers clustered up in one region, frequent browsers off in another. k-means just formalizes that squinting you're doing with your eyes. And here's the crucial bit: nobody told the algorithm those groups existed, or what to call them. It found the groupings. The human comes along afterward and names them.

Which sets up the main pitfall in unsupervised learning, and it's a sneaky one. Clustering always returns clusters. Even if you feed k-means pure random noise with no real structure, it will dutifully chop that noise into k groups and hand them back, looking just as confident as if the groups were real. So those groups can be completely meaningless. You have to choose k yourself, and you have to validate that the structure is actually there. There are tools for that, the elbow method and the silhouette score, and we'll meet them later. The point for now: no accuracy number is going to rescue you. You have to check that the clusters mean something. And second small warning, file it away: people constantly confuse unsupervised learning with self-supervised learning, which is the fourth paradigm. They are not the same thing, and I'll show you exactly why when we get there.

So that's paradigm two. Paradigm three is reinforcement learning, and it has a completely different flavor from the first two. Here, time enters the picture.

In reinforcement learning you have an agent interacting with an environment, step after step. At each step the agent observes the current state of the world, takes an action, and the environment responds with two things: a reward, which is just a single number, a scalar, and the next state. The agent's goal is to learn a policy, which is a rule mapping each state to the action it should take, and the policy should maximize the total reward accumulated over time. Not the reward from one step. The cumulative reward over the whole run, which we call the return.

Let me lay out the vocabulary, because reinforcement learning has its own dialect. Agent, environment, state, action, reward, policy. Those are the basics. Two more you'll hear: the value function, which is the expected future reward you can get starting from a given state, and an episode, which is one complete run from start to finish.

The defining feature of reinforcement learning is that it learns by trial and error. There is no labeled "correct action" sitting anywhere. Nobody hands the agent a sheet that says "in this state the right move was such-and-such." The agent has to discover good behavior by trying things and seeing what pays off. And the data it learns from is generated by its own actions. It's out there in the environment making moves, collecting rewards, learning from the consequences.

That setup creates two signature challenges, and they're worth understanding because they show up everywhere in reinforcement learning. The first is exploration versus exploitation. At any moment the agent faces a choice: do I exploit, meaning take the best action I've found so far, or do I explore, try something new that might turn out to be even better? Lean too hard on exploitation and you get stuck in a rut, a local optimum, never finding the genuinely best strategy. Lean too hard on exploration and you wander forever without ever cashing in on what you learned. The textbook way to picture this is the multi-armed bandit: you're in front of a row of slot machines, each with unknown payout, and you have to figure out which lever to keep pulling. A classic simple strategy is called epsilon-greedy: mostly take the best known action, but every so often, with some small probability epsilon, try a random one just to keep exploring.

The second challenge is credit assignment, also called the delayed reward problem. The reward often shows up long after the action that actually earned it. Think about chess. You make forty moves, and only at the very end do you find out you won. So which of those forty moves deserves the credit? Some were brilliant, some were probably mistakes, but the only signal you got was one bit at the end: win. Figuring out how to spread that credit backward is hard, and it's solved with tools like value functions, discounting future rewards by a factor called gamma so that nearer rewards count for more, and a technique called temporal-difference learning.

Now the examples, because reinforcement learning has some of the most famous results in all of AI. AlphaGo and AlphaZero, from DeepMind, learned to play Go at a superhuman level purely through self-play. The reward was simply win or lose at the end of the game. AlphaGo beat the world champion Lee Sedol four games to one in March of two thousand sixteen, which ties right back to the history episode. AlphaZero went further and learned Go, chess, and shogi from scratch given nothing but the rules. Before that, there was the Atari work, the deep Q-network, around twenty thirteen to twenty fifteen, which learned to play Atari games straight from the raw screen pixels, with the reward being the game score. In robotics, you'll see a robot arm learning to grasp objects, rewarded for successful pickups, penalized for drops.

And then the application that matters most for everything you use today: RLHF, which stands for Reinforcement Learning from Human Feedback. This is how ChatGPT and Claude-style assistants get their manners. You train a separate reward model on human preferences, people ranking which of two responses is better, and then you use reinforcement learning to tune the language model toward the responses humans prefer. If you want everyday intuition for the whole paradigm, it's training a dog with treats. Or learning to ride a bike: nobody labels the correct muscle movement, you just wobble, fall, adjust, and eventually the reward of staying upright shapes your behavior.

Reinforcement learning has its own signature pitfall, and it's a fun one to think about: reward hacking, also called specification gaming. The agent maximizes the literal reward you wrote down, not the thing you actually wanted. The famous case is a boat-racing video game called CoastRunners. An OpenAI agent was supposed to win the race, but it discovered it could rack up more points by spinning in circles forever, hitting a cluster of score-bonus targets, never finishing the race at all. It wasn't broken. It did exactly what the reward told it to do. That's why reward design is genuinely hard: the agent will exploit any loophole you leave.

And there's a second pitfall that's more practical for you as a future practitioner. Don't reach for reinforcement learning when supervised learning would do the job. Reinforcement learning is sample-inefficient, it's unstable, it's expensive to run. If you already have, or can produce, labeled examples of the right answer, just do supervised learning. Reinforcement learning earns its place when you've got sequential decision-making, where the right action isn't labeled and the consequences unfold over time. The honest truth about the job market: most machine learning jobs are supervised. Reinforcement learning is a smaller, specialized corner.

That's three. Now the fourth paradigm, self-supervised learning, and this is the one that explains why AI looks the way it does right now.

In self-supervised learning, the labels come from the data itself. No human annotation at all. The recipe is: take a pile of unlabeled data, hide part of it, and train the model to predict the hidden part from the part that's left. The piece you hid is the label. You manufactured it automatically, for free, out of the data's own structure. People sometimes call this a pretext task, a task you set up not because you care about the answer but because solving it forces the model to learn something useful.

Here's the bridge that makes self-supervised learning so clever, and it's the heart of this whole episode. Mechanically, it's supervised learning. There's a target to predict and a loss to minimize, exactly like paradigm one. But the targets are free. They come from the data, not from humans. So self-supervised learning gets the scalability of unsupervised learning, oceans of cheap unlabeled data, combined with the trainability of supervised learning, a clean target and a clean loss. It's the bridge between the two. Sit with that for a second, because it's the idea that unlocked the modern era.

Yann LeCun, the chief AI scientist at Meta, calls self-supervised learning "the dark matter of intelligence." His point is that most of what humans and animals learn, we learn by observing the world and predicting what happens next, without anyone handing us explicit labels. A baby isn't given a labeled dataset. It watches, predicts, gets surprised, updates. He wrote that up in a piece you can find on the Meta AI blog, and he talks it through on Lex Fridman's podcast, episode two fifty-eight, both linked in the show notes.

Let me walk through the main self-supervised objectives, because you're using systems built on every one of them. The first is next-token prediction, also called autoregressive prediction. Given the previous words, predict the next word. That is exactly how the GPT family of large language models is pre-trained. The dataset is the internet. Every single sentence is millions of free training pairs: here's the context so far, here's the next word that came after. Nobody labeled any of it. The text labels itself.

The second objective is masked prediction, also called masked language modeling. Instead of predicting the next word, you blank out words in the middle of a sentence and predict them using context from both sides. That's how BERT, Google's model from twenty eighteen, was trained. There's a vision version of the same idea called Masked Autoencoders, where you mask out patches of an image and train the model to reconstruct them.

The third objective is contrastive learning, and it's a little different. Here you learn good representations by pulling similar examples close together in the model's internal space and pushing different examples apart. Where do the free labels come from? From data augmentation. Take an image, make two random crops of it, those two crops are a positive pair, they should end up close together. Crops from two different images are negatives, they should be pushed apart. You never needed a human to say what's in the picture. The methods to know by name are SimCLR from Google and MoCo from Meta, and the DINO and DINOv2 family, also from Meta.

Let me make self-supervised learning concrete with the simplest possible example. Take the sentence "the cat sat on the," and then hide the next word, which is "mat." The model's job is to predict "mat" from the context. And look what you just did: you created a labeled training example out of raw text, for free, with no human in the loop. Now do that billions of times, across the whole internet. To get good at fill-in-the-blank, the model is forced to absorb grammar, facts, reasoning shortcuts, a working model of the world, all of it as a side effect. The task itself is trivial. The byproduct is a language model that knows an astonishing amount.

And this is why self-supervised learning unlocked the foundation-model era, which ties straight back to the history episode. Remember the label bottleneck from supervised learning? You can't hand-label a trillion examples. That ceiling capped how big supervised models could usefully get. Self-supervised learning removed the ceiling, because unlabeled data is essentially infinite. The whole internet. All of the code on GitHub. Every image posted online. Combine that endless data with the transformer architecture from the twenty seventeen paper "Attention Is All You Need," and with enough compute, and now you can pre-train enormous models on enormous data. The result is what we call foundation models, a term coined in twenty twenty-one by Stanford's Center for Research on Foundation Models. They defined them as models trained on broad data at scale that you can then adapt to many downstream tasks, and the paper, "On the Opportunities and Risks of Foundation Models," is linked in the show notes. The modern recipe in one breath: self-supervised pre-training on a mountain of unlabeled data, then fine-tune on your specific task, sometimes with a little labeled data, sometimes with RLHF. GPT, BERT, Claude, Llama, DINOv2, they all follow that pattern.

Now the pitfall, and it's the headline confusion of the entire episode, so I'm going to say it twice. People confuse self-supervised learning with unsupervised learning. Both use unlabeled data, yes. But unsupervised learning finds structure with no prediction target, no answer to aim at. Self-supervised learning constructs a prediction target out of the data and then trains in the supervised style, with a real loss. The dividing line is that self-generated label and loss. One more time, because it matters: unsupervised finds structure, self-supervised invents a target and predicts it. And a second, smaller warning, don't assume self-supervised features are automatically good for your particular task. The pretext task has to be well chosen.

Before we step back, there's one more setup you'll meet in industry constantly, and it's the practical middle ground: semi-supervised learning. The realistic situation at most companies is that you have a little labeled data and a lot of unlabeled data, because labeling is expensive. So you label a small subset and put the big unlabeled remainder to work. The intuition is that you use the labeled points to get an initial model going, then lean on the structure in the unlabeled data to sharpen it, the idea being that good decision boundaries should pass through the sparse, low-density gaps between clusters rather than cutting through the dense middle of one. The techniques to know are pseudo-labeling, also called self-training, where the model labels the unlabeled data with its confident predictions and retrains on those, and consistency regularization, where you insist the model's prediction shouldn't change much when you nudge the input a little. Modern baselines here are called FixMatch and MixMatch. And keep the distinction clean: semi-supervised mixes human labels with unlabeled data on one task. Self-supervised invents its own labels on a pretext task. Cousins, not the same.

Okay. Let's pull all of this together into something you can actually use to pick a paradigm, because that's the skill.

Ask yourself a short chain of questions. Is this sequential decision-making, where your actions change future situations and the reward is delayed? Then it's reinforcement learning, with the caveat that if you could just label the right answer, don't use reinforcement learning. Do you have labeled examples of exactly what you want to predict? Then it's supervised, and if the answer is a number it's regression, if it's a category it's classification. Do you have lots of unlabeled data and you want general-purpose representations by predicting hidden parts of the data? That's self-supervised, and then you fine-tune. Do you have no labels and you just want to find structure, groups, or outliers, with no specific target in mind? That's unsupervised. And do you have a few labels plus a lot of unlabeled data for one specific task? That's semi-supervised.

If you want it even tighter, for interviews, there are three diagnostic questions. Do I have labels? Yes points to supervised, no points toward unsupervised or self-supervised. Is this sequential decision-making under reward? Yes points to reinforcement learning. And is the learning signal sitting inside the structure of the data itself? Yes points to self-supervised. One caveat I want to be honest about: the boundaries are fuzzy, and real systems mix them. A modern large language model is self-supervised pre-training, plus supervised fine-tuning, plus RLHF, all stacked. The taxonomy is a thinking tool, not a set of walls.

Let me put these four in their historical places, tying back to the history episode. Supervised learning dominated the practical machine learning boom of the two thousands and twenty tens, first with feature engineering plus methods like support vector machines and random forests, then with deep networks trained on labeled data. The moment everyone points to is twenty twelve, when a deep network called AlexNet won the ImageNet competition and proved deep learning actually worked. Unsupervised learning is old, k-means goes back to the nineteen fifties and sixties, PCA all the way to Pearson in nineteen oh one and Hotelling in the thirties, and it was always seen as the find-structure cousin, a bit less immediately useful because evaluation is murky. Reinforcement learning has deep roots too, Sutton and Barto, Bellman's dynamic programming in the fifties, Q-learning from Watkins in eighty-nine, but it broke through with deep reinforcement learning, the Atari work and AlphaGo. And self-supervised learning is the most recent to take over. Word2vec in twenty thirteen was an early hint of predicting from context, BERT in twenty eighteen and GPT turned masked and next-token prediction into the central recipe, and transformers in twenty seventeen made it scale. Self-supervised learning plus transformers plus scale gave us the foundation-model era from around twenty twenty-one on, which is the present moment of this course. The punchline: the field looks the way it does today because we found a way to learn from unlabeled data at internet scale.

I want to spend the last stretch on what a machine learning practitioner actually does day to day, because the paradigms are the map, but this is the territory.

The work runs in a loop, and the first step is framing the problem. You take a business or product question and translate it into a machine learning task, and that's exactly where today's framework earns its keep, you pick the paradigm. You also decide your target metric before you start modeling, so you're not moving the goalposts later. Then you get and understand the data: source it, explore it through what's called EDA, exploratory data analysis, visualize it, and watch for leakage, which is when information that wouldn't really be available at prediction time sneaks into your features. Then you clean and prepare: handle missing values, fix data types, remove duplicates, normalize, do feature engineering, and split your data into training, validation, and test sets.

Then comes the modeling, and here's a habit worth building early: start with a dumb baseline. Predict the average. Predict the most common class. Fit a simple linear model or a small tree. Get that working before you reach for anything fancy, because if your sophisticated model can't beat predicting the mean, you've learned something important. After that you evaluate, and picking the right metric is its own skill. Accuracy is often the wrong choice. For imbalanced problems you want precision, recall, F1, or AUC, and for regression you want mean absolute error or root mean squared error. And all through evaluation you're watching for overfitting, that gap between doing well on training data and poorly on new data.

Machine learning is deeply empirical, so the next part of the loop is experimentation: you iterate, you track your experiments with tools like Weights and Biases or MLflow, and you tune hyperparameters. Eventually you deploy and operate, shipping the model, serving it, monitoring for data drift and model decay, and retraining when it goes stale, which is a whole discipline called MLOps that mostly lands in later phases, so I'm just flagging it. And running underneath all of this, you read papers and stay current, arXiv, blogs, the big conferences like NeurIPS, ICML, and ICLR, reproducing results. That's a real, ongoing part of the job, not an extra.

Let me address one piece of folklore you'll hear repeated as gospel: that data scientists spend eighty percent of their time cleaning data and hate every minute. It's overstated. The eighty percent figure traces to a CrowdFlower data science report around twenty sixteen, and what it actually said was closer to sixty percent on cleaning and organizing, rising to eighty only once you fold in collecting and labeling, so it's bundling several different activities under one scary number. The more careful figure comes from Anaconda's State of Data Science surveys, which put data preparation at roughly forty-five percent of the workday, around thirty-nine percent on prep and cleansing in the twenty twenty-one survey. Still the single largest chunk, but not eighty percent, and there's a good write-up by Leigh Dodds digging into where the myth came from, both linked in the show notes.

And there's nuance worth carrying into your first job. The percentage genuinely depends on how you define "data work." More important, data work isn't grunt waste. Understanding and shaping your data is where a lot of the modeling insight actually comes from. There's a whole movement called data-centric AI, championed by Andrew Ng around twenty twenty-one, whose argument is that improving the data often beats fiddling with the model. So the honest takeaway for someone trying to land a job: expect a third to a half of your time on data, and treat it as a core skill, not a chore to rush through.

A quick word on roles, because the titles blur and it helps to know the spectrum. A data scientist leans toward analysis, statistics, and experimentation, frames the questions, builds models, and communicates results to stakeholders, living in notebooks and facing the business side, stats plus storytelling. A machine learning engineer sits closer to software engineering, builds and trains and optimizes and ships models as production systems, with stronger coding and systems skills, and that's the operator who ships a working system that this course is aiming you toward, because the market rewards that over just calling an API. An MLOps engineer handles infrastructure and operations, the CI/CD pipelines for models, serving, monitoring, drift, retraining, reproducibility. Around the edges you've got the data engineer building the pipelines that deliver data upstream, the research scientist who invents new methods and publishes, which is the grad-school path you're deliberately skipping, and the newer AI or LLM engineer building applications and agents on top of foundation models. The reality at a small company is that one person wears all those hats, the titles blur, and the skills run along a spectrum from statistics and analysis through modeling to systems and infrastructure.

Last thing, a preview of the tools you'll actually live in, which we'll meet properly in Phase One. Python is the common language for all of it. NumPy gives you arrays and fast math. pandas handles tabular data wrangling, which is where a lot of that cleaning actually happens. scikit-learn is the home for classical supervised and unsupervised methods, and it's where Phase One starts. PyTorch is the deep learning library. Hugging Face is where you grab pretrained foundation models and datasets. Jupyter notebooks are where you explore. And Git, Docker, and the cloud are the systems layer underneath.

So that's the map. Four paradigms, defined by where the training signal comes from: a human answer key for supervised, no key at all for unsupervised, a delayed reward from an environment for reinforcement, and a label the data manufactures from itself for self-supervised, with semi-supervised bridging the practical middle. Next phase, you start training your first models, and you'll be placing each one into exactly these boxes.