Before transformers, language models were built on recurrent neural networks (RNNs) and their more capable variant, LSTMs. Both processed text the way a person reads word by word, left to right, carrying a compressed summary of everything read so far into the next step. That sequential design created two compounding problems: it was slow, because each word had to wait for the previous one to finish processing, and it was forgetful, the further back a piece of context sat, the harder it was for the model to retain it, since everything upstream had to survive being repeatedly compressed and passed forward.

Practically, this meant a model translating a long paragraph or answering a question about something mentioned early in a document would routinely lose the thread. This wasn’t a training data problem, it was a structural limit of the architecture itself.

The paper, and the announcement

In June 2017, eight researchers at Google, published “Attention Is All You Need,” introducing the Transformer architecture. The core move was to drop recurrence entirely and replace it with self-attention: instead of processing words in sequence, the model looks at the entire input at once and directly computes how relevant every word is to every other word, regardless of distance. Google’s own research blog post announcing the work, “Transformer: A Novel Neural Network Architecture for Language Understanding,” published by co-author Jakob Uszkoreit two months later, framed the practical win plainly: on top of better translation quality, the architecture needed less computation to train and was a far better fit for parallel hardware like GPUs and TPUs, cutting training time by up to an order of magnitude.

That parallelizability is the underrated part of the story. It’s not just that transformers understood language better, it’s that they could be trained at a scale RNNs never could, because you could throw more hardware at the problem and actually get a proportional speedup. That’s the direct reason “just add more data and compute” became a viable scaling strategy after 2017, and it’s the foundation the entire LLM era was built on.

Encoder vs. decoder

The Transformer paper describes two halves, an encoder, which builds a rich representation of the input, and a decoder, which generates output based on that representation. Different labs then took the architecture in different directions:

• BERT (Google, 2018) uses only the encoder half. It’s built to understand text deeply, good for classification, search relevance, and extracting meaning from existing text but it doesn’t generate new text.

• GPT and its lineage use only the decoder half, trained to predict the next token given everything before it. That’s what makes it a generation engine.

Knowing this distinction is a fast way to evaluate a vendor pitch: if a tool claims to be great at “understanding intent” versus “generating content,” the underlying architecture family is often a tell for which job it was actually built for.

Where this connects to a live problem: the context window

Context window is expensive, and it’s the direct legacy of solving one bottleneck (sequential processing) while creating another. A transformer processes everything in its context window at once, but every additional token in that window increases compute cost, and the model’s accuracy at recalling and reasoning over information doesn’t stay flat as the window fills, it degrades. Anthropic’s own engineering team calls this “context rot”: as token count increases, a model’s ability to accurately retrieve and reason over everything in that window gets worse, even though the window technically still “contains” the information.

This is exactly the problem “agent harnesses” exist to solve. Anthropic’s published pattern for long-running coding agents splits the work across two roles: an initializer agent sets up the environment and writes structured tests once, then hands off to a second agent that makes incremental progress across multiple separate context windows, using git commits and a persisted feature list as the state that survives when the context resets, instead of trying to cram an entire multi-hour task into one window. Anthropic also documented a concrete, very PM-relevant failure mode they called “context anxiety”: Claude Sonnet 4.5 would wrap up tasks prematurely because it sensed its context limit approaching, so the harness had to add explicit context resets to compensate and when the same harness was later run on a more capable model, that behavior had disappeared and the workaround became dead weight.

The generalizable takeaway: a transformer’s context window is not free memory. Every agentic product decision, what gets kept in context, what gets summarized to a file and reloaded later, what gets handed to a fresh sub-agent with a narrower scope is a product tradeoff between cost, latency, and accuracy, not a solved infrastructure detail.

What to take into your next agentic product conversation

Ask what’s actually being kept in the active context window versus what’s being retrieved on demand (RAG) versus what’s being summarized and reloaded. Those are three different cost and accuracy tradeoffs, and a team that hasn’t made that decision explicitly is usually the same team that will discover the “context rot” problem in production instead of in design.

Sources

• Vaswani, A. et al. (2017). Attention Is All You Need. arXiv:1706.03762, presented at NeurIPS 2017. arxiv.org/abs/1706.03762

• Uszkoreit, J. (2017, Aug 31). Transformer: A Novel Neural Network Architecture for Language Understanding. Google Research Blog. research.google/blog

• Anthropic (2025). Effective context engineering for AI agents. Anthropic Engineering Blog. anthropic.com/engineering/effective-context-engineering-for-ai-agents

• Anthropic. Scaling Managed Agents: Decoupling the brain from the hands anthropic.com/engineering/managed-agents

Every classic machine learning model: logistic regression, decision trees, gradient boosting; depends on a human deciding, in advance, what to measure. Someone has to sit down and say: for this customer, the relevant signals are tenure, return rate, spend velocity, complaint count. That list is the model’s entire universe. The algorithm is only ever as good as the feature list a person handed it.

Deep learning removes that constraint by learning its own representation of the data- pixels, tokens, raw sequences, without a human pre-naming the pattern. The practical question for a PM isn’t “ML or DL,” it’s narrower than that: can I name the feature that predicts the outcome, or can I not? If you can name it, you almost never need DL. If you can’t, because the signal lives in something unstructured like an image of a shoe or a purse, a free-text review, or a voice tone, that’s your signal to reach for DL.

Where this shows up in a marketplace product

Lets take an example of a trust-scoring for a B2B sourcing marketplace. For buyer-side risk (has this company defaulted before, what’s their order volume, how long have they been active), the features are obvious and structured: transaction history, payment terms, account age. A gradient-boosted model on those features would work fine; there was nothing for deep learning to add.

Seller trust is a different problem. The signal that a seller was likely to ghost an RFx or misrepresent capacity is not sitting in a column anywhere, it is buried in how they communicated: response latency patterns, the tone and specificity of their quotes, inconsistencies between what they claimed and what their catalog showed. There would be no spreadsheet column for “this quote reads like it was copy-pasted without reading the RFx.” That’s a feature engineering dead end for classic ML, and exactly the kind of pattern DL is built to extract from unstructured, multi-signal input.

The generalizable version of this test: if you’re building a trust, fraud, or quality signal and you keep listing structured proxies that only approximate the real thing (star rating as a proxy for review authenticity, order count as a proxy for reliability), you’re looking at a feature engineering ceiling. That’s usually the tell.

The nuance to be noted, DL doesn’t eliminate feature engineering

“DL means no more feature engineering” is wrong.

Google’s 2016 paper on the YouTube recommendation system is the clearest documented case of this. YouTube rebuilt their candidate generation and ranking system on deep neural networks specifically to escape hand-engineered features and their own paper says directly that despite that promise, the team still had to spend considerable engineering effort turning raw user and video data into usable inputs, because raw watch-history data doesn’t feed cleanly into a feedforward network on its own. They still hand-engineered signals like the transformed square and square-root versions of continuous features, because it gave the network more expressive power even though in theory a large enough network could learn those transformations itself.

DL reduces the burden of naming atomic features (edges, pixels, word co-occurrence) but doesn’t remove the PM’s job of deciding what raw data and context the model should even see, what the objective function optimizes for, and how noisy implicit signals (a click, a watch, a partial read) get turned into a training label. This is a product decision, not an engineering one.

What to take into a model-choice conversation

A useful gut check I use in scoping conversations: name the three most predictive features you’d hand-pick for this problem. If you can name them confidently and they’re sitting in structured data already, push back on any team defaulting to DL, you’re paying for complexity you don’t need. If the answer is “I know it when I see it”, a template-sounding review, a seller’s evasive quote, a product photo that doesn’t match the listing, that’s your signal the feature space itself needs to be learned, not engineered.

Sources

• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45530.pdf

• Acolyer, A. Deep neural networks for YouTube recommendations — the morning paper (plain-language walkthrough of the above paper, including the feature-engineering caveat). blog.acolyer.org

Here’s the shortcut I’ve started using. Almost every “which algorithm” question collapses into one of six questions about what the product actually needs to decide:

Let me walk through each one, with the examples that made them click for me.

Predict a number → Linear regression

This is the one you’ll run into constantly, because so much of product work is really about predicting a number: an ETA, a price, next month’s demand.

Take a food delivery ETA. The delivery time depends on a handful of things: distance, how long the kitchen takes, traffic, whether it’s raining. Linear regression just says: the outcome is a weighted sum of these inputs.

delivery time = (w1 × distance) + (w2 × prep_time) + (w3 × traffic) + (w4 × rain) + b

The model’s job is to find the weights that make its predictions match reality as closely as possible, using historical data (a classic 80/20 train-test split). Once it’s trained, the weights become a story: if traffic has the highest weight, geography and routing are your bottleneck. If prep time dominates, the fix is in the kitchen, not on the road.

DoorDash runs this kind of prediction across more than 2 billion orders a year, and their engineering team reported a ~20% relative improvement in ETA accuracy after refining their models, because an ETA people can trust is core to whether they come back.

The PM’s job here isn’t building the model, it’s deciding what error is tolerable. Users forgive a five-minute ETA miss. They don’t forgive fifteen. That tolerance number is a product call, not a data science one, and it’s what turns a metric into a ship decision.

Predict yes or no → Logistic regression

Some decisions aren’t a number on a scale, they’re binary. Approve the loan or not. Will this user churn or stay. Is this transaction fraud.

Logistic regression works like linear regression under the hood, but it squashes the output into a probability between 0 and 1. Then someone has to pick the cutoff: above what probability do we actually act? That threshold is where the real product thinking lives, because no classifier is ever 100% right. Every prediction lands in one of four buckets:

Precision is what you protect when a false alarm is expensive, think of a fraud model that’s too trigger-happy and starts declining good customers. Older rule-based fraud systems flagged 70–95% of alerts as false alarms; modern ML models with feedback loops bring that down to roughly 15–30%, which is the difference between wasting a compliance team’s week and actually catching fraud.

Recall is what you protect when a miss is dangerous, a disease screening tool that tells a sick person they’re fine is the failure mode you can’t accept, even at the cost of more false alarms.

This is why the PM sets the threshold, not the model: only the PM knows whether a false positive or a false negative costs the business more.

Explain the decision → Decision trees

Sometimes the output matters less than the reason behind it. If asked “ Why was this post flagged for review?”, a probability score isn’t an answer, a decision tree is.

A tree is basically a flowchart the model builds itself. At each split, it’s choosing the cut that produces the “purest” groups (using something called the Gini score, lower is cleaner). The result is a chain of if-then logic a human can actually trace and defend.

This is the trade-off worth remembering: trees are usually a little less accurate than the more complex models below, but in lending, hiring, or anything regulated, being able to explain the decision is often a harder requirement than squeezing out an extra percentage point of accuracy. That’s a call the PM has to make explicitly, not something to leave to whichever model scored best in a notebook.

Just maximize accuracy → Random forest

One tree is explainable but a bit fragile, it can latch onto quirks in your training data. A random forest fixes this by growing hundreds of trees, each trained on a different random slice of the data, and letting them vote. Individual mistakes cancel out and accuracy climbs.

The cost is that you lose the single clean story. A hundred trees voting isn’t something you can narrate to a VP the way one tree is, so this is the model you reach for once you’ve already decided accuracy matters more than explainability.

The one thing worth watching for as a PM: overfitting. If a tree is allowed to grow deep enough, it can start memorizing your training data instead of learning the actual pattern, splitting on something as oddly specific as an exact address. It’ll look flawless on the data it trained on and then fall apart on real users. So the only accuracy number worth trusting is the one measured on data the model has never seen.

No labels, find the groups → K-means clustering

Everything above assumed you had the “right answer” to train on, labelled supervised data to rely on. Clustering doesn’t. You hand the model raw behavior, nobody tells it what the segments are, and it finds the groups on its own.

K-means does this by picking a number of groups (k), sorting each data point into whichever group’s center it’s closest to, then repeating until the groups settle. This is the algorithm quietly running behind a lot of personalization you experience daily. Netflix groups viewers into taste clusters, and has estimated its recommendation engine is worth over $1 billion a year in retention, with roughly 80% of what people watch starting from a recommendation rather than a search.

Learn from live feedback → Multi-armed bandits

Last one, a normal A/B test splits traffic 50/50 and holds that split for weeks, which means half your users keep seeing the losing version the entire time, even once the data is already leaning hard one way.

A multi-armed bandit shifts traffic toward the winner while it’s still learning, instead of waiting for the test to “finish.” It’s constantly balancing exploring (keep testing to be sure) against exploiting (send more traffic to what’s already winning). Netflix uses exactly this to personalize the artwork for a title, a bandit can learn that a romance fan and a comedy fan should see two different thumbnails for the same movie, and keeps adjusting as more data comes in.

The PM’s job here is picking the reward the bandit chases. Point it at raw clicks and it will happily learn to produce clickbait. Point it at something that reflects real value, a completed order, a 30-day retention, and it optimizes for what actually matters.

The pattern underneath all six

Every one of these algorithms does the same kind of work: find the weights, split the data, group the points, chase the reward. What none of them do is decide what any of it should mean for the business. That’s still the PM’s job, every time:

• The metric: what “good enough” looks like for the user

• The threshold: which mistake you can afford

• The trade: accuracy versus explainability

• The name: turning a cluster number into a real segment and an action

• The reward: the one thing a live system is allowed to optimize for

One more thread I keep coming back to: in every example above, someone had to decide which inputs mattered in the first place, distance, prep time, credit history. For a long time that required domain experts, and it was a real bottleneck. Getting models to discover useful signals on their own, without a human hand-picking every feature, is a big part of what pushed the field toward deep learning. That’s probably its own post.

For now: the algorithm is the easy half. The judgment is still the job.

Now consider the flip side: a customer uploads a photo of a shoe and wants something similar. You could ask an engineer to manually code the attributes, color, silhouette, material, heel height. But what weight does color get versus shape? Who decides? And what if the right match shares a quality you never thought to name?

Two problems, same wall. The algorithm can only see what a human told it to look for. That’s exactly the limit deep learning was built to break.

What Deep Learning Actually Changed

In traditional ML, a human engineers the features. You decide what inputs matter and hand them to the model. It learns the weights; you chose the variables.

Deep learning inverts this. Instead of telling the model what to look at, you give it raw data: pixels, text, audio, and let it figure out what matters. The model learns its own internal representations through many stacked layers, each one finding progressively more abstract patterns in the layer before it. That’s what “deep” means: depth of layers, not complexity of math.

For the shoe problem, the first layer might learn edges. The next, shapes. The next, textures and material. By the output layer, the model has built a representation of “visual similarity” that no human defined and that often outperforms anything a human would have designed.

This is not a small shift. It expands the universe of problems that software can solve. If you cannot define the signal, you can still learn it.

The Text Problem And Why It Was Harder

Images have spatial structure. Text has something trickier: context that spans distance.

Consider: “The startup couldn’t raise its next round, the market had dried up.”

To understand “market” here, a model needs to have absorbed “startup,” “raise,” “round,” and their relationship. Earlier neural network architectures (RNNs) processed text sequentially, word by word, which meant by the time they reached the end of a long sentence, early context had faded.

The deeper problem: static word embeddings gave every word a fixed meaning. The word “bank” had one vector , whether you meant a riverbank or a financial institution. Context was invisible to the model.

Attention Is All You Need

In June 2017, a Google Brain team published “Attention Is All You Need.” The core contribution: self-attention, a mechanism where every word in a sequence simultaneously evaluates its relationship to every other word, regardless of distance.

The result: the word “bank” in “the startup’s bank account was frozen” produces a different internal representation than “bank” in “they camped on the river bank.” Same token, different meaning, because the model has read the whole sentence at once and weighted surrounding context into its understanding of each word.

Before this paper, models processed text like reading a novel one word at a time, forgetting the beginning by the end. After it, the model reads the whole page simultaneously, mapping how every word relates to every other, no matter how far apart.

The architectural implication: recurrence and convolution were eliminated entirely. The model relies solely on attention mechanisms, enabling all positions in a sequence to attend to all others at once. This also made training dramatically faster, parallelizable across GPUs in a way sequential models never could be.

Every major language model today: GPT, Claude, Gemini, BERT, LLaMA,is a transformer.

What This Means If You’re Building Products

Three things that are now just product decisions, not research problems:

1. You don’t need to define the signal. Visual search, fraud detection, fake review identification, if you have labeled examples and enough data, deep learning can find patterns you never named. Your job as a PM is to frame the outcome clearly, not engineer the features.

2. Language is now a structured input. Because transformers understand context, not just keywords, you can build products that reason about intent, not just match strings. A support bot that understands “this is broken” means something different from a user who bought yesterday versus one who bought two years ago.

3. The prompt is a product surface. LLMs are transformers trained to predict the next token. How you structure input, task, context, examples, constraints, directly shapes output quality.

Sources: https://research.google/pubs/attention-is-all-you-need/

Every time you skip a YouTube video, scroll past an Instagram reel, or close the app mid-scroll, something changes. Not just for that one video. For the AI model that decides what you see next.

That’s reinforcement learning (RL), and it’s a genuinely different way of training a model than the supervised learning most of us picture when we hear “AI.” There’s no labelled dataset saying “this is the correct answer.” There’s an agent, an environment, and a feedback loop that runs constantly, the model tries something, watches what happens, and adjusts.

RL runs on four ideas.

• State, what the system currently knows (about you, about the market, about the moment)

• Action, what it decides to do (recommend a reel, match a driver)

• Reward or punishment, what happens as a result

• Policy update, the system nudges its future behavior based on that signal

And then it repeats. Immediately. Every scroll, every ride request, every skip is another lap through that loop.

Instagram

When you open Instagram, content is suggested to you; from accounts, posts, or reels you may be interested in. One of the underlying AI systems decides what to show you and in what order, by predicting what you’re most likely to engage with.

That prediction leans on a mix of signals. Explicit ones, like whether you tapped “interested” or “not interested” on a reel, if you shared a reel, saved it, watched it completely. And implicit ones, who you follow, what kind of creator content you linger on, whether you rewatch something.

At a high level, the system:

1. Gathers inventory: pulls in a portion of the public content available on Instagram, photos and reels included.

2. Leverages signals: watch history, saves, shares, interested/not-interested taps, and a long tail of other inputs.

3. Ranks content: scores everything it has against what it knows about you.

4. Surfaces the top of that ranking: content predicted to hold the most value for you gets shown first.

Now, scoring every single reel against every single user directly would be absurdly expensive; millions of items, billions of users, recalculated constantly. So the system does something cleverer: it learns to represent both users and content as vectors, lists of numbers, in the same mathematical space. Finding a good recommendation becomes a matter of finding which content-vectors sit closest to your user-vector.

This is the two-tower model, and it’s the workhorse behind modern retrieval systems, Meta has publicly described using it for Instagram Explore, and versions of it are industry standard.

• The user tower compresses everything the system knows about you- recent watch history, likes, device, time of day; into one vector. Think of it as a GPS coordinate for your current interests, except in 128 dimensions instead of two.

• The item tower does the same for each reel: topic, creator, audio, engagement stats, age; compressed into a vector in that same space.

Here’s where the reward/punishment framing comes in. If you watch a reel all the way through, that’s a strong positive signal. The model treats it close to a reward. If you skip fast, or worse, close the app entirely, that’s a negative signal; closer to a punishment, because it risks losing your session altogether. Based on these outcomes, the system nudges its weights, this reel type worth more, that one worth less and repeats, at scale, across hundreds of millions of users, every few milliseconds.

The ranking step described above: two towers, retrieval, scoring- is mostly large-scale supervised prediction: “given this user and this content, what’s the probability of engagement?” The genuinely reinforcement-learning part shows up one layer deeper, in how platforms handle the sequential, long-term version of that problem. YouTube’s own researchers published on exactly this: they built a production recommender on REINFORCE, a policy-gradient RL algorithm, specifically to optimize for long-term satisfaction rather than the next single click, and to deliberately balance exploration (showing you something a little outside your lane, to learn if you’d like it) against exploitation (showing you more of what already works). That exploration piece is also the platform’s defense against the filter-bubble problem, if you only ever exploit, the model locks you into a narrower and narrower loop.

What should a PM actually take from this

This is the part that matters more than the mechanics, because as an AI PM, you’re rarely the one writing the RL algorithm. You’re the one deciding what the reward means.

• Reward design is a product decision, not a data science detail. If “reward = watch time,” you will get watch time, and you may also get rage-bait, clickbait, and a feed that keeps people scrolling but leaves them worse off. The PM’s job is defining what “good” actually looks like long-term, not just what’s easy to measure short-term.

• Exploration vs. exploitation is a tradeoff you own, not just an engineering parameter. Too much exploitation, filter bubbles and stale feeds. Too much exploration, an irrelevant, frustrating feed. Where that dial sits is a product call.

• The reward horizon is a product call too. Instagram’s lesson generalizes, optimizing for the next single action (the next match, the next click) can actively hurt the metric you actually care about a few steps later. Ask what the true north star is, and over what time window.

• Cold start doesn’t go away with RL. A new user has no history for the state representation to work with. Someone still has to decide the fallback, heuristics, popularity, demographic priors, until real signal accumulates.

• Because the output is probabilistic, evals matter more than they do in a typical dashboard review. You can’t eyeball whether an RL system is “working” the way you’d review a feature launch. You need a way to catch reward hacking or drift before it shows up as a PR problem.

Sources:

https://transparency.meta.com/features/explaining-ranking/ig-feed-recommendations/

https://paraschopra.github.io/explainers/recco/index.html