"It was too tired." What is "it"?

The trick behind every chatbot

Tap to start →

Old models forgot

Reading one word at a time, early words faded.

Who matters to me?

Each word scores every other word. "it" → animal.

Like a YouTube search

Query: what you type. Key: the title. Value: the video.

Attention Is All You Need

Every word looks at every word, at the same time. That made training fast.

The mistake

Thinking attention knows word order. Position is added in separately.

Can you get it?

Tap an answer. The full video explains why.

Cheat sheet

• Old: one word at a time
• Transformer: all at once
• Attention: who matters?
• Query, key, value
• Stack dozens of layers

Want the full lesson?

Free Generative AI course. Subscribe to Compiled Canvas on YouTube.