Attention Is All You Need: The Google Paper That Built ChatGPT
Episode 21 min 37s

Attention Is All You Need: The Google Paper That Built ChatGPT

In 2017, eight researchers, most of them at Google, published a paper with a catchy title and a radical idea: throw out the old way of processing language and rely on "attention" instead. We explain in plain language how attention lets an AI weigh every word against every other word, why the design made giant models like GPT possible, and how the "T" in ChatGPT traces back to a paper about translation. Plus the irony of history: Google invented the transformer, but OpenAI turned it into a household name.

This episode includes AI-generated content.

Community discussion