Exhibit 5.3
2017The Transformer
Google researchers publish “Attention Is All You Need”. The attention mechanism allows the model to relate each token to other tokens in the sequence.
Why it is in the museum
Language is treated as a prediction problem. From Shannon’s statistical sequences, the story leads to Transformers and large language models.
What supports this exhibit
Primary conference paper
The Transformer architecture and its use of self-attention.
Main source: Vaswani, A. et al. (2017), “Attention Is All You Need”, NeurIPS.
What to keep in mind: Primary source.
Ask the exhibit
This small guide uses only the information documented on this exhibit page. If your question goes beyond that evidence, it will say so instead of inventing an answer.
Start with one of the suggested questions above, or type your own.