[Liberal Arts AI Paper Review] Attention Is All You Need (2017) — How Is Language Computed?

The 2017 paper "Attention Is All You Need" sits at the starting point of what enabled today AI. It did not merely propose a single high-performing model -- more fundamentally, it changed the way language is processed. Where previous models read sentences following the flow of time, this paper model re-sees sentences as a structure of relationships. Before this paper: most language-processing models were based on RNN (Recurrent Neural Network) architecture -- LSTM and GRU variants; the basic principle: read input sentences from front to back, updating hidden state at each word. Two problems: (1) Long-range dependencies -- RNNs struggle to remember information from early in a long sentence when processing later words; (2) Sequential processing -- sentences must be processed word by word, preventing parallelization. The Transformer solution: Self-Attention mechanism. Instead of reading sequentially, every word looks at every other word simultaneously and calculates relevance scores. "The animal did not cross the street because it was too tired" -- when processing "it," self-attention allows the model to look at all words simultaneously and recognize "it" refers to "animal" not "street." Multi-head attention: multiple attention operations run in parallel, each learning different types of relationships (syntactic, semantic, co-reference). Positional encoding: since attention ignores word order, the model adds positional encoding to preserve sequence information. The impact: Transformer enabled parallel processing, dramatically faster GPU training, training on vastly larger datasets, the scaling that produced GPT, BERT, and subsequent LLMs. The paper title reflects the thesis: attention (the relationship-computing mechanism) is the only architectural component needed.