> ## Documentation Index
> Fetch the complete documentation index at: https://builditwithai.xyz/llms.txt
> Use this file to discover all available pages before exploring further.

# How Transformers Work

> The architecture behind modern AI (simplified, no math)

# How Transformers Work

## The Architecture Behind Modern AI

Transformers are the breakthrough technology that made modern AI possible. Introduced in 2017 in a paper called "Attention is All You Need," they revolutionized how AI processes and generates language.

<Note>
  **Don't worry:** We'll explain this without math or technical jargon. You don't need to understand the engineering to use AI effectively—but knowing the basics helps you use it better.
</Note>

## Why Transformers Changed Everything

Before transformers, AI processed text sequentially (word by word, in order). This was slow and struggled with long-range relationships.

**The Old Way (RNNs/LSTMs):**

```
The cat → sat → on → the → mat
(Process one word at a time, left to right)
```

**The Transformer Way:**

```
The cat sat on the mat
(Look at all words simultaneously and understand relationships)
```

This parallel processing made AI:

* **Faster** - Process entire sentences at once
* **Better at context** - Understand relationships between distant words
* **More scalable** - Can be trained on massive datasets

## The Key Innovation: Attention Mechanism

The "attention" mechanism is what makes transformers special. It allows the AI to focus on relevant parts of the input when generating each word.

### Simple Example

When generating a response to: "The trophy doesn't fit in the suitcase because **it** is too big"

The AI needs to figure out what "it" refers to. Attention helps it:

1. Look at all words in the sentence
2. Determine "it" most likely refers to "trophy" (not "suitcase")
3. Use this understanding to generate the correct response

**Why this matters for you:** This is why modern AI can:

* Understand context and nuance
* Handle long conversations
* Maintain coherence across paragraphs
* Answer questions about specific parts of documents

## How Transformers Process Your Prompt

When you send a prompt to ChatGPT or Claude, here's what happens (simplified):

### 1. Tokenization

Your text is broken into tokens (roughly 4 characters each)

```
"Hello, how are you?" → ["Hello", ",", " how", " are", " you", "?"]
```

### 2. Embedding

Each token is converted to numbers the AI can process

```
"Hello" → [0.23, -0.45, 0.67, ...] (hundreds of numbers)
```

### 3. Attention Layers

Multiple layers analyze relationships between all tokens

* Which words relate to which?
* What's the context?
* What's the intent?

### 4. Generation

The model predicts the next token, adds it, and repeats

```
"Hello, how are you?" → "I'm" → "doing" → "well" → "..."
```

## Why This Matters for Users

Understanding transformers helps you:

**1. Write Better Prompts**

* Provide clear context (attention mechanism uses it)
* Put important information early (though transformers handle long text well)
* Be specific about what you want

**2. Understand Limitations**

* Context windows (how much text the model can "see" at once)
* Why very long documents might need special handling
* Why the model sometimes loses track in very long conversations

**3. Choose the Right Tool**

* Different models have different context window sizes
* Some are optimized for speed, others for quality
* Understanding the architecture helps you pick the right one

## Key Concepts

<AccordionGroup>
  <Accordion title="Context Window">
    The maximum amount of text (input + output) the model can process at once.

    * GPT-4 Turbo: 128,000 tokens (\~96,000 words)
    * Claude 3: 200,000 tokens (\~150,000 words)
    * Gemini 1.5 Pro: 1,000,000 tokens (\~750,000 words)

    **Why it matters:** Determines how much text you can include in one conversation or document analysis.
  </Accordion>

  <Accordion title="Parameters">
    The internal settings that determine the model's behavior. More parameters generally mean more capability (but also more cost and slower speed).

    * GPT-4: \~1.7 trillion parameters
    * Claude 3 Opus: Undisclosed (estimated hundreds of billions)
    * Llama 3: 8B, 70B, 405B parameter versions

    **Why it matters:** Bigger isn't always better—choose based on your needs (speed vs. quality).
  </Accordion>

  <Accordion title="Training Data">
    The text the model learned from. Typically includes:

    * Books and articles
    * Websites and forums
    * Code repositories
    * Academic papers

    **Why it matters:** Models only know what they were trained on. They have a "knowledge cutoff" date.
  </Accordion>
</AccordionGroup>

## The Transformer Family Tree

Transformers evolved into different architectures:

**Encoder-Only** (BERT)

* Good at understanding and classifying text
* Used for: Search, sentiment analysis, classification

**Decoder-Only** (GPT, Llama)

* Good at generating text
* Used for: ChatGPT, writing assistants, code generation

**Encoder-Decoder** (T5, BART)

* Good at transforming text
* Used for: Translation, summarization

<Note>
  **For everyday use:** Most tools you'll use (ChatGPT, Claude, Gemini) are decoder-only models optimized for generation.
</Note>

## Curated Resources

<CardGroup cols={2}>
  <Card title="Attention is All You Need" icon="file" href="https://arxiv.org/abs/1706.03762">
    The original transformer paper (technical)
  </Card>

  <Card title="Visual Guide to Transformers" icon="image" href="https://jalammar.github.io/illustrated-transformer/">
    Jay Alammar's illustrated explanation
  </Card>

  <Card title="How GPT Works" icon="video" href="https://www.youtube.com/watch?v=wjZofJX0v4M">
    3Blue1Brown's visual explanation
  </Card>

  <Card title="Transformer Math" icon="book" href="https://www.datacamp.com/tutorial/demystifying-mathematics-concepts-deep-learning">
    For those who want the math (optional)
  </Card>
</CardGroup>

## Next Steps

Now that you understand the technology, let's explore what it can create:

<Card title="Text Generation" icon="arrow-right" href="/ai-101/text-generation">
  Learn how AI generates text and what you can do with it
</Card>