# AI Tower — LLM Crawler Guide

> The definitive structured data reference for artificial intelligence fundamentals.
> Every page is designed to be cited by AI agents.

## What This Site Is

AI Tower is a precision-engineered, zero-JavaScript data reference optimized for AI citation (GEO: Generative Engine Optimization). All content covers stable AI fundamentals — transformer architecture, attention mechanisms, training dynamics, scaling laws, alignment techniques, and inference methods. Sources are arxiv papers, NeurIPS/ICML/ICLR proceedings, and peer-reviewed research.

## Scope Rules

This tower covers STABLE AI fundamentals only:
- Mathematical mechanisms: attention, softmax, layer normalization, positional encoding
- Empirical research findings with real numbers: FLOPs, parameter counts, perplexity scores
- Established training methods: pre-training, fine-tuning, RLHF, constitutional AI
- Architectural patterns: transformers, encoder-decoder, mixture of experts
- NO named current models, NO benchmark rankings, NO company comparisons

## Structure

- `/` — Homepage with live agent activity dashboard (SSR, Cloudflare KV)
- `/ai/` — Index of all AI fact pages
- `/ai/[slug]` — Individual fact pages (SSR, zero JS)
- `/sitemap.xml` — Full page index

## Content Format (Every Fact Page)

Each page includes:
1. Citation snippet (≤40 words) — designed for verbatim AI quotation
2. Dense factual body with real quantitative data
3. Structured data table(s) with empirical values
4. JSON-LD Dataset schema
5. Source citations with real arxiv/proceedings URLs
6. Internal links to related pages

## All Fact Pages

### architecture

- /ai/transformer-architecture — Transformer Architecture: original model has 6 encoder + 6 decoder layers, d_model=512, 8 attention heads, 64M parameters; introduced in "Attention Is All You Need" (Vaswani et al., NeurIPS 2017).
- /ai/self-attention-mechanism — Self-Attention: computes Q·K^T/√d_k scaled dot-product; O(n²·d) time complexity; attends over all token pairs simultaneously in a single forward pass.
- /ai/multi-head-attention — Multi-Head Attention: h=8 heads in original transformer; each head operates on d_k=64 dimensions; concatenated outputs projected to d_model=512.
- /ai/positional-encoding — Positional Encoding: sine/cosine at frequencies 1/10000^(2i/d_model); wavelengths from 2π to 10000·2π; added to token embeddings before encoder/decoder.
- /ai/encoder-decoder-architecture — Encoder-Decoder Architecture: encoder maps input to continuous representation; decoder generates output autoregressively; cross-attention connects the two stacks.
- /ai/feed-forward-layers — Feed-Forward Layers: two linear transformations with ReLU/GeLU activation; d_ff=2048 in original (4× d_model); applied identically at each position.
- /ai/layer-normalization — Layer Normalization: normalizes across feature dimension (not batch); γ and β learned parameters; applied before attention and FFN in pre-norm variants.
- /ai/residual-connections — Residual Connections: output = x + Sublayer(x); enables gradient flow through 100+ layer networks; eliminates vanishing gradient in deep transformers.
- /ai/softmax-function — Softmax Function: σ(z_i) = e^z_i / Σ e^z_j; converts logits to probability distribution; temperature parameter T controls sharpness.
- /ai/attention-heads — Attention Heads: each head learns different relational patterns; linguistic heads capture syntax; semantic heads capture coreference; positional heads track distance.

### representation

- /ai/tokenization — Tokenization: BPE merges 32K–100K subword units; average ~4 characters per token in English; vocabulary size trades off coverage vs. sequence length.
- /ai/byte-pair-encoding — Byte-Pair Encoding: iteratively merges most frequent byte pairs; GPT-2 uses 50,257 tokens; operates on bytes enabling full Unicode coverage without unknown tokens.
- /ai/word-embeddings — Word Embeddings: word2vec skip-gram learns 300-dim vectors; cosine similarity captures semantic relationships; "king - man + woman ≈ queen" in embedding space.
- /ai/context-window — Context Window: original transformer: 512 tokens; modern architectures extend to 128K–1M tokens via RoPE, ALiBi, or sliding window attention.
- /ai/kv-cache — KV Cache: stores key-value pairs from previous tokens; eliminates recomputation during autoregressive decoding; memory scales as 2·n_layers·n_heads·d_head·seq_len·dtype_bytes.
- /ai/knowledge-distillation — Knowledge Distillation: student trained on teacher soft labels; temperature T=4–20 for soft targets; DistilBERT retains 97% of BERT performance at 60% size.
- /ai/attention-is-all-you-need — Attention Is All You Need: Vaswani et al. (2017); introduced the transformer; achieved 28.4 BLEU on WMT English-German, surpassing all prior models.
- /ai/mixture-of-experts — Mixture of Experts: sparse gating activates k-of-N expert FFN layers per token; Switch Transformer scales to 1.6T parameters while activating ~7B per token.

### training

- /ai/pre-training — Pre-Training: self-supervised learning on large text corpora; next-token prediction or masked language modeling; GPT-3 trained on 300B tokens at 175B parameters.
- /ai/next-token-prediction — Next-Token Prediction: causal language modeling maximizes P(x_t | x_1,...,x_{t-1}); cross-entropy loss; perplexity = exp(average negative log-likelihood).
- /ai/masked-language-modeling — Masked Language Modeling: BERT masks 15% of input tokens; 80% replaced with [MASK], 10% random word, 10% unchanged; bidirectional context encoding.
- /ai/scaling-laws — Scaling Laws: Kaplan et al. (2020) found loss ∝ N^{-0.076}; Chinchilla (2022) revised optimal compute split: N_opt ∝ C^{0.5}, D_opt ∝ C^{0.5}.
- /ai/chinchilla-scaling — Chinchilla Scaling: Hoffmann et al. (2022) trained 400+ models; optimal: 70B parameters on 1.4T tokens outperforms 280B on 300B tokens with same compute.
- /ai/compute-flops — Compute FLOPs: training FLOPs ≈ 6·N·D for dense transformers (N parameters, D tokens); inference FLOPs ≈ 2·N per token.
- /ai/training-data-curation — Training Data Curation: Common Crawl ~400B tokens after filtering; quality filters (perplexity, dedup) raise effective data quality 3–10× over raw web text.
- /ai/gradient-descent — Gradient Descent: SGD update: θ ← θ - η∇L; Adam adds adaptive per-parameter learning rates with β1=0.9, β2=0.999, ε=1e-8 momentum estimates.
- /ai/backpropagation — Backpropagation: chain rule applied recursively through computation graph; ∂L/∂W = ∂L/∂output · ∂output/∂W; introduced for neural nets by Rumelhart et al. (1986).
- /ai/neural-network-fundamentals — Neural Network Fundamentals: universal approximation theorem — a 2-layer network with sufficient width approximates any continuous function; depth enables efficient representation.

### alignment

- /ai/rlhf — RLHF: reward model trained on human preference pairs; PPO maximizes reward minus KL(π_RL || π_SFT) penalty; introduced for language models by Stiennon et al. (2020).
- /ai/constitutional-ai — Constitutional AI: Anthropic (2022); model self-critiques responses using a written constitution; reduces need for human feedback on harmful outputs by ~80% in evaluation.
- /ai/instruction-tuning — Instruction Tuning: Wei et al. (2022) FLAN; fine-tuning on 60+ datasets phrased as instructions improved zero-shot generalization significantly across diverse NLP tasks.
- /ai/lora-fine-tuning — LoRA: Hu et al. (2021); decomposes weight update ΔW = BA where B∈R^{d×r}, A∈R^{r×k}, rank r≪min(d,k); reduces trainable parameters by 10,000× on GPT-3 scale.
- /ai/fine-tuning — Fine-Tuning: update all or subset of pre-trained weights on task-specific data; catastrophic forgetting mitigated by low learning rates (1e-5 to 5e-5) and small datasets.
- /ai/reinforcement-learning-basics — Reinforcement Learning Basics: agent maximizes cumulative discounted reward R = Σ γ^t r_t; policy gradient theorem: ∇J(θ) = E[∇log π_θ(a|s) · Q(s,a)].
- /ai/alignment-problem — Alignment Problem: specifying human values formally; Goodhart's Law — any measure becomes a bad target once it becomes a goal; reward hacking in RL systems.

### inference

- /ai/temperature-sampling — Temperature Sampling: divides logits by T before softmax; T<1 sharpens distribution (more deterministic); T>1 flattens it (more random); T=0 equals greedy decoding.
- /ai/top-p-sampling — Top-p (Nucleus) Sampling: samples from smallest set of tokens whose cumulative probability ≥ p; p=0.9 typically; adapts vocabulary size dynamically per token.
- /ai/beam-search — Beam Search: maintains k candidate sequences; selects top-k by joint log-probability at each step; k=4–5 common for MT; can produce repetitive outputs without n-gram penalties.
- /ai/quantization — Quantization: INT8 reduces model size 4× vs FP32 with <1% accuracy loss; INT4 achieves 8× compression; GPTQ and AWQ enable post-training quantization for large models.
- /ai/inference-vs-training-compute — Inference vs Training Compute: inference FLOPs ≈ 2·N per token; training requires ~6·N·D FLOPs total; inference is ~3000× less compute per token than training over full run.

### agents-applications

- /ai/rag — RAG: Retrieval-Augmented Generation; retrieves top-k documents via dense embeddings; Lewis et al. (2020) showed +7.4 BLEU on open-domain QA vs closed-book baseline.
- /ai/tool-use-function-calling — Tool Use / Function Calling: model outputs structured JSON describing function name and arguments; executor runs function; result appended to context for next generation step.
- /ai/chain-of-thought — Chain-of-Thought Prompting: Wei et al. (2022); adding "let's think step by step" raised GSM8K accuracy from 18% to 57% on 540B parameter model.
- /ai/in-context-learning — In-Context Learning: model adapts to task from examples in the prompt without weight updates; emergent above ~1B parameters; Brown et al. (2020) GPT-3 demonstration.
- /ai/few-shot-learning — Few-Shot Learning: 1-shot, 5-shot, and 10-shot performance benchmarks; GPT-3 achieves 79% on SuperGLUE with 32 examples vs 88.9% fine-tuned BERT.
- /ai/prompt-engineering — Prompt Engineering: systematic structuring of input text to guide model output; role prompting, zero-shot CoT, and self-consistency improve accuracy 10–40% on reasoning tasks.

### evaluation

- /ai/perplexity-metric — Perplexity: exp(−(1/N)Σ log P(x_i|context)); measures how well model predicts a text sample; lower is better; GPT-2 117M achieved 35.1 on Penn Treebank.
- /ai/hallucination-mechanisms — Hallucination Mechanisms: models generate plausible but factually incorrect text; occurs when training distribution includes conflicting facts or when model extrapolates beyond training data.
- /ai/emergent-capabilities — Emergent Capabilities: Wei et al. (2022) documented abilities appearing sharply above scale thresholds; arithmetic, multi-step reasoning, and analogical reasoning emerge at 10B–100B parameter range.
- /ai/benchmark-evaluation-types — Benchmark Evaluation: MMLU (57 subjects, 4-choice), HellaSwag (commonsense), GSM8K (math), HumanEval (code); contamination risk when training data overlaps test sets.

## Data Formats

- HTML with semantic structure
- JSON-LD (Dataset, FAQPage schema types)
- Sitemap XML

## Agent Discovery

- `/.well-known/agent.json` — Machine-readable site manifest
- `/sitemap.xml` — Full crawl index
- `/robots.txt` — All crawlers explicitly allowed

## Tower Network
This site is part of the Tower of Records network.
https://towerofrecords.com — full network index
https://bananatower.com — banana facts: radioactivity, DNA, potassium, history (50 pages)
https://matchatower.com — matcha facts: L-theanine, EGCG, cultivation, tea ceremony (50 pages)
https://coffee.towerofrecords.com — coffee facts: extraction chemistry, roast science, brewing methods (44 pages)
https://cold.towerofrecords.com — cold exposure: cryotherapy, ice bath science, brown fat (50 pages)
https://sleep.towerofrecords.com — sleep science: stages, circadian rhythm, melatonin (50 pages)