AI/LLM Glossary: 120 Essential Terms

Looking for a fast, accurate guide to the most important words in AI and large language models? Here’s a clean, up-to-date glossary of 120 core terms, ordered roughly by how often people search, discuss, and use them in 2025.

Core LLM Concepts (1-10)

1. Large Language Model (LLM): A Transformer-based neural network trained to predict the next token and perform language tasks.
2. Transformer: The dominant neural architecture using attention instead of recurrence or convolutions.
3. RAG (Retrieval-Augmented Generation): Adds external documents to prompts so models can ground answers in retrieved evidence.
4. Vector Database: A database optimized for similarity search over embeddings (ANN indexes like HNSW/IVF, scalar filters, hybrid search).
5. Embedding: Numeric vector representation of text, images, or audio used for search, clustering, or retrieval.
6. Prompt: The input instruction and context given to an LLM to steer its behavior.
7. System Prompt: Hidden, high-priority instructions that set overall behavior and guardrails.
8. Agents: LLM programs that plan, call tools/APIs, and iterate toward a goal with feedback.
9. Tool/Function Calling: Structured requests where a model outputs JSON arguments to call external tools reliably.
10. Context Window: The maximum tokens the model attends to (input plus generated tokens).

Tokens & Text Processing (11-22)

11. Token: The unit a model reads/writes (sub-words, symbols, or bytes, depending on the tokenizer).
12. Tokenization (BPE/SentencePiece/WordPiece): Algorithms that split text into tokens for efficient modeling.
13. Self-Attention: Mechanism letting each token attend to others to compute contextualized representations.
14. Cross-Attention: Attention across two sequences (e.g., decoder attending to retrieved passages or image features).
15. Softmax: Converts logits into a probability distribution; temperature scales it to control randomness.
16. Logits: Raw scores before softmax; higher means more probable next tokens.
17. Temperature: Scales logits to control randomness; lower is more deterministic, higher is more diverse.
18. Top-p (Nucleus Sampling): Sample from the smallest set of tokens whose cumulative probability exceeds p.
19. Top-k Sampling: Sample from the k most probable tokens after renormalization.
20. Greedy Decoding: Always pick the highest-probability next token (most deterministic, least diverse).
21. Beam Search: Keep the best few partial sequences to approximate the overall best output.
22. Hallucination: Fluent but false or ungrounded output; mitigated by RAG, citations, and better evaluation.

Performance & Optimization (23-35)

23. Grounding: Tying answers to retrieved or trusted sources (documents, tools, databases) to improve accuracy.
24. KV Cache (Past Key-Values): Stores prior attention keys/values to speed up autoregressive decoding.
25. Speculative Decoding: A small “draft” model proposes tokens the target model verifies in parallel for speedups.
26. FlashAttention: Memory-efficient exact attention kernel that reduces HBM traffic and accelerates training/inference.
27. Mixture of Experts (MoE): Sparse layers route tokens to a few specialized experts, increasing parameters without proportional compute.
28. Router/Gating: Learns which expert(s) handle each token in MoE layers while balancing load.
29. Perplexity: Exponential of average negative log-likelihood; lower implies better next-token prediction.
30. MMLU: A broad knowledge benchmark used to evaluate model academic recall and reasoning.
31. GSM8K: Grade-school math word-problem benchmark for reasoning and step-by-step calculation.
32. HumanEval: Code-generation benchmark that executes tests against generated solutions.
33. Chain-of-Thought (CoT): Prompting or training to produce step-by-step reasoning before final answers.
34. Tree-of-Thought (ToT): Explore multiple reasoning branches, then select or vote on the best.
35. ReAct: Interleaves reasoning traces with actions (tool calls) to solve tasks transparently.

Training & Fine-Tuning (36-49)

36. Instruction Tuning: Supervised fine-tuning on instruction-response pairs to follow natural requests.
37. SFT (Supervised Fine-Tuning): General term for supervised training on labeled input-output pairs.
38. RLHF: Aligns models with human preferences using a reward model and reinforcement learning.
39. DPO (Direct Preference Optimization): Preference learning without an explicit reward model; simpler than RLHF.
40. RLAIF: Uses AI feedback instead of (or alongside) human feedback to scale preference learning.
41. LoRA: Low-rank adapters that fine-tune a small set of parameters on top of a frozen base model.
42. QLoRA: Memory-efficient fine-tuning with 4-bit quantization + LoRA adapters.
43. PEFT (Parameter-Efficient Fine-Tuning): Family of techniques (LoRA, adapters, prefix/prompt tuning) to update few parameters.
44. Prompt Tuning / Prefix Tuning: Learnable prompt vectors prepended to inputs instead of updating full weights.
45. Few-Shot / Zero-Shot: Solve tasks with a handful of examples—or none—purely from instructions.
46. Evaluation Harness: Scripts and datasets that run consistent, repeatable LLM evaluations.
47. Guardrails/Moderation: Policies and filters that block unsafe, copyrighted, or private content.
48. Function/Schema Validation: Forcing well-formed JSON or typed outputs (e.g., via adapters) to reduce parsing errors.
49. Context Overflow: When input exceeds the window; requires truncation, summarization, or long-context models.

Model Architecture & Optimization (50-66)

50. Long-Context Techniques: RoPE scaling, ALiBi, attention optimizations, and memory-efficient kernels.
51. Quantization (INT8/INT4/NF4): Lower precision weights/activations to reduce memory and speed up inference.
52. Distillation: Train a smaller “student” to mimic a larger “teacher” model’s behavior.
53. Pruning: Remove weights or neurons with minimal impact to compress models.
54. AdamW: Widely used optimizer combining Adam with decoupled weight decay.
55. Learning Rate Schedule: Warmup and decay strategies for stable training.
56. Batch Size / Micro-Batching: Number of examples per update; micro-batches simulate larger batches under memory limits.
57. Weight Decay: L2-style regularization to reduce overfitting during training.
58. Dropout: Randomly zero activations during training to improve generalization.
59. Label Smoothing: Softens targets to improve calibration and reduce overconfidence.
60. Residual Connections: Skip paths that ease optimization and stabilize deep networks.
61. LayerNorm / RMSNorm: Normalization layers; RMSNorm removes mean-centering for efficiency.
62. Positional Encoding (Sinusoidal/Learned): Injects position information so attention can model order.
63. RoPE (Rotary Positional Embeddings): Encodes relative positions via rotations applied inside attention.
64. ALiBi: Adds distance-based linear bias to attention for train-short, test-long extrapolation.
65. FFN (Feed-Forward Network): Per-token MLP sublayer inside each Transformer block.
66. SwiGLU/GEGLU: Gated activation variants used in modern FFNs for quality and efficiency.

Multimodal & Image Generation (67-79)

67. Multimodal (VLM): Models that accept or produce multiple modalities (text, image, audio, video).
68. Vision Transformer (ViT): Applies Transformer blocks to image patches for vision tasks.
69. Diffusion Model: Learns to denoise noise step-by-step to synthesize images or audio.
70. Latent Diffusion: Runs diffusion in a compressed latent space (via VAE) for speed and quality.
71. VAE (Variational Autoencoder): Encoder-decoder that learns a latent space used by latent diffusion.
72. U-Net: Encoder-decoder CNN backbone common in diffusion denoisers.
73. DiT (Diffusion Transformer): Transformer-based denoiser replacing U-Nets in diffusion pipelines.
74. Classifier-Free Guidance: Combines conditional and unconditional predictions to steer images toward prompts.
75. Guidance Scale: Strength of prompt adherence versus diversity in diffusion sampling.
76. Negative Prompt: Terms to avoid in image generation (e.g., “no text, no extra fingers”).
77. Sampler (DDIM/DPM-Solver): Numerical solvers that trade steps for speed vs. fidelity in diffusion.
78. Seed: Random initialization controlling reproducibility of generated outputs.
79. Inference Steps: Number of denoising steps; fewer are faster, more can improve detail (up to a point).

Vector Search & Retrieval (80-91)

80. HNSW: Hierarchical Navigable Small-World graph index for fast approximate nearest neighbor search.
81. FAISS: Popular similarity-search library for vector indexes on CPU/GPU.
82. Cosine Similarity: Measures angle between vectors; common for text embeddings.
83. Dot Product (Inner Product): Similarity via unnormalized vector projection; common in ANN indexes.
84. Euclidean Distance (L2): Straight-line distance between vectors; used in many ANN backends.
85. Hybrid Search (Sparse + Dense): Combine BM25 keywords with embeddings for recall and precision.
86. BM25: Classic keyword ranking function; strong baseline for sparse retrieval.
87. Reranker (Cross-Encoder): Re-scores candidates with a more accurate but slower model.
88. ColBERT (Late Interaction): Multi-vector retrieval that matches tokens for fine-grained relevance.
89. Chunking: Split long documents into overlap-aware segments for better retrieval and context fit.
90. Overlap / Stride: The sliding context shared between neighboring chunks to preserve continuity.
91. MMR (Maximal Marginal Relevance): Diversifies retrieved passages by balancing relevance and novelty.

Tools & Frameworks (92-100)

92. Embedding Model: The specific model used to compute vectors (domain-specific choices matter).
93. Semantic Caching: Store previous prompts/answers/vectors to speed up repeated or similar queries.
94. Vibe Coding: Iterative, AI-paired coding style—describe intent, let the agent build, then refine based on results.
95. Claude Code: Anthropic’s agentic coding tool that edits files, runs commands, and opens PRs from your terminal/IDE.
96. OpenAI Codex: OpenAI’s early code-generation model (2021) that powered initial GitHub Copilot; foundational but largely superseded by newer GPT-4-class coding models.
97. DSPy: Programmatic framework to define, evaluate, and optimize LLM pipelines with signatures, modules, and optimizers.
98. LangChain: Toolkit to compose LLM workflows (chains, tools, agents) with integrations to models and stores.
99. LlamaIndex: Data framework for RAG (ingestion, indexing, querying, evaluation) across vector stores.
100. Hugging Face: Open ecosystem for models, datasets, Spaces, and libraries like Transformers and Diffusers.

Advanced Inference & Serving (101-112)

101. Multi-Query Attention (MQA): Shares keys/values across heads to cut KV memory and speed decoding with minimal quality loss.
102. Grouped-Query Attention (GQA): Shares K/V within head groups—quality closer to MHA with MQA-like memory savings; used in modern LLMs.
103. Sliding-Window Attention (SWA): Limits attention to a moving window, enabling long contexts with linear memory/runtime growth.
104. PagedAttention (vLLM): Memory manager that pages KV cache blocks to keep GPUs highly utilized and reduce fragmentation.
105. Continuous Batching: Dynamically merges requests of different lengths to maximize throughput in serving.
106. Tensor Parallelism: Splits large matrix ops across GPUs (e.g., by columns/rows) to fit and accelerate massive models.
107. Pipeline Parallelism: Partitions layers across devices and streams micro-batches to keep all stages busy.
108. FSDP / ZeRO Sharding: Shards parameters, gradients, and optimizer states across workers to train models beyond single-GPU memory.
109. FP8: 8-bit floating-point formats (e4m3/e5m2) that reduce bandwidth and memory while preserving training quality with calibration.
110. AWQ: Activation-aware weight quantization that preserves salient channels for accurate 4-bit inference on GPUs.
111. GPTQ: Post-training quantization method that minimizes layer-wise error for accurate 4-bit weights.
112. KV Cache Quantization: Compresses past keys/values (e.g., 8-bit/4-bit) to boost throughput and extend context length.

Emerging Techniques & Methods (113-120)

113. Mamba (SSM): Selective state-space model with linear-time sequence processing and strong long-context performance.
114. GraphRAG: Builds/uses knowledge graphs with retrieval to ground answers in entities and relations, improving factuality.
115. HyDE: Generates hypothetical passages from the query, embeds them, then retrieves real documents matching those pseudo-answers.
116. Self-Consistency: Samples multiple reasoning paths (CoT) and votes—often boosts math and logic accuracy.
117. Reflexion: Agent loop that critiques its own steps, writes takeaways, and retries to improve results.
118. ORPO: Odds-ratio preference optimization—single-stage alignment alternative to RLHF with simpler training.
119. IPO (Implicit Preference Optimization): Preference learning that avoids reward models by matching optimal policies implicitly.
120. Structured Outputs (JSON/Schema): Constrains generations to schemas for reliable tool calls and API-safe responses.

Tip: Bookmark this page. As stacks evolve—new adapters, longer contexts, faster attention kernels—these fundamentals stay useful no matter which model or provider you pick.