Back to AI Glossary
Retrieval & Data

Embeddings

Dense numerical representations of text (or other data) in a high-dimensional vector space, where similar meanings are placed closer together.

Why it matters

Embeddings are the bridge between human language and machine computation. They power semantic search, RAG, recommendations, and virtually every AI feature that needs to understand meaning.

How embeddings work

An embedding model converts a piece of text — a word, sentence, or document — into a fixed-length array of floating-point numbers (typically 768–3072 dimensions). The model is trained so that semantically similar inputs produce vectors that are close together, measured by cosine similarity or dot product.

Use cases

  • Semantic search — find documents by meaning, not just keywords.
  • RAG retrieval — the retrieval step in RAG pipelines relies on embedding queries and documents into the same vector space.
  • Clustering & classification — group similar items or train lightweight classifiers on top of embeddings.
  • Anomaly detection — identify outliers in vector space.

Choosing an embedding model

Key factors: dimensionality (higher = more expressive but slower), context window (how much text it can embed at once), and domain fit. Popular choices include OpenAI's text-embedding-3, Cohere Embed v3, and open-source models like E5 and BGE.

Related Terms

Retrieval-Augmented Generation(RAG)— A technique that grounds a language model's output in external data by retrieving relevant documents before generating a response.Vector Database— A database optimized for storing, indexing, and querying high-dimensional vector embeddings, enabling fast similarity search at scale.

On the AI Radar

Embedding Fine-tuning— Adapting pre-trained embedding models to domain-specific data to improve retrieval accuracy in RAG pipelines and semantic search. Tooling has matured with Sentence Transformers v3 and Unsloth making the process accessible on consumer hardware.pgvector— An open-source PostgreSQL extension that adds vector similarity search with HNSW and IVFFlat indexing directly inside Postgres, allowing teams to store embeddings alongside relational data without a separate vector database.Pinecone— A fully managed, cloud-native vector database designed for similarity search at scale. Provides serverless infrastructure for storing and querying high-dimensional vector embeddings with sub-25ms latency, integrated inference for embedding and reranking, and dedicated read nodes for predictable performance.
Back to AI Glossary