Embedding Models: How Machines Understand Text

An embedding is a list of numbers that captures the meaning of text. Two pieces of text that mean similar things will have similar numbers. That's it. That's the entire concept.

"How do I reset my password?"  → [0.12, -0.45, 0.78, 0.33, ...]
"I forgot my login credentials" → [0.11, -0.43, 0.76, 0.31, ...]  ← similar!
"What's the weather today?"    → [0.89, 0.12, -0.34, 0.56, ...]  ← different!

These number lists (vectors) typically have 256 to 3072 dimensions. More dimensions = more nuance = more storage = slower search.

1 / 7

Ship it

Working embedding pipeline that converts documents to vectors

Architecture Patterns for LLM Apps

Vector Databases: Pinecone, Weaviate, Chroma

Rex — Embedding Models: How Machines Understand Text

Gemini Flash

Ask Rex anything about this lesson

Get help with concepts, run prompts, practice exercises, or ask what to do next.

LLM Orchestration & RAG Systems/Retrieval & Embeddings/Embedding Models: How Machines Understand Text

30 minLesson 4 of 13

Embedding Models: How Machines Understand Text

An embedding is a list of numbers that captures the meaning of text. Two pieces of text that mean similar things will have similar numbers. That's it. That's the entire concept.

"How do I reset my password?"  → [0.12, -0.45, 0.78, 0.33, ...]
"I forgot my login credentials" → [0.11, -0.43, 0.76, 0.31, ...]  ← similar!
"What's the weather today?"    → [0.89, 0.12, -0.34, 0.56, ...]  ← different!

These number lists (vectors) typically have 256 to 3072 dimensions. More dimensions = more nuance = more storage = slower search.

1 / 7

Ship it

Working embedding pipeline that converts documents to vectors

Architecture Patterns for LLM Apps

Vector Databases: Pinecone, Weaviate, Chroma

LLM Orchestration & RAG Systems/Retrieval & Embeddings/Embedding Models: How Machines Understand Text

30 minLesson 4 of 13

Embedding Models: How Machines Understand Text

An embedding is a list of numbers that captures the meaning of text. Two pieces of text that mean similar things will have similar numbers. That's it. That's the entire concept.

"How do I reset my password?"  → [0.12, -0.45, 0.78, 0.33, ...]
"I forgot my login credentials" → [0.11, -0.43, 0.76, 0.31, ...]  ← similar!
"What's the weather today?"    → [0.89, 0.12, -0.34, 0.56, ...]  ← different!

These number lists (vectors) typically have 256 to 3072 dimensions. More dimensions = more nuance = more storage = slower search.

1 / 7

Ship it

Working embedding pipeline that converts documents to vectors

Architecture Patterns for LLM Apps

Vector Databases: Pinecone, Weaviate, Chroma

Rex — Embedding Models: How Machines Understand Text

Gemini Flash

Ask Rex anything about this lesson

Get help with concepts, run prompts, practice exercises, or ask what to do next.

LLM Orchestration & RAG Systems/Retrieval & Embeddings/Embedding Models: How Machines Understand Text

30 minLesson 4 of 13

Embedding Models: How Machines Understand Text

An embedding is a list of numbers that captures the meaning of text. Two pieces of text that mean similar things will have similar numbers. That's it. That's the entire concept.

"How do I reset my password?"  → [0.12, -0.45, 0.78, 0.33, ...]
"I forgot my login credentials" → [0.11, -0.43, 0.76, 0.31, ...]  ← similar!
"What's the weather today?"    → [0.89, 0.12, -0.34, 0.56, ...]  ← different!

These number lists (vectors) typically have 256 to 3072 dimensions. More dimensions = more nuance = more storage = slower search.

1 / 7

Ship it

Working embedding pipeline that converts documents to vectors

Architecture Patterns for LLM Apps

Vector Databases: Pinecone, Weaviate, Chroma