Embeddings are numerical representations of text (or other data) that capture semantic meaning. In an embedding space, words and sentences with similar meanings have numerically similar representations — allowing computers to measure and compare meaning mathematically.
Try Lucy OS1 →Embedding models convert text into high-dimensional vectors (typically 1,536 or 3,072 numbers). Similar text → vectors that are close together in this space. 'Dog' and 'puppy' are close; 'dog' and 'democracy' are far. This mathematical representation of meaning is what enables semantic search — finding relevant documents by meaning rather than keyword match.
Lucy OS1 uses embeddings for memory retrieval. Every stored memory is converted to an embedding vector. When you start a conversation, Lucy embeds your first message and retrieves the stored memories with the most similar vectors — surfacing relevant context automatically.
Try Lucy OS1 →Similarity between embeddings is measured by cosine similarity or dot product. The higher the score, the more semantically similar the texts.
Modern embedding models produce vectors with 768-3072 dimensions. Higher dimensionality can capture more semantic nuance but requires more storage and computation.
Specialised models that convert text to embeddings. OpenAI text-embedding-3 and Cohere embed are common choices. Different models perform better for different languages and domains.
Finding the closest embeddings to a query vector in a large database. Approximate nearest neighbour (ANN) algorithms make this fast at scale.
What is an embedding vs a word vector?
Word vectors (Word2Vec, GloVe) embed individual words. Modern embeddings (from transformer models) embed entire sentences or passages, capturing context rather than just individual word meaning.
How are embeddings used in AI memory?
Each memory is stored with its embedding. When searching for relevant memories, the system embeds the query and finds the memories with highest cosine similarity — semantically related memories rise to the top.
Can embeddings capture meaning across languages?
Multilingual embedding models (like OpenAI's text-embedding-3 or Cohere multilingual) produce similar vectors for semantically equivalent sentences in different languages, enabling cross-lingual search.
Lucy OS1 puts these concepts to work in a real, streaming voice AI pipeline — Deepgram STT, GPT-4o-mini, and Cartesia TTS delivering natural voice conversation.
Start talking to Lucy →Welcome