Menttor
Library
LibrarySetting up Chroma

Setting up Chroma

Learn about Setting up Chroma as part of Vector Databases and RAG Systems Architecture

Table of Contents

Setting Up Chroma: Your Local Vector Database

This section guides you through the process of setting up Chroma, a popular open-source vector database, on your local machine. Understanding this setup is crucial for building RAG (Retrieval-Augmented Generation) systems and experimenting with AI applications that rely on semantic search.

Prerequisites

Before we begin, ensure you have the following installed:

  • Python: Version 3.7 or higher.
  • pip: Python's package installer.

Installation

Chroma can be easily installed using pip. Open your terminal or command prompt and run the following command:

bash
pip install chromadb

Running Chroma: In-Memory vs. Persistent Storage

Chroma offers two primary modes for operation: in-memory and persistent storage. Understanding the difference is key to managing your data effectively.

In-memory mode is for quick experimentation, while persistent storage saves your data between sessions.

In-memory mode is fast and requires no setup, but your data is lost when the application stops. Persistent storage saves your embeddings and metadata to disk, allowing you to resume your work later.

When you initialize Chroma without specifying a path, it defaults to an in-memory database. This is ideal for rapid prototyping and testing where data persistence isn't a concern. For any real-world application or longer-term projects, you'll want to configure Chroma to use persistent storage. This involves providing a directory path where Chroma will store its data files, including your vector embeddings and associated metadata.

Basic Usage: Creating a Client and Collection

Once installed, you can interact with Chroma using its Python client. Here's a simple example of how to initialize Chroma, create a collection, and add some data.

First, import the Chroma library:

python
400">"text-blue-400 font-medium">import chromadb

To start, create a client. For persistent storage, specify a path:

python
500 italic"># For persistent storage
client = chromadb.400">PersistentClient(path=400">"./chroma_db")
500 italic"># For 400">"text-blue-400 font-medium">in-memory 400">storage (data lost on exit)
500 italic"># client = chromadb.400">Client()

Next, get or create a collection. A collection is where you store your embeddings and associated metadata.

python
collection = client.400">get_or_create_collection(name=400">"my_documents")

Adding Data to a Collection

You can add data to your collection using the

code
add
method. This requires unique IDs for each item, the embeddings (which are typically generated by an embedding model), and optionally, the documents themselves and any metadata.

For this example, we'll use placeholder embeddings. In a real RAG system, these would come from an embedding model like Sentence-BERT or OpenAI's embeddings.

python
collection.400">add(
    embeddings=[
        [1.1, 2.3, 3.2, 4.5],
        [5.1, 6.2, 7.3, 8.4],
        [9.8, 7.6, 5.4, 3.2]
    ],
    documents=[
        400">"This is the first document.",
        400">"This document is about vector databases.",
        400">"Chroma is a great tool 400 font-medium400">">for AI applications."
    ],
    metadatas=[
        {400">"source": 400">"doc1"},
        {400">"source": 400">"doc2"},
        {400">"source": 400">"doc3"}
    ],
    ids=[400">"doc1", 400">"doc2", 400">"doc3"]
)

Querying Your Collection

To retrieve relevant documents, you'll query the collection with a query embedding. Again, this embedding would typically be generated by the same model used for your documents.

python
results = collection.400">query(
    query_embeddings=[[1.0, 2.0, 3.0, 4.0]],
    n_results=2
)
400">print(results)

The query method returns the n_results most similar items to your query_embeddings based on cosine similarity or Euclidean distance, depending on the embedding model's configuration.

Understanding Chroma's Architecture (Simplified)

Chroma's core functionality revolves around storing and efficiently searching high-dimensional vectors. When you add documents, they are first converted into numerical vector representations (embeddings) by an embedding model. These embeddings, along with the original text and metadata, are then indexed by Chroma for fast similarity searches.

Chroma's architecture involves a client-server model, though it can run entirely locally. The client interacts with the database, which manages collections. Each collection stores items, each consisting of a unique ID, an embedding vector, the original document text, and associated metadata. When a query is made, the query text is converted into an embedding, and Chroma uses an index (like HNSW or IVF) to find the nearest neighbor vectors in the collection, returning the corresponding documents and metadata.

📚

Text-based content

Library pages focus on text content

Next Steps: Integrating with RAG

With Chroma set up, you're ready to integrate it into a RAG pipeline. This typically involves:

  1. Document Loading and Chunking: Breaking down large documents into smaller, manageable pieces.
  2. Embedding Generation: Using an embedding model to convert text chunks into vectors.
  3. Storing in Chroma: Adding these embeddings, chunks, and metadata to your Chroma collection.
  4. Querying: When a user asks a question, embedding the question and searching Chroma for relevant document chunks.
  5. Augmenting the LLM: Providing the retrieved document chunks as context to a Large Language Model (LLM) to generate an informed answer.
What is the primary difference between Chroma's in-memory and persistent modes?

In-memory data is lost when the application stops, while persistent storage saves data to disk for future use.

What are the key components stored within a Chroma collection for each item?

Unique ID, embedding vector, document text, and metadata.

Learning Resources

Chroma Documentation - Getting Started(documentation)

The official documentation for Chroma, covering installation, basic usage, and core concepts.

Chroma GitHub Repository(documentation)

Explore the source code, contribute, or find issues and discussions related to Chroma development.

Building a RAG System with Chroma and LangChain(documentation)

Learn how to integrate Chroma with LangChain for building RAG applications.

Vector Databases Explained(blog)

A foundational article explaining what vector databases are and why they are important for AI applications.

Introduction to Embeddings(documentation)

Understand the concept of word embeddings, which are crucial for populating vector databases.

Retrieval-Augmented Generation (RAG) Explained(blog)

An overview of RAG systems, explaining how they combine retrieval with generative AI models.

ChromaDB: The AI-Native Open-Source Vector Database(video)

A video introduction to ChromaDB, showcasing its features and use cases.

Understanding Vector Similarity Search(blog)

Explains the underlying principles of how vector databases perform similarity searches.

Python Tutorial: Working with Files and Directories(documentation)

Essential Python knowledge for managing the persistent storage path for Chroma.

What are Vector Databases?(blog)

Another excellent resource explaining the fundamental concepts and benefits of vector databases.