Chroma DB on a VDS: Vector Database for RAG

What is Chroma?

Chroma is an open-source vector database built specifically for AI applications. It embeds directly into Python without extra services (embedded mode) or runs as a standalone server. It's ideal for RAG systems with small to medium data volumes.

Installation

>pip install chromadb

Embedded mode (no server)

import chromadb

# Persistent storage on disk
client = chromadb.PersistentClient(path="./my_chroma_db")

# Creating a collection
collection = client.get_or_create_collection("documents")

# Adding documents
collection.add(
    documents=["A VDS server provides dedicated resources", "Shared hosting is suitable for small websites"],
    metadatas=[{"source": "faq"}, {"source": "faq"}],
    ids=["doc1", "doc2"]
)

# Search
results = collection.query(
    query_texts=["which hosting should I choose for high load?"],
    n_results=2
)
print(results["documents"])

Server mode (for production)

docker run -d \
  --name chroma \
  -p 8000:8000 \
  -v chroma_data:/chroma/chroma \
  --restart always \
  chromadb/chroma

Integration with LangChain

from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import OllamaEmbeddings

embeddings = OllamaEmbeddings(model="nomic-embed-text")
db = Chroma(persist_directory="./chroma_db", embedding_function=embeddings)

# Search for similar documents
docs = db.similarity_search("user question", k=3)
for doc in docs:
    print(doc.page_content)

Common errors when working with Chroma

«Collection expecting embedding with dimension»

InvalidDimensionException: Embedding dimension 384 does not match collection dimensionality 768

As with other vector databases, the dimension is fixed the first time vectors are added to a collection. If you later switch to an embedding model with a different dimension, the old collection will not work with the new vectors — you need a new collection.

PersistentClient does not keep data between runs

If the path passed to PersistentClient(path=...) resolves to a different relative directory on each run (for example, the script is launched from different working directories), Chroma creates a new empty store every time. Use an absolute path instead.

Server mode is unreachable from outside the container

curl: (7) Failed to connect to IP port 8000: Connection refused

Check that port -p 8000:8000 is published and that the chroma_data volume is mounted — without it, server-mode data is also lost when the container is recreated.

«database is locked» under concurrent access in embedded mode

sqlite3.OperationalError: database is locked

Chroma's embedded mode stores data in SQLite directly on the filesystem, and SQLite handles concurrent writes from multiple processes poorly. If several application workers access the same persist_directory, switch to server mode — there, access to storage goes through a single process over the network.

Embedded and server modes: which to choose

The difference between the two modes is not search capability, but who talks to storage and how: a single process directly, or multiple clients over a network.

When Chroma is not a good fit

Chroma's simplicity at the start turns into limitations wherever load or data requirements outgrow a small project.

• Data volume grows toward millions of vectors with heavy search load — Qdrant's production-oriented Rust architecture suits that scale better
• Data already lives in PostgreSQL and you want a single database without a separate service — pgvector is the more logical choice
• You need complex filtering on nested metadata during search — Chroma's capabilities here are more limited than specialized vector databases
• Horizontal scaling across multiple nodes is critical — Chroma was not designed for that from the start
• Several application processes need to write to the same collection at once — embedded mode on SQLite was not built for that

Verifying the result

Before relying on Chroma in production, check both search quality and how well it survives a process restart.

1. Add test documents to a collection and run a query with meaningfully related text
2. Confirm the most relevant document comes back at the top of the results
3. In server mode, check API availability from outside the container via curl
4. Restart the process or container and confirm previously added documents are still there
5. If several workers access the database, confirm server mode is in use rather than a shared SQLite file

Related: a more powerful database for production — Qdrant on a VDS, a comparison of all the options — vector databases, a ready-made document Q&A system — RAG on a VDS.