Configuring Chroma Collections

Chroma collections have a configuration that determines how their embeddings index is constructed and used. We use default values for these index configurations that should give you great performance for most use cases out-of-the-box.

The embedding function you choose to use in your collection also affects its index construction, and is included in the configuration.

When you create a collection, you can customize these index configuration values for different data, accuracy and performance requirements. Some query-time configurations can also be customized after the collection's creation using the .modify function.

HNSW Index Configuration#

In Single Node Chroma collections, we use an HNSW (Hierarchical Navigable Small World) index to perform approximate nearest neighbor (ANN) search.

The HNSW index parameters include:

space defines the distance function of the embedding space, and hence how similarity is defined. The default is l2 (squared L2 norm), and other possible values are cosine (cosine similarity), and ip (inner product).

Distance	parameter	Equation	Intuition
Squared L2	l2	$d = \sum\left(A_i-B_i\right)^2$	measures absolute geometric distance between vectors, making it suitable when you want true spatial proximity.
Inner product	ip	$d = 1.0 - \sum\left(A_i \times B_i\right)$	focuses on vector alignment and magnitude, often used for recommendation systems where larger values indicate stronger preferences
Cosine similarity	cosine	$d = 1.0 - \frac{\sum\left(A_i \times B_i\right)}{\sqrt{\sum\left(A_i^2\right)} \cdot \sqrt{\sum\left(B_i^2\right)}}$	measures only the angle between vectors (ignoring magnitude), making it ideal for text embeddings or cases where you care about direction rather than scale

You should make sure that the space you choose is supported by your collection's embedding function. Every Chroma embedding function specifies its default space and a list of supported spaces.

ef_construction determines the size of the candidate list used to select neighbors during index creation. A higher value improves index quality at the cost of more memory and time, while a lower value speeds up construction with reduced accuracy. The default value is 100.
ef_search determines the size of the dynamic candidate list used while searching for the nearest neighbors. A higher value improves recall and accuracy by exploring more potential neighbors but increases query time and computational cost, while a lower value results in faster but less accurate searches. The default value is 100. This field can be modified after creation.
max_neighbors is the maximum number of neighbors (connections) that each node in the graph can have during the construction of the index. A higher value results in a denser graph, leading to better recall and accuracy during searches but increases memory usage and construction time. A lower value creates a sparser graph, reducing memory usage and construction time but at the cost of lower search accuracy and recall. The default value is 16.
num_threads specifies the number of threads to use during index construction or search operations. The default value is multiprocessing.cpu_count() (available CPU cores). This field can be modified after creation.
batch_size controls the number of vectors to process in each batch during index operations. The default value is 100. This field can be modified after creation.
sync_threshold determines when to synchronize the index with persistent storage. The default value is 1000. This field can be modified after creation.
resize_factor controls how much the index grows when it needs to be resized. The default value is 1.2. This field can be modified after creation.

For example, here we create a collection with customized values for space and ef_construction:

collection = client.create_collection(
    name="my-collection",
    embedding_function=OpenAIEmbeddingFunction(model_name="text-embedding-3-small"),
    configuration={
        "hnsw": {
            "space": "cosine",
            "ef_construction": 200
        }
    }
)

Fine-Tuning HNSW Parameters

In the context of approximate nearest neighbors search, recall refers to how many of the true nearest neighbors were retrieved.

Increasing ef_search normally improves recall, but slows down query time. Similarly, increasing ef_construction improves recall, but increases the memory usage and runtime when creating the index.

Choosing the right values for your HNSW parameters depends on your data, embedding function, and requirements for recall, and performance. You may need to experiment with different construction and search values to find the values that meet your requirements.

For example, for a dataset with 50,000 embeddings of 2048 dimensions, generated by

Python

embeddings = np.random.randn(50000, 2048).astype(np.float32).tolist()

we set up two Chroma collections:

The first is configured with ef_search: 10. When querying using a specific embedding from the set (with id = 1), the query takes 0.00529 seconds, and we get back embeddings with distances:

[3629.019775390625, 3666.576904296875, 3684.57080078125]

The second collection is configured with ef_search: 100 and ef_construction: 1000. When issuing the same query, this time it takes 0.00753 seconds (about 42% slower), but with better results as measured by their distance:

[0.0, 3620.593994140625, 3623.275390625]

In this example, when querying with the test embedding (id=1), the first collection failed to find the embedding itself, despite it being in the collection (where it should have appeared as a result with a distance of 0.0). The second collection, while slightly slower, successfully found the query embedding itself (shown by the 0.0 distance) and returned closer neighbors overall, demonstrating better accuracy at the cost of performance.

Embedding Function Configuration#

The embedding function you choose when creating a collection, along with the parameters you instantiate it with, is persisted in the collection's configuration. This allows us to reconstruct it correctly when you use collection across different clients.

You can set your embedding function as an argument to the "create" methods, or directly in the configuration:

Install the openai and cohere packages:

Terminal

pip install openai cohere

Creating collections with embedding function and custom configuration:

Python

import os
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction, CohereEmbeddingFunction

# Using the `embedding_function` argument
openai_collection = client.create_collection(
    name="my_openai_collection",
    embedding_function=OpenAIEmbeddingFunction(
        model_name="text-embedding-3-small"
    ),
    configuration={"hnsw": {"space": "cosine"}}
)

# Setting `embedding_function` in the collection's `configuration`
cohere_collection = client.get_or_create_collection(
    name="my_cohere_collection",
    configuration={
        "embedding_function": CohereEmbeddingFunction(
            model_name="embed-english-light-v2.0",
            truncate="NONE"
        ),
        "hnsw": {"space": "cosine"}
    }
)

Note: Many embedding functions require API keys to interface with the third party embeddings providers. The Chroma embedding functions will automatically look for the standard environment variable used to store a provider's API key. For example, the Chroma OpenAIEmbeddingFunction will set its api_key argument to the value of the OPENAI_API_KEY environment variable if it is set.

If your API key is stored in an environment variable with a non-standard name, you can configure your embedding function to use your custom environment variable by setting the api_key_env_var argument. In order for the embedding function to operate correctly, you will have to set this variable in every environment where you use your collection.

Python

cohere_ef = CohereEmbeddingFunction(
    api_key_env_var="MY_CUSTOM_COHERE_API_KEY",
    model_name="embed-english-light-v2.0",
    truncate="NONE",
)

Delete Data

Query And Get

Edit this page on GitHub

Configuring Chroma Collections

HNSW Index Configuration#

What is an HNSW index?

Fine-Tuning HNSW Parameters

Embedding Function Configuration#