Hosted Chroma
Chroma Cloud, our fully managed hosted service, is in early access. Fill out the survey to jump the waitlist and get the best retrieval experience. Full access coming Q1 2025.
If you are using Chroma in production, please fill out this form, and we will add you to a dedicated Slack workspace for supporting production users. This is the best place to
- Get support with building with Chroma in prod.
- Stay up-to-date with exciting new features.
- Get swag!
We would love to help you think through the design of your system, or if you would be a good fit for our upcoming distributed cloud service.
A Simple GCP Deployment#
You can deploy Chroma on a long-running server, and connect to it remotely.
For convenience, we have provided a very simple Terraform configuration to experiment with deploying Chroma to Google Compute Engine.
Chroma and its underlying database need at least 2GB of RAM, which means it won't fit on the instances provided as part of the GCP "always free" tier. This template uses an e2-small instance, which costs about two cents an hour, or $15 for a full month, and gives you 2GiB of memory. If you follow these instructions, GCP will bill you accordingly.
In this guide we show you how to secure your endpoint using Chroma's native authentication support. Alternatively, you can put it behind GCP API Gateway or add your own authenticating proxy. This basic stack doesn't support any kind of authentication; anyone who knows your server IP will be able to add and query for embeddings.
By default, this template saves all data on a single volume. When you delete or replace it, the data will disappear. For serious production use (with high availability, backups, etc.) please read and understand the Terraform template and use it as a basis for what you need, or reach out to the Chroma team for assistance.
Step 1: Set up your GCP credentials#
In your GCP project, create a service account for deploying Chroma. It will need the following roles:
- Service Account User
- Compute Admin
- Compute Network Admin
- Storage Admin
Create a JSON key file for this service account, and download it. Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of your JSON key file:
Step 2: Install Terraform#
Download Terraform and follow the installation instructions for you OS.
Step 3: Configure your GCP Settings#
Create a chroma.tfvars file. Use it to define the following variables for your GCP project ID, region, and zone:
Step 4: Initialize and deploy with Terraform#
Download our GCP Terraform configuration to the same directory as your chroma.tfvars file. Then run the following commands to deploy your Chroma stack.
Initialize Terraform:
Plan the deployment, and review it to ensure it matches your expectations:
If you did not customize our configuration, you should be deploying an e2-small instance.
Finally, apply the deployment:
Customize the Stack (optional)#
If you want to use a machine type different from the default e2-small, in your chroma.tfvars add the machine_type variable and set it to your desired machine:
After a few minutes, you can get the IP address of your instance with
Step 5: Chroma Client Set-Up#
Once your Compute Engine instance is up and running with Chroma, all you need to do is configure your HttpClient to use the server's IP address and port 8000. Since you are running a Chroma server on GCP, our thin-client package may be enough for your application.
Step 5: Clean Up (optional).#
To destroy the stack and remove all GCP resources, use the terraform destroy command.
Note
This will destroy all the data in your Chroma database, unless you've taken a snapshot or otherwise backed it up.
Authentication with GCP#
By default, the Compute Engine instance created by our Terraform configuration will run with no authentication. There are many ways to secure your Chroma instance on GCP. In this guide we will use a simple set-up using Chroma's native authentication support.
You can learn more about authentication with Chroma in the Auth Guide.
Static API Token Authentication#
Customize Chroma's Terraform Configuration#
Security Note
Current implementation of static API token auth supports only ENV based tokens. Tokens must be alphanumeric ASCII strings. Tokens are case-sensitive.
If, for example, you want the static API token to be "test-token", set the following variables in your chroma.tfvars. This will set Authorization: Bearer test-token as your authentication header.
To use X-Chroma-Token: test-token type of authentication header you can set the ChromaAuthTokenTransportHeader parameter:
Client Set-Up#
Add the CHROMA_CLIENT_AUTH_CREDENTIALS environment variable to your local environment, and set it to the token you provided the server (test-token in this example):
We will use Chroma's Settings object to define the authentication method on the client.
If you are using a custom CHROMA_AUTH_TOKEN_TRANSPORT_HEADER (like X-Chroma-Token), add it to your Settings:
Observability with GCP#
Chroma is instrumented with OpenTelemetry hooks for observability. We currently only exports OpenTelemetry traces. These should allow you to understand how requests flow through the system and quickly identify bottlenecks.
Tracing is configured with four environment variables:
CHROMA_OTEL_COLLECTION_ENDPOINT: where to send observability data. Example:api.honeycomb.com.CHROMA_OTEL_SERVICE_NAME: Service name for OTel traces. Default:chromadb.CHROMA_OTEL_COLLECTION_HEADERS: Headers to use when sending observability data. Often used to send API and app keys. For example{"x-honeycomb-team": "abc"}.CHROMA_OTEL_GRANULARITY: A value from the OpenTelemetryGranularity enum. Specifies how detailed tracing should be.
To enable tracing on your Chroma server, simply define the following variables in your chroma.tfvars: