Cassandra 5 shipped two features that change how we model tables: a native vector<float, n> type with approximate-nearest-neighbour (ANN) search, and Storage-Attached Indexing (SAI) as the general-purpose secondary index. Teams are now arriving with RAG and personalization projects and asking whether they can keep embeddings in the cluster they already run.
Often, yes. Sometimes, no. Here is how we reason about it, and what the two features actually do to your read path.
SAI, briefly
SAI replaces the older SASI and legacy secondary indexes. It builds per-SSTable index structures that live with the data, so index files are created and dropped by the same compaction lifecycle as the SSTables themselves. That single fact fixes most of what made 2i painful: no separate hidden table, no independent repair story, index memory and disk that scale with the data files.
What SAI does not change is the physics of a distributed query. An SAI query without a partition key restriction is still a scatter-gather across every replica set in the datacenter. Each node consults its local index, returns candidates, and the coordinator merges. That works, and it works far better than 2i did — but its latency is bounded by the slowest node in the fan-out, and its cost grows with cluster size, not with result size.
So the modeling rule survives Cassandra 5 intact:
- Partition key first. Design a table for your primary access pattern.
- SAI for the secondary filters inside a known partition, or for low-frequency queries where a cluster-wide fan-out is acceptable.
- Denormalized second table for any secondary access pattern that runs at request volume.
SAI made the middle option viable. It did not make the third option obsolete. If a query is on your latency-critical path and runs thousands of times a second, it still deserves its own table.
One practical note: SAI indexes are cheap to add and easy to accumulate. Every index is write amplification. We routinely find tables with five or six indexes where two are queried and the rest were added during a debugging session in 2023. Audit system_schema.indexes against your actual query log and drop what nobody reads.
The vector type and ANN
The vector path is built on SAI: you declare a column, index it, and query with ORDER BY ... ANN OF.
CREATE TABLE docs.chunks (
tenant_id text,
doc_id uuid,
chunk_no int,
body text,
embedding vector<float, 1536>,
PRIMARY KEY ((tenant_id), doc_id, chunk_no)
);
CREATE CUSTOM INDEX ON docs.chunks (embedding)
USING 'StorageAttachedIndex'
WITH OPTIONS = {'similarity_function': 'dot_product'};
SELECT doc_id, chunk_no, body
FROM docs.chunks
WHERE tenant_id = 'acme'
ORDER BY embedding ANN OF [ ... ]
LIMIT 10;
The index is a graph structure (JVector's DiskANN-style implementation) per SSTable. Two consequences matter operationally:
Recall is per-SSTable and approximate. Each SSTable's graph is searched independently and the results are merged. More SSTables means more graph searches per query — the same compaction-debt story that governs the rest of Cassandra, with a steeper slope. Watch SSTables-per-read on vector tables the way you watch it on hot read tables, because ANN latency tracks it closely.
Dimensionality costs memory. A 1536-dimension float vector is ~6 KB before index overhead. Ten million chunks is ~60 GB of raw vectors plus graph structures, and graph search wants those pages resident. Vector tables push you toward more RAM per node than your operational tables ever needed. Size the hardware against the embedding footprint, not the row count.
The restriction shown above — WHERE tenant_id = 'acme' — is the design that makes vector search on Cassandra genuinely good. Constraining the ANN search to one partition turns a cluster-wide graph traversal into a local one. If your product is multi-tenant, or per-user, or per-document-collection, that restriction is free and the fit is excellent: one store, one replication model, one operational surface, embeddings co-located with the metadata you need to render results.
When we say don't
We tell teams to keep embeddings elsewhere in three situations.
Unfiltered global search over a large corpus. If every query is "nearest 10 across all 500 million vectors" with no partition restriction, you are asking a distributed store to do the one thing its topology is worst at. A purpose-built vector index on fewer, larger nodes will beat it on both latency and cost.
Heavy re-embedding cycles. Swapping models rewrites every vector. That is a full-table rewrite plus complete index rebuild plus the compaction backlog that follows — feasible, but plan the capacity and the window, and expect elevated read latency throughout. Teams still iterating on embedding models weekly should not carry that in their production operational cluster.
Recall you need to tune aggressively. Cassandra's ANN exposes far fewer knobs than a dedicated engine. If your relevance work involves sweeping graph parameters, quantization schemes, and hybrid scoring weights, you want a system built for that experimentation.
What's left after those three is a large and legitimate category: applications that already run Cassandra or Astra DB, need per-tenant or per-entity similarity search at moderate corpus size, and would rather not add a second stateful system with its own failure modes and its own on-call rotation. For them, moving embeddings into the existing cluster removes an operational surface instead of adding one — which is usually worth more than a percentage point of recall.
Before you commit
A short checklist we run with clients evaluating this:
- Can every ANN query restrict to a partition? If not, keep looking.
- What is the vector footprint per node at 12-month projected volume, against current RAM?
- Are you on 5.x already? Vector search is a 5.0 feature; on 4.1 this conversation starts with an upgrade, and the upgrade has its own runbook and its own risk.
- Who owns re-embedding? Model changes are schema-scale events. Decide the process before the first one.
- What is your recall target, and how will you measure it? "Good enough" is a number, or it is an argument you will have later during an incident.
If you are weighing vectors in Cassandra against a dedicated store, or you are already on 5.x and your ANN latency is worse than the benchmarks suggested, describe the cluster and the query shape in an email. Those conversations are usually short, and the answer is often that the schema needs one change rather than the platform needing another component.