GoKawiilSay you are building search over your team's internal documentation — guides, runbooks, postmortems. You have a table with auto embeddings : you insert text, Manticore runs the model and fills the vector column for you. (If that is new to you, start with vector search in Manticore .) You load a 4,000-word document. The insert succeeds. The search works. Everything looks fine.
Except the model you picked has a 512-token input window, and that document is about 5,000 tokens long. The model read the first 380 words and threw away the other 3,600. Nothing in the document past that point can ever be retrieved, and nothing anywhere told you. The embedding may not represent the document as a whole either.
Until now, you would usually split the document into several pieces yourself, create embeddings for each one, and then work out how to combine the results if you wanted document search rather than chunk search. Manticore now handles this in the table definition: add chunk_strategy to the vector column in CREATE TABLE , and Manticore splits each document into chunks, embeds every chunk, and searches all of them:
DROP TABLE IF EXISTS docs; CREATE TABLE docs ( title text, content text, chunks float_vector_array knn_type = 'hnsw' hnsw_similarity = 'cosine' model_name = 'Xenova/all-MiniLM-L6-v2' from = 'title,content' chunk_strategy = 'sentence' max_tokens = '256' overlap_tokens = '32' );
That is the whole feature. No ingest pipeline, no splitter library, no second table for chunks, no GROUP BY to fold chunk hits back into documents.
TL;DR
Five strategies : truncate (the old default), mean , fixed , recursive , sentence . Set with chunk_strategy on a model-backed vector column.
: (the old default), , , , . Set with on a model-backed vector column. truncate and mean produce one vector per document and work on a float_vector column. fixed , recursive and sentence produce many, so they need a float_vector_array column.
produce one vector per document and work on a column. produce many, so they need a column. A document is still one search result. Chunks compete individually, and Manticore returns the document once, with knn_dist() reporting the distance to its closest chunk. k counts documents, not chunks.
Chunks compete individually, and Manticore returns the document once, with reporting the distance to its closest chunk. counts documents, not chunks. Tuning knobs : max_tokens (chunk size), overlap_tokens (shared tokens between neighbors), max_chunks (ceiling per document).
... continue reading