GoKawiilManticore Search adds automatic document chunking to its vector search engine
Manticore Search now supports built-in chunking for embedding-based vector search, letting users add a chunk_strategy option directly to a table's vector column instead of manually splitting long documents before indexing. The database can split text into pieces, embed each chunk, and search across all of them within a single table, eliminating the need for a separate chunking pipeline or secondary table. Five strategies are offered: truncate, mean, fixed, recursive, and sentence, with the latter three requiring a float_vector_array column to store multiple vectors per document.
GoKawiil's interpretation of the reporting above, not reported fact.
Embedding models have fixed token limits, so long documents like internal wikis or postmortems were often silently truncated, causing search systems to miss large portions of content without any warning. By moving chunking logic into the database layer, Manticore removes a common source of hidden retrieval failures and simplifies architecture for teams building semantic search over lengthy text. This positions Manticore competitively against vector databases and search engines that still require external preprocessing for effective long-document retrieval.
- Manticore Search can now automatically chunk long documents during indexing via a chunk_strategy setting.
- Five chunking strategies are available, with fixed, recursive, and sentence requiring a float_vector_array column for multiple embeddings per document.
- The feature addresses silent truncation issues where embedding models ignored content beyond their token limits.
Source: manticoresearch.com, 2026-09-17
Published there as: “Better Vector Search for Long Documents: Chunking Inside Manticore Search”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.