xerj

xerj-org

8.5k stars454 forksRustApache-2.0

ai-searchai-search-algorithmsai-search-engineai-search-freeai-search-moduleai-search-optimizationai-searchenginebm25-searchelastic-searchelasticsearch

A Rust AI search engine that automatically indexes folders so agents retrieve exact passages instead of grepping files.

It turns mixed local files into typed search indexes without schemas or pipelines, letting coding agents retrieve relevant passages rather than load whole files. Point it at a folder, then search by text, structure, vector similarity, or hybrid queries. It also accepts Elasticsearch API clients and provides MCP tools for assistants.

For Coding agents and their operators working with unfamiliar private code, and developers using Elasticsearch clients

Service

Use cases

Reuse proven code patterns from similar open-source repositories
Answer questions about a codebase with file and line passages
Audit files for security risks across mixed document types
Investigate incidents from logs with Elasticsearch-style aggregations
Recall project knowledge between sessions through namespaced agent memory

Key features

Automatic folder indexing: Code, PDFs, SQLite, CSV, DOCX, EPUB, mail, and logs
AST-aware source records: Tree-sitter preserves symbols and file line numbers
Elasticsearch API compatibility: Supports indices, bulk, search, aggregations, kNN, scroll, and reindex
Full-text, vector, and hybrid retrieval: Uses Elasticsearch knn and semantic query syntax
MCP search and memory tools: Includes xerj_search, vector, hybrid, and memory operations
Reference-coding workflow: Measured 2.7x fewer output tokens than grep

Under the hood

4 parts

XERJ keeps its engine independent from Elasticsearch while exposing the Elasticsearch 8.x HTTP protocol as a compatibility bridge. Folder ingestion is a separate HTTP client feature, so autoindex can create typed indices against an endpoint without linking the engine.

docs/ARCHITECTURE.md (Bird's-eye view, Crate responsibilities); engine/crates/xerj-autoindex/src/lib.rs (module documentation)

Search engine
Combines storage, full-text search, vectors, and aggregations into indexes.
engine/crates/xerj-engine/src/
Autoindex
Infers datasets and bulk-indexes folders through Elasticsearch-compatible HTTP.
engine/crates/xerj-autoindex/src/
API layer
Serves Elasticsearch-compatible and native HTTP endpoints through Axum.
engine/crates/xerj-api/
Index storage
Records mutations in WAL, writes segments, and atomically swaps read snapshots.
engine/crates/xerj-storage/src/index_store.rs

Built with

RustTokio 1Axum 0.7Serde 1Rayon 1.10tree-sitterLZ4Zstd

Generated 2 days ago

Analyze another repo