API Reference

Complete public surface of shipit_agent.rag. Every class is importable either from shipit_agent.rag or from the top-level shipit_agent package (with the RAG prefix where there is a name collision with shipit_agent.memory).

Top-level imports

from shipit_agent.rag import (
    RAG,
    # Types
    Document, Chunk, IndexFilters, RAGContext, RAGSource,
    SearchQuery, SearchResult,
    # Components
    DocumentChunker, HashingEmbedder, CallableEmbedder, Embedder,
    InMemoryVectorStore, VectorStore,
    InMemoryBM25Store, KeywordStore,
    LLMReranker, Reranker,
    HybridSearchPipeline,
    # Tools
    RAGSearchTool, RAGFetchChunkTool, RAGListSourcesTool,
    # Utilities
    TextExtractor, RAGDependencyError, RAGIngestError,
)

# Or from the top-level package — the RAG namespace is prefixed to avoid
# colliding with shipit_agent.memory's VectorStore / SearchResult names:
from shipit_agent import (
    RAG,
    RAGChunk, RAGContext, RAGDocument, RAGIndexFilters,
    RAGSearchQuery, RAGSearchResult, RAGSource,
    DocumentChunker, HashingEmbedder, CallableEmbedder,
    HybridSearchPipeline, LLMReranker,
)

Dataclasses

Document

Chunk

IndexFilters

Method:

• matches(chunk: Chunk) -> bool — return True iff the chunk passes every configured filter.

SearchQuery

SearchResult

RAGContext

Methods:

• to_prompt_context(max_chars: int = 8000) -> str — render as a prompt-ready block with [N] citation markers.
• to_dict() -> dict — JSON-friendly representation.

RAGSource

Method:

• to_dict() -> dict — JSON-friendly representation.

RAG facade

class RAG:
    def __init__(
        self,
        *,
        vector_store: VectorStore,
        embedder: Any,
        keyword_store: KeywordStore | None = None,
        reranker: Reranker | None = None,
        chunker: DocumentChunker | None = None,
        auto_embed_on_add: bool = True,
    ) -> None: ...

Class methods

• RAG.default(*, embedder, reranker=None) -> RAG — in-memory hybrid index with sensible defaults. Recommended starting point.

Indexing

• index_text(text, *, document_id=None, title=None, source=None, metadata=None) -> list[Chunk]
• index_file(path, *, document_id=None, title=None, metadata=None) -> list[Chunk]
• index_document(document: Document) -> list[Chunk]
• index_documents(documents: Iterable[Document]) -> list[Chunk]
• reindex(document_id, *, content=None) -> list[Chunk]
• delete_document(document_id) -> None

Query

• search(query, *, top_k=5, filters=None, hybrid_alpha=0.5, enable_reranking=False, enable_recency_bias=False, chunks_above=0, chunks_below=0) -> RAGContext
• fetch_chunk(chunk_id, *, chunks_above=0, chunks_below=0) -> Chunk | None
• list_sources() -> list[str]
• count() -> int

Source tracking

• begin_run() -> None — open a per-thread tracker scope.
• end_run() -> list[RAGSource] — close the scope and return collected sources, indexed [1..n] in first-seen order.
• current_sources() -> list[RAGSource] — peek at sources captured so far inside the active scope.

Agent integration helpers

• as_tools() -> list[Tool] — return the three RAG tools bound to this instance.
• prompt_section() -> str — return the system-prompt addition that describes the RAG tools and citation rules. Override to customise.

Components

DocumentChunker

DocumentChunker(target_tokens=512, overlap_tokens=64, title_prefix_chars=64)

Methods:

• chunk(document: Document) -> list[Chunk]
• chunk_many(documents: Iterable[Document]) -> list[Chunk]

HashingEmbedder

HashingEmbedder(dimension=384, seed="shipit-rag")

Stdlib-only, deterministic, L2-normalised. Method: embed(texts, *, kind="passage") -> list[list[float]].

CallableEmbedder

CallableEmbedder(fn=callable, dimension=int)

Wraps any function with the signature (list[str]) -> list[list[float]]. Validates dimensions and length on every call.

InMemoryVectorStore

Pure-Python VectorStore backed by a dict, with cosine_similarity search. Methods follow the VectorStore protocol plus all_chunks() for inspecting the full corpus.

InMemoryBM25Store

Pure-Python KeywordStore implementing BM25 with k1=1.5, b=0.75 defaults. Configurable via the constructor.

LLMReranker

LLMReranker(llm, batch_size=10)

Reuses any LLM with a complete(messages=...) method to score candidates 0–10. Scores are normalised to [0, 1].

HybridSearchPipeline

HybridSearchPipeline(
    vector_store: VectorStore,
    embedder: Embedder,
    keyword_store: KeywordStore | None = None,
    reranker: Reranker | None = None,
)

Method: search(query: SearchQuery) -> RAGContext.

Pipeline order: embed → vector + keyword (parallel) → RRF fusion → optional recency bias → optional reranking → context expansion.

Agent tools

RAGSearchTool

name="rag_search". Parameters:

Returns a JSON payload with query, total_found, and a results list containing citation, chunk_id, document_id, source, score, and text for each hit. The tool's metadata["rag_chunk_ids"] field exposes the captured chunk ids for downstream consumers.

RAGFetchChunkTool

name="rag_fetch_chunk". Parameters:

Returns a JSON payload with the chunk text plus optional context_above / context_below arrays of neighbouring chunks.

RAGListSourcesTool

name="rag_list_sources". No parameters. Returns {"sources": ["readme.md", "manual.pdf", ...]}.

Errors

• RAGDependencyError(RuntimeError) — raised when an optional dependency required by an extractor or adapter is missing.
• RAGIngestError(RuntimeError) — raised by TextExtractor.extract when a file cannot be read or parsed. Carries path and cause attributes.

See also

• Overview
• Standalone RAG
• RAG + Agent
• RAG + Deep Agents
• Adapters