01 // Project dossier

RagBench

RAG Quality Lab

FastAPIAPI surface + ablation endpoints
Postgres 16 + pgvectorVector + relational store
FTS + RRF (k=60)Hybrid retrieval fusion
bge-small-en-v1.5Local embeddings (CPU-friendly)
OllamaOpenAI-compatible local LLM via LLM_MODEL
React + Vite + TSBench / Probe / Corpus UI

01 // What it is

A self-hostable **retrieval quality bench**: ablate dense vs keyword vs hybrid on a fixed golden set, tag failures by cause, probe bad cases side-by-side. Hybrid retrieval is commodity plumbing; the product surface is Bench / Probe / Corpus (not chat-first).

02 // Why it exists

Industry is saturated with doc-chat products (AnythingLLM, Onyx) and eval *libraries* (Ragas, DeepEval). The portfolio gap is a small full system where **measurement is the product surface**.

03 // Surfaces

Surface

Role

Bench

Bench (/)

Ablation matrix, metrics, failure taxonomy, worst items

Probe

Dual-config diagnosis + live strip

Corpus

Seed/list handbook + golden summary

04 // Stack

FastAPIAPI surface + ablation endpoints
Postgres 16 + pgvectorVector + relational store
FTS + RRF (k=60)Hybrid retrieval fusion
bge-small-en-v1.5Local embeddings (CPU-friendly)
OllamaOpenAI-compatible local LLM via LLM_MODEL
React + Vite + TSBench / Probe / Corpus UI

05 // Honest status

Honest status

MVP code landed. Vertical slice works: compose DB → migrate → seed Northstar handbook → probe with citations → ablation API. Still need a full judged ablation (Ollama up) before publishing metrics or resume numbers. No lift, recall@k, faithfulness, or abstain numbers exist yet — this page will be updated when they do.

06 // Screenshots

Screenshot pending judged evals