AutoRAG and refrag
AutoRAG provides a comprehensive framework for systematically evaluating and optimizing RAG pipelines, while REFRAG offers a specialized technique for improving retrieval precision through LLM-powered representations that could be integrated as a retrieval component within AutoRAG's optimization workflow.
About AutoRAG
Marker-Inc-Korea/AutoRAG
AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation
Provides end-to-end RAG pipeline optimization through YAML-driven configuration, encompassing document parsing, semantic chunking, and QA dataset generation with support for multiple parsing/chunking strategies simultaneously. Uses grid-search and metric-driven evaluation across retriever-generator combinations to identify optimal module configurations, with results tracked in a dashboard for deployment-ready pipeline export. Integrates with LlamaIndex, LangChain, and local embedding models, supporting both cloud APIs (OpenAI) and GPU-accelerated inference for custom models.
About refrag
Shaivpidadi/refrag
REFRAG: LLM-powered representations for better RAG retrieval. Improve precision, reduce context size, same speed.
Implements micro-chunking (16-32 tokens) with fast encoder-only indexing and query-time compression policies that dynamically mark top-ranked chunks as RAW and lower-ranked ones as compressed keywords. It's model-agnostic and integrates with any LLM via context preparation, supporting sentence-transformers embeddings and currently using in-memory storage with planned vector DB support.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work