RAG Configuration Wizard: Automated Selection of LLM, Embedding Model, and ReRanker for Enterprise Use Cases
Description
This project develops an interactive configuration tool — the RAG Configuration Wizard — that helps organizations identify the optimal combination of components for a Retrieval-Augmented Generation (RAG) pipeline. Given a specific use case, the wizard evaluates and recommends the best-fitting "triad" of LLM, embedding model, and (optionally) reranker, taking into account hardware constraints, data characteristics, and use-case requirements. The tool is developed in the context of the Frainderl project and is intended to be reusable across enterprise scenarios.
Motivation
- Building a production-ready RAG pipeline requires choosing at least two to three tightly interdependent components — LLM, embedding model, and optionally a reranker. No single combination is universally optimal: the best configuration depends on available hardware (GPU VRAM, CPU-only inference, edge deployment), data modality and structure (plain text, images, structured records, mixed corpora), retrieval latency requirements, and domain-specific language patterns. Currently, this selection is largely manual, expert-driven, and time-consuming.
- A systematic, data-driven wizard that accepts sample data or a data source link and returns ranked configuration recommendations would significantly reduce time-to-deployment for RAG systems and lower the barrier for organizations without deep ML expertise.
Goal
The main objective is a working RAG Configuration Wizard that takes as input:
- a description of available hardware,
- sample data or a link to a data source,
- a brief use-case description — and outputs a ranked list of LLM + embedding + reranker combinations with supporting benchmark results and a plain-language recommendation.
Approach
- The use case (Frainderl / enterprise RAG) is predefined; the configuration space and evaluation methodology are open to the student's design decisions.
- The wizard will be implemented in Python, leveraging the Hugging Face ecosystem for model loading and a framework for automated RAG evaluation. A lightweight UI will make the tool accessible to non-technical stakeholders.
- The solution should enable any organization to input their constraints and data and receive an actionable, evidence-backed component recommendation without requiring deep knowledge of the underlying ML stack.
Activity Distribution
- 30% programming
- 30% analysis
- 20% theory
- 20% evaluation
Preferred Skills
- Interest in NLP, large language models, and information retrieval
- Familiarity with embedding models and vector databases (e.g., FAISS, ChromaDB)
- Experience with or interest in RAG frameworks
- Basic knowledge of benchmarking methodologies
- Curiosity about practical AI deployment and hardware-aware model selection