RAG Configuration Wizard: Automated Selection of LLM, Embedding Model, and ReRanker for Enterprise Use Cases

Description

This project develops an interactive configuration tool — the RAG Configuration Wizard — that helps organizations identify the optimal combination of components for a Retrieval-Augmented Generation (RAG) pipeline. Given a specific use case, the wizard evaluates and recommends the best-fitting "triad" of LLM, embedding model, and (optionally) reranker, taking into account hardware constraints, data characteristics, and use-case requirements. The tool is developed in the context of the Frainderl project and is intended to be reusable across enterprise scenarios.

Motivation

  • Building a production-ready RAG pipeline requires choosing at least two to three tightly interdependent components — LLM, embedding model, and optionally a reranker. No single combination is universally optimal: the best configuration depends on available hardware (GPU VRAM, CPU-only inference, edge deployment), data modality and structure (plain text, images, structured records, mixed corpora), retrieval latency requirements, and domain-specific language patterns. Currently, this selection is largely manual, expert-driven, and time-consuming.
  • A systematic, data-driven wizard that accepts sample data or a data source link and returns ranked configuration recommendations would significantly reduce time-to-deployment for RAG systems and lower the barrier for organizations without deep ML expertise.

Goal

The main objective is a working RAG Configuration Wizard that takes as input:

  1. a description of available hardware,
  2. sample data or a link to a data source,
  3. a brief use-case description — and outputs a ranked list of LLM + embedding + reranker combinations with supporting benchmark results and a plain-language recommendation.

Approach

  • The use case (Frainderl / enterprise RAG) is predefined; the configuration space and evaluation methodology are open to the student's design decisions.
  • The wizard will be implemented in Python, leveraging the Hugging Face ecosystem for model loading and a framework for automated RAG evaluation. A lightweight UI will make the tool accessible to non-technical stakeholders.
  • The solution should enable any organization to input their constraints and data and receive an actionable, evidence-backed component recommendation without requiring deep knowledge of the underlying ML stack.

Activity Distribution

  • 30% programming
  • 30% analysis
  • 20% theory
  • 20% evaluation

Preferred Skills

  • Interest in NLP, large language models, and information retrieval
  • Familiarity with embedding models and vector databases (e.g., FAISS, ChromaDB)
  • Experience with or interest in RAG frameworks
  • Basic knowledge of benchmarking methodologies
  • Curiosity about practical AI deployment and hardware-aware model selection
Apply