Uploads from CampusX

Currently Playing: Mastering G-Eval: The Deterministic LLM-as-a-Judge Framework Explained | CampusX

This lecture was taken live for insiders, join here: https://youtu.be/j_G30FLmCcw 📱 Grow with us: CampusX' LinkedIn: https://www.linkedin.com/company/campusx-official CampusX on Instagram for daily tips: https://www.instagram.com/campusx.official My LinkedIn: https://www.linkedin.com/in/nitish-singh-03412789 Discord: https://discord.gg/PsWu8R87Z8 E-mail us at support@campusx.in Chapters: 00:00 - Introduction & Recap of the 3-Tier RAG Eval Suite 03:15 - Count-Based vs. Judgment-Based Evaluation Metrics 08:18 - Why Claim-Counting Fails for Correctness and Style 13:28 - The Two Core Flaws of Standard LLM-as-a-Judge 21:40 - How Probability Jitter Causes High Evaluation Variance 24:15 - Introducing G-Eval: The 2023 Breakthrough Framework 26:00 - G-Eval Innovation 1: Converting Criteria to Steps via Chain of Thought 30:35 - G-Eval Innovation 2: Probability-Weighted Scoring via Logprobs 43:05 - Implementing Custom G-Eval Metrics in DeepEval (`GEval` Class) 48:38 - Evaluating Metric 1: Correctness (Factuality vs. Expected Output) 58:20 - Evaluation Steps vs. Explicit Scoring Rubrics 01:11:18 - Evaluating Metric 2: Completeness (Addressing Multi-Part Queries) 01:15:05 - Improving Completeness Through Systematic Prompt Engineering 01:18:28 - Evaluating Metric 3: Tone & Style (Reference-Free Evaluation) 01:22:15 - Handling Over-Correction in Evaluator Rubrics 01:25:20 - Summary & Next Steps: Safety and Operational Audits