Watch and track your favorite playlist.
Curated by: CampusX (12 videos)
In this session of our LLM Evaluation Masterclass, we pivot from Application Evals to Model Evaluations (Model Evals). You will learn how frontier labs measure raw model intelligence and how you, as an AI engineer, can read these standardised benchmarks to optimise cost, latency, and system performance. This lecture was taken live for insiders, join here: https://youtu.be/j_G30FLmCcw Slides: https://1drv.ms/o/c/85452F67DAA1111C/IgCIRXGQVQm8R7-eF2h5lZ4NAayeX_9R3V4XHDDmsXYMnrk 📱 Grow with us: CampusX' LinkedIn: https://www.linkedin.com/company/campusx-official CampusX on Instagram for daily tips: https://www.instagram.com/campusx.official My LinkedIn: https://www.linkedin.com/in/nitish-singh-03412789 Discord: https://discord.gg/PsWu8R87Z8 E-mail us at support@campusx.in Chapters 00:00 - Course Playlist Recap & Shifting to Model-Level Evaluations 02:26 - Why AI Engineers Need Model Evals: Sourcing the Brain of Your App 04:42 - Sourcing the Architecture: Proprietary APIs vs. Self-Hosted Open Source 05:44 - The 4 Structural Steps of an Industry-Standard Model Evaluation 08:33 - Standardised Global Benchmarks vs. Custom Evaluation Datasets 09:48 - Zomato Case Study: Balancing Token Cost, Latency, and Accuracy Matrix 14:00 - The ROI Framework: Squeezing Maximum Performance Out of Cheap Models 17:00 - Introducing the 8 Core LLM Capabilities Sourced by Frontier Labs 17:46 - Pillar 1: Knowledge & Reasoning (Factual Recall Across 57 Subjects) 19:26 - Pillar 2: Coding & Software Engineering (Functional Generation & Bug Fixing) 21:54 - Pillar 3: Mathematics (Grade School Logic to Advanced Symbolic Problems) 23:14 - Pillar 4: Long Context Management (Evaluating Information Retrieval at Scale) 26:12 - Pillar 5 & 6: Vision/Multimodality and Agentic Tool & API Calling 27:10 - Pillar 7 & 8: Safety Alignment (Jailbreak Defense) & Instruction Following 27:39 - Closing Words: Why Investing Time in Deep Theory Prevents Practical Failures