Uploads from Programming Throwdown

Currently Playing: 188: World Models

Intro topic: Running News/Links: • Flow matching versus diffusion models • https://youtu.be/firXjwZ_6KI?is=QMq8DcCsXTTktOuE  • OpenCV 5 • https://opencv.org/opencv-5/ • Claude fable beats pokemon with no harness • https://youtu.be/Ty_50J84fMY?si=EJ1KjZCZegipfCsV  • Can the stockmarket swallow Anthropic, SpaceX and OpenAI? (https://www.economist.com/finance-and-economics/2026/06/01/can-the-stockmarket-swallow-anthropic-spacex-and-openai)   • https://archive.ph/nKEVw Book of the Show • Patrick • Strength of the Few - James Islington • https://amzn.to/4pmr10T • Jason • Descender - Jeff Lemire • https://amzn.to/3QKhp3l Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show • Patrick • No Man’s Sky • Jason • Paperlib https://paperlib.app/en/  Topic: World Models • Making decisions with AI • Action-Value (called a Q model): What is the long-term value of making a decision at a position • Policy: What action should I take (must be a distribution) • Value (called a V model): What is the value of a position (depends on policy) • Advantage/Disadvantage: difference in value given two policies • When advantage is +, do that more. • Model-Free • Look at the current situation and suggest an action • Run that action in the real world and measure the effect • Use that measurement to suggest better actions next time • Model-Based • Observe rollouts (sequences of situations) and learn the dynamics • Choose an action, use your dynamics model to measure the consequence • Potentially do MPC (try many actions and choose the best) • World Models • Observe many many rollouts and learn a full forward model (how to create the input in the future) • Train a policy & value inside the world model • Deploy the policy and fine-tune based on the real world