Watch and track your favorite playlist.
Curated by: pouria (9 videos)
How AI models like Claude and Gemini run on a GPU and a TPU. We open up NVIDIA's H100, follow one matrix multiplication through its 16,896 CUDA cores, then run the same calculation through the systolic arrays of Google's TPU v5e. This video is built for learning! To keep things clear, some details have been oversimplified. Chapters 0:00 Intro 1:13 One word at a time 1:47 Neurons, weights, and parameters 2:36 From graph to matrices 3:40 Matrix multiplication 4:09 Why GPUs and TPUs 5:15 Opening up the H100 6:32 CUDA cores and tensor cores 7:52 High bandwidth memory 8:32 Loading the model 9:25 Kernels, threads, and warps 11:19 Following one thread 13:01 The memory bottleneck 14:09 The idea behind the TPU 14:30 Inside the TPU v5e 15:43 Running the systolic array 16:54 Why everyone still buys GPUs Contact tajpouria.dev@gmail.com https://github.com/tajpouria References How to Think About GPUs https://jax-ml.github.io/scaling-book/gpus/ How to Think About TPUs https://jax-ml.github.io/scaling-book/tpus/ NVIDIA H100 Tensor Core GPU https://www.nvidia.com/en-us/data-center/h100/ Transformers, the tech behind LLMs | Deep Learning Chapter 5 (3Blue1Brown) https://www.youtube.com/watch?v=wjZofJX0v4M How to Optimize a CUDA Matmul Kernel https://siboehm.com/articles/22/CUDA-MMM #gpu #tpu #ai #nvidia #llm