GPU Mode is an open community dedicated to teaching GPU programming for AI and machine learning. The lecture series covers everything from CUDA fundamentals to cutting-edge topics like Flash Attention, quantized training, and multi-GPU communication.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/gpu-mode/lectures/llms.txt
Use this file to discover all available pages before exploring further.
Get Set Up
Install PyTorch, CUDA, and the tools you need to run lecture code locally
CUDA Fundamentals
Start from the PMPP book and build up to real kernel writing
Triton Guide
Write GPU kernels in Python with Triton
Flash Attention
IO-aware attention that powers modern LLMs