Inside Mixture of Experts: How Sparse Routing Scales LLMs

AI Tools Kit

AI Tools Kit provides free developer tools for working with AI language models. Built by developers, for developers.

GRPO vs PPO: How Modern LLMs Learn from Feedback

GRPO replaced PPO in post-training for DeepSeek-R1 and Gemini. Covers the math, training pipeline, design trade-offs, and when each method wins.

LLM Architecture

Inside LLM Training: The Transformer Pipeline Explained

The full LLM pre-training pipeline: tokenization, attention computation, cross-entropy loss, backpropagation, AdamW optimizer, and the architectural choices behind billion-parameter scale.

Inside Mixture of Experts: How Sparse Routing Scales LLMs

Related Articles

GRPO vs PPO: How Modern LLMs Learn from Feedback

Inside LLM Training: The Transformer Pipeline Explained