MARL, Part 4: Multi-Agent Policy Gradients
Joint policy gradients, deterministic actor–critic updates, and the sequential importance corrections behind HAPPO.
Blog
MARL, Part 3: Centralized Training, Decentralized Decisions
Local information constraints, centralized critics, and why an agent’s memory matters even when training observes the state.
MARL, Part 2: Cooperation and Credit Assignment
Shared rewards, counterfactual policy gradients, and additive or monotonic value decomposition for cooperative agents.
MARL, Part 1: Foundations and Markov Games
Joint policies, best responses, and Nash equilibria, with a minimax Bellman equation and contraction proof for zero-sum Markov games.