Data science, career and other topics
15 June 2026
MiniMax Sparse Attention is a practical sparse-attention design for million-token LLMs - it uses a lightweight learned indexer to select relevant KV blocks and performs exact attention only over those blocks. The paper is important because it connects architecture, training stability, and GPU kernels into a deployable long-context system, powering the open-weight MiniMax-M3 model.
10 June 2026
A hands-on look at MiniMax M3 through Claude Code — what its new MiniMax Sparse Attention (MSA) is and how it differs from the lightning-attention and full-attention designs of earlier MiniMax models, plus three real tasks: auditing and refactoring an old idle game, debugging two stubborn UI bugs from screenshots, and turning years of Spotify history into music recommendations.
09 June 2026
A review of Mike X Cohen 50 ML Projects To Understand LLMs, a hands-on book that uses code, statistics, and controlled experiments to investigate what happens inside GPT-2 — covering what it does well, who it is for, and a few small nitpicks.
01 June 2026
A review of Gamma-World, NVIDIA's generative multi-agent world model that produces shared, action-controllable video rollouts for multiple independently acting agents. It places agents at the vertices of a regular simplex in rotary space for permutation symmetry, routes cross-agent interaction through learnable hub tokens to keep attention linear in the number of agents, and distills a diffusion teacher into a causal student that streams at 24 FPS.
18 May 2026
An evaluation of MiniMax M2.7 used through Claude Code on three workflows I run regularly — writing code for a Kaggle competition submission, drafting and auditing Obsidian vault notes, and refactoring an old PyTorch project — with Claude Opus 4.7 as the comparison baseline.
24 April 2026
DeepSeek V4 pairs a hybrid sparse-attention stack with on-policy distillation across domain specialists to bring 1M-token inference to frontier quality at a fraction of the FLOPs and KV cache of its predecessor.
Type at least 2 characters to search...