Data science, career and other topics
03 February 2025
A new architecture that pairs attention with a learnable long-term memory module, scaling to 2M+ tokens and outperforming Transformers on language modeling, reasoning, genomics, and time series.
27 January 2025
How pure reinforcement learning (without supervised fine-tuning) can teach LLMs to reason, producing open-source models that rival OpenAI-o1 on math and coding benchmarks.
13 January 2025
My review of the paper STAR Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution
06 January 2025
Coconut lets LLMs reason in latent space instead of generating text tokens, enabling breadth-first exploration of reasoning paths and better performance on tasks requiring backtracking.
28 December 2024
12 years of studying foreign languages with Anki
23 December 2024
BERT rebuilt with modern tricks — 2 trillion training tokens, 8192 context length, Flash Attention, and rotary embeddings — delivering state-of-the-art classification and retrieval with the best speed/memory efficiency among encoders.
Type at least 2 characters to search...