Efficiency & Scaling
3 posts tagged Efficiency & Scaling. All topics
2026
3 notes-
01
AI Auto-Research and Automated Paper Writing: Systems, Open-Source Frameworks, and Empirical Failure Modes
Between 2024 and 2026, AI auto-research systems evolved from fixed-template scripts (The AI Scientist v1) to template-free tree-search agents (AI Scientist-v2, AI-Researcher, Agent Laboratory, Zochi, Kosmos, EvoScientist) and single-file overnight ratchets (karpathy/autoresearch, AIDE², RD-Agent). Reviewing 50+ papers, repositories, and benchmarks alongside 5,932 controlled runs from our own studies shows where autonomous loops deliver genuine leverage—and where LLM-judge Goodharting, shared-prior ceilings, rubric leakage, multi-hop citogenesis, cache-write bill inversions, and subset-denominator bugs manufacture illusory progress.
-
02
Generative Modeling via Drifting: Training-Time Pushforward Evolution for One-Step Generation
Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He (MIT & Harvard, arXiv:2602.04770) introduce Drifting Models, moving iterative distribution evolution from inference time to training time. Driven by a kernelized attraction-repulsion drifting field in self-supervised feature space, a single-pass DiT-L generator reaches 1.54 FID in latent space and 1.61 FID in pixel space on ImageNet 256×256 at 1-NFE, while matching 100-step Diffusion Policy on robotic control.
-
03
You Don't Need Long Context: Compaction, Selective Reading, and Semantic State Space Models
NVIDIA researchers evaluate whether frontier language models require 200K–1M token contexts (arXiv:2611.05306). Serving at 16K instead of 256K boosts H100 decode throughput by 3.6x–7x, End-of-Conversation Compaction and Selective Reading match or beat full-context baselines across 13 benchmarks, and Single-Action Compaction solves 70.2%–82.4% of SWE-bench Verified at a 7K–8K median peak context.