← LEARN LLM · RESEARCH INDEX 3 ENTRIES

Efficiency & Scaling

3 posts tagged Efficiency & Scaling. All topics

2026

3 notes
  1. 01
    Research report··34 min read

    AI Auto-Research and Automated Paper Writing: Systems, Open-Source Frameworks, and Empirical Failure Modes

    Between 2024 and 2026, AI auto-research systems evolved from fixed-template scripts (The AI Scientist v1) to template-free tree-search agents (AI Scientist-v2, AI-Researcher, Agent Laboratory, Zochi, Kosmos, EvoScientist) and single-file overnight ratchets (karpathy/autoresearch, AIDE², RD-Agent). Reviewing 50+ papers, repositories, and benchmarks alongside 5,932 controlled runs from our own studies shows where autonomous loops deliver genuine leverage—and where LLM-judge Goodharting, shared-prior ceilings, rubric leakage, multi-hop citogenesis, cache-write bill inversions, and subset-denominator bugs manufacture illusory progress.

  2. 02
    Deep dive··17 min read

    Generative Modeling via Drifting: Training-Time Pushforward Evolution for One-Step Generation

    Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He (MIT & Harvard, arXiv:2602.04770) introduce Drifting Models, moving iterative distribution evolution from inference time to training time. Driven by a kernelized attraction-repulsion drifting field in self-supervised feature space, a single-pass DiT-L generator reaches 1.54 FID in latent space and 1.61 FID in pixel space on ImageNet 256×256 at 1-NFE, while matching 100-step Diffusion Policy on robotic control.

  3. 03
    Deep dive··17 min read

    You Don't Need Long Context: Compaction, Selective Reading, and Semantic State Space Models

    NVIDIA researchers evaluate whether frontier language models require 200K–1M token contexts (arXiv:2611.05306). Serving at 16K instead of 256K boosts H100 decode throughput by 3.6x–7x, End-of-Conversation Compaction and Selective Reading match or beat full-context baselines across 13 benchmarks, and Single-Action Compaction solves 70.2%–82.4% of SWE-bench Verified at a 7K–8K median peak context.