Inside SIA, the Self-Improving AI Agent Updating Both Scaffold and Weights

Those updates are what set SIA apart.

• Harness updates modify the agent's scaffold, which includes the system prompt, tool-dispatch logic, retry policies, and answer-extraction code . This is akin to a developer improving the agent's strategy and workflow without touching the brain underneath.
• Weight updates directly fine-tune the model's parameters, changing the fundamental capability of the Task-Agent itself .

The paper's central thesis is that combining both levers is more powerful than iterating on the scaffold alone, a claim it backs with benchmarks across three unrelated domains .

What the Benchmarks Actually Show

The research paper evaluates SIA on three contrasting tasks to demonstrate generality: a 191-class Chinese legal charge classification problem (LawBench), low-level GPU kernel optimization using Triton on an H100, and single-cell RNA denoising in biology .

The reported improvements are substantial but highly domain-specific.

• 56.6% accuracy gain on LawBench over the baseline . A separate source notes SIA scored 0.701 on LawBench, compared to Claude Code's 0.173 .
• 91.9% runtime reduction on GPU kernel optimization versus the baseline . The paper reports SIA-W+H kernels ran at 1,017 µs compared to prior SOTA's 1,161 µs .
• 502% improvement on single-cell RNA denoising over baseline .

It is important to distinguish these peer-reviewed gains from the public announcement's claim that SIA "accelerates superintelligence by 350X" as indicated by an OpenAI-designed benchmark . That specific figure does not appear in the arXiv paper and is only sourced from the company's Business Wire press release . The academic sources report the three domain benchmarks above .

How It Differs from Other Self-Improving Agents

The self-improving agent space has grown crowded, but SIA carves out a distinct position alongside frameworks like Nous Research's Hermes Agent.

• Scaffold-only approaches typically let an LLM tweak its prompt or tool-use policy but leave model weights untouched. SIA's differentiator is the combination: the paper reports that scaffold-plus-weight updates outperform scaffold-only iteration on every benchmark tested .
• Nous Research's Hermes Agent operates with a built-in learning loop that creates reusable "skills" from successful task completions, storing them for future reuse. Agents with 20 or more self-created skills complete similar research tasks roughly 40% faster than fresh instances, though the improvement is domain-specific and does not claim cross-domain generalization . Hermes improves through accumulated procedural knowledge and memory, while SIA's paper emphasizes direct weight updates alongside harness changes .

Hexo Labs describes SIA as "the world's first agent that learns from itself instead of human actions" . The paper's novelty claim rests on the dual update mechanism for a task-specific agent .

The Grant Program for Researchers

Alongside the code release under an MIT license, Hexo Labs announced Frontier Research Grants to give academic and independent researchers access to SIA, compute infrastructure, and direct collaboration with the lab's team . The program aims to put self-improving AI tools into the hands of domain experts working on ambitious problems, with Hexo arguing that open-source access is a guardrail rather than a risk . The lab has already partnered with researchers at Stanford University and the University of Cambridge, among others .

SIA represents a concrete step toward AI systems that don't just perform tasks but actively improve their own machinery. Whether the 350x superintelligence claim holds up to broader scrutiny remains to be seen, but the underlying research—a self-improving loop that edits both code and weights—marks a notable new capability in the open-source agent landscape.