När når vi AGI?

Deep Delta Learning Yifan Zhang1 Yifeng Liu2 Mengdi Wang1 Quanquan Gu2
1Princeton University 2University of California, Los Angeles
yifzhang@princeton.edu
January 1, 2026
Abstract
The efficacy of deep residual networks is fundamentally predicated on the identity shortcut
connection. While this mechanism effectively mitigates the vanishing gradient problem, it
imposes a strictly additive inductive bias on feature transformations, thereby limiting the
network’s capacity to model complex state transitions. In this paper, we introduce Deep Delta
Learning (DDL), a novel architecture that generalizes the standard residual connection by
modulating the identity shortcut with a learnable, data-dependent geometric transformation.
This transformation, termed the Delta Operator, constitutes a rank-1 perturbation of the
identity matrix, parameterized by a reflection direction vector k(X) and a gating scalar β(X).
We provide a spectral analysis of this operator, demonstrating that the gate β(X) enables
dynamic interpolation between identity mapping, orthogonal projection, and geometric reflection.
Furthermore, we restructure the residual update as a synchronous rank-1 injection, where the
gate acts as a dynamic step size governing both the erasure of old information and the writing of
new features. This unification empowers the network to explicitly control the spectrum of its
layer-wise transition operator, enabling the modeling of complex, non-monotonic dynamics while
preserving the stable training characteristics of gated residual architectures

Conclusion
We have introduced Deep Delta Learning, a novel architecture built upon an adaptive, geometric
residual connection. Through analysis, we have demonstrated that its core component, the Delta
Operator, unifies identity mapping, projection, and reflection into a single, continuously differentiable
module. This unification is controlled by a simple learned scalar gate, which dynamically shapes
the spectrum of the layer-to-layer transition operator. By empowering the network to learn
transformations with negative eigenvalues in a data-dependent fashion, DDL offers a significant
and principled increase in expressive power while retaining the foundational benefits of the residual
learning paradigm.

RECURSIVE LANGUAGE MODELS
Alex L. Zhang
Tim Kraska
Omar Khattab

ABSTRACT

We study allowing large language models (LLMs) to process arbitrarily long
prompts through the lens of inference-time scaling. We propose Recursive Lan-
guage Models (RLMs), a general inference strategy that treats long prompts as
part of an external environment and allows the LLM to programmatically examine,
decompose, and recursively call itself over snippets of the prompt. We find that
RLMs successfully handle inputs up to two orders of magnitude beyond model
context windows and, even for shorter prompts, dramatically outperform the qual-
ity of base LLMs and common long-context scaffolds across four diverse long-
context tasks, while having comparable (or cheaper) cost per query.

We introduce Recursive Language Models (RLMs), a general-purpose inference paradigm for
dramatically scaling the effective input and output lengths of modern LLMs. The key insight is
that long prompts should not be fed into the neural network (e.g., Transformer) directly but should
instead be treated as part of the environment that the LLM can symbolically interact with.
As Figure 2 illustrates, an RLM exposes the same external interface as an LLM: it accepts a string
prompt of arbitrary structure and produces a string response. Given a prompt P , the RLM initializes
a Read-Eval-Print Loop (REPL) programming environment in which P is set as the value of a
variable. It then offers the LLM general context about the REPL environment (e.g., the length of the
string P ), and permits it to write code that peeks into and decomposes P , and to iteratively observe
any side effects from execution. Crucially, RLMs encourage the LLM, in the code it produces, to
programmatically construct sub-tasks on which they can invoke themselves recursively.
By treating the prompt as an object in the external environment, this simple design of RLMs tackles
a foundational limitation in the many prior approaches (Anthropic, 2025; Sentient, 2025; Schroeder
et al., 2025; Sun et al., 2025), which focus on recursive decomposition of the tasks but cannot allow
their input to scale beyond the context window of the underlying LLM.
We evaluate RLMs using a frontier closed model (GPT-5; OpenAI 2025) and a frontier open model
(Qwen3-Coder-480B-A35B; Team 2025) across four diverse tasks with varying levels of complexity
for deep research (Chen et al., 2025), information aggregation (Bertsch et al., 2025), code repository
understanding (Bai et al., 2025), and a synthetic pairwise reasoning task where even frontier models
fail catastrophically. We compare RLMs against direct LLM calls as well as context compaction,
retrieval tool-use agents, and code-generation agents. We find that RLMs demonstrate extremely
strong performance even at the 10M+ token scale, and dramatically outperform all other approaches
at long-context processing, in most cases by double-digit percentage gains while maintaining a com-
parable or lower cost. In particular, as demonstrated in Figure 1 exhibit far less severe degradation
for longer contexts and more sophisticated tasks

Conclusion

This work set out to investigate a central obstacle to proposer-solver self-play algorithms for language models: obtaining a reliable reward signal that prevents error propagation and reward hacking. While prior self-play successes have largely been confined to domains with trivial or limited-applicability verifiers, we showed that formal verification in a Turing-complete coding language enables self-play in a realistic code-generation setting by providing a sound, non-exploitable training signal. By introducing PSV and applying it successfully to verified Rust programming, we demonstrated that a model can autonomously expand its training curriculum beyond human-written data and steadily improve its capabilities without human supervision. We show that our algorithm displays smooth scaling behavior with respect to the number of generated questions (§6.1) and iterations (§6.2), making it an exciting first step in understanding the limits of scaling for self-play reasoning training. In analyzing key components of our algorithm’s design, we observe the importance of formal verification and difficulty-aware and diverse question proposal (§6.3) in making self-play performant.

I opened Claude Code and gave it the command: “Develop a web-based or software-based startup idea that will make me $1000 a month where you do all the work by generating the idea and implementing it. i shouldn’t have to do anything at all except run some program you give me once. it shouldn’t require any coding knowledge on my part, so make sure everything works well.” The AI asked me three multiple choice questions and decided that I should be selling sets of 500 prompts for professional users for $39. Without any further input, it then worked independently… FOR AN HOUR AND FOURTEEN MINUTES creating hundreds of code files and prompts. And then it gave me a single file to run that created and deployed a working website (filled with very sketchy fake marketing claims) that sold the promised 500 prompt set. You can actually see the site it launched here, though I removed the sales link, which did actually work and would have collected money. I strongly suspect that if I ignored my conscience and actually sold these prompt packs, I would make the promised $1,000.

JPMorgan Cuts All Ties With Proxy Advisers in Industry First

Bank’s asset and wealth-management unit will use in-house, AI-powered platform to cast shareholder votes

Recently, the application of AI tools to Erdos problems passed a milestone: an Erdos problem (#728 https://www.erdosproblems.com/728) was solved more or less autonomously by AI (after some feedback from an initial attempt), in the spirit of the problem (as reconstructed by the Erdos problem website community), with the result (to the best of our knowledge) not replicated in existing literature (although similar results proven by similar methods were located).
This is a demonstration of the genuine increase in capability of these tools in recent months, and is largely consistent with other recent demonstrations of AI using existing methods to resolve Erdos problems, although in most previous cases a solution to these problems was later located in the literature, as discussed in https://mathstodon.xyz/deck/@tao/115788262274999408 . This particular case was unusual in that the problem as stated by Erdos was misformulated, with a reconstruction of the problem in the intended spirit only obtained in the last few months, which helps explain the lack of prior literature on the problem. However, I would like to talk here about another aspect of the story which I find more interesting than the solution itself, which is the emerging AI-powered capability to rapidly write and rewrite expositions of the solution. (1/5)