Anthropic, one of the leading advanced artificial intelligence laboratories, warned investors in a report that technology could begin independently creating and improving its subsequent generations faster than experts predicted. This was reported by RBC-Ukraine with reference to the company's documentation. In it, developers directly point to the risk that the evolution of model capabilities will significantly outpace the ability of engineers to implement safety and alignment measures.
What is Recursive Self-Improvement
Recursive self-improvement (RSI) occurs at the moment when artificial intelligence becomes so effective at training similar systems that the process of technological development begins to move with unprecedented acceleration. According to Anthropic's internal estimates, the company may be approaching the RSI threshold set in its responsible scaling policy. At the same time, the developers emphasize that none of the current models have crossed this limit yet, and previously the company recorded a sharp increase in the performance of new models only after it had already occurred.
Two Polar Assessments of the Same Process
Critics and supporters of RSI view this process differently. Pessimists consider recursive self-improvement to be a critical limit after which algorithms will spiral out of human control and begin developing independently. Optimists, conversely, see this tool as a source of unprecedented productivity for scientific discoveries and engineering breakthroughs. Anthropic emphasizes that AI models are rapidly progressing in programming, mathematics, and cybersecurity, and against this backdrop, the issue of controlling their ability to independently improve is becoming increasingly acute.
Human Control and Revision of Safety Policy
Anthropic's management promises to "maintain meaningful human control and direct involvement in the investigation process and the creation of subsequent generations of AI." Complete system autonomy in designing new architectures, according to the company, is not planned for now. Security policies are being reviewed to prevent the loss of control over technological development. Thus, even when approaching the RSI threshold, the company declares that a human will remain in the decision-making loop.
Context: Warnings from OpenAI and at the UN Level
Anthropic's position is not isolated. OpenAI CEO Sam Altman, at a UN Security Council meeting dedicated to artificial intelligence, stated that the risk lies in the fact that everything is happening so fast that humans can no longer monitor what is happening or intervene if necessary. According to him, the automation of the AI development process itself could lead to a sharp acceleration in technology development, and systems are already becoming more autonomous, which is why state and international institutions need to take this speed into account when formulating regulatory approaches.
The combination of these statements indicates that the issue of recursive self-improvement is ceasing to be hypothetical: leading laboratories are openly discussing it as a practical scenario requiring a revision of security policies and international regulatory mechanisms.
