---
title: "Google Research: Neural Networks Store 95–98% of Facts but Cannot Retrieve Up to a Third of Them"
description: "Google Research showed that LLMs encode 95–98% of facts but fail to retrieve 26–34% of them. The new knowledge profiling method separates real knowledge gaps from access failures, including the word-order paradox."
date: 2026-08-19T23:52:07.000Z
lang: en
url: https://xab.info/en/posts/google-research-knowledge-profiling-llm-en
tags: [google-research, llm, knowledge-profiling, ai, large-language-models]
publisher: "XAB.info"
---

# Google Research: Neural Networks Store 95–98% of Facts but Cannot Retrieve Up to a Third of Them

![Robotic arm extracting a glowing memory block from server racks — illustration of how Google neural networks store 95–98% of facts but cannot retrieve up to a third of them](https://xab.info/media/2026/08/20/google-research-knowledge-profiling-llm/google-research-knowledge-profiling-llm-1.webp)

## 🎯 Key Points

- LLMs encode 95–98% of facts but fail to retrieve 26–34% when asked directly — this is an access failure, not a knowledge gap
- The knowledge profiling method classifies facts into 5 categories on a sample of 2,150 facts × 10 questions
- Reversing the 'subject–object' word order sharply degrades retrieval
- Developers should test systems on rephrased questions, not on a single template

Researchers at Google Research published a paper showing that state-of-the-art large language models (LLMs) encode 95 to 98% of the tested facts, yet when asked directly they are unable to retrieve 26–34% of them. The authors' key conclusion is that this is not an error in the knowledge itself, but a failure in the mechanism for accessing already stored information. In other words, the model 'knows' the fact but cannot 'pull it out' without additional cues. This fundamentally changes the approach to diagnosing fact-generation errors: the problem shifts from the plane of training and data volume to the plane of retrieval architecture.

### The 'Knowledge Profiling' Method

To systematize the problem, the authors proposed a method called knowledge profiling. Instead of evaluating the accuracy of each question in isolation, the system classifies each fact into five categories: not encoded; encoded but not retrieved; retrieved directly; retrieved with reasoning; inferred without encoding. This taxonomy allows real knowledge gaps to be separated from technical access limitations. To build the sample, the researchers used 2,150 facts from 'Wikipedia,' each accompanied by 10 formulated questions, yielding more than 21,000 'fact–question' pairs for analysis.

### The 'Empty Shelf' and 'Lost Key' Metaphors

The central metaphor of the paper is simple and intuitive. An 'empty shelf' means that the fact never made it into the model at all — it is not present in its parameters. A 'lost key' means that the fact is present, but the model cannot find it without additional cues. The difference between these two states is fundamental from an engineering standpoint: in the first case, a larger model or a broader coverage of training data is required; in the second, it is enough to improve the methods for retrieving existing knowledge without touching the model itself. This is precisely why accurate diagnosis of the error class is critically important for optimizing the costs of system development.

### The Word-Order Paradox

The most unexpected result concerns word order. When a question reverses the familiar 'subject–object' direction, retrieval quality drops sharply. For example, a model easily answers a question about who wrote a particular book, but may fail to handle the rephrasing 'which book did this author write.' The authors emphasize that this is not a knowledge gap but a limitation of the access mechanism: the model stores the fact in a particular orientation and struggles to retrieve it in the reverse direction. At the same time, the Google Research paper does not claim that word order is the sole cause of failures, but rather provides a structured method for measuring this specific class of errors.

### Practical Takeaways for Developers

For developers building applications based on language models, the paper's findings are practical in nature. Factual systems must be tested on rephrased questions, with different word orders, and in a recognition format, rather than relying on a single canonical phrasing template. The knowledge profiling method allows the share of 'lost keys' to be quantified and targeted work to be done on improving retrieval. Published against the backdrop of the active deployment of LLMs in corporate and consumer products, the work sets a new standard for verifying fact generation: accuracy on 'convenient' questions is no longer a sufficient criterion for model reliability.

## 🔍 Fact-Check Verification

- [Google figures out why smart neural networks 'blank out' on simple questions](https://www.ixbt.com/news/2026/08/19/v-google-vyjasnili-pochemu-umnye-nejroseti-tupjat-na-prostyh-voprosah.html) - Источник от 19.08.2026 подтверждает тему работы Google Research о сбое доступа к закодированным фактам и метод профилирования знаний. Точные проценты и размер выборки взяты из предоставленной основы; независимой верификации цифр во втором источнике нет.

## ❓ FAQ

### Q: What is knowledge profiling?
**A:** It is a Google Research method in which each fact is classified into five categories: not encoded, encoded but not retrieved, retrieved directly, retrieved with reasoning, inferred without encoding. It allows real knowledge gaps to be separated from access failures.

### Q: What share of facts is not retrieved by the model?
**A:** According to the paper, models encode 95–98% of the tested facts but cannot retrieve 26–34% of them when asked directly.

### Q: What sample did the researchers use?
**A:** The sample consisted of 2,150 facts from 'Wikipedia,' each accompanied by 10 questions.

### Q: Why does word order affect the result?
**A:** Reversing the 'subject–object' direction sharply degrades retrieval: this is a limitation of the access mechanism, not a knowledge gap in the model.

### Q: What should LLM-application developers do?
**A:** Test factual systems on rephrased questions, with different word orders, and in a recognition format, rather than relying on a single canonical template.