In October 2026, Google Research described an offline “sleep” stage that moves short-term context knowledge toward longer-term model parameters. Its reported math scores invite a narrower question: how consistent are the gains across models, tasks, baselines, and components? A published AIPOCH Open-Science case explores that question through a .science package containing the recorded analysis, files, and conversation. The case examines reported scores; it does not reproduce the paper's model-training experiments.
The headline becomes a comparison question
The LLM Sleep headline is broad; the math tables allow a more specific question. AIPOCH Open-Science traces the reported scores to particular models, tasks, baselines, and component comparisons so a reader can inspect both the pattern and its exceptions.
Google Research's introduction describes memory consolidation and Dreaming as two stages of the framework. The paper, arXiv version 2, places the math comparisons in its evaluation of consolidation without Dreaming. The case focuses on Tables 1 and 2 within that scope.
That scope matters. A score table can show a numerical advantage within a reported evaluation; it cannot by itself establish long-term retention, the value of Dreaming, or repeated-run reliability.
Figure 1. The original case diagram shows the wider Wake–Consolidate–Dream workflow. The paper's math tables evaluate consolidation without Dreaming.
Six settings show positive but uneven gains
Sleep has the highest reported score in each of Table 2's six model–task settings. Its lead varies by baseline and task, so one gain cannot stand for every comparison.
Table 2 of the paper reports Qwen3-1.7B and Qwen3-8B on AIME-24, AIME-25, and HMMT-25. Tables 1 and 2 label the metric average@16. Differences below are absolute points on that reported scale, not relative percentage gains. The published case audit calculates 24 Sleep-minus-baseline contrasts from Table 2, all positive, ranging from +0.2 to +6.4 points. Those are descriptive contrasts from reported scores, not 24 independent reruns.
One small comparison is especially instructive: Qwen3-1.7B on AIME-25 is 40.2 for Sleep versus 40.0 for OPSD, a +0.2 point lead. Reusing the same Sleep scores against four baselines does not create independent replications, and the paper does not quantify repeated-run uncertainty for this small difference. The case also notes different listed training-step counts for GRPO and Sleep, so the table does not establish an equal-compute contest.
Figure 2. Original case heatmap of Table 2's 24 Sleep-minus-baseline differences. Each cell is a descriptive score contrast, not an independent replication.
One ablation interrupts an all-components-help story
In one reported Qwen3-8B AIME-25 comparison, the no-semantic-reward configuration scores slightly higher than full Sleep. The table therefore does not show uniformly positive full-minus-ablation differences. It does not establish a repeatable benefit from removing semantic reward.
Table 1 shows full Sleep at 69.0 and the no-semantic-reward variant at 69.2: a −0.2 point full-minus-variant contrast. The case audit records eight positive component contrasts and this one negative contrast. Like the +0.2 point baseline lead, this small difference has no reported repeated-run uncertainty.
That is the kind of detail a research package makes worth checking. A reader can compare the component rows across tasks without turning one score ordering into a stable causal effect.
Figure 3. Original case ablation heatmap. The highlighted cell is the −0.2 point contrast in Table 1; the table does not establish a repeatable component effect.
Follow the evidence in the .science package
The AIPOCH Open-Science .science package lets a reader trace a reported result through the source paper, comparison CSVs, saved report, and recorded conversation. Follow the small positive difference and the ablation exception to see exactly what the archive supports.
The published case names sleep_reasoning_report.md and describes the retained CSVs and scripts. The saved report predates later checks recorded in the conversation: a revised report and unified script were not saved. Read those updates alongside the earlier report rather than treating it as the final consolidated audit. The AIPOCH Open-Science .science guide says imported sessions are read-only; in the desktop app, Fork creates a separate writable continuation.
A focused reading route is: open sleep_reasoning_report.md with the later conversation checks beside it, check sleep_table2_comparisons.csv for the 40.2-versus-40.0 comparison, then check sleep_table1_ablation_comparisons.csv for 69.0-versus-69.2. The package also inventories sleep_reasoning_results.csv and sleep_opsd_expansion_comparisons.csv; the latter helps keep OPSD-plus-expansion separate from the Sleep ablation. These are archived records, not fresh model-training results. An imported verification record reports the sender's checks, not a rerun on the reader's computer.
Figure 4. Conceptual reading guide through the paper, CSVs, and saved report. Consult the later conversation checks alongside that report; this path does not represent a new experiment.
FAQ
Did the case reproduce the paper's training experiments?
No. The case audits reported paper scores and archived calculations; it does not retrain or independently evaluate the models.
####Do 24 positive differences prove reliable improvement? No. They reuse scores within six settings, and the published case does not provide repeated-run uncertainty for the small leads.
####Can a reader continue the investigation? Yes. After importing the package, a reader can inspect the read-only record and, in the desktop app, use Fork to start a separate writable session, as described in the AIPOCH Open-Science guide.
Explore the published LLM Sleep case in AIPOCH Open-Science and follow one comparison from paper table to archived CSV before deciding what the headline supports.