Open Science Ranks #1 in BiomniBench-DA Public 50. Here Is What the Workbench Behind the Score Actually Does
AIPOCH Open Science ranks #1 in the supplied BiomniBench-DA Public-50 comparison with a 79.05% dual-judge mean. See the result and product workflow.
Today, AIPOCH is sharing a new internal result for Open Science: 79.05% on the 50-task set labeled “Public 50” in our BiomniBench-DA comparison workbook, using gpt-5.6-sol (xhigh).
That is the highest displayed value in the supplied workbook. It is not an official first-place listing, and the AIPOCH run is not yet present on an external results page.
The tempting takeaway is the rank. The more useful takeaway is the system being tested. A scientific agent is not only a model responding to a prompt. It also needs a workbench that can hold project context, read files, reach scientific data sources, execute code, preserve artifacts, expose tool activity, and keep a researcher in control.
Open Science is an open-source, local-first, model-agnostic, self-hosted AI research workbench for reproducible scientific discovery. The result reported here does not prove that any single Open Science feature caused the score. It does give us a concrete reason to explain what BiomniBench-DA evaluates, how the 79.05% value was recorded, and what the Open Science workbench contributes to a research workflow.
Here is what BiomniBench actually measures
BiomniBench is a process-level evaluation framework for LLM agents working on real-world biomedical research tasks. Instead of checking only whether an agent reaches one expected final answer, it scores the analytical trajectory against task-specific rubrics designed with domain experts. The first release, BiomniBench-DA, focuses on data analysis and contains 100 tasks across 17 task types, five disease areas, and a general-biology category, according to the May 2026 bioRxiv preprint. The paper is a preprint, so its claims should be read as research that has not yet completed journal peer review.
Why evaluate the trajectory? A plausible final result can still come from the wrong file, an unsuitable statistical method, missed confounders, or unsupported interpretation. The reverse can also happen: an agent may follow a defensible analysis path but reach a different answer from one reference trace. BiomniBench-DA is designed to retain more of that process signal.
The framework examines six parts of a run:
| Evaluation dimension | What the rubric examines |
|---|---|
| Data handling | Whether the correct data are loaded, joined, filtered, and prepared |
| Method selection | Whether the analytical method fits the research question and data |
| Statistical rigor | Whether tests, thresholds, corrections, and reporting are applied appropriately |
| Biological interpretation | Whether the interpretation is grounded in the specific biological context |
| Scientific reasoning | Whether the analytical chain is coherent and its limits are acknowledged |
| Source reliability | Whether claims are supported by identifiable, credible sources |
That makes BiomniBench-DA relevant to research workbenches, not only language models. The model still matters, but so do the files it can access, the tools it can call, the code it can run, the context it retains, and the evidence a reviewer can inspect afterward.
One naming detail matters here. The paper describes a 100-task release. “Public 50” is the label used in AIPOCH’s supplied workbook for the 50-task set evaluated in this run. This article does not present “Public 50” as the name of an official public leaderboard.
The result
AIPOCH evaluated the aipoch/open-science harness with GPT-5.6 Sol at xhigh reasoning effort. The workbook records two aggregate judge scores for the same run and an equal-weight mean.
| Field | Recorded value |
|---|---|
| Evaluation owner | AIPOCH |
| Harness | aipoch/open-science |
| Model | GPT-5.6 Sol (xhigh) |
| Task-set label | Public 50 |
| Gemini 3.1 Pro judge score | 81.04% |
| DeepSeek v4-pro judge score | 77.06% |
| Dual-judge mean | 79.05% |
| Result status | Internal; no public result URL in the workbook |
| External listing status | Not yet listed |
The calculation is direct:
(81.04 + 77.06) / 2 = 79.05
The two judges recorded in the workbook are Gemini 3.1 Pro and DeepSeek v4-pro. The workbook contains aggregate values, not the per-task score sheet or full grading protocol. This article therefore reports the recorded result without reconstructing evidence that was not supplied.
How the number compares with the collected reference results
The 79.05% internal result is the highest displayed value in AIPOCH’s comparison workbook. That sentence needs three qualifiers: the workbook combines more than one judge-coverage pattern, the AIPOCH row has not been independently verified, and the difference from the nearest displayed value is small.
The nearest displayed reference value is 78.80% for OmicOS with gpt-5.5. The workbook marks that value as DeepSeek-only, while Open Science’s 79.05% is the mean of two judges. The numerical difference is 0.25 percentage points, but the supplied material contains no repeated runs, confidence intervals, task-level resampling, or judge-variance analysis. The margin should not be described as a statistically established lead.
The workbook also retains these collected mixed values: Claude/CSSwitch at 68.20%, EvoScientist at 65.30%, Biomni at 63.05%, synthetic-sciences/openscience at 62.50%, ai4s-research/open-science at 62.10%, and Wisp Science at 61.20%. They provide context inside the supplied comparison, not proof of a universal ordering across every possible model, judge, or evaluation configuration.
The externally accessible BiomniBench-AI4S comparison repository does not currently include the AIPOCH run. Until a public submission or independent reproduction is available, the accurate description is: highest displayed value in AIPOCH’s supplied workbook, from an internal run.
Here is what Open Science actually does
Open Science brings research tasks, files, execution, scientific data access, and reviewable artifacts into one local desktop workspace. It is available for macOS, Windows, and Linux, and its Apache-2.0 source code can be inspected and forked.
The product is organized around projects and sessions rather than one-off chats. A project can keep source files, conversation branches, generated reports, tables, figures, notebook history, previews, and run records together. That structure matters when a research question changes halfway through an analysis or when a result needs to be revisited days later.
The setup is designed around explicit choices:
- the researcher creates a project and states the goal, inputs, constraints, expected outputs, and review criteria;
- the researcher selects an agent runtime, model provider, model, and approval profile;
- the agent can read project files, search, call permitted connectors, and execute approved Python, R, or shell work;
- generated artifacts remain attached to the session and project file library;
- the researcher can inspect tool activity, preview outputs, review provenance, and revise an earlier prompt on a separate branch.
This is workflow support for scientific work. Open Science does not turn an agent output into a scientific conclusion, and it does not remove the need for domain review, statistical checking, or verification against primary evidence.
The research loop
The Open Science repository documents a practical loop that moves from a research request to an inspectable artifact without separating the conversation from the work that produced it.
- Create a project.
- Attach the relevant files and define the research goal.
- Choose the agent runtime, model, and approval mode.
- Review file reads, searches, connector calls, and code execution as the task runs.
- Open the generated report, table, figure, code, or notebook output.
- Inspect the available provenance and branch the conversation if another analytical direction is needed.
The loop can support literature work, data preparation, code execution, statistical analysis, simulation, visualization, and research writing. The exact path depends on the task and the skills, connectors, model, and execution environments the researcher has configured.
Why this matters: the model is only one layer
Model names dominate agent comparisons because they fit neatly into a table. Scientific work does not. A long-horizon analysis also depends on how the agent receives data, whether it can execute methods, how it reaches external sources, what permissions govern its actions, and whether the resulting files can be reviewed later.
Open Science treats those elements as parts of the same operating environment:
1. Persistent projects keep the research context together
Open Science stores projects, sessions, drafts, files, previews, and run history locally so work can continue across restarts. Completed prompts can be edited into selectable branches without deleting the original downstream path. The artifact produced on one branch remains associated with that branch rather than being blended with a later alternative.
This can help researchers compare two analytical directions while preserving the context that produced each one. It also avoids treating the final response as the only durable record of a run.
2. Model-agnostic operation separates the workbench from one provider
Open Science supports multiple agent backends, including Claude Code, OpenCode, Codex, and CodeBuddy. The repository documents built-in cloud providers, compatible custom gateways, and existing Claude or Codex subscription logins, with model and provider compatibility checked against the selected agent backend.
That separation lets a researcher choose a model for a session without moving the surrounding project, files, previews, and artifact history into a different application. Provider choices and availability can change, so the installed application’s provider picker and connection test remain the current source of truth.
3. Execution turns a suggestion into reviewable work
Open Science can support approved execution through persistent Python and R kernels, stateless shell commands, and remote SSH compute targets. Notebook code and output remain in run history, while the Variables view can show names, types, shapes, and previews from a live Python or R namespace in read-only form.
Execution is still governed by the researcher. Package installation, commands, network access, connector calls, and remote work may require approval depending on the selected permission profile. The resulting analysis remains a starting point for expert review.
4. Scientific skills and connectors make the workspace domain-aware
The Open Science repository currently documents 18 featured file-based research skills and 24 built-in research connectors. The listed skills include literature review, protein-structure and protein-design workflows, single-cell tools, environment inspection, and remote compute. The connector catalog covers sources and services for literature, genes, genomes, variants, structures, protein annotation, expression, omics archives, chemistry, clinical trials, cancer models, and other research resources.
Skills remain readable files that researchers can inspect, create, import, enable, or select explicitly with /. Connectors remain behind per-tool permission controls. The wider AIPOCH medical-research-skills repository also publishes more than 500 file-based medical and scientific research skills that can be inspected and paired with Open Science from GitHub.
Catalogs change as the product develops. The installed app and the latest repository documentation should be treated as the source of truth for what is available in a specific release.
5. Provenance keeps the output connected to available evidence
Open Science stores generated artifacts as immutable, checksummed versions. Its Provenance view can expose the evidence the application was able to verify at creation time, including producer code and execution history, referenced inputs, an observed environment inventory, the producing conversation branch, and version-scoped reviewer findings.
Missing evidence is marked unavailable rather than inferred. That boundary is important: provenance can make a result more inspectable, but it does not establish that the method was correct or that the scientific interpretation is valid.
6. Permission profiles keep the researcher in the loop
Open Science documents three approval profiles: Ask for approval, Auto-approve edits, and Full access. The narrowest profile asks before edits, commands, network access, and connector calls; broader profiles are intended for clearly scoped and trusted work.
Project data, settings, artifact versions, and provenance evidence are stored locally. External data flow can still occur when a selected model provider, web search, connector, or remote service receives the context needed for a task. Researchers should review sensitive inputs, tool parameters, provider policies, and screenshots before allowing external calls or public sharing.
What the 79.05% result does not prove
The internal result shows that one recorded Open Science configuration received a 79.05% dual-judge mean on the 50-task set labeled Public 50. It does not establish a universal product ranking.
Four limits remain visible in the supplied evidence:
- No task-level material: the workbook does not include per-task scores, agent traces, or the full grading protocol.
- No uncertainty analysis: it contains no repeated runs, intervals, or variance estimates.
- Mixed comparison conditions: model choice, judge coverage, and provenance differ across retained rows.
- No feature attribution: the workbook cannot show whether a particular Open Science capability caused the recorded result.
The result also does not show performance on unrelated research questions, wet-lab work, clinical decisions, or every possible BiomniBench-DA configuration. It should be read as one evaluation record inside a disclosed setup.
What we can say
The evidence supports a precise statement: AIPOCH Open Science with gpt-5.6-sol (xhigh) achieved a 79.05% dual-judge mean in AIPOCH’s internal run across the 50-task set labeled Public 50. That is the highest displayed value in AIPOCH’s supplied comparison workbook.
The broader product story is separate from that score. Open Science provides an open, local-first workspace where researchers can combine files, models, code execution, scientific skills, connectors, approvals, previews, and provenance in one inspectable workflow. Those capabilities are documented in the public repository; scientific validity still depends on the people reviewing the methods and outputs.
What is next
The next evidence step is public reproducibility. A public AIPOCH result package would need enough task-level material, configuration detail, and grading information for others to inspect or reproduce the run. Until that exists, the internal status should remain attached to the 79.05% figure.
The product itself is already open for inspection. Researchers can review the Open Science source repository, check current installers and release notes, or visit the AIPOCH Open Science product page.
Frequently Asked Questions
What is BiomniBench?
BiomniBench is a process-level evaluation framework for LLM agents performing biomedical research tasks. Its data-analysis release scores the analytical trajectory across data handling, method selection, statistical rigor, biological interpretation, scientific reasoning, and source reliability.
What score did AIPOCH Open Science receive?
AIPOCH Open Science received a 79.05% dual-judge mean in AIPOCH’s internal BiomniBench-DA Public-50 run. The value is the equal-weight average of 81.04% from Gemini 3.1 Pro and 77.06% from DeepSeek v4-pro.
Is the AIPOCH result an official first-place listing?
No. It is the highest displayed value in AIPOCH’s supplied comparison workbook, but the run is internal and is not currently present on the external comparison page reviewed for this article.
What does Open Science provide for researchers?
Open Science provides a local project workspace that can support model-based agent sessions, file access, Python and R execution, shell work, scientific skills, data connectors, artifact previews, permission controls, and provenance for researcher review. It does not replace scientific judgment or independent verification.
Does local-first mean no data ever leaves the computer?
No. Project state is stored locally by default, but prompts and necessary context may be sent to a selected model provider, search service, connector, or remote compute service. Researchers remain responsible for reviewing sensitive data and external service policies.
Disclaimer
The evaluation figures reproduce aggregate values from a user-supplied workbook and should be interpreted only within the setup and disclosure limits described above.
Open Science is a research workflow tool. It does not replace researcher judgment, and researchers remain fully responsible for evaluating the accuracy, completeness, and appropriateness of any outputs generated. All outputs require independent verification and expert interpretation before use in any research context.
References and external links in this article are provided for informational purposes. AIPOCH does not endorse and is not responsible for the content of third-party sources.