Evidence Insight

openalex-db

Access the OpenAlex database (240M+ scholarly works) for bibliometric analysis, literature search, and citation tracking; use when you need to query works/authors/institutions/concepts without an API key.

90100Total Score
Core Capability
85 / 100
Functional Suitability
11 / 12
Reliability
9 / 12
Performance & Context
8 / 8
Agent Usability
14 / 16
Human Usability
8 / 8
Security
9 / 12
Maintainability
9 / 12
Agent-Specific
17 / 20
Medical Task
20 / 20 Passed
97You need to search scholarly works by keyword and filter by properties (e.g., open access, year, venue)
4/4
93You want to look up authors and analyze their publication output and citation impact
4/4
91Query OpenAlex entities: works, authors, institutions, concepts
4/4
91Keyword search with filtering support
4/4
91End-to-end case for Query OpenAlex entities: works, authors, institutions, concepts
4/4

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS
✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS
✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS
✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASS
Research Veto✅ PASS — Applicable
DimensionResultDetail
Scientific IntegrityPASS
The archived evaluation kept the skill tied to retrieved records or indexed source material rather than invented scientific claims.
Practice BoundariesPASS
The package stayed in retrieval, extraction, or evidence-organization scope rather than drifting into unsupported interpretation.
Methodological GroundPASS
No methodological-grounding issue was recorded for openalex-db in the archived evaluation.
Code UsabilityPASS
The packaged retrieval surface remained understandable at the command and parameter level in the archived review.

Core Capability85 / 100 — 8 Categories

Functional Suitability
Related legacy finding for openalex-db: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
11 / 12
92%
Reliability
Reliability was softened by the legacy issue 'Improve stress-case output rigor'. Stress and boundary scenarios show weaker consistency
9 / 12
75%
Performance & Context
No point loss was recorded for performance context in the legacy audit.
8 / 8
100%
Agent Usability
The archived evaluation left some headroom for openalex-db under agent usability.
14 / 16
88%
Human Usability
No point loss was recorded for human usability in the legacy audit.
8 / 8
100%
Security
The legacy audit deducted points for openalex-db in security.
9 / 12
75%
Maintainability
The archived evaluation left some headroom for openalex-db under maintainability.
9 / 12
75%
Agent-Specific
Agent specific was softened by the legacy issue 'Improve stress-case output rigor'. Stress and boundary scenarios show weaker consistency
17 / 20
85%
Core Capability Total85 / 100

Medical TaskExecution Average: 92.6 / 100 — Assertions: 20/20 Passed

97
Canonical
You need to search scholarly works by keyword and filter by properties (e.g., open access, year, venue)
4/4 ✓
93
Variant A
You want to look up authors and analyze their publication output and citation impact
4/4 ✓
91
Edge
Query OpenAlex entities: works, authors, institutions, concepts
4/4 ✓
91
Variant B
Keyword search with filtering support
4/4 ✓
91
Stress
End-to-end case for Query OpenAlex entities: works, authors, institutions, concepts
4/4 ✓
97
Canonical✅ Pass
You need to search scholarly works by keyword and filter by properties (e.g., open access, year, venue)

You need to search scholarly works by keyword and filter by... was evaluated as a bounded documentation path, not as a runnable script workflow.

Basic 36/40|Specialized 60/60|Total 97/100
✅A1The openalex-db output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
93
Variant A✅ Pass
You want to look up authors and analyze their publication output and citation impact

You want to look up authors and analyze their publication output... was evaluated as a bounded documentation path, not as a runnable script workflow.

Basic 34/40|Specialized 59/60|Total 93/100
✅A1The openalex-db output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
91
Edge✅ Pass
Query OpenAlex entities: works, authors, institutions, concepts

The archived run for Query OpenAlex entities: works, authors, institutions, concepts remained guidance-driven rather than command-driven.

Basic 33/40|Specialized 58/60|Total 91/100
✅A1The openalex-db output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
91
Variant B✅ Pass
Keyword search with filtering support

Keyword search with filtering support was evaluated as a bounded documentation path, not as a runnable script workflow.

Basic 32/40|Specialized 59/60|Total 91/100
✅A1The openalex-db output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
91
Stress✅ Pass
End-to-end case for Query OpenAlex entities: works, authors, institutions, concepts

This stress case stayed inside the documented workflow and remained instruction-led.

Basic 29/40|Specialized 60/60|Total 91/100
✅A1The openalex-db output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
Medical Task Total92.6 / 100

Key Strengths

  • Primary routing is Evidence Insight with execution mode B
  • Static quality score is 85/100 and dynamic average is 79.6/100
  • Assertions and command execution outcomes are recorded per input for human review
  • Execution verification summary: Script verification 1/1; adjustment=5. openalex_client.py: OK