Evidence Insight

clinical-study-info-extractor

Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs. Use when the user wants to extract details from specific PMIDs.

86100Total Score
Core Capability
77 / 100
Functional Suitability
9 / 12
Reliability
9 / 12
Performance & Context
8 / 8
Agent Usability
12 / 16
Human Usability
7 / 8
Security
8 / 12
Maintainability
9 / 12
Agent-Specific
15 / 20
Medical Task
20 / 20 Passed
96Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs
4/4
92Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs
4/4
90Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs
4/4
90Packaged executable path(s): scripts/utils.py
4/4
90End-to-end case for Scope-focused workflow aligned to: Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs. Use when the user wants to extract details from specific PMIDs
4/4

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS
✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS
✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS
✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASS
Research Veto✅ PASS — Applicable
DimensionResultDetail
Scientific IntegrityPASS
Scientific content remained anchored to fetched metadata or source-linked evidence in the legacy review.
Practice BoundariesPASS
The package stayed in retrieval, extraction, or evidence-organization scope rather than drifting into unsupported interpretation.
Methodological GroundPASS
The legacy audit preserved a method-grounded interpretation of the Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs workflow.
Code UsabilityPASS
The packaged retrieval surface remained understandable at the command and parameter level in the archived review.

Core Capability77 / 100 — 8 Categories

Functional Suitability
The archived deduction in functional suitability traces back to: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
9 / 12
75%
Reliability
Reliability was softened by the legacy issue 'Improve stress-case output rigor'. Stress and boundary scenarios show weaker consistency
9 / 12
75%
Performance & Context
No point loss was recorded for performance context in the legacy audit.
8 / 8
100%
Agent Usability
The archived evaluation left some headroom for clinical-study-info-extractor under agent usability.
12 / 16
75%
Human Usability
A modest deduction remained in human usability for clinical-study-info-extractor in the archived review.
7 / 8
88%
Security
A modest deduction remained in security for clinical-study-info-extractor in the archived review.
8 / 12
67%
Maintainability
A modest deduction remained in maintainability for clinical-study-info-extractor in the archived review.
9 / 12
75%
Agent-Specific
Related legacy finding for clinical-study-info-extractor: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
15 / 20
75%
Core Capability Total77 / 100

Medical TaskExecution Average: 91.6 / 100 — Assertions: 20/20 Passed

96
Canonical
Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs
4/4 ✓
92
Variant A
Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs
4/4 ✓
90
Edge
Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs
4/4 ✓
90
Variant B
Packaged executable path(s): scripts/utils.py
4/4 ✓
90
Stress
End-to-end case for Scope-focused workflow aligned to: Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs. Use when the user wants to extract details from specific PMIDs
4/4 ✓
96
Canonical✅ Pass
Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs

The archived run for Batch extracts and verifies structured information (PMID, title,... remained guidance-driven rather than command-driven.

Basic 35/40|Specialized 60/60|Total 96/100
✅A1The clinical-study-info-extractor output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
92
Variant A✅ Pass
Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs

This variant a case stayed inside the documented workflow and remained instruction-led.

Basic 33/40|Specialized 59/60|Total 92/100
✅A1The clinical-study-info-extractor output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
90
Edge✅ Pass
Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs

Batch extracts and verifies structured information (PMID, title,... was evaluated as a bounded documentation path, not as a runnable script workflow.

Basic 32/40|Specialized 58/60|Total 90/100
✅A1The clinical-study-info-extractor output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
90
Variant B✅ Pass
Packaged executable path(s): scripts/utils.py

This variant b case stayed inside the documented workflow and remained instruction-led.

Basic 31/40|Specialized 59/60|Total 90/100
✅A1The clinical-study-info-extractor output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
90
Stress✅ Pass
End-to-end case for Scope-focused workflow aligned to: Batch extracts and verifies structured information (PMID, title, abstract, methodology, results, etc.) from clinical research literature using PMIDs. Use when the user wants to extract details from specific PMIDs

This stress case stayed inside the documented workflow and remained instruction-led.

Basic 28/40|Specialized 60/60|Total 90/100
✅A1The clinical-study-info-extractor output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
Medical Task Total91.6 / 100

Key Strengths

  • Primary routing is Evidence Insight with execution mode B
  • Static quality score is 77/100 and dynamic average is 78.6/100
  • Assertions and command execution outcomes are recorded per input for human review
  • Execution verification summary: Script verification 1/1; adjustment=5. utils.py: OK