Evidence Insight
research-paper-downloader
Download academic papers from open-access sources when the user provides a DOI/arXiv ID or requests a keyword-based paper search, and return the saved PDF path.
87100Total Score
Core Capability
88 / 100
Functional Suitability
11 / 12
Reliability
10 / 12
Performance & Context
8 / 8
Agent Usability
14 / 16
Human Usability
8 / 8
Security
10 / 12
Maintainability
10 / 12
Agent-Specific
17 / 20
Medical Task
15 / 20 Passed
88The user provides a DOI (e.g., 10.1038/s41586-020-2649-2) and wants the corresponding open-access PDF
3/4
86The user provides an arXiv ID (e.g., 1706.03762) and wants the preprint PDF
3/4
86Multi-source open-access retrieval with priority-based fallback until a PDF is found
3/4
86Supports three input modes:
3/4
86End-to-end case for Multi-source open-access retrieval with priority-based fallback until a PDF is found
3/4
Veto GatesRequired pass for any deployment consideration
Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASSResearch Veto✅ PASS — Applicable
| Dimension | Result | Detail |
|---|---|---|
| Scientific Integrity | PASS | The archived evaluation kept the skill tied to retrieved records or indexed source material rather than invented scientific claims. |
| Practice Boundaries | PASS | The legacy review kept this workflow on the evidence-access side of the boundary, not the advice-giving side. |
| Methodological Ground | PASS | The older review treated the package logic as methodologically aligned with its stated workflow. |
| Code Usability | PASS | Code usability passed because the search or lookup workflow still exposed a usable entrypoint and output expectation. |
Core Capability88 / 100 — 8 Categories
Functional Suitability
Related legacy finding for research-paper-downloader: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
11 / 12
92%
Reliability
Related legacy finding for research-paper-downloader: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
10 / 12
83%
Performance & Context
No point loss was recorded for performance context in the legacy audit.
8 / 8
100%
Agent Usability
A modest deduction remained in agent usability for research-paper-downloader in the archived review.
14 / 16
88%
Human Usability
No point loss was recorded for human usability in the legacy audit.
8 / 8
100%
Security
A modest deduction remained in security for research-paper-downloader in the archived review.
10 / 12
83%
Maintainability
The archived evaluation left some headroom for research-paper-downloader under maintainability.
10 / 12
83%
Agent-Specific
Agent specific was softened by the legacy issue 'Stabilize executable path and fallback behavior'. Some inputs only reached PARTIAL due to execution gaps or weak boundary handling
17 / 20
85%
Core Capability Total88 / 100
Medical TaskExecution Average: 86.4 / 100 — Assertions: 15/20 Passed
88
Canonical
The user provides a DOI (e.g., 10.1038/s41586-020-2649-2) and wants the corresponding open-access PDF
3/4 ✓
86
Variant A
The user provides an arXiv ID (e.g., 1706.03762) and wants the preprint PDF
3/4 ✓
86
Edge
Multi-source open-access retrieval with priority-based fallback until a PDF is found
3/4 ✓
86
Variant B
Supports three input modes:
3/4 ✓
86
Stress
End-to-end case for Multi-source open-access retrieval with priority-based fallback until a PDF is found
3/4 ✓
88
Canonical✅ Pass
The user provides a DOI (e.g., 10.1038/s41586-020-2649-2) and wants the corresponding open-access PDF
The packaged path for The user provides a DOI (e.g., 10.1038/s41586-020-2649-2) and wants... remained intelligible, but the observed run timed out before completion.
Basic 33/40|Specialized 55/60|Total 88/100
✅A1The research-paper-downloader output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Variant A✅ Pass
The user provides an arXiv ID (e.g., 1706.03762) and wants the preprint PDF
The The user provides an arXiv ID (e.g., 1706.03762) and wants the... path is defined clearly, but this run was interrupted by a timeout.
Basic 31/40|Specialized 55/60|Total 86/100
✅A1The research-paper-downloader output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Edge✅ Pass
Multi-source open-access retrieval with priority-based fallback until a PDF is found
The Multi-source open-access retrieval with priority-based fallback... path is defined clearly, but this run was interrupted by a timeout.
Basic 30/40|Specialized 56/60|Total 86/100
✅A1The research-paper-downloader output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Variant B✅ Pass
Supports three input modes:
The workflow for Supports three input modes: is present, though the archived execution was cut short by a timeout.
Basic 29/40|Specialized 57/60|Total 86/100
✅A1The research-paper-downloader output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Stress✅ Pass
End-to-end case for Multi-source open-access retrieval with priority-based fallback until a PDF is found
The packaged path for End-to-end case for Multi-source open-access retrieval with... remained intelligible, but the observed run timed out before completion.
Basic 26/40|Specialized 60/60|Total 86/100
✅A1The research-paper-downloader output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
Medical Task Total86.4 / 100
Key Strengths
- Primary routing is Evidence Insight with execution mode B
- Static quality score is 88/100 and dynamic average is 73.6/100
- Assertions and command execution outcomes are recorded per input for human review
- Execution verification summary: Script verification 1/1; adjustment=5. download.py: OK