Data Analysis

batch-effect-correction

Corrects batch effects in merged bulk expression matrices with sample-level batch metadata using sva::ComBat(), preserves biological group structure, applies limma normalization, and generates paired before-and-after QC plots (boxplot, PCA, hierarchical clustering).

89100Total Score
Core Capability
90 / 100
Functional Suitability
12 / 12
Reliability
11 / 12
Performance & Context
7 / 8
Agent Usability
15 / 16
Human Usability
8 / 8
Security
11 / 12
Maintainability
11 / 12
Agent-Specific
15 / 20
Medical Task
24 / 25 Passed
92ComBat correction on 2-batch 2-group expression matrix
5/5
90Custom metadata columns with log_transform disabled
5/5
87Batch with only 1 sample (below ComBat minimum)
5/5
87Expression matrix with extra QC sample columns not in metadata
5/5
85Expression matrix with negative values and forced log-transform
4/5

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS
✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS
✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS
✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASS
Research Veto✓ PASS — Applicable
DimensionResultDetail
Scientific IntegrityPASS
No fabrication of statistical values; ComBat and normalizeBetweenArrays outputs derive from sva and limma package computations on user data
Practice BoundariesPASS
No medical diagnoses; skill corrects technical batch variation without clinical conclusions
Methodological GroundPASS
ComBat with biological group as covariate (mod matrix) is the correct approach for preserving biological signal during batch correction; post-ComBat limma normalization is methodologically sound
Code UsabilityPASS
Syntactically correct; sva::ComBat() called with proper mod matrix; limma::normalizeBetweenArrays() applied post-correction; OMP thread limit set at startup for reproducibility; no infinite loops; clean exit codes

Core Capability90 / 100 — 8 Categories

Functional Suitability
Covers all promised use cases: auto/yes/no log transform, custom metadata columns, extra sample handling, paired QC plots, and both core correction steps (ComBat + normalizeBetweenArrays)
12 / 12
100%
Reliability
Nine SKILL_* codes including SKILL_INVALID_DATA, SKILL_INVALID_TYPE, and SKILL_TIMEOUT; validate_design() enforces ComBat design requirements; align_inputs() handles extra samples gracefully; one minor gap: negative value error does not identify source genes/samples
11 / 12
92%
Performance & Context
SKILL.md is 291 lines within budget; references directory for algorithm details; OMP thread limit env vars set for cross-platform reproducibility; one minor gap: no guidance on dataset size limits before ComBat performance degrades
7 / 8
88%
Agent Usability
Agent Response Contract specifies 5 structured outputs; When to Read External Files table guides navigation; four examples cover common scenarios including custom columns and disabled transforms; feedback design excellent with GC snapshot logging
15 / 16
94%
Human Usability
Natural trigger language; When Not to Use clearly excludes scRNA-seq, raw FASTQ, DEG without batch labels, and single-batch datasets; prerequisites section clear; extra sample subset behavior explicitly documented
8 / 8
100%
Security
No hardcoded credentials; no eval/exec; OMP env vars set at top level are safe system-level settings; one minor gap: output_dir path not checked for traversal
11 / 12
92%
Maintainability
Clean 7-script modular structure; input_functions.R separated for testability; implementation checklist present; one gap: no unit tests in polished package
11 / 12
92%
Agent-Specific
Precise trigger language for ComBat batch correction; SKILL.md under 500 lines; idempotent via set.seed; escape hatches present; nine structured error codes; minor gap: GC snapshot logged to agent stdout may add noise; composability docs missing
15 / 20
75%
Core Capability Total90 / 100

Medical TaskExecution Average: 88.2 / 100 — Assertions: 24/25 Passed

92
Canonical
ComBat correction on 2-batch 2-group expression matrix
5/5 ✓
90
Variant A
Custom metadata columns with log_transform disabled
5/5 ✓
87
Edge
Batch with only 1 sample (below ComBat minimum)
5/5 ✓
87
Variant B
Expression matrix with extra QC sample columns not in metadata
5/5 ✓
85
Stress
Expression matrix with negative values and forced log-transform
4/5 ✓
92
CanonicalPass
ComBat correction on 2-batch 2-group expression matrix

Full 4-step workflow documented; auto log-transform correct; 9 output files produced; Agent Response Contract complete

Basic 37/40|Specialized 55/60|Total 92/100
A1Output routes to CLI with required parameters
A2Auto log-transform detection logic documented and implemented
A3SKILL_* codes cover all failure modes including SKILL_INVALID_DATA and SKILL_TIMEOUT
A4Paired before/after QC plots documented and produced
A5Agent Response Contract specifies 5 structured outputs including QC assessment
Pass rate: 5 / 5
90
Variant APass
Custom metadata columns with log_transform disabled

Custom column resolution correct; log_transform no bypasses log2; example in SKILL.md

Basic 36/40|Specialized 54/60|Total 90/100
A1Custom column names resolved via validate_metadata_columns()
A2log_transform no correctly bypasses log2 transformation
A3Custom column name example documented in SKILL.md
A4SKILL_MISSING_COLUMNS fires when custom column name not found
A5set.seed applied for reproducibility
Pass rate: 5 / 5
87
EdgePass
Batch with only 1 sample (below ComBat minimum)

SKILL_INVALID_DATA correctly triggered; ComBat design requirement enforced; minimum requirements documented

Basic 35/40|Specialized 52/60|Total 87/100
A1SKILL_INVALID_DATA triggered for batch with only 1 sample
A2Rationale for minimum batch size documented
A3Minimum 2 per group also enforced
A4Minimum sample requirements documented in Input Format section
A5Error exits cleanly with status 1
Pass rate: 5 / 5
87
Variant BPass
Expression matrix with extra QC sample columns not in metadata

Extra columns ignored with warning; subset behavior documented; SKILL_SAMPLE_MISMATCH for reverse case

Basic 35/40|Specialized 52/60|Total 87/100
A1Extra expression columns trigger warning, not error
A2Analysis continues with metadata-matched samples only
A3Subset behavior documented in SKILL.md Input Format
A4Corrected output matrix contains only matched samples
A5SKILL_SAMPLE_MISMATCH fires when metadata samples absent from matrix
Pass rate: 5 / 5
85
StressPass
Expression matrix with negative values and forced log-transform

SKILL_INVALID_DATA correctly triggered; source genes/samples not identified in error message

Basic 34/40|Specialized 51/60|Total 85/100
A1SKILL_INVALID_DATA triggered for negative values with log_transform yes
A2Error message clearly states negative values cannot be log-transformed
A3auto mode would not trigger this for log-scale data
A4Error message identifies which genes or samples contain negative values
A5No silent coercion of negative values
Pass rate: 4 / 5
Medical Task Total88.2 / 100

Key Strengths

  • Most complete error handling of the five skills: nine SKILL_* codes covering SKILL_INVALID_DATA, SKILL_INVALID_TYPE, SKILL_EMPTY_FILE, SKILL_RUNTIME_ERROR, and SKILL_TIMEOUT
  • OMP/BLAS thread limit env vars set at startup ensure cross-platform reproducibility for ComBat's numerical computations
  • Graceful handling of extra expression columns (warn and continue) matches real-world pipeline usage where QC samples may be present
  • Auto log-transform detection with documented thresholds (max > 50 or 99th quantile > 16) prevents silent misclassification
  • Comprehensive paired before/after QC plots (boxplot, PCA, hierarchical clustering) provide full batch correction assessment