Data Analysis

batch-effect-correction

Corrects batch effects in merged bulk expression matrices with sample-level batch metadata using sva::ComBat(), preserves biological group structure, applies limma normalization, and generates paired before-and-after QC plots (boxplot, PCA, hierarchical clustering).

89100Total Score

Core Capability

90 / 100

Functional Suitability

12 / 12

Reliability

11 / 12

Performance & Context

7 / 8

Agent Usability

15 / 16

Human Usability

8 / 8

Security

11 / 12

Maintainability

11 / 12

Agent-Specific

15 / 20

Medical Task

24 / 25 Passed

92ComBat correction on 2-batch 2-group expression matrix

5/5

90Custom metadata columns with log_transform disabled

5/5

87Batch with only 1 sample (below ComBat minimum)

5/5

87Expression matrix with extra QC sample columns not in metadata

5/5

85Expression matrix with negative values and forced log-transform

4/5

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed

✓

Operational Stability

System remains stable across varied inputs and edge cases

PASS

✓

Structural Consistency

Output structure conforms to expected skill contract format

PASS

✓

Result Determinism

Equivalent inputs produce semantically equivalent outputs

PASS

✓

System Security

No prompt injection, data leakage, or unsafe tool use detected

PASS

Research Veto✅ PASS — Applicable

Dimension	Result	Detail
Scientific Integrity	PASS	No fabrication of statistical values; ComBat and normalizeBetweenArrays outputs derive from sva and limma package computations on user data
Practice Boundaries	PASS	No medical diagnoses; skill corrects technical batch variation without clinical conclusions
Methodological Ground	PASS	ComBat with biological group as covariate (mod matrix) is the correct approach for preserving biological signal during batch correction; post-ComBat limma normalization is methodologically sound
Code Usability	PASS	Syntactically correct; sva::ComBat() called with proper mod matrix; limma::normalizeBetweenArrays() applied post-correction; OMP thread limit set at startup for reproducibility; no infinite loops; clean exit codes

Core Capability90 / 100 — 8 Categories

Functional Suitability

Covers all promised use cases: auto/yes/no log transform, custom metadata columns, extra sample handling, paired QC plots, and both core correction steps (ComBat + normalizeBetweenArrays)

12 / 12

100%

Reliability

Nine SKILL_* codes including SKILL_INVALID_DATA, SKILL_INVALID_TYPE, and SKILL_TIMEOUT; validate_design() enforces ComBat design requirements; align_inputs() handles extra samples gracefully; one minor gap: negative value error does not identify source genes/samples

11 / 12

92%

Performance & Context

SKILL.md is 291 lines within budget; references directory for algorithm details; OMP thread limit env vars set for cross-platform reproducibility; one minor gap: no guidance on dataset size limits before ComBat performance degrades

7 / 8

88%

Agent Usability

Agent Response Contract specifies 5 structured outputs; When to Read External Files table guides navigation; four examples cover common scenarios including custom columns and disabled transforms; feedback design excellent with GC snapshot logging

15 / 16

94%

Human Usability

Natural trigger language; When Not to Use clearly excludes scRNA-seq, raw FASTQ, DEG without batch labels, and single-batch datasets; prerequisites section clear; extra sample subset behavior explicitly documented

8 / 8

100%

Security

No hardcoded credentials; no eval/exec; OMP env vars set at top level are safe system-level settings; one minor gap: output_dir path not checked for traversal

11 / 12

92%

Maintainability

Clean 7-script modular structure; input_functions.R separated for testability; implementation checklist present; one gap: no unit tests in polished package

11 / 12

92%

Agent-Specific

Precise trigger language for ComBat batch correction; SKILL.md under 500 lines; idempotent via set.seed; escape hatches present; nine structured error codes; minor gap: GC snapshot logged to agent stdout may add noise; composability docs missing

15 / 20

75%

Core Capability Total90 / 100

Medical TaskExecution Average: 88.2 / 100 — Assertions: 24/25 Passed

Canonical

ComBat correction on 2-batch 2-group expression matrix

5/5 ✓

Variant A

Custom metadata columns with log_transform disabled

5/5 ✓

Edge

Batch with only 1 sample (below ComBat minimum)

5/5 ✓

Variant B

Expression matrix with extra QC sample columns not in metadata

5/5 ✓

Stress

Expression matrix with negative values and forced log-transform

4/5 ✓

Canonical✅ Pass

ComBat correction on 2-batch 2-group expression matrix

Full 4-step workflow documented; auto log-transform correct; 9 output files produced; Agent Response Contract complete

Basic 37/40|Specialized 55/60|Total 92/100

✅A1Output routes to CLI with required parameters

✅A2Auto log-transform detection logic documented and implemented

✅A3SKILL_* codes cover all failure modes including SKILL_INVALID_DATA and SKILL_TIMEOUT

✅A4Paired before/after QC plots documented and produced

✅A5Agent Response Contract specifies 5 structured outputs including QC assessment

Pass rate: 5 / 5

Variant A✅ Pass

Custom metadata columns with log_transform disabled

Custom column resolution correct; log_transform no bypasses log2; example in SKILL.md

Basic 36/40|Specialized 54/60|Total 90/100

✅A1Custom column names resolved via validate_metadata_columns()

✅A2log_transform no correctly bypasses log2 transformation

✅A3Custom column name example documented in SKILL.md

✅A4SKILL_MISSING_COLUMNS fires when custom column name not found

✅A5set.seed applied for reproducibility

Pass rate: 5 / 5

Edge✅ Pass

Batch with only 1 sample (below ComBat minimum)

SKILL_INVALID_DATA correctly triggered; ComBat design requirement enforced; minimum requirements documented

Basic 35/40|Specialized 52/60|Total 87/100

✅A1SKILL_INVALID_DATA triggered for batch with only 1 sample

✅A2Rationale for minimum batch size documented

✅A3Minimum 2 per group also enforced

✅A4Minimum sample requirements documented in Input Format section

✅A5Error exits cleanly with status 1

Pass rate: 5 / 5

Variant B✅ Pass

Expression matrix with extra QC sample columns not in metadata

Extra columns ignored with warning; subset behavior documented; SKILL_SAMPLE_MISMATCH for reverse case

Basic 35/40|Specialized 52/60|Total 87/100

✅A1Extra expression columns trigger warning, not error

✅A2Analysis continues with metadata-matched samples only

✅A3Subset behavior documented in SKILL.md Input Format

✅A4Corrected output matrix contains only matched samples

✅A5SKILL_SAMPLE_MISMATCH fires when metadata samples absent from matrix

Pass rate: 5 / 5

Stress✅ Pass

Expression matrix with negative values and forced log-transform

SKILL_INVALID_DATA correctly triggered; source genes/samples not identified in error message

Basic 34/40|Specialized 51/60|Total 85/100

✅A1SKILL_INVALID_DATA triggered for negative values with log_transform yes

✅A2Error message clearly states negative values cannot be log-transformed

✅A3auto mode would not trigger this for log-scale data

❌A4Error message identifies which genes or samples contain negative values

✅A5No silent coercion of negative values

Pass rate: 4 / 5

Medical Task Total88.2 / 100

Key Strengths

Most complete error handling of the five skills: nine SKILL_* codes covering SKILL_INVALID_DATA, SKILL_INVALID_TYPE, SKILL_EMPTY_FILE, SKILL_RUNTIME_ERROR, and SKILL_TIMEOUT
OMP/BLAS thread limit env vars set at startup ensure cross-platform reproducibility for ComBat's numerical computations
Graceful handling of extra expression columns (warn and continue) matches real-world pipeline usage where QC samples may be present
Auto log-transform detection with documented thresholds (max > 50 or 99th quantile > 16) prevents silent misclassification
Comprehensive paired before/after QC plots (boxplot, PCA, hierarchical clustering) provide full batch correction assessment