Analytical Method Validation

---
name: analytical-method-validation
description: Plans and evaluates analytical method validation, verification, and transfer using Python scripts (plan_validation, check_response, check_accuracy_precision, check_detection_limits, check_bioanalytical_run, compare_methods) under ICH Q2(R2)/Q14, ICH M10, USP 1220/1225/1226, CLSI EP, or ISO/IEC 17025. Covers protocols, linearity, accuracy and precision, DL/QL, M10 run checks, TOST equivalence, Deming and Passing-Bablok. Use when designing a validation protocol with pre-stated acceptance criteria. Use when evaluating linearity, precision, or detection limit data. Use when checking an ICH M10 bioanalytical run or ISR. Use when comparing or transferring a method between labs. Does not decide a method is validated; USP, CLSI, ISO text is not supplied.
license: MIT
compatibility: Requires Python 3.11+. Scripts use only the standard library - no numpy, scipy, or network access. Statistical distributions are computed from first principles so results are reproducible in any conforming interpreter.
allowed-tools: Read Write Edit Bash
metadata:
  version: '1.1'
  category: ideation-and-design
  maintainer: Kalaris Labs
  last-reviewed: '2026-07-27'
---

# Analytical Method Validation

## When to use

Any time the question is whether an analytical procedure is fit for its intended purpose:
designing a validation study, evaluating validation data, verifying a compendial procedure,
transferring a procedure to another laboratory or instrument, or defending any of these in a
report.

## The two rules

**1. Establish which framework governs before designing anything.** The same assay validates
differently under ICH Q2(R2), USP <1225>, ICH M10, CLSI EP, and ISO/IEC 17025. They differ in
which characteristics are required, how the studies are laid out, and whether numeric acceptance
criteria are supplied at all. Blending them produces a protocol that satisfies none of them.

**2. State acceptance criteria before collecting data.** Criteria chosen after seeing results are
not acceptance criteria, and deciding them post hoc is a standing audit finding. ICH Q2(R2)
deliberately supplies almost no numeric criteria — they have to come from the specification, the
analytical target profile (ICH Q14 section 3), or development data. ICH M10 is the exception: it
supplies explicit numbers, and they differ between chromatographic assays and ligand binding
assays.

## Scope

This skill plans studies, computes the statistics correctly, and structures the documentation. It
does **not** decide that a procedure is validated, release a batch, accept or reject a run, close
an investigation, or substitute for the analyst, the technical reviewer, the quality unit, or the
regulator. Every script reports; none of them concludes.

## Copyright boundary

ICH guidelines are published openly and licensed for reuse with acknowledgement, so their
requirements are encoded directly in this skill. **USP general chapters, CLSI EP documents, and
ISO standards are copyrighted and paywalled.** For those, this skill supplies the designation,
scope, and where to obtain an authorised copy — never the text, never invented thresholds. Do not
ask an agent to retrieve, transcribe, or reconstruct their content. If a number matters and it
lives in a paywalled document, read it from the authorised copy.

## Frameworks

```bash
cd skills/analytical-method-validation/scripts
python3 plan_validation.py --list-frameworks
```

| Key | Governs | Numeric criteria supplied |
| --- | --- | --- |
| `ich-q2r2` | Release and stability testing of drug substances and products | Almost none — you derive them |
| `ich-m10` | Bioanalytical concentration measurement (PK, TK, BE) | Yes, and they differ by modality |
| `usp-1220` | Compendial procedure lifecycle, three stages | Paywalled |
| `usp-1225` / `usp-1226` | Validation / verification of compendial procedures | Paywalled |
| `clsi` | Clinical laboratory measurement procedures (EP series) | Paywalled |
| `iso-17025` | Lab-developed and modified methods under accreditation | No — "to the extent necessary" |

**Q2(R2) replaced Q2(R1) in November 2023 and restructured the characteristics.** Range is now
the parent characteristic (section 3.2), containing *response* (linearity) and *validation of
lower range limits* (DL/QL). Accuracy and precision are section 3.3 and may be evaluated in
combination against a single criterion. Robustness is treated as a development activity and
cross-refers to ICH Q14. Multivariate procedures are addressed explicitly (2.5 and 3.2.2.3), and
Annex 2 adds worked examples for techniques Q2(R1) never covered — quantitative ¹H-NMR, NIR,
quantitative LC/MS, qPCR, biological assays, and particle size. A Q2(R1)-shaped protocol — a flat
list of linearity, range, accuracy, precision, specificity, LOD, LOQ, robustness — is out of date.
Note also the error correction dated 30 November 2023 to Table 5 and Tables 6–11.

## Scripts

```bash
cd skills/analytical-method-validation/scripts
```

| Script | Question answered |
| --- | --- |
| `plan_validation.py` | Which framework, which characteristics, what study layout, what protocol? |
| `check_response.py` | Does the calibration model actually hold across the range? |
| `check_accuracy_precision.py` | What is the recovery, and how much of the variability is between days? |
| `check_detection_limits.py` | What are DL and QL by each allowed approach, and do they serve the reporting threshold? |
| `check_bioanalytical_run.py` | Does this run meet ICH M10 for its modality? |
| `compare_methods.py` | Are two procedures equivalent, at a pre-stated margin? |

All take `--format table|tsv|json`. Provenance, guideline citations, and caveats go to stderr;
data goes to stdout, so `> out.tsv` keeps them separate. Exit code is `0` for no findings, `1`
when findings were raised, `2` for bad input — so any of them can gate a workflow.

## Workflow

### 1. Fix the framework and the required characteristics

```bash
python3 plan_validation.py --framework ich-q2r2 --attribute assay --technique hplc --range-use assay
```

Q2(R2) Table 1 decides what is required from the *measured attribute*, not from the technique. For
an assay: specificity, response, accuracy, repeatability, intermediate precision. For a limit
test: specificity and DL only. For an identity test: specificity alone. Attributes accepted include
`assay`, `impurity` (quantitative), `impurity-limit`, and `identity`.

Reportable range comes from the specification. Q2(R2) Table 2 gives worked examples — 80–120% of
declared content for an assay, 70–130% for content uniformity, reporting threshold to 120% of the
specification for an impurity.

### 2. Generate the protocol and fill in the criteria

```bash
python3 plan_validation.py --framework ich-q2r2 --attribute impurity --protocol > protocol.md
```

Every bracketed field is a decision to make and record *before* data collection. The protocol
skeleton deliberately refuses to pre-fill acceptance criteria for Q2(R2) work, because there is no
defensible default.

### 3. Evaluate the response

```bash
python3 check_response.py -i calibration.csv --max-back-calc-error 2
```

Input is `level,response`, one row per injection; repeated rows at the same level are replicates,
and supplying them is what makes the linearity test possible.

Real output from a curve that a coefficient of determination would wave through:

```
statistic                           value
distinct levels                     5
slope                               166.6000
intercept                           2495.0000
intercept CI includes 0             no
coefficient of determination (r2)   0.9830
lack-of-fit F                       469.5294
lack-of-fit p                       1.5139e-06
runs test p                         0.0492

level     n  mean_response  mean_back_calculated  relative_error_pct
50.0000   2  10075.0000     45.4982               -9.0036
75.0000   2  15150.0000     75.9604               1.2805
100.0000  2  20050.0000     105.3721              5.3721
125.0000  2  24050.0000     129.3818              3.5054
150.0000  2  26450.0000     143.7875              -4.1417
```

r² = 0.983 and the model is unusable: −9.0% back-calculated error at the bottom of the range,
lack-of-fit p = 1.5 × 10⁻⁶, non-random residual signs. **r² is not evidence of linearity** — it
rises with range and is nearly insensitive to curvature. The lack-of-fit F test against pure error
and the residual pattern are the evidence, which is why Q2(R2) 3.2.2.1 asks for an analysis of the
deviation of points from the line rather than a correlation coefficient alone.

Add `--weight 1/x2` for a wide-range curve. The script flags heteroscedasticity when the residual
variance in the top third of the range exceeds the bottom third by more than 10×, because an
unweighted fit then biases exactly the low end where a reporting threshold lives.

### 4. Evaluate accuracy and precision

```bash
python3 check_accuracy_precision.py -i ap.csv --accuracy-limit 2 --rsd-limit 1.0 --design-check assay
```

Input is `level,measured,group`, where `group` is the intermediate-precision factor — day, analyst,
or instrument.

```
level  component                       sd      rsd_pct  df      ci90_low_sd  ci90_high_sd
100    repeatability (within group)    0.0707  0.0707   3       0.0438       0.2065
100    between-group                   1.6515  1.6515   2       n/a          n/a
100    intermediate precision (total)  1.6530  1.6530   2.0037  0.9554       7.2821
```

Repeatability of 0.07% RSD looks superb; intermediate precision is 1.65%, twenty-three times
larger, because the variability lives entirely between days. Reporting the within-day figure as
the procedure's precision would understate routine performance by more than an order of magnitude.
This is why the script fits a one-way random-effects model rather than pooling.

Two traps the script handles for you:

- **Precision is estimated within each level, never pooled across levels.** Pooling 80/100/120%
  results into one standard deviation turns the range itself into apparent imprecision. The script
  reports per level, plus a level-independent view as percent of nominal.
- **`--require-ci-within-limit`** enforces that the whole confidence interval sits inside the
  limit, not just the mean. Q2(R2) 3.3.1.4 asks for the interval to be *compatible with* the
  criterion; a mean that scrapes inside on six replicates has not demonstrated much.

### 5. Establish DL and QL, and confirm them

```bash
python3 check_detection_limits.py --calibration lowcal.csv --blanks blanks.csv \
    --confirm-ql 0.05 --confirm-data ql_check.csv --reporting-threshold 0.05
```

```
approach                                          sigma   slope      DL      QL
sd-and-slope (sigma = residual SD of regression)  7.2816  5033.3490  0.0048  0.0145
sd-and-slope (sigma = SD of y-intercept)          4.3303  5033.3490  0.0028  0.0086
sd-and-slope (sigma = SD of 8 blanks)             3.7702  5033.3490  0.0025  0.0075
```

The same data give QL estimates spanning 1.9×, purely from the choice of σ. Q2(R2) 3.2.3.5
therefore requires the limit **and the approach used to determine it** to be reported, and an
estimated limit to be confirmed with samples at or near it. For an impurity procedure the QL must
be at or below the reporting threshold. Reaching for `3.3σ/slope` reflexively, reporting one number
with no named approach, and never confirming it are three separate findings.

### 6. Bioanalytical runs under ICH M10

```bash
python3 check_bioanalytical_run.py --modality chromatographic --run run1.csv
python3 check_bioanalytical_run.py --modality lba --isr isr.csv
python3 check_bioanalytical_run.py --modality lba --criteria
```

`--modality` is mandatory and has no default, because the criteria genuinely differ:

| | Chromatographic | Ligand binding assay |
| --- | --- | --- |
| Calibration tolerance | ±15%, ±20% at LLOQ | ±20%, ±25% at LLOQ and ULOQ |
| Accuracy / precision | ±15% / ≤15% CV (±20% / ≤20% at LLOQ) | ±20% / ≤20% CV (±25% / ≤25% at LLOQ and ULOQ) |
| A&P design | 4 QC levels, 5 replicates/run, ≥3 runs over ≥2 days | 5 QC levels, 3 replicates/run, ≥6 runs over ≥2 days |
| Total error | no such criterion | ≤30%, ≤40% at LLOQ and ULOQ |
| ISR agreement | ±20% for ≥2/3 of repeats | ±30% for ≥2/3 of repeats |

Applying the ±15% chromatographic numbers to a ligand binding assay, or importing the LBA total-error
criterion into a chromatographic method, are both common and both wrong.

The run check enforces the per-level rule that gets missed: at least 2/3 of *all* QCs **and** at
least 50% at *each* level. A run can pass the overall fraction while a single level fails
completely.

```
finding: QC level high: 0/2 within tolerance (0%); M10 requires at least 50% at each level
```

### 7. Transfer and method comparison

```bash
python3 compare_methods.py -i paired.csv --margin 2 --relative --slope-tolerance 0.05
```

```
mean difference (%)                       1.4646
TOST margin                               2.0000
TOST p-value                              1.0528e-13
90% CI (TOST)                             1.44127 to 1.48797
equivalent at stated margin               yes
--- for contrast only ---
paired t-test p (NOT equivalence)         0.0000
OLS slope (biased here)                   1.0396
Deming slope                              1.0398
Passing-Bablok slope                      1.0351
```

Two errors this replaces:

- **"p > 0.05, no significant difference, therefore the methods are equivalent."** Failing to
  detect a difference is not evidence of equivalence, and on a small transfer dataset that outcome
  is close to guaranteed. TOST tests the hypothesis that matters — that the true difference lies
  inside a pre-stated margin. Here the t test says the difference is highly significant *and* TOST
  says the methods are equivalent at ±2%; both are true, and only one answers the question.
- **Ordinary least squares for method comparison.** OLS assumes the reference values carry no
  error, which is false when comparing two procedures, and biases the slope toward zero. Deming
  (with a stated error-variance ratio) and Passing–Bablok (non-parametric, outlier-resistant) are
  the appropriate regressions and are reported side by side with OLS for contrast.

The script also flags proportional bias — when the difference trends with concentration, a single
mean bias and its limits of agreement are misleading regardless of how tight they look.

## What this skill exists to prevent

1. Validating against ICH Q2(R1)'s structure three years after Q2(R2) replaced it.
2. Acceptance criteria written after the data were seen.
3. r² presented as evidence of linearity.
4. Repeatability reported as the procedure's precision, with the between-day component invisible.
5. One DL/QL number with no named approach and no confirmation.
6. Chromatographic M10 criteria applied to a ligand binding assay, or the reverse.
7. A t test's non-significance presented as equivalence at a method transfer.

## References

- `references/framework-selection.md` — which framework governs, and the questions that decide it
- `references/ich-q2r2.md` — structure, Table 1 and Table 2, per-characteristic recommended data
- `references/ich-m10-bioanalytical.md` — the full chromatographic and LBA criteria side by side
- `references/compendial-and-clsi.md` — USP, CLSI and ISO designations, scope, and how to cite them
- `references/statistics.md` — the statistical methods, why each one, and the common errors
- `references/source-ledger.md` — provenance and research dates for every claim in this skill

## Assets

- `assets/validation-protocol-template.md` — protocol structure with criteria stated up front
- `assets/validation-report-template.md` — report structure with raw-data traceability

## Related skills

- `iso-standards-readiness`: Prepares and structurally reviews readiness evidence for ISO management-system and laboratory-competence standards - ISO 13485 medical devi…