Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
Installs into .claude/skills of the current project.
Are you the author of Llm Evaluation?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/dokhacgiakhoa-llm-evaluation)
---
version: 4.1.0-fractal
name: llm-evaluation
description: Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
---
# LLM Evaluation
Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.
## Do not use this skill when
- The task is unrelated to llm evaluation
- You need a different domain or tool outside this scope
## Instructions
- Clarify goals, constraints, and required inputs.
- Apply relevant best practices and validate outcomes.
- Provide actionable steps and verification.
- If detailed examples are required, open `resources/implementation-playbook.md`.
## Use this skill when
- Measuring LLM application performance systematically
- Comparing different models or prompts
- Detecting performance regressions before deployment
- Validating improvements from prompt changes
- Building confidence in production systems
- Establishing baselines and tracking progress over time
- Debugging unexpected model behavior
## Core Evaluation Types
## 🧠 Knowledge Modules (Fractal Skills)
### 1. [1. Automated Metrics](./sub-skills/1-automated-metrics.md)
### 2. [2. Human Evaluation](./sub-skills/2-human-evaluation.md)
### 3. [3. LLM-as-Judge](./sub-skills/3-llm-as-judge.md)
### 4. [BLEU Score](./sub-skills/bleu-score.md)
### 5. [ROUGE Score](./sub-skills/rouge-score.md)
### 6. [BERTScore](./sub-skills/bertscore.md)
### 7. [Custom Metrics](./sub-skills/custom-metrics.md)
### 8. [Single Output Evaluation](./sub-skills/single-output-evaluation.md)
### 9. [Pairwise Comparison](./sub-skills/pairwise-comparison.md)
### 10. [Annotation Guidelines](./sub-skills/annotation-guidelines.md)
### 11. [Inter-Rater Agreement](./sub-skills/inter-rater-agreement.md)
### 12. [Statistical Testing Framework](./sub-skills/statistical-testing-framework.md)
### 13. [Regression Detection](./sub-skills/regression-detection.md)
### 14. [Running Benchmarks](./sub-skills/running-benchmarks.md)
