Skip to content
Back to skills

Measure Relevance

ASecurity

Compute NDCG, MRR, and precision@k against the judgment list with a baseline. Reach for this before any tuning.

  • 7 stars
  • 0 votes
  • 0 copies
  • 1 view
  • Added September 23, 2026
ai-agents

Security analysis

A100/100

Scanned September 23, 2026

npx -y skills add mcorbett51090/RavenClaude --skill measure-relevance --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Measure Relevance?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Measure Relevance
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/mcorbett51090-measure-relevance/badge)](https://www.skillsdirectory.com/skills/mcorbett51090-measure-relevance)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: measure-relevance
description: "Compute NDCG, MRR, and precision@k against the judgment list with a baseline. Reach for this before any tuning."
---

# Skill: Measure relevance

'These results look better' is one session, not a metric (§3 #1).

## Step 1 — Load the judgment list
Graded relevance labels for a representative query set (§3 #3).

## Step 2 — Compute NDCG
DCG = Σ rel_i / log2(i+1) (rank i from 1); NDCG = DCG/IDCG via `search_relevance_engineering_calc.py relevance` (§3 #1).

## Step 3 — Compute MRR + precision@k
MRR for first-relevant; precision@k for top-k quality (§3 #1).

## Step 4 — Baseline before tuning
Record the baseline; every change reports a before/after delta (§3 #1 #3).

## Output
An NDCG/MRR/precision@k read against the judgment list with a baseline. Traverse Tree 1 in the decision-trees file.

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

Comments

Loading comments…