Installs into .claude/skills of the current project.
Are you the author of Agreement Statistics?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/vectorspacelab-agreement-statistics)
---
name: agreement_statistics
description: Compute ID accuracy, pairwise ID/OOD agreement, and clipped probit statistics for agreement-on-the-line experiments.
---
# Agreement Statistics
Use this skill after a prediction table has been validated. It preserves the paper's estimator boundary by computing OOD agreement from predictions only and never using OOD labels.
## Inputs
- Normalized prediction table from `prediction_table_contracts`.
- Optional clipping epsilon for probit transforms.
## Outputs
- Per-model ID accuracies.
- Pairwise ID and OOD agreement fractions.
- Probit-transformed accuracies and agreements.
## Workflow
1. Compute ID accuracy from ID labels and ID predictions.
2. Compute unordered pairwise agreement on ID predictions.
3. Compute unordered pairwise agreement on OOD predictions.
4. Clip probabilities before inverse-normal transformation.
## Validation
Run `python tests/test_agreement_statistics.py` from this skill directory.
## Limitations
This skill does not fit the agreement line or solve ALine-D.
