Aeropath Airway Segmentation Eval

---
name: aeropath-airway-segmentation-eval
description: This benchmark evaluates 3D medical image segmentation models on contrast-enhanced CT scans containing severe airway pathologies. It probes a model's ability to accurately segment complex, distorted bronchial trees and maintain topological completeness down to small airway generations despite anatomical anomalies like tumors and emphysema. Use when the user wants to benchmark on AeroPath, or asks about evaluating this task. Reports DSC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2311.01138
  bibtex_key: stoverud2023aeropath
  confidence: high
---

# aeropath-airway-segmentation-eval

> AeroPath: An airway segmentation benchmark dataset with challenging pathology — Støverud et al. (2023) (arXiv:2311.01138, 2023)

## What this evaluates

This benchmark evaluates 3D medical image segmentation models on contrast-enhanced CT scans containing severe airway pathologies. It probes a model's ability to accurately segment complex, distorted bronchial trees and maintain topological completeness down to small airway generations despite anatomical anomalies like tumors and emphysema.

## Datasets

- **AeroPath** — total 27; splits: test (27); repo https://github.com/raidionics/AeroPath

## Metrics

- `DSC` **(primary)** — range: percent
  - Dice Similarity Coefficient, calculated as 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Measures voxel-wise overlap between the predicted airway mask and the annotated ground truth.
- `TD` — range: percent
  - Tree Detection rate, representing the percentage of correctly detected airway tree length or generations relative to the ground truth.
- `BD` — range: percent
  - Branch Detection rate, representing the percentage of correctly detected individual airway branches relative to the ground truth.

## Input / output format

**Input**: 3D contrast-enhanced computed tomography (CT) volumes of the chest.

**Output**: 3D binary segmentation mask indicating airway voxels.

## Scoring recipe

```python
def compute_metrics(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    union = np.sum(pred_mask) + np.sum(gt_mask)
    dsc = (2 * intersection / union) * 100 if union > 0 else 0.0
    tp = np.sum((pred_mask == 1) & (gt_mask == 1))
    fp = np.sum((pred_mask == 1) & (gt_mask == 0))
    fn = np.sum((pred_mask == 0) & (gt_mask == 1))
    tn = np.sum((pred_mask == 0) & (gt_mask == 0))
    precision = (tp / (tp + fp)) * 100 if (tp + fp) > 0 else 0.0
    sensitivity = (tp / (tp + fn)) * 100 if (tp + fn) > 0 else 0.0
    specificity = (tn / (tn + fp)) * 100 if (tn + fp) > 0 else 0.0
    return {'DSC': dsc, 'Precision': precision, 'Sensitivity': sensitivity, 'Specificity': specificity}
```

## Common pitfalls

- Ground truth annotations for some cases were partly generated by baseline tools (FAST + BronchiNet), which may artificially inflate DSC and other metrics for those specific scans.
- Post-processing steps (e.g., connecting fragmented branches, removing islands) improve visual/topological continuity but do not substantially change voxel-wise metrics like DSC.
- Models may produce false positives by segmenting adjacent structures with similar CT intensities (e.g., esophagus, lung tissue) if not carefully constrained.

## Evidence (verbatim from paper)

> The PW method has the best overall DSC (84.98 ± 3.24) of all tested methods. Adding ensemble to the PW model causes a drop in DSC and precision with 0.5 and 4%, respectively. On the other hand, TD and BD increases with 2.4 and 4.1%, respectively.

## Citation

```bibtex
@misc{stoverud2023aeropath,
  title={AeroPath: An airway segmentation benchmark dataset with challenging pathology},
  author={Støverud et al. (2023)},
  year={2023},
  note={arXiv:2311.01138}
}
```

- arXiv: 2311.01138