Youtube Implicit Rec Eval

---
name: youtube-implicit-rec-eval
description: Evaluates recommender systems on implicit feedback datasets, testing their ability to rank relevant items for users. It probes model versatility across cold-start, offline, and instant recommendation scenarios using side information and sequential context features. Use when the user wants to benchmark on YouTube Implicit Feedback Subset, or asks about evaluating this task. Reports NDCG@100.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1611.04666
  bibtex_key: bayer2016generic
  confidence: high
---

# youtube-implicit-rec-eval

> A Generic Coordinate Descent Framework for Learning from Implicit Feedback — Bayer et al. (2016) (arXiv:1611.04666, 2016)

## What this evaluates

Evaluates recommender systems on implicit feedback datasets, testing their ability to rank relevant items for users. It probes model versatility across cold-start, offline, and instant recommendation scenarios using side information and sequential context features.

## Datasets

- **YouTube Implicit Feedback Subset** — total ?; splits: train (-1), test (-1)

## Metrics

- `NDCG@100` **(primary)** — range: [0, 1]
  - Normalized Discounted Cumulative Gain at rank 100. Computes the sum of graded relevance scores discounted by log2(rank+1), normalized by the ideal DCG.
- `Recall@100` — range: [0, 1]
  - Fraction of relevant items in the ground truth that appear in the top 100 recommended items.

## Input / output format

**Input**: User ID, interaction history, and optional side/context features (age, gender, country, device, previously watched video, all previously watched videos, user ID).

**Output**: Ranked list of top 100 recommended video IDs.

## Scoring recipe

```python
def compute_metrics(preds, gold, k=100):
    rel = [1 if item in gold else 0 for item in preds[:k]]
    dcg = sum(r / math.log2(i + 2) for i, r in enumerate(rel))
    ideal_rel = sorted(rel, reverse=True)
    idcg = sum(r / math.log2(i + 2) for i, r in enumerate(ideal_rel))
    ndcg = dcg / idcg if idcg > 0 else 0.0
    recall = sum(rel) / len(gold) if gold else 0.0
    return ndcg, recall
```

## Common pitfalls

- The paper reports relative improvements over a Popularity baseline rather than absolute metric values.
- The dataset is a proprietary YouTube subset; exact train/test split sizes and the tuning holdout are not publicly disclosed.
- Different evaluation scenarios (Cold-Start, Offline, Instant) use fundamentally different train/eval partitioning strategies.

## Evidence (verbatim from paper)

> We measure the recall and NDCG for the top 100 returned videos. Note that we report relative improvements over the Popularity recommender.

## Citation

```bibtex
@misc{bayer2016generic,
  title={A Generic Coordinate Descent Framework for Learning from Implicit Feedback},
  author={Bayer et al. (2016)},
  year={2016},
  note={arXiv:1611.04666}
}
```

- arXiv: 1611.04666