Bankertoolbench Eval

---
name: bankertoolbench-eval
description: Evaluates AI agents' ability to execute end-to-end investment banking workflows, requiring multi-file deliverable generation (Excel, PowerPoint, reports) using specialized financial tools and data sources. It probes financial judgment, cross-artifact consistency, tool-use fidelity, and professional presentation standards under realistic constraints. Use when the user wants to benchmark on BankerToolBench, or asks about evaluating this task. Reports rubric score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.11304
  bibtex_key: lau2026bankertoolbench
  confidence: high
---

# bankertoolbench-eval

> BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows — Lau et al. (2026) (arXiv:2604.11304, 2026)

## What this evaluates

Evaluates AI agents' ability to execute end-to-end investment banking workflows, requiring multi-file deliverable generation (Excel, PowerPoint, reports) using specialized financial tools and data sources. It probes financial judgment, cross-artifact consistency, tool-use fidelity, and professional presentation standards under realistic constraints.

## Datasets

- **BankerToolBench** — total 100; splits: train (80), test (20); repo https://github.com/harbor-framework/harbor

## Metrics

- `rubric score` **(primary)** — range: [0, 1]
  - Average score across 100+ expert-crafted criteria per task. Each criterion is evaluated as a binary pass/fail based on professional deliverable standards, then averaged to produce a single task-level metric.

## Input / output format

**Input**: A structured prompt template specifying the agent's role ('investment banker'), available tools (MCP tools for VDR/SEC EDGAR, Python libraries), directory layout, execution protocol, formatting guidelines, security constraints, and a specific <TASK> instruction.

**Output**: Multi-file deliverables (e.g., .xlsx, .pptx, .pdf, .docx) saved to the `banker Workspace/deliverables/` directory. Intermediate scripts go to `banker Workspace/`.

## Scoring recipe

```python
def compute_rubric_score(predictions, rubric_criteria):
    scores = []
    for criterion in rubric_criteria:
        # Binary check: 1 if deliverable meets criterion, 0 otherwise
        met = verifier_or_human_check(predictions, criterion)
        scores.append(1.0 if met else 0.0)
    return sum(scores) / len(scores)
```

## Common pitfalls

- Excel formulas written by agents must be recalculated using headless LibreOffice before being read with `openpyxl`, otherwise they return `None` or stale values.
- Agents must strictly use the provided MCP tools (VDR, SEC EDGAR) for financial data and are explicitly forbidden from using web search or external APIs.
- Only files saved to `banker Workspace/deliverables/` are reviewed for grading; intermediate outputs in other directories are ignored.

## Evidence (verbatim from paper)

> Table 4 shows that for Claude Opus 4.6, the spread in output quality achieved across the three harnesses is very small (the average rubric score only varies by 0.3).

## Citation

```bibtex
@misc{lau2026bankertoolbench,
  title={BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows},
  author={Lau et al. (2026)},
  year={2026},
  note={arXiv:2604.11304}
}
```

- arXiv: 2604.11304