Vendor Evaluation

---
name: vendor-evaluation
description: Evaluate vendors against weighted criteria, a proof of concept that tests your real workload, and a clear-eyed accounting of exit costs before you sign. Use when choosing a paid tool or service you will depend on and a wrong pick is expensive to undo.
---

# Vendor evaluation

Choosing a vendor is choosing a dependency you will live with for years and pay
to leave. The evaluation goes wrong when it is run on demos and vibes: the
polished sales walkthrough decides it, criteria get invented to justify the
favorite, and no one asks what it costs to get back out. A disciplined
evaluation tests the vendor on your work and prices the exit before it prices
the entry.

## Method

1. **Set weighted criteria before you see a single demo.** List what matters:
   functionality, reliability, security posture, support, total cost, and how
   hard it is to leave. Assign weights that sum to a fixed total, and write them
   down first. Criteria invented after the demo just rationalize the vendor who
   demoed best.
2. **Score against requirements, not against each other.** Rate every vendor on
   each criterion against your bar, so a weak field cannot make a mediocre
   option look strong by comparison. Separate must-haves, which are pass or
   fail, from nice-to-haves that earn weighted points.
3. **Design the proof of concept around your real workload.** Feed it your
   actual data shapes, your peak volume, your awkward edge cases, and your
   integration points. A POC on the vendor's sample data proves their demo
   works. Define the success metrics and the time box before you start so the
   POC ends in a verdict, not an open-ended trial.
4. **Cost the exit, not just the entry.** Estimate what it takes to migrate off:
   data export formats, proprietary interfaces you would rewrite, retraining,
   and contract lock-in. High switching cost is a real price even if it never
   appears on an invoice. Prefer vendors whose data you can get back in a usable
   form.
5. **Verify security and compliance with evidence, not the questionnaire
   alone.** Ask for the SOC 2 or ISO report, the data processing terms,
   subprocessor list, breach history, and where data is stored. A vendor who
   cannot produce these for a serious deal is telling you something.
6. **Reference-check past the list they gave you.** Talk to the references, then
   find a customer they did not hand you and ask what breaks at month six:
   support response times, hidden costs, the feature that was "on the roadmap."
   Sales sells the roadmap; a real customer tells you the product.

## Checks

- Were the weighted criteria fixed before demos, or shaped to fit a favorite?
- Did the POC run on your workload and data, or on the vendor's canned scenario?
- Can you state, in dollars and weeks, what leaving this vendor would cost?

## Boundaries

This picks the vendor; it does not negotiate the contract terms or run
procurement and legal review, which have their own owners and thresholds.
Approval limits and required security sign-offs are company policy: route spend
and data agreements through the process your organization requires rather than
deciding them here.