Create Test Run

---
name: create-test-run
description: >-
  Create a Kobiton test run from a test case or suite, then offer to monitor it.
  When the user gives only partial details (or just a test case id), fill the
  rest with sensible defaults that match the createTestRun schema, show a summary
  of what will run, and ask to proceed or customize before creating. After the
  run is created, offer monitoring in a single prompt — monitor + auto-open live
  remediation (only when the org's live-remediation flag is ON), monitor only,
  or don't monitor — and hand off to the monitor-test-run skill if chosen. Use
  when the user asks to "create / kick off / start / run a test run", "run test
  case X on N devices", or similar. Wraps the createTestRun MCP tool (and
  getOrgSettings / listDevices for defaults); delegates the watch to
  monitor-test-run.
allowed-tools: >-
  Read, Skill
version: 1.0.0
author: Kobiton Inc.
license: MIT
compatibility: >-
  Runs on any MCP-aware host, including hosts with no local filesystem — this
  is the plugin's only pure-MCP skill, needing no local file, binary, or shell.
  Uses the Kobiton MCP tools createTestRun, getTestCase/getTestSuite,
  listDevices, and getOrgSettings; requires an authenticated Kobiton MCP
  connection. Delegates monitoring to the monitor-test-run skill (same plugin),
  which does need a local poller — so on a filesystem-less host, create the run
  and report its id rather than offering to watch it.
tags: [testing, test-run, create, monitoring, live-remediation, kobiton]
---

## Prerequisites

**Needs only an authenticated Kobiton MCP connection** — no local filesystem, no credentials file, no
binary, no shell. It is the plugin's only pure-MCP skill, so it is the only one that works where the
host supplies nothing else. **The monitoring hand-off in Step 5 is the exception**: `monitor-test-run`
runs a local poller, so where you can't run a local command at all, create the run and report its id
instead of offering to watch it (Step 5 carries the wording). Where you *can*, hand off and let that
skill check its own credentials and streaming options — it degrades rather than refusing. See the Skill
compatibility matrix in `CLAUDE.md`.

## Overview

Turn a "run this" request into a created test run with as little friction as the
user wants, then offer to watch it. Two phases:

1. **Build + confirm the run.** Resolve what to run (test case or suite), on which
   devices, with what app — filling any unspecified field with a documented
   default — then show a one-screen summary and create on confirmation.
2. **Offer monitoring once.** After creation, present the monitor choice in a
   single prompt and delegate to `monitor-test-run` if the user wants it.

> **Tool naming.** Kobiton MCP tools are referenced by bare name (`createTestRun`,
> `getOrgSettings`, `listDevices`, `getTestCase`, `getTestSuite`). The host
> resolves the prefix (`mcp__plugin_automate_kobiton__createTestRun`,
> `mcp__kobiton__createTestRun`, etc.).

## Inputs

| Input | Required | Notes |
|-------|----------|-------|
| test case id **or** test suite id | one of | A test case id → `TEST_CASE` selection; a suite id → `TEST_SUITE`. If the user named neither, ask for it (it's the one thing with no sensible default). |
| device count / specific devices | no | Default: 1 device matching the test case/suite platform. The user may say "3 devices", name models, or give UDIDs. |
| run name / app version / allocation | no | All defaulted (see Step 2). |

## Steps

### 1. Resolve the target and its platform

- A **test case id** → fetch it (`getTestCase`) to learn its platform and the app under test. Selection
  will be `TEST_CASE` with `testCaseSelections: [{ testCaseId, version }]` (default to the latest version
  if the user didn't pin one).
- A **test suite id** → fetch it (`getTestSuite`) for platform + member test cases. Selection will be
  `TEST_SUITE` with `testSuiteId`.
- If the user gave neither a case nor a suite id, ask for one — there's no sensible default for *what* to
  run. Everything else can be defaulted.

### 2. Fill defaults for anything unspecified

Apply these defaults so a bare request ("run test case X") becomes a complete, valid `createTestRun`
payload. **Use the exact enum values below — they are upper-case and case-sensitive; the API rejects
lower-case variants (`test_case`, `specific_devices`, …).**

| Field | Default | Notes |
|-------|---------|-------|
| `testSelection.type` | `TEST_CASE` (case) / `TEST_SUITE` (suite) | per Step 1 |
| test case version | latest version of the case | from `getTestCase` |
| `deviceSelection.type` | `INDIVIDUAL_DEVICES` | prefer explicit devices over a bundle (avoids stale-bundle surprises) |
| devices | **1** available device matching the target's platform | call `listDevices(platform=<platform>, available=true)` and pick online, unbooked, non-cloud devices; if the user asked for N, pick N distinct ones. Pass each as `{ udid, isCloud }`. |
| `appSelections` | the test case's app under test, latest version | derive `appPackage` + `appVersionId` from `getTestCase` (the case records its app); omit only if the target carries no app. |
| `deviceAllocationStrategy` | `CROSS_DEVICE` | "All Permutations" — run each test case on each device (see label map below) |
| `name` | `"<test case/suite name> — <N> device(s) — <YYYY-MM-DD HH:mm>"` | human-readable; the user can override |
| `description` | omit | optional |

Enum reference (exact API values — upper-case, case-sensitive):
- `testSelection.type`: `TEST_CASE` | `TEST_SUITE`
- `deviceSelection.type`: `INDIVIDUAL_DEVICES` | `DEVICE_BUNDLE`
- `deviceAllocationStrategy`: `CROSS_DEVICE` | `SINGLE_DEVICE`

**Allocation-strategy labels (show the human label to the user, send the enum to the API).** Mirror the
Portal's "Device Allocation Strategy" dropdown wording:

| API enum | Show to the user |
|----------|------------------|
| `CROSS_DEVICE` | **All Permutations** — run each test case on each device |
| `SINGLE_DEVICE` | **Random Allocation** — run each test case once, randomly chosen from the selected devices |

Never surface the bare enum (`CROSS_DEVICE`) in the summary or prompts — it's meaningless to the user.

### 3. Show the summary and confirm (skip the prompt only if the user already gave full, explicit details)

Post a compact summary of exactly what will be created, e.g.:

```
About to create this test run:
  Test case : API Demos  (id 019ef40b…, version 2, Android)
  Devices   : 3 × Android (Pixel 8 · Galaxy S24 Ultra · Pixel 4)
  App       : io.appium.android.apis  (version 73202)
  Allocation: All Permutations — run each test case on each device
  Name      : "API Demos — 3 devices — 2026-06-24 11:20"
```

(Allocation shows the human label, not the `CROSS_DEVICE` enum.)

Then ask to **proceed or customize** in one prompt — offer "Proceed", "Change devices", "Change
allocation", "Change name", and let the user free-type any other tweak. When the user wants to change
allocation, present the two human-labeled choices (All Permutations / Random Allocation) and map their
pick back to the enum. Re-summarize and re-ask only if they change something. If the user's original
request was already fully explicit (every field named), you may create directly and just state what you
created.

### 4. Create the run

Call `createTestRun` with the assembled payload (camelCase keys as in the tool schema; the exact enum
values from Step 2). On success it returns the test run id + queued sessions. Report the id and a
one-line confirmation.

If `createTestRun` returns a validation error, fix it from the error text and the Step-2 enum reference
rather than guessing — do not retry the same body.

### 5. Offer monitoring — one prompt, then delegate

**First, decide whether monitoring is even possible here.** `monitor-test-run` runs a bundled Node
poller, so it needs a local filesystem, a shell, and `~/.kobiton/.credentials`. You can't probe for
those from this skill (it has no shell of its own), so judge from what your host has given you: **if
you have no way to run a local command, monitoring is out** — don't present the choices below, because
offering a watch you can't perform is the trap the matrix in `CLAUDE.md` warns about. Close out instead:

> Run created (`<testRunId>`) — <portal link>. I can't watch it from here (monitoring needs to run a
> local poller, which isn't available in this environment), so follow it in the portal — or ask me again
> from a host with a shell and I'll monitor it for you.

Then stop. If you *can* run local commands, continue below and let `monitor-test-run` handle the rest:
it checks for the credentials file itself and reports the `/automate:setup` remedy, and it degrades to a
foreground loop on hosts without a streamed-output affordance rather than refusing. Don't pre-empt
either of those decisions here.

Read the live-remediation flag first: call `getOrgSettings` once and note `live_remediation_enabled`
(`flagOn`). This decides whether the auto-open option is meaningful.

Then offer monitoring in a **single** message (don't fire multiple separate questions — that's the
annoying part). Present the choices inline and let the user pick one:

- **Monitor + auto-open live remediation** — *only list this option when `flagOn = true`.* Watches the
  run and, when an execution blocks, automatically opens the live-remediation window for the device.
- **Monitor only** — watches the run and surfaces blockers / the final summary, but doesn't auto-open
  windows (you print the URL instead).
- **Don't monitor** — stop here; give the user the test run id (and the portal link if handy) so they can
  watch it themselves later.

Phrase it as one prompt, e.g. (flag ON):

> Run created (`<testRunId>`). Want me to monitor it? **(a)** monitor + auto-open the live-remediation
> window when a blocker hits, **(b)** monitor only (I'll surface blockers + the URL), or **(c)** don't
> monitor. Reply a / b / c.

(flag OFF — drop option **a**, since there's no live window to open):

> Run created (`<testRunId>`). Want me to monitor it? **(b)** monitor only (I'll surface blockers + the
> portal URL), or **(c)** don't monitor. Reply b / c.

On the user's answer:

- **(a) or (b)** → invoke the **`monitor-test-run`** skill for `<testRunId>`. Pass along the auto-open
  intent: for **(a)**, the user has already opted into auto-open, so tell monitor-test-run to skip its own
  up-front auto-open question and treat `autoOpen = yes`; for **(b)**, `autoOpen = no`. monitor-test-run
  owns the watch loop (the bundled poller, blocker surfacing, post-mortem) from here.
- **(c)** → done. Report the test run id and how to watch later (`monitor-test-run <testRunId>`), then
  stop.

## Errors

| Condition | Handling |
|-----------|----------|
| No test case/suite id given | Ask for one — it's the only field with no default. |
| `createTestRun` validation error (bad enum, missing pair) | Correct from the error + the Step-2 enum reference; don't blind-retry. |
| No available devices for the platform | Tell the user; offer to widen (cloud devices) or wait. Don't create a run that can't dispatch. |
| `getOrgSettings` fails before the monitor offer | Assume `flagOn = false` (drop the auto-open option); still offer monitor-only / don't-monitor. |

## Notes

- This skill **creates** the run and **hands off** the watch; it does not implement monitoring itself —
  that's `monitor-test-run` (same plugin), which runs the bundled emit-on-change poller.
- Default to the smallest run that satisfies the request (1 device unless asked for more) — creating real
  device sessions consumes minutes/concurrency.
- Prefer `INDIVIDUAL_DEVICES` with explicit `{ udid, isCloud }` over a device bundle, so the exact devices
  are known and a stale/oversized bundle can't surprise the run.