Testdriver:Find

---
name: testdriver:find
description: Locate UI elements using natural language
---
<!-- Generated from find.mdx. DO NOT EDIT. -->

## Overview

Locate UI elements on screen using AI-powered natural language descriptions. Returns an `Element` object that can be interacted with.

## Syntax

```javascript
const element = await testdriver.find(description)
const element = await testdriver.find(description, options)
```

## Parameters

<ParamField path="description" type="string" required>
  Natural language description of the element to find
</ParamField>

<ParamField path="options" type="object | number">
  Optional configuration for finding and caching
  
  <Expandable title="properties">
    <ParamField path="cacheKey" type="string">
      Custom cache key for storing element location. Use this to prevent cache pollution when using dynamic variables in prompts, or to share cache across tests.
    </ParamField>
    
    <ParamField path="cacheThreshold" type="number" default={0.05}>
      Similarity threshold (0-1) for cache matching. Lower values require more similarity. Set to -1 to disable cache.
    </ParamField>
    
    <ParamField path="timeout" type="number" default={10000}>
      Maximum time in milliseconds to poll for the element. Retries every 5 seconds until found or timeout expires. Defaults to `10000` (10 seconds). Set to `0` to disable polling and make a single attempt.
    </ParamField>
    
    <ParamField path="confidence" type="number">
      Minimum confidence threshold (0-1). If the AI's confidence score for the found element is below this value, the find will be treated as a failure (`element.found()` returns `false`). Useful for ensuring high-quality matches in critical test steps.
    </ParamField>
    
    <ParamField path="type" type="string">
      Element type hint that wraps the description for better matching. Accepted values:
      - `"text"` — Wraps the prompt as `The text "..."`
      - `"image"` — Wraps the prompt as `The image "..."`
      - `"ui"` — Wraps the prompt as `The UI element "..."`
      - `"any"` — No wrapping, uses the description as-is (default behavior)
    </ParamField>
    
    <ParamField path="zoom" type="boolean" default={false}>
      Two-phase zoom mode for better precision in crowded UIs with many similar elements. Disabled by default.
    </ParamField>
    
    <ParamField path="verify" type="boolean" default={false}>
      Enable AI verification of the located element. When `true`, a second AI call checks that the coordinates returned actually correspond to the requested element, catching hallucinated or incorrect positions. Disabled by default for lower latency. Defaults to the global `verify` option set on the SDK constructor when not specified per call.
    </ParamField>
    
    <ParamField path="ai" type="object">
      AI sampling configuration for this find call (overrides global `ai` config from constructor).
      
      <Expandable title="properties">
        <ParamField path="temperature" type="number">
          Controls randomness. `0` = deterministic. Default: `0` for find verification.
        </ParamField>
        
        <ParamField path="top" type="object">
          Sampling parameters
          
          <Expandable title="properties">
            <ParamField path="p" type="number">
              Top-P (nucleus sampling). Range: 0-1.
            </ParamField>
            
            <ParamField path="k" type="number">
              Top-K sampling. `1` = most deterministic.
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

## Returns

`Promise<Element>` - Element instance that has been automatically located

## Examples

### Basic Element Finding

```javascript
// Find by role
const button = await testdriver.find('submit button');
const input = await testdriver.find('email input field');

// Find by text content
const link = await testdriver.find('Contact Us link');
const heading = await testdriver.find('Welcome heading');

// Find by visual appearance
const icon = await testdriver.find('red warning icon');
const image = await testdriver.find('company logo image');
```

### Finding with Context

```javascript
// Provide location context
const field = await testdriver.find('username input in the login form');
const button = await testdriver.find('delete button in the top right corner');

// Describe nearby elements
const input = await testdriver.find('input field below the email label');
const checkbox = await testdriver.find('checkbox next to "Remember me"');

// Describe visual position
const menu = await testdriver.find('hamburger menu icon in the top left');
```

### Interacting with Found Elements

```javascript
// Find and click
const submitBtn = await testdriver.find('submit button');
await submitBtn.click();

// Find and verify
const message = await testdriver.find('success message');
if (message.found()) {
  console.log('Success message appeared');
}

// Find and extract info
const price = await testdriver.find('product price');
console.log('Price location:', price.coordinates);
console.log('Price text:', price.text);
```

## Element Object

The returned `Element` object provides:

### Methods

- `found()` - Check if element was located
- `click(action)` - Click the element
- `hover()` - Hover over the element
- `doubleClick()` - Double-click the element
- `rightClick()` - Right-click the element
- `find(newDescription)` - Re-locate with optional new description

### Properties

- `coordinates` - Element position `{x, y, centerX, centerY}`
- `x`, `y` - Top-left coordinates
- `centerX`, `centerY` - Center coordinates
- `text` - Text content (if available)
- `screenshot` - Base64 screenshot (if available)
- `confidence` - AI confidence score
- `width`, `height` - Element dimensions
- `boundingBox` - Complete bounding box

See [Elements Reference](/v7/elements) for complete details.

### JSON Serialization

Elements can be safely serialized using `JSON.stringify()` for logging and debugging. Circular references are automatically removed:

```javascript
const element = await testdriver.find('login button');

// Safe to stringify - no circular reference errors
console.log(JSON.stringify(element, null, 2));

// Output includes useful debugging info:
// {
//   "description": "login button",
//   "coordinates": { "x": 100, "y": 200, "centerX": 150, "centerY": 225 },
//   "found": true,
//   "threshold": 0.01,
//   "x": 100,
//   "y": 200,
//   "cache": {
//     "hit": true,
//     "strategy": "pixel-diff",
//     "createdAt": "2025-12-09T10:30:00Z",
//     "diffPercent": 0.0023,
//     "imageUrl": "https://..."
//   },
//   "similarity": 0.98,
//   "confidence": 0.95,
//   "selector": "button#login",
//   "aiResponse": "Found the blue login button..."
// }
```

This is useful for:
- Debugging element detection issues
- Logging test execution details
- Sharing element information across processes
- Analyzing cache performance

## Best Practices

<Check>
  **Be specific in descriptions**
  
  More specific descriptions improve accuracy:
  
  ```javascript
  // ✅ Good
  await testdriver.find('blue submit button below the email field');
  
  // ❌ Too vague
  await testdriver.find('button');
  ```
</Check>

<Check>
  **Always check if found**
  
  Verify elements were located before interacting:
  
  ```javascript
  const element = await testdriver.find('login button');
  if (!element.found()) {
    throw new Error('Login button not found');
  }
  await element.click();
  ```
</Check>

<Check>
  **Include visual or positional context**
  
  ```javascript
  // Include color
  await testdriver.find('red error icon');
  
  // Include position
  await testdriver.find('search button in the top navigation bar');
  
  // Include nearby text
  await testdriver.find('checkbox next to "I agree to terms"');
  ```
</Check>

## Confidence Threshold

Require a minimum AI confidence score for element matches. If the confidence is below the threshold, `find()` treats the result as not found:

```javascript
// Require at least 90% confidence
const element = await testdriver.find('submit button', { confidence: 0.9 });

if (!element.found()) {
  // AI found something but wasn't confident enough
  throw new Error('Could not confidently locate submit button');
}

await element.click();
```

This is useful for:
- Critical test steps where an incorrect click could cause cascading failures
- Distinguishing between similar elements (e.g., multiple buttons)
- Failing fast when the UI has changed unexpectedly

```javascript
// Combine with timeout for robust polling with confidence gate
const element = await testdriver.find('success notification', {
  confidence: 0.85,
  timeout: 15000,
});
```

<Tip>
  The `confidence` value is a float between 0 and 1 (e.g., `0.9` = 90%). The AI returns its confidence with each find result, which you can also read from `element.confidence` after a successful find.
</Tip>
## Element Type

Use the `type` option to hint what kind of element you're looking for. This wraps your description into a more specific prompt for the AI, improving match accuracy — especially when users provide short or ambiguous descriptions.

```javascript
// Find text on the page
const label = await testdriver.find('Sign In', { type: 'text' });
// AI prompt becomes: The text "Sign In"

// Find an image
const logo = await testdriver.find('company logo', { type: 'image' });
// AI prompt becomes: The image "company logo"

// Find a UI element (button, input, checkbox, etc.)
const btn = await testdriver.find('Submit', { type: 'ui' });
// AI prompt becomes: The UI element "Submit"

// No wrapping — same as omitting the option
const el = await testdriver.find('the blue submit button', { type: 'any' });
```

| Type | Prompt sent to AI |
|------|----|
| `"text"` | `The text "..."` |
| `"image"` | `The image "..."` |
| `"ui"` | `The UI element "..."` |
| `"any"` | Original description (no wrapping) |

<Tip>
  This is particularly useful for short descriptions like `"Submit"` or `"Login"` where the AI may not know whether to look for a button, a link, or visible text. Specifying `type` removes the ambiguity.
</Tip>
## Polling for Dynamic Elements

By default, `find()` polls for up to 10 seconds (retrying every 5 seconds) until the element is found. You can customize this with the `timeout` option:

```javascript
// Uses default 10s timeout - polls every 5 seconds
const element = await testdriver.find('login button');
await element.click();

// Custom timeout - wait up to 30 seconds
const element = await testdriver.find('login button', { timeout: 30000 });
await element.click();

// Disable polling - single attempt only
const element = await testdriver.find('login button', { timeout: 0 });
```

The `timeout` option:
- Defaults to `10000` (10 seconds)
- Retries finding the element every 5 seconds
- Stops when the element is found or the timeout expires
- Logs progress during polling
- Returns the element (check `element.found()` if not throwing on failure)
- Set to `0` to disable polling and make a single attempt

## Zoom Mode

Zoom mode is **disabled by default**. It uses a two-phase approach for better precision when locating elements, especially in crowded UIs with many similar elements.

To enable zoom for a specific find call, pass `zoom: true`:

```javascript
// Enable zoom for better precision in crowded UIs
const extensionsBtn = await testdriver.find('extensions puzzle icon in Chrome toolbar', { zoom: true });
await extensionsBtn.click();

// Without zoom (default)
const largeButton = await testdriver.find('big hero button');
```

### How Zoom Mode Works

1. **Phase 1**: AI identifies the approximate location of the element
2. **Phase 2**: A 30% crop of the screen is created around that location
3. **Phase 3**: AI performs precise location on the zoomed/cropped image
4. **Result**: Coordinates are converted back to absolute screen position

This two-phase approach gives the AI a higher-resolution view of the target area, improving accuracy when multiple similar elements are close together.

<Tip>
  You may want to enable zoom with `zoom: true` when:
  - Targeting small elements in crowded UIs with many similar elements
  - You need extra precision for closely spaced UI elements
</Tip>

## Verify Mode

Verify mode is **disabled by default**. When enabled, a second AI call checks that the coordinates returned by `find()` actually correspond to the requested element, catching hallucinated or incorrect positions.

```javascript
// Enable verification for critical interactions
const deleteBtn = await testdriver.find('delete account button', { verify: true });
await deleteBtn.click();
```

### How Verify Mode Works

1. **Phase 1**: AI locates the element and returns coordinates
2. **Phase 2**: A second AI call examines the screenshot at those coordinates to confirm the element matches the description
3. **Result**: If verification fails, the find is retried or marked as not found

### Combining Zoom and Verify

For maximum accuracy, enable both `zoom` and `verify` together. This is useful for critical interactions where clicking the wrong element could cause cascading failures:

```javascript
// Maximum accuracy: zoom for precision + verify to catch hallucinations
const element = await testdriver.find('small cancel icon next to the subscription', {
  zoom: true,
  verify: true,
});
await element.click();
```

## Cache Options

Control caching behavior to optimize performance, especially when using dynamic variables in prompts.

### Custom Cache Key

Use `cacheKey` to prevent cache pollution when prompts contain variables:

```javascript
// ❌ Without cacheKey - creates new cache entry for each email value
const email = 'user@example.com';
await testdriver.find(`input for ${email}`); // Cache miss every time

// ✅ With cacheKey - reuses cache regardless of variable
const email = 'user@example.com';
await testdriver.find(`input for ${email}`, {
  cacheKey: 'email-input'
});

// Also useful for dynamic IDs, names, or other changing data
const orderId = generateOrderId();
await testdriver.find(`order ${orderId} status`, {
  cacheKey: 'order-status'  // Same cache for all orders
});
```

### Cache Threshold

Control how similar a cached result must be to reuse it:

```javascript
// Default: 95% similarity required
await testdriver.find('submit button');

// Strict threshold - 99% similarity required
await testdriver.find('submit button', {
  cacheThreshold: 0.01
});

// Disable cache entirely for this call
await testdriver.find('submit button', {
  cacheThreshold: -1
});

// Combine cacheKey with threshold
await testdriver.find('submit button', {
  cacheKey: 'submit-btn',
  cacheThreshold: 0.01
});
```

<Tip>
  By default, TestDriver auto-generates a cache key from the SHA-256 hash of your test file. When you modify your test file, the hash changes automatically, invalidating stale cache entries.
</Tip>

### Manual Polling (Alternative)

If you need custom polling logic:

```javascript
async function waitForElement(testdriver, description, timeout = 30000) {
  const startTime = Date.now();
  
  while (Date.now() - startTime < timeout) {
    const element = await testdriver.find(description);
    if (element.found()) return element;
    await new Promise(r => setTimeout(r, 1000));
  }
  
  throw new Error(`Element "${description}" not found after ${timeout}ms`);
}

// Usage
const button = await waitForElement(testdriver, 'submit button', 10000);
await button.click();
```

## Use Cases

<AccordionGroup>
  <Accordion title="Form Fields">
    ```javascript
    const emailField = await testdriver.find('email input field');
    await emailField.click();
    await testdriver.type('user@example.com');
    
    const passwordField = await testdriver.find('password input');
    await passwordField.click();
    await testdriver.type('MyP@ssw0rd');
    ```
  </Accordion>
  
  <Accordion title="Buttons and Links">
    ```javascript
    const submitBtn = await testdriver.find('submit button');
    await submitBtn.click();
    
    const cancelLink = await testdriver.find('cancel link');
    await cancelLink.click();
    
    const menuIcon = await testdriver.find('hamburger menu icon');
    await menuIcon.click();
    ```
  </Accordion>
  
  <Accordion title="Dynamic Content">
    ```javascript
    // Wait for loading to complete
    let content;
    for (let i = 0; i < 30; i++) {
      content = await testdriver.find('results table');
      if (content.found()) break;
      await new Promise(r => setTimeout(r, 1000));
    }
    
    // Interact with loaded content
    const firstRow = await testdriver.find('first row in the results table');
    await firstRow.click();
    ```
  </Accordion>
  
  <Accordion title="Complex UI Elements">
    ```javascript
    // Modals and dialogs
    const modal = await testdriver.find('confirmation dialog');
    if (modal.found()) {
      const confirmBtn = await testdriver.find('confirm button in the dialog');
      await confirmBtn.click();
    }
    
    // Dropdown menus
    const dropdown = await testdriver.find('country dropdown');
    await dropdown.click();
    
    const option = await testdriver.find('United States option');
    await option.click();
    ```
  </Accordion>
</AccordionGroup>

## Complete Example

```javascript
import { beforeAll, afterAll, describe, it, expect } from 'vitest';
import TestDriver from 'testdriverai';

describe('Element Finding', () => {
  let testdriver;

  beforeAll(async () => {
    client = new TestDriver(process.env.TD_API_KEY);
    await testdriver.auth();
    await testdriver.connect();
  });

  afterAll(async () => {
    await testdriver.disconnect();
  });

  it('should find and interact with elements', async () => {
    await testdriver.focusApplication('Google Chrome');
    
    // Find login form elements
    const usernameField = await testdriver.find('username input field');
    expect(usernameField.found()).toBe(true);
    
    await usernameField.click();
    await testdriver.type('testuser');
    
    // Find with context
    const passwordField = await testdriver.find('password input below username');
    await passwordField.click();
    await testdriver.type('password123');
    
    // Find button
    const submitBtn = await testdriver.find('green submit button');
    expect(submitBtn.found()).toBe(true);
    
    console.log('Button location:', submitBtn.centerX, submitBtn.centerY);
    
    await submitBtn.click();
    
    // Wait for success message
    let successMsg;
    for (let i = 0; i < 10; i++) {
      successMsg = await testdriver.find('success notification');
      if (successMsg.found()) break;
      await new Promise(r => setTimeout(r, 1000));
    }
    
    expect(successMsg.found()).toBe(true);
  });
});
```

## Related Methods

- [`click()`](/v7/click) - Click on found elements
- [`hover()`](/v7/hover) - Hover over elements
- [`assert()`](/v7/assert) - Verify element states
- [Elements Reference](/v7/elements) - Complete Element API

---

## findAll()

Locate **all elements** matching a description, rather than just one.

### Syntax

```javascript
const elements = await testdriver.findAll(description, options)
```

### Parameters

<ParamField path="description" type="string" required>
  Natural language description of elements to find
</ParamField>

<ParamField path="options" type="object | number">
  Optional cache options (same as `find()`)
  
  <Expandable title="properties">
    <ParamField path="cacheKey" type="string">
      Cache key for storing element location
    </ParamField>
    
    <ParamField path="cacheThreshold" type="number" default={-1}>
      Similarity threshold (0-1) for cache matching. Set to -1 to disable cache.
    </ParamField>
  </Expandable>
</ParamField>

### Returns

`Promise<Element[]>` - Array of Element instances

### Examples

#### Basic Usage

```javascript
// Find all matching elements
const buttons = await testdriver.findAll('button');
console.log(`Found ${buttons.length} buttons`);

// Interact with specific element
if (buttons.length > 0) {
  await buttons[0].click(); // Click first button
}

// Iterate over all
for (const button of buttons) {
  console.log(`Button at (${button.x}, ${button.y})`);
}
```

#### Finding Multiple Items

```javascript
// Find all list items
const items = await testdriver.findAll('list item');

// Find specific item by index
const thirdItem = items[2];
await thirdItem.click();

// Check all items
for (let i = 0; i < items.length; i++) {
  console.log(`Item ${i + 1}: ${items[i].text || 'No text'}`);
}
```

#### With Caching

```javascript
// Cache element locations for faster subsequent runs
const menuItems = await testdriver.findAll('menu item', {
  cacheKey: 'main-menu-items'
});

// First run: ~2-3 seconds (AI call)
// Subsequent runs: ~100ms (cache hit)
```

#### Empty Results

```javascript
// Returns empty array if nothing found (doesn't throw error)
const errors = await testdriver.findAll('error message');

if (errors.length === 0) {
  console.log('No errors found - test passed!');
} else {
  console.log(`Found ${errors.length} errors`);
}
```

### Differences from find()

| Feature | find() | findAll() |
|---------|--------|-----------|
| Return type | Single `Element` | Array of `Element[]` |
| If nothing found | Throws `ElementNotFoundError` | Returns empty array `[]` |
| Chainable | ✅ Yes: `await find('button').click()` | ❌ No (returns array) |
| Use case | One specific element | Multiple similar elements |
| Cache support | ✅ Yes | ✅ Yes |

### Use Cases

<AccordionGroup>
  <Accordion title="Table Rows">
    ```javascript
    // Find all rows in a table
    const rows = await testdriver.findAll('table row');
    
    // Click every row
    for (const row of rows) {
      await row.click();
      await new Promise(r => setTimeout(r, 500)); // Wait between clicks
    }
    
    // Or click specific row
    await rows[2].click(); // Click third row
    ```
  </Accordion>
  
  <Accordion title="Checkboxes/Radio Buttons">
    ```javascript
    // Find all checkboxes
    const checkboxes = await testdriver.findAll('checkbox');
    
    // Check all boxes
    for (const checkbox of checkboxes) {
      await checkbox.click();
    }
    
    // Or select first unchecked
    const unchecked = checkboxes[0];
    await unchecked.click();
    ```
  </Accordion>
  
  <Accordion title="Navigation Links">
    ```javascript
    // Find all navigation links
    const navLinks = await testdriver.findAll('navigation link');
    
    // Validate all are present
    expect(navLinks.length).toBeGreaterThan(0);
    
    // Click specific link by text
    const homeLink = navLinks.find(link => 
      link.text?.toLowerCase().includes('home')
    );
    
    if (homeLink) {
      await homeLink.click();
    }
    ```
  </Accordion>
  
  <Accordion title="Conditional Interactions">
    ```javascript
    // Check if any error messages exist
    const errors = await testdriver.findAll('error message');
    
    if (errors.length > 0) {
      console.log(`Found ${errors.length} validation errors`);
      
      // Log each error location
      errors.forEach((error, i) => {
        console.log(`Error ${i + 1} at (${error.x}, ${error.y})`);
      });
    } else {
      console.log('Form validation passed!');
    }
    ```
  </Accordion>
</AccordionGroup>

### Complete Example

```javascript
import { test, expect } from 'vitest';
import { chrome } from 'testdriverai/presets';

test('select multiple items from list', async (context) => {
  const { testdriver } = await chrome(context, {
    url: 'https://example.com/products'
  });
  
  // Find all product cards
  const products = await testdriver.findAll('product card');
  
  expect(products.length).toBeGreaterThan(0);
  console.log(`Found ${products.length} products`);
  
  // Click first 3 products
  const productsToSelect = Math.min(3, products.length);
  
  for (let i = 0; i < productsToSelect; i++) {
    await products[i].click();
    console.log(`Selected product ${i + 1}`);
    await new Promise(r => setTimeout(r, 500)); // Brief pause
  }
  
  // Verify selections
  const selectedBadges = await testdriver.findAll('selected badge');
  expect(selectedBadges.length).toBe(productsToSelect);
});
```

### Best Practices

<Check>
  **Handle empty arrays gracefully**
  
  ```javascript
  // ✅ Good - check length first
  const items = await testdriver.findAll('list item');
  if (items.length > 0) {
    await items[0].click();
  }
  
  // ❌ Bad - may throw error
  const items = await testdriver.findAll('list item');
  await items[0].click(); // Error if array is empty!
  ```
</Check>

<Check>
  **Use find() for single elements**
  
  ```javascript
  // ✅ Use find() when you need exactly one
  const submitBtn = await testdriver.find('submit button');
  await submitBtn.click();
  
  // ❌ Unnecessary - findAll() returns array
  const buttons = await testdriver.findAll('submit button');
  await buttons[0].click(); // Extra array handling
  ```
</Check>

<Check>
  **Cache for performance**
  
  ```javascript
  // First run - slow (AI call)
  const items = await testdriver.findAll('menu item', {
    cacheKey: 'menu-items'
  });
  
  // Subsequent runs - fast (cache hit)
  // ~10-20x faster than without cache
  ```
</Check>