Semgrep

---
name: semgrep
description: "Expert agent for Semgrep static analysis. Covers rule syntax (patterns, metavariables, taint tracking), custom rule authoring, autofix, Semgrep Supply Chain, Semgrep Secrets, CI/CD integration, and Semgrep Cloud. WHEN: \"Semgrep\", \"semgrep rule\", \"semgrep pattern\", \"metavariable\", \"semgrep-action\", \"semgrep supply chain\", \"semgrep secrets\", \"custom SAST rule\", \"SARIF\", \"semgrep scan\"."
license: MIT
---

# Semgrep

This skill covers Semgrep — the semantic code analysis tool with an open-source engine. It covers the Semgrep OSS engine, Semgrep Code (Pro rules + Cloud Platform), Semgrep Supply Chain (SCA with reachability), and Semgrep Secrets.

## How to Approach Tasks

1. **Identify the product tier:**
   - **Semgrep OSS:** Free, open-source (LGPL 2.1), community rules, no cloud
   - **Semgrep Code:** Pro rules (proprietary), cross-file analysis, Cloud Platform
   - **Semgrep Supply Chain:** SCA with reachability analysis
   - **Semgrep Secrets:** Secret detection with validity checking
2. **Classify the task:** Running scans, writing rules, CI/CD integration, understanding findings, false positive management.
3. **Apply pattern-first reasoning** -- Semgrep excels at precise pattern-based rules. Guide users toward correct metavariable usage and pattern composition.

## Core Architecture

Semgrep operates on ASTs (Abstract Syntax Trees) rather than regex. This gives it:
- **Syntax-aware matching:** `$X + $Y` matches addition in any form, not just literally `a + b`
- **Semantic equivalence:** Recognizes equivalent code structures (e.g., `not x == y` and `x != y`)
- **Language-specific rules:** Rules are language-specific (a Python rule won't match Java)
- **Multi-language rule sets:** A single YAML file can contain rules for multiple languages

**Engine:** Rust core with language-specific parsers (tree-sitter based for most languages).

## Rule Syntax

### Basic Rule Structure

```yaml
rules:
  - id: my-rule-id
    patterns:
      - pattern: $SINK(request.args.get(...))
    message: >
      User input flows to $SINK without validation.
      Consider using parameterized queries or input validation.
    severity: ERROR           # ERROR, WARNING, INFO
    languages: [python]
    metadata:
      cwe: "CWE-89: SQL Injection"
      owasp: "A03:2021 - Injection"
      confidence: HIGH
```

### Pattern Operators

**`pattern`** — Matches code literally (with metavariables):
```yaml
pattern: requests.get($URL)
```

**`patterns`** — All patterns must match (AND):
```yaml
patterns:
  - pattern: $FUNC($INPUT)
  - pattern-inside: |
      def $FUNC(...):
        ...
  - metavariable-regex:
      metavariable: $INPUT
      regex: request\..*
```

**`pattern-either`** — Any pattern matches (OR):
```yaml
pattern-either:
  - pattern: eval($X)
  - pattern: exec($X)
  - pattern: os.system($X)
```

**`pattern-not`** — Exclude matches:
```yaml
patterns:
  - pattern: subprocess.call($CMD, ...)
  - pattern-not: subprocess.call($CMD, shell=False, ...)
```

**`pattern-not-inside`** — Exclude matches within a context:
```yaml
patterns:
  - pattern: hashlib.md5(...)
  - pattern-not-inside: |
      # ... non-security use (e.g., checksum of public data)
      $X = hashlib.md5(...)
```

**`pattern-inside`** — Match only within a context:
```yaml
patterns:
  - pattern: $QUERY.execute($SQL)
  - pattern-inside: |
      class $CLASS(View):
        ...
```

### Metavariables

Metavariables capture matched code for use in messages or further patterns:

```yaml
# $X matches any single expression or identifier
# $...X matches any sequence of arguments (variadic)
# $X == $X matches same expression on both sides (self-comparison bug)

patterns:
  - pattern: |
      if $COND:
        ...
      if $COND:
        ...
  message: Duplicate condition $COND — likely a bug
```

**Metavariable types:**
- `$X` — Any single AST node (expression, identifier, statement)
- `$...X` — Zero or more items (function arguments, list elements)
- `"$X"` — String literal binding

**Metavariable filters:**

```yaml
# Regex filter on captured value
metavariable-regex:
  metavariable: $ALGO
  regex: (md5|sha1|des|rc4)

# Type filter (requires type inference, Pro)
metavariable-type:
  metavariable: $INPUT
  type: str

# Pattern filter — $X must match a sub-pattern
metavariable-pattern:
  metavariable: $QUERY
  patterns:
    - pattern: "SELECT ..."
```

### Taint Analysis

Semgrep Pro supports interprocedural taint tracking:

```yaml
rules:
  - id: sql-injection
    mode: taint
    pattern-sources:
      - pattern: request.args.get(...)
      - pattern: request.form.get(...)
    pattern-sanitizers:
      - pattern: escape($X)
      - pattern: parameterize($X)
    pattern-sinks:
      - pattern: db.execute($QUERY)
      - pattern: cursor.execute($QUERY)
    message: SQL injection via user input
    severity: ERROR
    languages: [python]
```

**Taint mode differences:**
- OSS: intra-file taint only (within one file)
- Pro: cross-file, cross-function taint (follows calls across the project)

### Autofix

Rules can include automatic fix suggestions:

```yaml
rules:
  - id: use-secrets-manager
    pattern: os.environ["$KEY"]
    fix: get_secret("$KEY")
    message: Use secrets manager instead of environment variables
    severity: WARNING
    languages: [python]
```

Fix is applied with `semgrep --autofix`. Recommended workflow: run in CI, generate diff, create PR with fixes.

### Focus Metavariable

For taint rules, highlight which part of the match is the source of the finding:

```yaml
focus-metavariable: $SINK
```

---

## Running Semgrep

### CLI Usage

```bash
# Install
pip install semgrep
# or
brew install semgrep

# Scan with OWASP Top 10 ruleset
semgrep scan --config p/owasp-top-ten .

# Scan with specific rule file
semgrep scan --config my-rules.yaml src/

# Scan and output SARIF (for CI/CD)
semgrep scan --config p/default --sarif > results.sarif

# Fast PR scan (only changed files)
semgrep scan --config p/default --baseline-commit origin/main .

# Auto-fix
semgrep scan --config my-rules.yaml --autofix src/
```

### Rule Registry (Semgrep Registry)

Community rules at `semgrep.dev/r`:
- `p/owasp-top-ten` — OWASP Top 10 2021 rules
- `p/default` — Semgrep recommended defaults
- `p/django` — Django-specific security rules
- `p/flask` — Flask-specific security rules
- `p/express` — Express.js security rules
- `p/java` — Java security rules
- `p/golang` — Go security rules
- `p/javascript` — JavaScript security rules
- `p/python` — Python security rules
- `p/docker` — Dockerfile rules
- `p/terraform` — Terraform IaC rules

3,000+ community rules available. Pro rules (proprietary, higher signal) require Semgrep Code subscription.

---

## CI/CD Integration

### GitHub Actions

```yaml
- uses: actions/checkout@v4
  with:
    fetch-depth: 0   # Required for baseline comparison

- name: Semgrep SAST
  uses: semgrep/semgrep-action@v1
  with:
    config: >-
      p/owasp-top-ten
      p/default
    # For PRs: only report new findings vs. base branch
    generateSarif: "1"
  env:
    SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}

- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: semgrep.sarif
  if: always()
```

### GitLab CI

```yaml
semgrep:
  image: semgrep/semgrep
  script:
    - semgrep ci --config p/owasp-top-ten --sarif --output semgrep.sarif
  artifacts:
    reports:
      sast: semgrep.sarif
  rules:
    - if: $CI_MERGE_REQUEST_ID
```

### Pre-commit Hook

```yaml
# .pre-commit-config.yaml
repos:
  - repo: https://github.com/returntocorp/semgrep
    rev: v1.70.0
    hooks:
      - id: semgrep
        args: ['--config', 'p/default', '--error']
```

---

## Semgrep Supply Chain (SCA)

Semgrep Supply Chain adds Software Composition Analysis with **reachability analysis** — it determines whether a vulnerable function in a dependency is actually called by your code.

**Reachability analysis value:**
- Standard SCA: "Your dependency lodash 4.17.20 has CVE-2021-23337"
- Supply Chain: "Your dependency lodash 4.17.20 has CVE-2021-23337 AND your code calls the vulnerable `_.template()` function on line 42" (reachable)

**Supported package managers:** npm, yarn, pnpm, pip, pipenv, poetry, Maven, Gradle, Gemfile, Go modules, Cargo, Composer.

**Running Supply Chain:**
```bash
semgrep ci --supply-chain
# or
semgrep scan --config p/default --supply-chain .
```

**SARIF output includes** finding type `supply-chain` with reachability status.

---

## Semgrep Secrets

Detects hardcoded secrets with:
- **Pattern matching:** Regex for secret patterns (API key formats, connection strings)
- **Entropy analysis:** High entropy strings likely to be secrets
- **Validity checking (Pro):** Actually tests if a detected secret is currently valid (e.g., pings AWS, GitHub, Stripe APIs)

```bash
semgrep ci --secrets
```

**Validity status levels:**
- `valid` — Secret was confirmed active (immediate remediation required)
- `invalid` — Secret detected but no longer active
- `unknown` — Cannot determine validity (no checker for this secret type)

---

## Writing Production-Quality Rules

### Rule Design Principles

1. **Specificity over coverage:** Better to have a rule that matches exactly what you want with 5% false positives than one that catches 10% more with 40% false positives.

2. **Use `pattern-not` aggressively:** Eliminate known safe patterns explicitly.

3. **Add `metadata`:** Include `cwe`, `owasp`, `confidence`, `likelihood`, `impact` for integration with security platforms.

4. **Test your rules:**
```yaml
rules:
  - id: no-eval
    pattern: eval($X)
    message: Avoid eval
    severity: ERROR
    languages: [javascript]
    # Inline test cases
    tests:
      - code: |
          eval(userInput);   # ruleid: no-eval
          eval("static");    # ok: no-eval (but keep this in mind)
```

### Rule Testing

```bash
# Run tests for all rules in directory
semgrep --test rules/

# Test specific rule file
semgrep --test rules/my-rule.yaml
```

Test case annotations:
- `# ruleid: rule-id` — This line SHOULD be flagged by rule-id
- `# ok: rule-id` — This line should NOT be flagged
- `# todoruleid: rule-id` — Should eventually be flagged (tracks as warning, not failure)

### Performance Considerations

- Rules with `pattern-inside` are more expensive (require context search)
- Taint rules (Pro) are expensive for large codebases — use specific sources/sinks
- `metavariable-regex` is applied after pattern match, so it's cheap to add
- Avoid overly broad patterns like `$X($Y)` as base patterns — too many matches

---

## Semgrep Cloud Platform

Semgrep Cloud provides:
- **Findings dashboard:** Centralized view across all repositories
- **Policy management:** Set blocking conditions per severity/confidence
- **Pro rules:** Proprietary high-confidence rules (fewer false positives than community)
- **Assistant (AI):** Auto-triage findings (mark as true positive/false positive)
- **Notifications:** Slack, email, Jira integration

**Authentication:**
```bash
semgrep login   # Opens browser OAuth flow
semgrep ci      # Sends results to Cloud Platform (requires SEMGREP_APP_TOKEN)
```

## Common Issues

**Rule matches nothing expected:**
- Check language is correct (`languages: [python]` not `languages: [py]`)
- Use `semgrep --pattern '$PATTERN' --lang python file.py` to test interactively
- Check indentation in YAML (YAML is sensitive; use `|` for multiline patterns)

**Too many false positives:**
- Add `pattern-not` for known safe patterns
- Use `metavariable-regex` to narrow match scope
- For taint rules: ensure sanitizers are correctly specified

**Taint not following across functions:**
- Intra-file taint is OSS; cross-file requires Pro
- Verify sink and source patterns match exactly how the functions are called in code

**SARIF upload not showing in GitHub Security tab:**
- Ensure `github/codeql-action/upload-sarif` step runs even on failure: `if: always()`
- Check SARIF output is valid (use `semgrep --sarif > out.sarif` and validate schema)