Qradar

---
name: qradar
description: "Expert agent for IBM QRadar SIEM. Provides deep expertise in AQL query development, offense management, DSM configuration, event pipeline tuning, custom rule building, reference sets/maps, Ariel database, and on-premises deployment architecture. WHEN: \"QRadar\", \"AQL\", \"Ariel query\", \"QRadar offense\", \"DSM\", \"QRadar rule\", \"building block\", \"reference set\", \"QRadar tuning\", \"IBM SIEM\"."
license: MIT
---

# IBM QRadar

This skill covers IBM QRadar SIEM (on-premises, 7.5.x). It has deep knowledge of:

- Ariel Query Language (AQL) for event and flow search
- Offense management (automatic grouping of correlated events)
- Device Support Module (DSM) configuration for log parsing
- Event pipeline (parsing, normalization, correlation, offense creation)
- Custom rules and building blocks
- Reference sets and reference maps (dynamic lookups)
- Ariel database internals (time-series columnar storage)
- Deployment architecture (Console, Event Processor, Flow Processor, Data Node)
- QRadar apps and extensions (App Exchange)
- MITRE ATT&CK mapping with QRadar Use Case Manager

**Important context:** IBM divested QRadar SaaS to Palo Alto Networks (now part of Cortex XSIAM). On-premises QRadar continues under IBM at version 7.5.x. This agent covers the on-premises product.

## How to Approach Tasks

1. **Classify** the request:
   - **Investigation** -- AQL query development, offense drill-down
   - **Detection engineering** -- Custom rules, building blocks, reference sets
   - **Data onboarding** -- DSM configuration, log source management
   - **Architecture** -- Deployment sizing, distributed architecture, HA
   - **Troubleshooting** -- Pipeline issues, parsing errors, performance problems
   - **Migration** -- Planning migration to other SIEM platforms

2. **Gather context** -- QRadar version, deployment size (EPS), distributed vs. all-in-one, existing DSM coverage

3. **Analyze** -- Apply QRadar-specific reasoning, especially around the offense management model

4. **Recommend** -- Provide actionable guidance with AQL queries and configuration steps

## Core Expertise

### AQL (Ariel Query Language)

AQL is a SQL-like language for querying QRadar's Ariel database:

```sql
-- Basic event search: failed logins in the last hour
SELECT sourceip, username, COUNT(*) as attempt_count
FROM events
WHERE LOGSOURCETYPENAME(logsourcetypeid) = 'Microsoft Windows Security Event Log'
  AND qidname(qid) = 'Login Failure'
  AND INOFFENSE(*)
  AND starttime > NOW() - 1 * 60 * 60 * 1000
GROUP BY sourceip, username
HAVING COUNT(*) > 10
ORDER BY attempt_count DESC
LAST 1 HOURS
```

**AQL key functions:**

| Function | Purpose | Example |
|---|---|---|
| `LOGSOURCETYPENAME()` | Get log source type name | `LOGSOURCETYPENAME(logsourcetypeid) = 'Linux OS'` |
| `qidname()` | Get QID (event) name | `qidname(qid) = 'Login Failure'` |
| `CATEGORYNAME()` | Get event category name | `CATEGORYNAME(category) = 'Authentication'` |
| `INOFFENSE()` | Filter events in offenses | `INOFFENSE(offense_id)` or `INOFFENSE(*)` |
| `REFERENCETABLE()` | Query reference table | `REFERENCETABLE('threat_ips', sourceip)` |
| `REFERENCESETCONTAINS()` | Check reference set membership | `REFERENCESETCONTAINS('blocked_ips', sourceip)` |
| `DATEFORMAT()` | Format timestamps | `DATEFORMAT(starttime, 'yyyy-MM-dd HH:mm:ss')` |
| `UTF8()` | Convert payload to string | `UTF8(payload)` |
| `ASSETHOSTNAME()` | Get asset hostname | `ASSETHOSTNAME(sourceip)` |

**AQL for flows:**
```sql
-- Top talkers by bytes
SELECT sourceip, destinationip, SUM(sourcebytes + destinationbytes) as total_bytes
FROM flows
WHERE starttime > NOW() - 24 * 60 * 60 * 1000
GROUP BY sourceip, destinationip
ORDER BY total_bytes DESC
LIMIT 20
LAST 24 HOURS
```

### Offense Management

QRadar's offense model automatically groups related events into offenses:

**Offense lifecycle:**
```
Events arrive
    |
    v
Rules evaluate (custom + IBM-provided)
    |
    v
Rule matches --> Create or update offense
    |
    ├── New offense: assign offense source (IP, user, etc.)
    ├── Existing offense: add contributing events
    └── Magnitude calculation: severity x relevance x credibility
    |
    v
Offense displayed in dashboard
    |
    v
Analyst triage:
    ├── Investigate (drill into events, flows, assets)
    ├── Assign to analyst
    ├── Add notes
    ├── Protect (prevent auto-close)
    ├── Close (with closing reason: false positive, non-issue, policy violation, resolved)
    └── Escalate (export to ticketing system or SOAR)
```

**Offense magnitude formula:**
```
Magnitude = (Severity + Relevance + Credibility) / 3  (weighted, scale 1-10)

Severity: How dangerous is the event type? (from QID mapping)
Relevance: Is the target asset known and important? (from asset database)
Credibility: How trustworthy is the log source? (from log source credibility)
```

### Event Pipeline

The QRadar event pipeline processes events through multiple stages:

```
Log Source --> Parsing (DSM) --> Normalization --> Rule Evaluation --> Storage (Ariel)
                                                        |
                                                        v
                                                  Offense Creation/Update
```

**Pipeline stages:**

1. **Collection** -- Events received via syslog, JDBC, API, file, WinCollect agent
2. **Parsing (DSM)** -- Device Support Module extracts fields from raw log
3. **Normalization** -- Map to QRadar's internal schema (QID, category, severity)
4. **Coalescing** -- Identical events within a time window are merged (count incremented)
5. **Rule evaluation** -- Custom rules and building blocks evaluate against events
6. **Offense management** -- Matched rules create or update offenses
7. **Storage** -- Events written to Ariel database (time-series columnar)

### DSM (Device Support Module)

DSMs define how QRadar parses logs from specific devices:

**DSM components:**
- **Log source type** -- Identifies the device/application
- **QID map** -- Maps raw event IDs to QRadar normalized event names and categories
- **Property extraction** -- Regex-based field extraction from raw log
- **Event categorization** -- Maps events to QRadar's high-level/low-level categories

**Custom DSM creation workflow:**
1. Collect sample logs from the device
2. Create log source type (Admin > DSM Editor)
3. Define event properties (regex or JSON path extraction)
4. Map to QIDs (event names and categories)
5. Test with log source auto-detection
6. Deploy to production

### Custom Rules and Building Blocks

**Building blocks** are reusable rule components (don't generate offenses on their own):
```
Building Block: "Local Network Addresses"
  Test: when source IP is in 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16

Building Block: "Administrative Accounts"
  Test: when username matches regex "^(admin|root|sa|svc_.*)"
```

**Custom rules** combine tests and building blocks to detect threats:
```
Rule: "Brute Force Against Administrative Account"
  Tests:
    - when the event category is "Authentication"
    - AND when the event QID is "Login Failure"
    - AND when the username matches building block "Administrative Accounts"
    - AND when at least 10 events are seen with the same source IP
          in 5 minutes
  Actions:
    - Create offense, indexed on source IP
    - Set severity: High
    - Annotate with MITRE ATT&CK: T1110 (Brute Force)
```

**Rule response actions:**
- Create offense (new or contribute to existing)
- Send email notification
- Add to reference set (e.g., add IP to "suspicious_ips" set)
- Execute custom action (script, API call)
- Generate syslog event (forward to other systems)

### Reference Sets and Reference Maps

Dynamic lookup tables used in rules and searches:

| Type | Structure | Use Case |
|---|---|---|
| **Reference Set** | Single column (values) | IP blocklist, known-bad hashes, VIP usernames |
| **Reference Map** | Key-value pairs | IP-to-country, user-to-department |
| **Reference Map of Sets** | Key to set of values | User-to-allowed-IPs, department-to-subnets |
| **Reference Table** | Multi-column table | Full asset inventory, TI feed with confidence/expiry |

**API management:**
```bash
# Add to reference set
curl -X POST "https://qradar/api/reference_data/sets/blocked_ips" \
  -H "SEC: <api_token>" \
  -d '{"value": "203.0.113.50"}'

# Bulk load reference set from file
curl -X POST "https://qradar/api/reference_data/sets/bulk_load/blocked_ips" \
  -H "SEC: <api_token>" \
  -H "Content-Type: text/plain" \
  --data-binary @ip_list.txt
```

**Use in AQL:**
```sql
SELECT sourceip, destinationip, qidname(qid)
FROM events
WHERE REFERENCESETCONTAINS('blocked_ips', sourceip)
LAST 24 HOURS
```

### Deployment Architecture

| Component | Role | Scaling |
|---|---|---|
| **Console** | Web UI, offense management, reporting, API | Single (HA optional) |
| **Event Processor (EP)** | Event parsing, rule evaluation | Add EPs for higher EPS |
| **Event Collector (EC)** | Remote event collection | Deploy at remote sites |
| **Flow Processor (FP)** | Network flow analysis | Add for flow-heavy environments |
| **Data Node** | Additional Ariel storage | Scale search and storage capacity |

**Typical sizing:**

| Deployment | EPS | Components |
|---|---|---|
| **Small** | < 5,000 | All-in-one appliance |
| **Medium** | 5,000-25,000 | Console + 1-2 Event Processors |
| **Large** | 25,000-100,000 | Console + 4-8 EPs + Data Nodes |
| **Enterprise** | 100,000+ | Console + 8+ EPs + 4+ Data Nodes + Event Collectors at sites |

## Common Pitfalls

1. **Coalescing hides events** -- QRadar coalesces identical events (same QID, source IP, dest IP, username). A single coalesced event may represent hundreds of raw events. Always check `eventcount` field.
2. **License by EPS** -- QRadar licenses by events per second. Exceeding license causes event drops. Monitor with Admin > System Monitoring.
3. **DSM parsing failures** -- Unparsed events show as "Unknown" log source type. Check parsing status in Log Activity > "Unknown" category.
4. **Rule ordering** -- Rules evaluate in order. A rule that matches first and generates an offense can prevent later, more specific rules from contributing context. Plan rule ordering.
5. **Reference set size** -- Large reference sets (>100K entries) impact rule evaluation performance. Use reference tables with indexed columns for large datasets.
6. **Offense clutter** -- Without tuning, QRadar generates thousands of low-magnitude offenses. Set minimum magnitude thresholds for the offense dashboard and tune rules aggressively.

## Migration Considerations

Given IBM's divestiture of QRadar SaaS, many organizations are evaluating migration:

**Common migration targets:**
- **XSIAM** -- Palo Alto's converged platform (received QRadar SaaS customer base)
- **Sentinel** -- For Azure-centric environments
- **Splunk** -- For complex, large-scale deployments
- **Elastic Security** -- For open-source preference or cost optimization

**Migration planning:**
1. Export custom rules, building blocks, and reference data
2. Map QRadar DSM categories to target platform normalization (CIM, ECS, ASIM)
3. Convert AQL saved searches to target query language (SPL, KQL, EQL)
4. Migrate offense workflows to target incident/notable event model
5. Parallel run for 30-60 days to validate detection parity

## Reference Files

Load these for deep knowledge:
- `references/architecture.md` -- Ariel database internals, event pipeline, DSM architecture, deployment patterns, HA configuration