Insaiai Intelligent Editing

---
name: insaiai-intelligent-editing
version: 1.0.0
description: Use when performing video/audio processing tasks including transcoding, filtering, streaming, metadata manipulation, or complex filtergraph operations with FFmpeg.
triggers:
  - ffmpeg
  - ffprobe
  - video processing
  - audio conversion
  - codec
  - transcoding
  - filter_complex
  - h264
  - h265
  - mp4
  - mkv
  - hardware acceleration
role: specialist
scope: implementation
output-format: shell-command
---

# inSaiAI Intelligent Editing

Comprehensive guide for professional video and audio manipulation using FFmpeg and FFprobe.

## Core Concepts

FFmpeg is the leading multimedia framework, able to **decode, encode, transcode, mux, demux, stream, filter and play** almost anything that humans and machines have created. It is a command-line tool that processes streams through a complex pipeline of demuxers, decoders, filters, encoders, and muxers.

## Common Operations

```bash
# Basic Transcoding (MP4 to MKV)
ffmpeg -i input.mp4 output.mkv

# Change Video Codec (to H.265/HEVC)
ffmpeg -i input.mp4 -c:v libx265 -crf 28 -c:a copy output.mp4

# Extract Audio (No Video)
ffmpeg -i input.mp4 -vn -c:a libmp3lame -q:a 2 output.mp3

# Resize/Scale Video
ffmpeg -i input.mp4 -vf "scale=1280:720" output.mp4

# Cut Video (Start at 10s, Duration 30s)
ffmpeg -i input.mp4 -ss 00:00:10 -t 00:00:30 -c copy output.mp4

# Fast Precise Cut (Re-encoding only the cut points is complex, so standard re-encoding is safer for precision)
ffmpeg -ss 00:00:10 -i input.mp4 -to 00:00:40 -c:v libx264 -crf 23 -c:a aac output.mp4

# Concatenate Files (using demuxer)
# Create filelist.txt: file 'part1.mp4' \n file 'part2.mp4'
ffmpeg -f concat -safe 0 -i filelist.txt -c copy output.mp4

# Speed Up/Slow Down Video (2x speed)
ffmpeg -i input.mp4 -filter_complex "[0:v]setpts=0.5*PTS[v];[0:a]atempo=2.0[a]" -map "[v]" -map "[a]" output.mp4
```

---

## Processing Categories & When to Use

### Codecs & Quality
| Option | Use When |
|-----------|----------|
| `-c:v libx264` | Standard H.264 encoding (best compatibility) |
| `-c:v libx265` | H.265/HEVC encoding (best compression/quality) |
| `-crf [0-51]` | Constant Rate Factor (lower is higher quality, 18-28 recommended) |
| `-preset` | Encoding speed vs compression (ultrafast, medium, veryslow) |
| `-c:a copy` | Pass-through audio without re-encoding (saves time/quality) |

### Filters & Manipulation
| Filter | Use When |
|-----------|----------|
| `scale` | Changing resolution (e.g., `scale=1920:-1` for 1080p width) |
| `crop` | Removing edges (e.g., `crop=w:h:x:y`) |
| `transpose` | Rotating video (1=90deg CW, 2=90deg CCW) |
| `fps` | Changing frame rate (e.g., `fps=30`) |
| `drawtext` | Adding text overlays/watermarks |
| `overlay` | Picture-in-picture or adding image watermarks |
| `fade` | Adding fade-in/out effects (e.g., `fade=in:0:30` for first 30 frames) |
| `volume` | Adjusting audio levels (e.g., `volume=1.5` for 150% volume) |
| `setpts` | Changing video speed (e.g., `setpts=0.5*PTS` for double speed) |
| `atempo` | Changing audio speed without pitch shift (0.5 to 2.0) |

### Inspection & Metadata
| Tool/Option | Use When |
|-----------|----------|
| `ffprobe -v error -show_format -show_streams` | Getting detailed technical info of a file |
| `-metadata title="Name"` | Setting global metadata tags |
| `-map` | Selecting specific streams (e.g., `-map 0:v:0 -map 0:a:1`) |

---

## Advanced: Complex Filtergraphs

Use `filter_complex` when you need to process multiple inputs or create non-linear filter chains.

```bash
# Example: Adding a watermark at the bottom right
ffmpeg -i input.mp4 -i watermark.png -filter_complex "overlay=main_w-overlay_w-10:main_h-overlay_h-10" output.mp4

# Example: Vertical Stack (2 videos)
ffmpeg -i top.mp4 -i bottom.mp4 -filter_complex "vstack=inputs=2" output.mp4

# Example: Side-by-Side (2 videos)
ffmpeg -i left.mp4 -i right.mp4 -filter_complex "hstack=inputs=2" output.mp4

# Example: Grid (4 videos 2x2)
ffmpeg -i v1.mp4 -i v2.mp4 -i v3.mp4 -i v4.mp4 -filter_complex "[0:v][1:v]hstack=inputs=2[top];[2:v][3:v]hstack=inputs=2[bottom];[top][bottom]vstack=inputs=2" output.mp4

# Example: Fade Transition (Simple crossfade between two clips)
# Requires manual offset calculation, using xfade is better
ffmpeg -i input1.mp4 -i input2.mp4 -filter_complex "xfade=transition=fade:duration=1:offset=9" output.mp4
```

## Pro Editing Tips & Techniques

### 1. High-Quality GIF Creation
Standard conversion often results in poor colors. Use a palette for best results:
```bash
ffmpeg -i input.mp4 -vf "fps=15,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif
```

### 2. Audio Mixing (Background Music + Voice)
Mix background music at 30% volume with the main audio:
```bash
ffmpeg -i voice.mp4 -i bgm.mp3 -filter_complex "[1:a]volume=0.3[bg];[0:a][bg]amix=inputs=2:duration=first" -c:v copy output.mp4
```

### 3. Video Stabilization
Two-pass process to fix shaky footage:
```bash
# Pass 1: Analyze
ffmpeg -i shaky.mp4 -vf vidstabdetect -f null -
# Pass 2: Transform
ffmpeg -i shaky.mp4 -vf vidstabtransform,unsharp=5:5:0.8:3:3:0.4 output.mp4
```

### 4. Color Correction & Enhancement
Adjust brightness, contrast, and saturation:
```bash
# brightness=0.05, contrast=1.1, saturation=1.2
ffmpeg -i input.mp4 -vf "eq=brightness=0.05:contrast=1.1:saturation=1.2" output.mp4
```

### 5. Automatic Thumbnail Sheet
Create a 3x3 grid of frames:
```bash
ffmpeg -i input.mp4 -vf "select='not(mod(n,100))',scale=320:-1,tile=3x3" -frames:v 1 preview.png
```

### 6. Remove Silence from Audio
Automatically cut silent parts from the beginning and end:
```bash
ffmpeg -i input.mp4 -af silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB:stop_periods=1:stop_silence=0.1:stop_threshold=-50dB output.mp4
```

### 7. Hardsub Burning
Burn SRT/ASS subtitles directly into the video stream:
```bash
# Burn SRT
ffmpeg -i input.mp4 -vf "subtitles=subs.srt" output.mp4
# Burn ASS (supports advanced styling)
ffmpeg -i input.mp4 -vf "ass=subs.ass" output.mp4
```

### 8. Target File Size Compression
Calculate bitrate to fit a specific file size (e.g., 50MB for 60s video):
```bash
# Bitrate = (TargetSize_in_bits) / (Duration_in_seconds)
# 50MB = 400,000 bits. For 60s, bitrate ≈ 6600k
ffmpeg -i input.mp4 -b:v 6000k -maxrate 6000k -bufsize 12000k -c:a aac -b:a 128k output.mp4
```

### 9. Scene Change Detection
Extract frames where a scene change is detected (threshold 0.4):
```bash
ffmpeg -i input.mp4 -filter_complex "select='gt(scene,0.4)',metadata=print:file=scenes.txt" -vsync vfr scene_%03d.png
```

### 10. Extracting Frames at Specific Intervals
Extract one frame every 5 seconds:
```bash
ffmpeg -i input.mp4 -vf "fps=1/5" img_%03d.jpg
```

### 11. Batch Processing (Shell Snippet)
Convert all `.mov` files in a directory to `.mp4`:
```bash
for f in *.mov; do ffmpeg -i "$f" "${f%.mov}.mp4"; done
```

### 12. Live Streaming (RTMP)
Push a local file to a streaming server (YouTube/Twitch):
```bash
ffmpeg -re -i input.mp4 -c:v libx264 -preset veryfast -b:v 3000k -maxrate 3000k -bufsize 6000k -pix_fmt yuv420p -g 60 -c:a aac -b:a 128k -f flv rtmp://a.rtmp.youtube.com/live2/YOUR_STREAM_KEY
```

## Hardware Acceleration

| Platform | Codec | Command |
|----------|-------|---------|
| NVIDIA (NVENC) | H.264 | `-c:v h264_nvenc` |
| Intel (QSV) | H.264 | `-c:v h264_qsv` |
| Apple (VideoToolbox) | H.265 | `-c:v hevc_videotoolbox` |

## Constraints & Error Handling

- **Stream Mapping**: Always use `-map` for complex files to ensure you get the right audio/subtitle tracks.
- **Seeking**: Put `-ss` *before* `-i` for fast seeking (input seeking), or *after* `-i` for accurate seeking (output seeking).
- **Format Support**: Ensure the output container (extension) supports the codecs you've chosen.