DART

> **MANDATORY EXECUTION POLICY** — READ BEFORE PROCEEDING
>
> You MUST run the **actual model binary or package** described in this document.
> If the model fails to import, compile, or execute, you MUST:
> 1. Check `diagnostics/triplets.yaml` for a matching error pattern
> 2. Try the fix suggested in the triplet's `remedy` section
> 3. If still failing, report the error to the user with full details
>
> You MUST NOT substitute a simplified Python formula, regression equation,
> or hand-coded approximation in place of the real model.
>
>
> Before starting, run: `python preflight_check.py` (in this KI directory)
> to verify that the model binary/package and required data are available.
>
> **DEBUGGING PROTOCOL** — When something goes wrong, follow this order:
> 1. **Check triplets** — `diagnostics/triplets.yaml` may already cover this error
> 2. **Read official docs** — The model's own documentation for expected formats/units
> 3. **Find working examples** — Check `outputs/` or the model's shipped test data
> 4. **Fix the tool** — With knowledge of what "correct" looks like
>
> Do NOT write custom debug scripts. The answers are in the docs and examples.

<!-- KI-MAP:BEGIN (projected by generate_skill_map.py — edit the KI, not this table) -->
## KI map — what to read, and when

| when you need | read | why |
|---|---|---|
| FIRST, always | `preflight_check.py` | run it (`python preflight_check.py`): proves env/binary/data are usable and emits a machine-readable `PREFLIGHT_REPORT=` line. Do not debug a run that never had a healthy environment. |
| to run the pipeline stages | `tools/` (4 tools) | the executable pipeline. Read each tool's argparse (`--help`) before composing a command; SKILL.md's stage table says which tool serves which stage. |
| before running a stage | `docs/s*_*.md` (5 stage docs) | per-stage procedure, verification and traps — the how-to that SKILL.md's overview compresses. |
| on ANY error, before debugging | `diagnostics/triplets.yaml` (20 entries) | symptom → diagnosis → remedy for this model's known failure modes. Check here FIRST; the answer usually exists. Never renumber or rewrite entries. |
| to know what an output IS | `dag.yaml` | the model's identity: every output's medium, units, `validation_rank` (1 = the headline variable) and observability. Scoring and obs-binding read THIS — when asked 'what does this model predict', the dag is the answer, not a guess. |
| when building inputs / parsing outputs | `docs/format_spec.yaml` | exact I/O shapes + `known_issues`, projected from dag + triplets. Regenerate with `ki_tools_common/generate_format_spec.py` after changing either — never hand-edit. |
| to judge a run's skill | `docs/validation_convention.yaml` | how this model's field judges it validated: per-`dag_variable` metrics, directions and CITED pass-bands. A run is graded against these, not against intuition. |
| for claims and thresholds | `docs/gathered_papers.json` (17 papers) + `docs/papers_index.md` | the literature this KI is judged by; each entry's `text_path` is fetched full text in the central paper cache. `role: benchmark` marks the model's own skill paper. |
| for a machine-readable summary | `knowledge_infrastructure.yaml` | the manifest (package, pipeline, validation tier, counts) — projected by `ki_tools_common/generate_ki_manifest.py`; regenerate after structural changes, never hand-edit. |

*Projected 2026-08-17 from the KI's actual contents — 9 components present. Refresh: `python3 ki_tools_common/generate_skill_map.py --ki_dir <this KI>`.*
<!-- KI-MAP:END -->

<!-- KI-TOOL-INDEX:BEGIN (projected by generate_skill_map.py — the discoverability contract: every public tool, exact path; PURPOSE stays human-authored elsewhere) -->
### Executable tool index (projected — complete by construction)

Every public tool in this KI, by exact path. What each is FOR lives in the
human-written Tool Inventory above; `--help` on any of these prints its arguments.

| tool (exact path) | invocation |
|---|---|
| `tools/convert_obs_to_dart.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/convert_obs_to_dart.py --help` |
| `tools/generate_input_nml.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/generate_input_nml.py --help` |
| `tools/parse_dart_output.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/parse_dart_output.py --help` |
| `tools/run_dart.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/run_dart.py --help` |

*4 public tools; `_`-prefixed helpers and packaging files excluded.*
<!-- KI-TOOL-INDEX:END -->

# DART — Data Assimilation Research Testbed

**Package**: DART Knowledge Infrastructure
**Model**: DART (Data Assimilation Research Testbed)
**Version**: Manhattan Release
**Domain**: Ensemble Data Assimilation
**Language**: Fortran 90+ with NetCDF I/O
**Created by**: auto_dissect pipeline
**Tools**: 4 | **Docs**: 5 | **Diagnostics**: 20 | **Lines**: ~3000
**Validation**: Lorenz 63 OSSE

---

## Data Preparation

### Input data

**Input Source**: DART is a data assimilation framework — it ingests observations to update model states.
The observation source depends on the coupled model (discharge for hydrology, temperature for lakes, etc.).

**Data Validation Reference**: See `data_ki/ObservedQ/SKILL.md` for discharge observations.
See `data_ki/FLUXNET/SKILL.md` for flux tower observations.
See  for flux tower observations.



## Data Preparation

### Forcing data

**Data Sources**: Use `from ki_tools_common.load_forcing import load_daily_forcing` for CMFD/MSWX/NASA POWER.

**Data Validation Reference**: Framework models use data from the coupled models.


## Overview

DART is an open-source community facility for ensemble data assimilation (DA)
developed by the Data Assimilation Research Section (DAReS) at NSF NCAR.
It provides a modular, flexible framework for combining observations with
numerical models to estimate the state of physical systems.

DART supports 50+ models spanning atmosphere, ocean, land, ice, and idealized
dynamical systems. It implements multiple assimilation algorithms including
the Ensemble Adjustment Kalman Filter (EAKF), Ensemble Kalman Filter (EnKF),
and the Quantile-Conserving Ensemble Filtering Framework (QCEFF) for
non-Gaussian distributions.

Key capabilities include:
- Generating initial conditions for forecasts
- Producing reanalyses (retrospective state estimates)
- Conducting observing system experiments (OSEs)
- Conducting observing system simulation experiments (OSSEs)
- Diagnosing model error and observation impact

DART reads observations from obs_seq files (ASCII or binary), state vectors
from NetCDF, and configuration from Fortran namelists (input.nml). All
output is NetCDF (state diagnostics) or obs_seq format (observation diagnostics).

---

## Installation

### Dependencies

| Dependency | Purpose | Required |
|---|---|---|
| gfortran >= 9 or Intel ifort | Fortran compiler | Yes |
| NetCDF-Fortran + NetCDF-C | I/O for state files | Yes |
| MPI (mpif90) | Parallel execution | Optional (for large models) |
| Perl | mkmf build tool | Yes |
| Make | Build system | Yes |
| Python 3 | pytools, diagnostics | Optional |
| MATLAB | Traditional diagnostics | Optional |
| NCO (ncdump, ncview) | NetCDF inspection | Optional |

### Build Steps

```bash
# 1. Clone repository
git clone https://github.com/NCAR/DART.git

# 2. Configure compiler template
cd DART/build_templates
cp mkmf.template.gfortran mkmf.template
# Edit mkmf.template: set NETCDF path, compiler flags

# 3. Build the simplest model (Lorenz 63)
cd ../models/lorenz_63/work
./quickbuild.sh nompi

# 4. Quick validation
./perfect_model_obs    # generates obs_seq.out
./filter               # runs ensemble assimilation
```

### Quick Validation

After building, run `perfect_model_obs` then `filter` in `models/lorenz_63/work/`.
Filter should complete without errors. If using MATLAB, run `plot_total_err` —
the RMSE should stay around 2.0 and not grow unbounded.

### Build System

DART uses `mkmf` (a Perl script) to generate Makefiles from Fortran source
dependencies. Each model has a `work/` directory with:
- `quickbuild.sh` — automated build script
- `input.nml` — Fortran namelist configuration
- `path_names_*` — source file lists for each executable

The `quickbuild.sh` script:
1. Sets DART root directory
2. Sources `build_templates/buildfunctions.sh`
3. Runs `preprocess` to generate obs_def_mod.f90 and obs_kind_mod.f90
4. Builds all executables via mkmf + make

---

## Pipeline

DART operates through a staged pipeline. Each stage has specific executables,
input files, and output files.

| # | Stage | Executable | Input | Output | Description |
|---|---|---|---|---|---|
| 1 | Preprocess | `preprocess` | DEFAULT_obs_def_mod.F90, obs type files | obs_def_mod.f90, obs_kind_mod.f90 | Generate observation type modules |
| 2 | Build | `quickbuild.sh` | path_names_*, mkmf.template | Executables | Compile all programs |
| 3 | Obs Definition | `create_obs_sequence` | interactive / stdin | obs_seq.in (template) | Define observation types and locations |
| 4 | Obs Replication | `create_fixed_network_seq` | obs_seq.in, input.nml | obs_seq.in (time series) | Replicate obs template across time |
| 5 | Obs Conversion | obs converters | Raw data (HDF/BUFR/CSV) | obs_seq.out | Convert real observations to DART format |
| 6 | Truth Run | `perfect_model_obs` | obs_seq.in, initial state | obs_seq.out, true state | Generate synthetic observations (OSSE) |
| 7 | Assimilation | `filter` | obs_seq.out, ensemble states | analysis states, obs_seq.final | Run ensemble DA |
| 8 | Diagnostics | `obs_diag` | obs_seq.final | obs_diag_output.nc | Compute observation-space statistics |
| 9 | Visualization | MATLAB/Python | NetCDF output files | Plots | Analyze and visualize results |

**Dependencies**: Stage 1 must precede Stage 2. Stages 3-4 (synthetic) OR Stage 5
(real) feed into Stage 6 or 7. Stage 8 requires Stage 7 output.

---

## 6. Output Description

**Source**: `dag.yaml`. The dag is the source of truth for output identity,
units, validation rank, and observability. If this section ever disagrees with
`dag.yaml`, the dag wins.

**Headline output**: `analysis ensemble mean (posterior state)` — Posterior ensemble-mean estimate in the coupled host model's physical medium (atmosphere, ocean, land, ice, etc.) and state space after assimilation and posterior inflation; the primary analysis product seeding the next cycle. (`host-model state units`)

| Output variable (dag `var`) | Validation rank | Unit | Description |
|---|---:|---|---|
| `analysis ensemble mean (posterior state)` | 1 | `host-model state units` | Posterior ensemble-mean estimate in the coupled host model's physical medium (atmosphere, ocean, land, ice, etc.) and state space after assimilation and posterior inflation; the primary analysis product seeding the next cycle. |
| `analysis ensemble spread (posterior sd)` | not provided in extracted facts | see `dag.yaml` | Named dag output; read `dag.yaml` for the authoritative description. |
| `observation-space innovations and QC (obs_seq.final)` | not provided in extracted facts | see `dag.yaml` | Named dag output; read `dag.yaml` for the authoritative description. |
| `binned observation-space diagnostics (bias, rmse, totalspread)` | not provided in extracted facts | see `dag.yaml` | Named dag output; read `dag.yaml` for the authoritative description. |

The primary product that downstream agents should treat as this KI's judged
prediction is `analysis ensemble mean (posterior state)`, not a diagnostic plot
or a hand-selected variable from a coupled host model.

---

## Tools Reference

| Tool | Stage | Script | Lines | Purpose |
|---|---|---|---|---|
| convert_obs_to_dart | s5 | tools/convert_obs_to_dart.py | ~250 | Convert CSV observations to DART obs_seq text format |
| generate_input_nml | s2-s7 | tools/generate_input_nml.py | ~350 | Generate input.nml namelist for any DART program |
| run_dart | s6-s7 | tools/run_dart.py | ~200 | Execute DART programs with validation |
| parse_dart_output | s8-s9 | tools/parse_dart_output.py | ~250 | Extract NetCDF output to CSV for analysis |

---

## Critical Domain Knowledge

### #1 — Observation Sequence File Format

DART observations are stored in `obs_seq` files — a custom format that is
either ASCII (human-readable) or binary (machine-dependent). The file
contains:
- **Header**: number of copies, QC fields, observation count
- **Copy metadata**: labels like "observations", "truth", "prior ensemble mean"
- **Observations**: location, time, type, error variance, values, QC

**CRITICAL**: The error variance field stores the **variance** (sigma^2),
NOT the standard deviation. If you provide standard deviation instead of
variance, assimilation weights will be wrong — observations will be
trusted sqrt(sigma) times too much or too little.

**CRITICAL**: Time in obs_seq files is stored as (days, seconds) pairs
relative to a base calendar. The `advance_time` utility converts between
human-readable dates and DART time format.

### #2 — Namelist Configuration (input.nml)

All DART programs read from `input.nml` using Fortran namelist I/O. Key
namelists include:

| Namelist | Program | Key Parameters |
|---|---|---|
| `&filter_nml` | filter | ens_size, inf_flavor, cutoff, stages_to_write |
| `&perfect_model_obs_nml` | perfect_model_obs | obs_seq_in/out, async |
| `&preprocess_nml` | preprocess | obs_type_files, quantity_files |
| `&assim_tools_nml` | filter | cutoff, sort_obs_inc, sampling_error_correction |
| `&obs_sequence_nml` | all | write_binary_obs_sequence |
| `&model_nml` | model-specific | varies per model |
| `&location_nml` | filter | varies per location module |
| `&cov_cutoff_nml` | filter | select_localization (1=Gaspari-Cohn) |

**CRITICAL**: Fortran namelist format requires:
- Ampersand prefix: `&namelist_name`
- Slash terminator: `/`
- Boolean values: `.true.` / `.false.` (with dots)
- Strings: single-quoted `'value'`
- Arrays: comma-separated `1.0, 1.0`

### #3 — Ensemble Size and Inflation

The ensemble size (`ens_size`) controls the number of model realizations.
Typical values: 20-80 for low-order models, 40-100 for GCMs.

**Inflation** counteracts ensemble collapse (filter divergence) by
artificially increasing ensemble spread. Options (inf_flavor):
- 0: No inflation
- 2: Spatially/temporally varying (Anderson 2009) — most common
- 3: Spatially uniform, temporally varying
- 4: Relaxation to Prior Spread (RTPS) — posterior only
- 5: Enhanced with Inverse Gamma (El Gharamti 2018)

**CRITICAL**: inf_flavor is a 2-element array: `[prior, posterior]`.
Setting `inf_flavor = 2, 0` applies adaptive prior inflation only.
The inflation restart files (`prior_inf_mean.nc`, `prior_inf_sd.nc`)
must be present if `inf_initial_from_restart = .true.`.

### #4 — Localization

Localization reduces spurious long-range correlations by tapering the
ensemble covariance with distance. The `cutoff` parameter in
`&assim_tools_nml` defines the half-width of the Gaspari-Cohn function.

**CRITICAL**: The cutoff units depend on the location module:
- `oned`: fraction of unit circle [0, 1]
- `threed_sphere`: **radians** (NOT degrees). 0.2 rad ≈ 1146 km.
  To convert: cutoff_rad = cutoff_km / 6371.0
- `threed_cartesian`: same units as the coordinate system

**CRITICAL**: Setting cutoff too large (e.g., 1000000.0) effectively
disables localization. This works for small models (Lorenz 63) but
causes filter divergence for realistic models.

### #5 — Vertical Coordinate Handling

For 3D models on the sphere, observations and state variables may use
different vertical coordinates:

| which_vert | Coordinate | Units |
|---|---|---|
| -2 | Undefined | N/A (column-integrated) |
| -1 | Surface | N/A |
| 1 | Model level | Integer level number |
| 2 | Pressure | Pascals (Pa) |
| 3 | Height | Meters (m) |
| 4 | Scale height | Unitless (-ln(p/p0)) |

**CRITICAL**: Pressure must be in **Pascals**, not hPa or mb.
1 hPa = 100 Pa. If observations provide pressure in hPa,
multiply by 100 before ingestion.

### #6 — NetCDF State File Format

DART state files are NetCDF with dimensions matching the model grid.
For simple models (Lorenz), the state is a 1D vector. For complex models
(CAM, WRF), the state includes multiple variables on structured grids.

Output stages controlled by `stages_to_write`:
- `preassim`: After prior inflation, before assimilation
- `analysis`: After assimilation + posterior inflation
- `output`: Final restart state for next cycle

Each stage produces:
- `{stage}_mean.nc` — Ensemble mean
- `{stage}_sd.nc` — Ensemble standard deviation
- `{stage}_member_NNNN.nc` — Individual members (if output_members=.true.)

### #7 — Observation Quality Control

DART assigns QC values to observations during assimilation:

| QC Value | Meaning |
|---|---|
| 0 | Assimilated successfully |
| 1 | Evaluated only (not assimilated) |
| 2 | Posterior forward operator failed |
| 3 | Not used (outside time window) |
| 4 | Prior forward operator failed |
| 5 | Not used (not selected in obs_kind_nml) |
| 6 | Incoming QC rejected |
| 7 | Outlier rejected (failed outlier_threshold test) |
| 8 | Vertical conversion failed |

**CRITICAL**: A high fraction of QC > 0 usually indicates:
- Forward operator bugs (QC 4) — check model_mod interpolation
- Wrong obs locations (QC 8) — verify vertical coordinates
- Overly tight outlier threshold — increase outlier_threshold in filter_nml

### #8 — Missing Data Convention

DART uses a sentinel value for missing data:
- `MISSING_R8 = -888888.0` (double precision)
- `MISSING_R4 = -888888.0` (single precision)
- `MISSING_I = -888888` (integer)

**CRITICAL**: Do NOT use NaN, -9999, or other conventions. DART
checks for exact equality with -888888.0. Using a different sentinel
will cause silent data corruption.

### #9 — Model Async Modes

The `async` parameter in `&filter_nml` controls how filter advances the model:

| async | Mode | Description |
|---|---|---|
| 0 | Synchronous | filter calls model adv_1step() directly (Fortran subroutine) |
| 2 | Shell command | filter writes state, calls adv_ens_command, reads result |
| 4 | Parallel shell | Like 2, but launches multiple model advances |

**CRITICAL**: For built-in models (Lorenz, simple_advection), use `async = 0`.
For external models (WRF, CAM), use `async = 2` or `async = 4` with a
shell script that reads filter output, runs the model, and writes DART input.

---

## Unit Trap Table

| Variable | DART Expected | Common Source | Conversion | Trap |
|---|---|---|---|---|
| Obs error | Variance (σ²) | Std dev (σ) | Square it | Wrong assimilation weights |
| Localization cutoff | Radians (3D sphere) | Degrees or km | deg × π/180 or km/6371 | Spurious correlations |
| Pressure (vertical) | Pascals (Pa) | hPa / mb | × 100 | Wrong vertical localization |
| Longitude | Radians [0, 2π] internal | Degrees [-180, 360] | × π/180 | Wrong obs locations |
| Latitude | Radians [-π/2, π/2] internal | Degrees [-90, 90] | × π/180 | Wrong obs locations |
| Time | (days, seconds) pair | ISO datetime | advance_time utility | Time mismatch |
| Temperature | Kelvin (model-dependent) | Celsius | + 273.15 | Bias in assimilation |
| Wind speed | m/s | knots or km/h | × 0.5144 or / 3.6 | Wind obs rejected |
| Missing data | -888888.0 | NaN, -9999 | Replace with -888888.0 | Silent corruption |
| Inflation | 2-element array | Scalar | [prior, posterior] | Only prior inflated |
| Ensemble spread | From ensemble SD | From variance | sqrt(variance) | Wrong spread diagnostics |
| Rain rate | Model-dependent | mm/day vs m/day | Check model docs | 1000x error possible |

---

## 8. Unit Table

**Source**: `dag.yaml` for output units, with conversion traps summarized from
the DART workflow notes above. Do not infer physical units for coupled host
states from the word "analysis"; read the host model state metadata and the dag.

| Variable or quantity | Source unit | DART / output unit | Conversion or rule | Source / note |
|---|---|---|---|---|
| `analysis ensemble mean (posterior state)` | coupled host-model state units | `host-model state units` | No universal conversion; preserve the coupled host model's physical medium and state-space units. | `dag.yaml` rank-1 output |
| `analysis ensemble spread (posterior sd)` | ensemble variance or member states | see `dag.yaml` | If starting from variance, take `sqrt(variance)` before treating it as spread. | Existing unit trap table |
| Observation error | standard deviation may be supplied by users | DART obs error variance | Square standard deviation before writing obs_seq error variance. | Existing critical domain knowledge |
| Pressure vertical coordinate | hPa / mb are common in observations | Pascals (Pa) | Multiply hPa or mb by `100`. | Existing critical domain knowledge |
| 3D sphere localization cutoff | km or degrees are common user inputs | radians | `cutoff_rad = cutoff_km / 6371.0`; degrees require `deg * pi / 180`. | Existing critical domain knowledge |
| Longitude and latitude | degrees are common in raw observations | DART internal radians | Convert degrees with `pi / 180`. | Existing unit trap table |
| Time | ISO datetime or model time stamps | DART `(days, seconds)` pair | Use DART `advance_time`; do not hand-roll calendar offsets. | Existing unit trap table |
| Missing data | NaN, -9999, or dataset sentinel | `-888888.0` for real values, `-888888` for integers | Replace non-DART sentinels before ingestion. | Existing critical domain knowledge |

For the other dag outputs, the extracted facts list names the variables but does
not provide units. Treat `dag.yaml` as required reading before parsing
`obs_seq.final` or binned diagnostics in an automated scoring workflow.

---

## 11. Validated Results

**Source**: `docs/validation_convention.yaml` for metrics, directions, bands,
and citations. The convention is the source of truth for pass bands; null bands
are written here as `no cited threshold`, never replaced by remembered values.

### Validation Bar For Headline Output

| Dag variable | Metric | Direction | Convention band | Convention cites |
|---|---|---|---|---|
| `analysis ensemble mean (posterior state)` | `nse` | maximize | no cited threshold | `[]` |
| `analysis ensemble mean (posterior state)` | `rmse` | minimize | no cited threshold | `[]` |
| `analysis ensemble mean (posterior state)` | `csi` | maximize | no cited threshold | `[]` |
| `analysis ensemble mean (posterior state)` | `rmse` | minimize | no cited threshold | `[]` |
| `analysis ensemble spread (posterior sd)` | `pbias` | zero_centered | no cited threshold | `[]` |
| `analysis ensemble spread (posterior sd)` | `spread_error_ratio` | zero_centered | no cited threshold | `[]` |

The convention records no cited satisfactory thresholds for these bars. A run
may report NSE, RMSE, CSI, PBIAS, or spread-error ratio, but this KI must not
label the value satisfactory/good/very good unless the convention later supplies
a cited threshold.

### Lorenz 63 OSSE

**Model**: Lorenz 63 (3-variable chaotic system)
**Parameters**: σ=10.0, r=28.0, b=8/3
**Ensemble size**: 20 members
**Observations**: All 3 state variables observed every hour
**Observation error**: σ = sqrt(8.0)
**Assimilation**: EAKF, no inflation, no localization
**Duration**: 1000 assimilation cycles

| Metric | Value | Expected | Status |
|---|---|---|---|
| RMSE (total) | ~2.0 | < 4.0 | PASS |
| Filter divergence | None | None | PASS |
| Ensemble spread | ~2.0 | ~RMSE | PASS |
| Build time | < 30s | < 60s | PASS |
| Runtime | < 5s | < 30s | PASS |

### Key Findings

1. The Lorenz 63 OSSE validates that the build system, preprocess,
   perfect_model_obs, and filter are all functioning correctly.
2. RMSE stabilizes around 2.0 after initial spinup, confirming
   proper ensemble initialization and assimilation.
3. No inflation is needed for this simple model — ensemble spread
   is maintained by model nonlinearity.

---

## Calibration Parameters

For realistic models, the key tuning parameters in priority order:

| Priority | Parameter | Namelist | Range | Controls |
|---|---|---|---|---|
| 1 | ens_size | filter_nml | 20-100 | Sampling error, computational cost |
| 2 | cutoff | assim_tools_nml | 0.01-1.0 rad | Localization radius |
| 3 | inf_flavor(1) | filter_nml | 0,2,3,4,5 | Prior inflation scheme |
| 4 | inf_initial(1) | filter_nml | 1.0-1.2 | Initial inflation magnitude |
| 5 | inf_damping(1) | filter_nml | 0.5-1.0 | Inflation persistence |
| 6 | outlier_threshold | filter_nml | 2.0-5.0 | Observation rejection |
| 7 | sampling_error_correction | assim_tools_nml | .true./.false. | Small-ensemble bias fix |
| 8 | inf_flavor(2) | filter_nml | 0,4 | Posterior inflation |

---

## Data Requirements

| Data | Source | Format | Purpose |
|---|---|---|---|
| Model state | Model output | NetCDF | Ensemble initial conditions |
| Observations | Converters | obs_seq (ASCII/binary) | Assimilation input |
| NetCDF library | System package | Library | Required for I/O |
| mkmf.template | build_templates/ | Text | Compiler configuration |
| input.nml | User-created | Fortran namelist | Runtime configuration |

---

## Quick Start (Lorenz 63 OSSE)

```bash
# 1. Configure compiler
cd DART/build_templates
cp mkmf.template.gfortran mkmf.template
export NETCDF=/usr   # or wherever NetCDF is installed

# 2. Build Lorenz 63
cd ../models/lorenz_63/work
./quickbuild.sh nompi

# 3. Generate synthetic observations
./perfect_model_obs

# 4. Run ensemble assimilation
./filter

# 5. Examine output
ncdump -h preassim_mean.nc
ncdump -h analysis_mean.nc

# 6. Run observation diagnostics
./obs_diag
ncdump -h obs_diag_output.nc
```

---

## Diagnostic Triplets Summary

20 diagnostic triplets covering 6 failure domains:

| Domain | Count | Silent |
|---|---|---|
| unit_conversion | 5 | 5 |
| namelist_format | 4 | 2 |
| file_format | 3 | 2 |
| runtime | 3 | 0 |
| ensemble_config | 3 | 2 |
| observation_handling | 2 | 1 |
| **Total** | **20** | **12 (60%)** |

See `diagnostics/triplets.yaml` for full details.

---

## Supported Models (50+)

| Category | Models |
|---|---|
| Atmospheric | CAM-FV, CAM-SE, WRF, MPAS-ATM, ECHAM, LMDZ, pangu |
| Oceanic | MOM6, POP, ROMS, MITgcm, MPAS-OCN, FESOM |
| Land/Hydro | CLM, Noah, wrf_hydro, pywatershed |
| Coupled | CESM, CICE |
| Idealized | Lorenz 63/84/96, 9var, bgrid_solo, ikeda, sqg, seir |

---

## File Structure

```
ki/
├── SKILL.md                          # This file — consolidated knowledge
├── tools/
│   ├── convert_obs_to_dart.py        # CSV → obs_seq converter
│   ├── generate_input_nml.py         # Namelist generator
│   ├── run_dart.py                   # Execution wrapper
│   └── parse_dart_output.py          # NetCDF → CSV parser
├── docs/
│   ├── s1_preprocessing_and_build.md # Build system documentation
│   ├── s2_observation_preparation.md # Observation ingestion
│   ├── s3_ensemble_initialization.md # Ensemble setup
│   ├── s4_assimilation_execution.md  # Running filter
│   └── s5_diagnostics.md            # Output analysis
└── diagnostics/
    └── triplets.yaml                 # 20 symptom→diagnosis→remedy entries
```