High Rank Connectivity Scaffolds Rnn

---
name: high-rank-connectivity-scaffolds-rnn
description: Use when analyzing high-rank RNN connectivity scaffolds.
category: ai_collection
trigger_words: [low-rank RNN, high-rank connectivity, singular value decomposition, path integration, recurrent neural network, mode decomposition, core scaffold, generalisation, error correction, latent dynamics]
arxiv: 2609.35207
---

# High-Rank Connectivity Scaffolds in Recurrent Neural Networks

**Source**: Hawes & Nolan (University of Edinburgh), arXiv:2609.35207 (28 Sep 2026), q-bio.NC.
Full paper: https://arxiv.org/abs/2609.35207

## Core Claim

Low-rank connectivity generates the dominant low-dimensional task manifold, but **high-rank connectivity modes are NOT residual complexity** — they form a distributed "scaffold" that corrects errors in the low-dimensional representation, and are *necessary* for behavioural precision and generalisation to novel input distributions.

**Core/scaffold decomposition**: SVD of the recurrent weight matrix W_hh yields unit-rank modes ordered by singular value. Leading modes (1-2) = core (location-tracking manifold); later weak modes = scaffold (error correction).

## Key Experimental Facts (1D path-integration task)

RNN agents trained with PPO on a linear-track location memory task (mice-analogue protocol, MEC-dependent):

| Reconstruction (rank k) | Training speed dist. | Novel speed dist. (0.5x / 2x) |
|---|---|---|
| k = 5 | >75% reward, R²>0.65 decode | 0% uncued reward for k<128 |
| k ≥ 224 | >99% reward | first-stop accuracy restored |
| Full rank | >99% | >99% |

- First 2 singular values average 5.2x larger than the rest (skewed spectrum), yet full-rank is functionally required.
- Modes 1-2: strong location info, no speed info; their left singular vectors match PC1/PC2 axes (cosine similarity ~1). Modes >2: carry speed + partial location info.
- **LDAC (Low-Dimensional Activity Clamp)**: use PCA invertibility to project modified PC values back to neural space. Clamping PC1/2 to manifold values for specific track locations *causally* directs stopping behaviour (stops everywhere / nowhere). >99% performance needs first 15 PCs on training distribution; novel distributions need >35 PCs.
- PC6 explains only 4.1% variance but clamping it cuts uncued reward by 93% — variance explained ≠ causal importance.
- Fixed-point structure: point attractor ahead of position during movement; line attractor preserving location during stops.

## The Error-Correction Mechanism (the scaffold's job)

1. Single-step speed perturbation → modes 1/2 deviate from expected location value (overshoot/undershoot, location-dependent).
2. Full-rank network corrects the error at the *next* timestep.
3. Rank-2 network cannot correct: error persists.
4. **Mode-to-mode connectivity** (M^T W_hh M in left-singular-vector coordinates): strong self-recurrence in modes 1-2 (location memory); weak distributed inputs from later modes onto modes 1-2 (correction).
5. Causal test: zeroing self-recurrence of mode 1 or 2 → 0% uncued reward. Zeroing later→mode1/2 inputs → 0% uncued reward. Zeroing later-mode self-recurrence or inputs onto later modes → no effect.

## 2D Generalisation

Context-dependent 2D navigation: rank 16 suffices for cued reward; rank 64 for uncued; but location decoding R²<0.7 and navigation efficiency remain impaired until high-rank modes restored. Same perturbation-correction signature (errors >1.0 activity units at rank 2 vs <0.2 full rank).

## Why It Matters

- **Reconciles population-level and circuit-level views**: same neurons/synapses implement both the low-dim core and high-rank scaffold — roles separated in *connectivity mode space*, not anatomically.
- Low-dimensional activity does NOT imply a low-complexity circuit. Neural heterogeneity can serve robustness without altering dominant latent dynamics.
- Manifold-level explanations are incomplete: neurons with similar PC loadings differ markedly in causal influence; <2% neuron populations selected by behavioural effect predictably shift stopping, PC-loading-selected populations need to be much larger.

## Reusable Methods

### 1. Rank-truncation generalisation test
```python
U, S, Vt = np.linalg.svd(Whh)
def rank_k(Whh, U, S, Vt, k):
    return U[:, :k] @ np.diag(S[:k]) @ Vt[:k, :]
# Evaluate: training-dist performance vs novel-dist performance as f(k)
# The k-gap between the two curves = scaffold contribution
```

### 2. LDAC causal manipulation
```python
# project activity to PC space, zero/clamp selected PCs, invert back
pcs = (h - mean) @ loadings          # loadings: neurons x PCs
pcs[:, clamp_idx] = target_values
h_clamped = pcs @ loadings.T + mean  # back to neural space
```

### 3. Mode-space connectivity
```python
M = U  # stack left singular vectors
W_mode = M.T @ Whh @ M   # rows/cols = modes
# diagonal = mode self-recurrence; off-diagonal = inter-mode drive
```

### 4. Single-step perturbation recovery test
Perturb input one step, measure |mode activity − expected| at t+1 for rank-k vs full rank. Full rank recovers; low rank does not → scaffold carries correction.

## Applicability Triggers

- Any trained-RNN analysis using low-rank truncation (SVD rank-k reconstruction) to claim a mechanism — check generalisation before concluding low rank suffices.
- Continuous integration tasks: path integration, evidence accumulation, perceptual estimation, oculomotor control, motor timing (error-accumulation regime).
- Interpreting MEC/hippocampal models where grid/ramp tuning emerges from training.
- Model-based neuroscience debates: "low-dimensional dynamics = low-rank circuit" inference is invalid in precision-demanding tasks.

## Related Skills (contrast/extension)

- [[low-rank-rnn-learning-dynamics]] — framework this paper extends beyond simple tasks
- [[fixed-point-compositionality-low-rank-gluing]] — compositional low-rank gluing
- [[dynamical-alignment-snn-paradox-resolution]], [[gain-vs-off-manifold-decomposition]] — related perturbation analyses

## Limitations (from authors)

- Open why RL converges to superimposed low+high-rank solutions.
- Biological verification requires perturbing specific connectivity modes (mode-specific optogenetics), not just latent-space observation.
- Tested on RL-trained unconstrained RNNs; biologically constrained models untested.