Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
de7531b
Initial Driver System (#1335)
jlarson4 May 27, 2026
2e236da
vLLM Batches (#1338)
jlarson4 May 28, 2026
d8b471e
vLLM Driver Bugs (#1343)
jlarson4 May 28, 2026
5522c9f
Merge remote-tracking branch 'origin/dev' into dev-4.x
jlarson4 May 29, 2026
59a1a97
Pass through bridge changes
jlarson4 May 29, 2026
21a88b9
Merge remote-tracking branch 'origin/dev' into dev-4.x
jlarson4 Jun 6, 2026
91ea21d
Inspect Driver (#1367)
jlarson4 Jun 8, 2026
2113730
Merge remote-tracking branch 'origin/main' into dev-4.x
jlarson4 Jun 8, 2026
58f4f5b
Merge remote-tracking branch 'origin/dev' into dev-4.x
jlarson4 Jun 24, 2026
3e8da1e
Setup a parity script
jlarson4 Jul 6, 2026
bb5be71
Close gap that prevent falcon attention in inspect
jlarson4 Jul 6, 2026
0b20b6a
fixed some logit resolution issues,add logit reconstruction demo for …
jlarson4 Jul 6, 2026
2bb3c02
Inspect & vLLM driver Features & Bugs (#1492)
jlarson4 Jul 7, 2026
ddc3980
vLLM & Inspect Driver Correctness Improvements (#1515)
jlarson4 Jul 15, 2026
21cd02e
Multi-GPU testing for vLLM systems (#1517)
jlarson4 Jul 16, 2026
fc02adc
Merge remote-tracking branch 'origin/dev' into dev-4.x
jlarson4 Jul 20, 2026
003b20f
lock fixed
jlarson4 Jul 20, 2026
44a26ab
fix re-export
jlarson4 Jul 20, 2026
d92683d
Initial Setup for HookedTransformers Deprecation (#1538)
jlarson4 Jul 24, 2026
596e584
Comment cleanup jul 26 (#1541)
jlarson4 Jul 24, 2026
e6818b2
Merge remote-tracking branch 'origin/dev' into dev-4.x
jlarson4 Jul 28, 2026
1b2eaa4
Release cleanup 3.6.0 Release port cleanup for 4.x (#1550)
jlarson4 Jul 29, 2026
cd89db7
fix(bridge): make tokenizer assignment re-run wiring logic (#1569)
MdSadiqMd Jul 31, 2026
909c6ca
Document gradient support limits during migration (#1571)
lntutor Jul 31, 2026
aa26c17
docs: add bridge capability migration recipes (#1567)
happykawayigt Jul 31, 2026
b232ab5
fix(bridge): validate boot_native config type (#1573)
Austin1serb Jul 31, 2026
29bc91f
Test Hooked from_pretrained deprecation warnings (#1575)
noor-ahmadi Jul 31, 2026
29802d3
flaky test cleanup (#1583)
jlarson4 Jul 31, 2026
4d0ccfb
fix(bridge): add audio-specific guard for start_at_layer (#1563)
MdSadiqMd Jul 31, 2026
d70b1a4
feat(native): add param-free pre-norm (LNPre/RMSPre) and fold_ln supp…
MdSadiqMd Jul 31, 2026
b27b7c6
Honor native bridge weight initialization config (#1577)
happykawayigt Aug 3, 2026
6773425
fix(bridge): expand grouped K/V heads in QK/OV and composition circui…
TravisHaa Aug 3, 2026
84251ee
refactor(train): relocate to tools/training.py for bridge compatibili…
MdSadiqMd Aug 4, 2026
a6e0033
fix(bridge): make native TransformerBridge state_dict()/load_state_di…
LightWork666 Aug 4, 2026
b3093c4
docs: add boot_native train-from-scratch recipe to migration guide (#…
priyanka25aug Aug 4, 2026
5495284
Fix native bridge state dict round trips (#1595)
ArS377 Aug 4, 2026
29a6baf
fix(bridge): remove orphaned convert_weights override from nanogpt ad…
sohv Aug 4, 2026
3b04dc6
feat(utilities): add one-time converter for legacy TL-format checkpoi…
LightWork666 Aug 4, 2026
2cec140
fix(bridge): remove dead _enable_ht_attention and its exclusive helpe…
sohv Aug 4, 2026
2985420
Reanchoring tests and benchmarks (#1603)
jlarson4 Aug 5, 2026
f17dff3
fix(tests): un-quarantine encoder acceptance suites, sync quarantine …
sohv Aug 5, 2026
855967f
docs: demonstrate W_pos migration (#1572)
lntutor Aug 5, 2026
69f5bd2
docs: migrate executable doctests to TransformerBridge (#1576)
Austin1serb Aug 6, 2026
7ebeab9
Fix masked causal loss in TransformerBridge (#1608)
emerardd Aug 6, 2026
21993bf
fix(bridge): derive position_ids from attention_mask for left-padded …
sohv Aug 6, 2026
02d46ec
feat(bridge): support disk device_map offload targets
LightWork666 Aug 6, 2026
5fcba00
update
jlarson4 Aug 7, 2026
5abc247
fix(bridge): address disk-offload review feedback
LightWork666 Aug 7, 2026
39166fe
fix(bridge): stop_at_layer never stops on native Bridge (#1632)
LightWork666 Aug 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
8 changes: 3 additions & 5 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ jobs:
- name: Install dependencies
run: |
uv lock --check
uv sync
uv sync --extra inspect
- name: Authenticate HuggingFace
if: env.HF_TOKEN != ''
run: uv run python -c "import os; from huggingface_hub import login; login(token=os.environ['HF_TOKEN'])"
Expand Down Expand Up @@ -141,7 +141,7 @@ jobs:
- name: Install dependencies
run: |
uv lock --check
uv sync
uv sync --extra inspect
- name: MPS Availability Check
run: |
uv run python -c "
Expand All @@ -159,9 +159,7 @@ jobs:
--ignore=tests/unit/model_bridge/test_optimizer_compatibility.py
--ignore=tests/unit/model_bridge/test_gpt_oss_moe.py
--ignore=tests/unit/model_bridge/test_component_inspection.py
--ignore=tests/unit/model_bridge/test_key_analysis.py
--ignore=tests/unit/model_bridge/test_benchmark_gated_hooks_fire.py
--ignore=tests/unit/model_bridge/test_weight_processing_adapter_paths.py
--ignore=tests/unit/model_bridge/test_bridge_generate_kv_cache.py
--ignore=tests/unit/model_bridge/test_bridge_vs_hooked_transformer_patching.py
--ignore=tests/unit/model_bridge/compatibility/
Expand Down Expand Up @@ -296,7 +294,7 @@ jobs:
- name: Install dependencies
run: |
uv lock --check
uv sync
uv sync --extra inspect
- name: Authenticate HuggingFace
if: env.HF_TOKEN != ''
run: uv run python -c "import os; from huggingface_hub import login; login(token=os.environ['HF_TOKEN'])"
Expand Down
16 changes: 6 additions & 10 deletions .github/workflows/oracle-parity.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,11 @@
name: Oracle Parity

# Runs the @pytest.mark.slow oracle parity suite for JacobianLens.
# Not part of the standard PR gate — requires ~4 GB VRAM and network access.
# Triggered manually or automatically when the test file or this workflow changes.
# Not part of the standard PR gate — requires ~4 GB VRAM (or a long CPU run)
# and network access, so it is manual-trigger only. Run it after changing
# tests/integration/test_jacobian_lens_oracle_parity.py or this workflow.
on:
workflow_dispatch:
push:
branches:
- main
- dev*
paths:
- "tests/integration/test_jacobian_lens_oracle_parity.py"
- ".github/workflows/oracle-parity.yml"

permissions:
contents: read
Expand All @@ -21,6 +15,8 @@ jobs:
name: Oracle Parity Checks
runs-on: ubuntu-latest
timeout-minutes: 60
env:
HAS_HF_TOKEN: ${{ secrets.HF_TOKEN != '' }}
steps:
- uses: actions/checkout@v4

Expand All @@ -46,7 +42,7 @@ jobs:
uv pip install "git+https://github.com/anthropics/jacobian-lens.git@${ORACLE_COMMIT}"

- name: Authenticate HuggingFace
if: env.HF_TOKEN != ''
if: ${{ env.HAS_HF_TOKEN == 'true' }}
run: uv run python -c "import os; from huggingface_hub import login; login(token=os.environ['HF_TOKEN'])"
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
Expand Down
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Sub-folder rules: [tests/AGENTS.md](tests/AGENTS.md) · [supported_architectures

## 1. What this repo is

**TransformerLens** — mechanistic-interpretability library. Loads 9,000+ models across 50+ architecture families (see [supported_models.json](transformer_lens/tools/model_registry/data/supported_models.json)) and exposes internal activations through a hook system for caching, editing, and ablating intermediate state. Built on HuggingFace `transformers`.
**TransformerLens** — mechanistic-interpretability library. Loads 15,000+ models across 140+ architecture families (see [supported_models.json](transformer_lens/tools/model_registry/data/supported_models.json)) and exposes internal activations through a hook system for caching, editing, and ablating intermediate state. Built on HuggingFace `transformers`.

## 2. Two systems live in this repo

Expand All @@ -31,9 +31,9 @@ Sub-folder rules: [tests/AGENTS.md](tests/AGENTS.md) · [supported_architectures
| **`TransformerBridge`** | v3 — default for new work | [transformer_lens/model_bridge/](transformer_lens/model_bridge/) | Raw HF weights by default; `bridge.enable_compatibility_mode()` for HT-equivalent | [transformer_lens/tools/model_registry/data/supported_models.json](transformer_lens/tools/model_registry/data/supported_models.json) |
| **`HookedTransformer`** | Legacy, maintenance mode, deprecated in 3.0 | [transformer_lens/HookedTransformer.py](transformer_lens/HookedTransformer.py) + [transformer_lens/components/](transformer_lens/components/) | Folds LayerNorm + centres weights → does NOT match HF | [transformer_lens/supported_models.py](transformer_lens/supported_models.py) (**HT-only**) |

> ⚠ The **HookedTransformer acceptance suite is quarantined** ([test_hooked_transformer.py](tests/acceptance/test_hooked_transformer.py), [test_hooked_encoder.py](tests/acceptance/test_hooked_encoder.py), [test_hooked_encoder_decoder.py](tests/acceptance/test_hooked_encoder_decoder.py); see [QUARANTINES.md](tests/QUARANTINES.md)). HT changes land untested at the acceptance level — extra manual care required.
> ⚠ `HookedTransformer` has no acceptance suite of its own — `test_hooked_transformer.py` was removed by the reanchoring in #1603, which moved Bridge tests onto frozen goldens. Changes to `HookedTransformer` itself land untested at the acceptance level; extra manual care required. The `HookedEncoder` (BERT) and `HookedEncoderDecoder` (T5) acceptance suites do cover those two classes. See [QUARANTINES.md](tests/QUARANTINES.md) for the remaining skips.

Bridge architecture-adapter pattern: each HF architecture has one file in [supported_architectures/](transformer_lens/model_bridge/supported_architectures/) mapping HF module paths to canonical names. Bridge hooks are architecture-native (e.g. `blocks.{i}.hook_out`); HT-style aliases live in [bridge.py](transformer_lens/model_bridge/bridge.py).
Bridge architecture-adapter pattern: each HF architecture has one file in [supported_architectures/](transformer_lens/model_bridge/supported_architectures/) mapping HF module paths to canonical names. Bridge hooks are architecture-native (e.g. `blocks.{i}.hook_out`); HT-style aliases live in [bridge_core.py](transformer_lens/model_bridge/bridge_core.py).

**Mirroring rule:** if you change `HookedTransformer` behaviour that has a `TransformerBridge` counterpart, update both in the same PR. [supported_models.py](transformer_lens/supported_models.py) is HT-only — Bridge-only models go in the Bridge registry data file.

Expand Down Expand Up @@ -107,7 +107,7 @@ Python: **>=3.10, <4.0**. CI tests 3.10, 3.11, 3.12. Format/type/docstring check
## 5. Hook naming — HT vs Bridge

- **HT canonical**: uniform across architectures — `hook_embed`, `blocks.{i}.hook_resid_pre`, `blocks.{i}.attn.hook_q`, `blocks.{i}.hook_resid_post`.
- **Bridge-native**: architecture-shaped — `blocks.{i}.hook_out`, `blocks.{i}.attn.q.hook_out`. HT aliases registered via `build_alias_to_canonical_map()` in [bridge.py](transformer_lens/model_bridge/bridge.py).
- **Bridge-native**: architecture-shaped — `blocks.{i}.hook_out`, `blocks.{i}.attn.q.hook_out`. HT aliases registered via `build_alias_to_canonical_map()` in [bridge_core.py](transformer_lens/model_bridge/bridge_core.py).

Prefer Bridge-native names in new code. Raw-HF-forward drivers comparing against `boot_transformers` must match its load configuration (fp32, eager attention) and probe for optional features like `resid_mid` rather than assume.

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ interpretability](https://distill.pub/2020/circuits/zoom-in/) of GPT-2 Style lan
goal of mechanistic interpretability is to take a trained model and reverse engineer the algorithms
the model learned during training from its weights.

TransformerLens lets you load in 9,000+ open source language models across 50+ architecture families,
TransformerLens lets you load in 15,000+ open source language models across 140+ architecture families,
and exposes the internal activations of the model to you. You can cache any internal activation in the
model, and add in functions to edit, remove or replace these activations as the model runs.

Expand Down Expand Up @@ -52,7 +52,7 @@ logits, activations = bridge.run_with_cache("Hello World")

> Gated models (Llama, Mistral, Gemma, ...) require `HF_TOKEN` in your environment. See [Environment Variables](https://TransformerLensOrg.github.io/TransformerLens/content/getting_started.html#environment-variables) for the full list.

`TransformerBridge` is the recommended 3.0 path and supports 9,000+ models across 50+ architecture families (see [`supported_models.json`](transformer_lens/tools/model_registry/data/supported_models.json) for the full inventory). By default it preserves raw HuggingFace weights – logits and activations match HF, *not* legacy `HookedTransformer` (which folds LayerNorm and centers weights by default). Call `bridge.enable_compatibility_mode()` after booting for HookedTransformer-equivalent numerics. The legacy `HookedTransformer.from_pretrained` API is still available but deprecated — see the [Migrating to TransformerLens 3](https://TransformerLensOrg.github.io/TransformerLens/content/migrating_to_v3.html) guide.
`TransformerBridge` is the recommended path and supports 15,000+ models across 140+ architecture families (see [`supported_models.json`](transformer_lens/tools/model_registry/data/supported_models.json) for the full inventory). By default it preserves raw HuggingFace weights – logits and activations match HF, *not* legacy `HookedTransformer` (which folds LayerNorm and centers weights by default). Call `bridge.enable_compatibility_mode()` after booting for HookedTransformer-equivalent numerics. The legacy `HookedTransformer.from_pretrained` API is still available but deprecated — see the [Migrating to TransformerLens 3](https://TransformerLensOrg.github.io/TransformerLens/content/migrating_to_v3.html) guide.

## Key Tutorials

Expand Down
Loading