Agentic test data harness for Python — deterministic in CI, AI-powered in dev.
# ❌ Everyone does this. It's brittle and misses real-world edge cases.
user = User(name="Test User", email="test@test.com", bio="Lorem ipsum...")
# ❌ factory_boy is great — but it's static. No surprises, no edge cases.
UserFactory.create(role="admin")
# ❌ Writing 500 of them by hand? Not happening.Hardcoded fixtures rot. AI-generated fixtures are unpredictable in CI.
FixtureForge solves both — same codebase, two behaviors:
Dev mode → AI generates rich, realistic, edge-case-aware fixtures
CI mode → same fixtures, frozen with seed=42, 100% reproducible
pip install fixtureforgefrom fixtureforge import Forge
from pydantic import BaseModel
class User(BaseModel):
id: int
name: str
email: str
bio: str
forge = Forge() # auto-detects AI provider from env vars
users = forge.create_batch(User, count=50, context="SaaS platform users")FixtureForge routes each field to the cheapest generator automatically:
id→ sequential counter (free)name,email→ Faker (free)bio→ single batched AI call for all 50 records (1 API call, not 50)
No AI key? No problem. Pure Faker mode works out of the box:
forge = Forge(use_ai=False, seed=42) # deterministic, zero network, CI-safe
users = forge.create_batch(User, count=500)Every field is classified into a tier. Only semantic fields hit the AI:
| Tier | Fields | Generator | Cost |
|---|---|---|---|
| Structural | id, user_id, order_id |
Counters + FK registry | Free |
| Standard | name, email, phone, address |
Faker | Free |
| Computed | @computed_field |
Pydantic | Free |
| Semantic | bio, description, review, message |
LLM (batched) | API tokens |
100 users with 2 semantic fields = 2 API calls, not 200.
# .github/workflows/test.yml
- name: Run tests
env:
FORGE_SEED: 42 # identical output every run
# No AI key needed — FixtureForge auto-detects and falls back to Faker
run: pytestIn dev, export any provider key and AI kicks in automatically:
export ANTHROPIC_API_KEY=sk-ant-... # → Claude
export OPENAI_API_KEY=sk-... # → GPT
export GOOGLE_API_KEY=... # → Gemini
export GROQ_API_KEY=... # → Groq (fast + cheap)# Step 1: generate customers
customers = forge.create_batch(Customer, count=10)
# Step 2: Order.customer_id auto-resolves to a real customer.id
orders = forge.create_batch(Order, count=100)
# → every order.customer_id is valid. No manual wiring.Generate multiple models in parallel. The first model warms the AI cache;
every subsequent model inherits it — ~90% cheaper per additional model.
results = forge.swarm(
models=[User, Order, Product, Payment],
counts=[10, 50, 100, 30],
contexts=["SaaS users", "E-commerce orders", None, None],
)
# {
# "User": [...10 users...],
# "Order": [...50 orders...],
# "Product": [...100 products...],
# "Payment": [...30 payments...],
# }5 models ≈ cost of 1.5 models.
forge.memory.add_rule("financial", "Users under 18 get restricted account type")
forge.memory.add_rule("user", "Israeli phone numbers use format 05x-xxx-xxxx")
forge.memory.add_rule("orders", "Max 3 active loans per customer at any time")
# Rules inject into AI prompts automatically on every generation call
users = forge.create_batch(User, count=50, context="Israeli SaaS platform")Rules survive across sessions. Update a rule — next call respects it immediately.
Skeptical Memory validates stored rules against the live schema before every call.
# conftest.py
from fixtureforge.pytest_plugin import forge_fixture, forge_swarm_fixture
from pydantic import BaseModel
class User(BaseModel):
id: int; name: str; email: str
forge_fixture(User, count=10, seed=42) # → fixture: "users"
forge_swarm_fixture([User, Order], counts=[5, 20], seed=42) # → "swarm_data"# test_users.py
def test_signup(users):
for user in users:
assert "@" in user.email
def test_full_flow(swarm_data):
users = swarm_data["User"]
orders = swarm_data["Order"]The forge fixture is auto-available in every test with zero config.
# Be explicit
forge = Forge(provider_name="anthropic", model="claude-haiku-4-5-20251001")
forge = Forge(provider_name="openai", model="gpt-4o-mini")
forge = Forge(provider_name="gemini", model="gemini-2.0-flash")
forge = Forge(provider_name="groq", model="llama-3.3-70b-versatile")
forge = Forge(provider_name="ollama", model="llama3.2") # local, zero cost
forge = Forge(use_ai=False) # pure Faker# Seed + Interpolation: generates ~1 000 unique AI values, tiles to 100 000
dataset = forge.create_large(Order, count=100_000, seed_ratio=0.01)
# Streaming — one record at a time, never loads all into memory
for user in forge.create_stream(User, count=1_000_000, filename="users.json"):
process(user)from fixtureforge.core.exporter import DataExporter
users = forge.create_batch(User, count=100)
DataExporter.export(users, "users.json") # format inferred from extension
DataExporter.export(users, "users.csv")
DataExporter.export(users, "users.sql")| FixtureForge | factory_boy | Faker | hypothesis | |
|---|---|---|---|---|
| AI-powered context | ✅ | ❌ | ❌ | ❌ |
| Deterministic (seed=) | ✅ | ✅ | ✅ | ✅ |
| FK relationships | Auto | Manual | ❌ | ❌ |
| Batched AI calls | ✅ | ❌ | ❌ | ❌ |
| Coverage gap analysis | ✅ | ❌ | ❌ | Partial |
| Large datasets (100k+) | ✅ | Manual | Manual | ❌ |
| pytest plugin | ✅ | ❌ | ❌ | ❌ |
| Multi-LLM support | ✅ | ❌ | ❌ | ❌ |
| Permission gates | ✅ | ❌ | ❌ | ❌ |
| CI-safe (zero network) | ✅ | ✅ | ✅ | ✅ |
FixtureForge is not a replacement for Faker — it uses Faker internally for standard fields.
It adds the layer between "I need realistic data" and "I need it to feel like production."
# Core (deterministic mode, no AI)
pip install fixtureforge
# With your preferred provider
pip install "fixtureforge[anthropic]" # Claude
pip install "fixtureforge[openai]" # GPT
pip install "fixtureforge[gemini]" # Gemini
pip install "fixtureforge[all]" # All providersRequirements: Python 3.11+ · pydantic ≥ 2.5 · faker ≥ 22.0
For teams with compliance requirements — GDPR, SOC2, HIPAA, multi-tenant SaaS:
| Community | Enterprise | |
|---|---|---|
| AI generation + Faker | ✅ | ✅ |
| pytest plugin | ✅ | ✅ |
| Deterministic seeding | ✅ | ✅ |
| Cryptographic Provenance Envelope | ❌ | ✅ |
| PII Airgap — fail-closed scanner | ❌ | ✅ |
| Contextual Tenant Enclaves | ❌ | ✅ |
| Cross-tenant FK violation detection | ❌ | ✅ |
| Presidio / custom scanner support | ❌ | ✅ |
from fixtureforge.enterprise import ForgeEnterprise
forge = ForgeEnterprise(use_ai=False)
users = forge.create_batch(User, count=10)
users[0].model_dump()
# { "id": 1, "name": "...", "bio": "...",
# "forge_metadata": {
# "forge_id": "abc-123",
# "provenance_hash": "sha256:029773ed...", ← immutable audit stamp
# "tenant_id": "tenant-acme",
# "source": "faker",
# ...
# }
# }
with forge.isolate_tenant("tenant-acme"):
acme_users = forge.create_batch(User, count=5)
# FK references from tenant-acme can NEVER resolve to tenant-xyz recordsAccess: yaniv2809@gmail.com
| Component | Status |
|---|---|
Core (Forge, create_batch) |
✅ Stable |
| DataSwarms | ✅ Stable |
| ForgeMemory | ✅ Stable |
| pytest plugin | ✅ v2.3.0 |
| Anthropic / OpenAI / Gemini / Groq / Ollama | ✅ |
| Literal / Enum / date / datetime / UUID / Decimal | ✅ v2.2.0 |
MCP Server (fixtureforge-mcp) |
✅ v2.3.0 |
assert_semantic_match |
✅ v2.2.0 |
SmartFailureAnalyzer |
✅ v2.2.0 |
| Enterprise Edition | ✅ (access by request) |
| ForgeDream (coverage analysis) | 🔜 Feature-flagged |
| Async support | 🔜 Planned |
- Docs: https://yaniv2809.github.io/fixtureforge/
- PyPI: https://pypi.org/project/fixtureforge/
- Repository: https://github.com/Yaniv2809/fixtureforge
- Issues: https://github.com/Yaniv2809/fixtureforge/issues
- Discussion: #1
FixtureForge ships a built-in MCP server so Claude Desktop and other MCP-capable AI tools can generate test data directly in a conversation.
pip install "fixtureforge[mcp]"{
"mcpServers": {
"fixtureforge": {
"command": "fixtureforge-mcp"
}
}
}Restart Claude Desktop — FixtureForge appears in the tool list automatically.
| Tool | What it does |
|---|---|
generate |
Generate N records for a model described by {field: type} |
swarm |
Generate multiple models in parallel with shared AI cache |
add_rule |
Persist a domain rule injected into every AI prompt |
export |
Generate records and save to .json, .csv, or .sql |
"Use fixtureforge to generate 20 User records with id, name, email and bio fields, then export them to users.json"
FixtureForge ships an official agent skill so AI coding assistants (Claude Code, Cursor, GitHub Copilot, Devin, etc.) know exactly how to use the library.
The skill lives at skills/fixtureforge/SKILL.md
and covers: when to use FixtureForge, install & setup, core API, CI determinism,
FK wiring, pytest plugin, common mistakes, and provider names.
Agents working in a repo that uses FixtureForge will automatically pick up the
skill and avoid the most common mistakes (hardcoded generators, manual FK wiring,
use_ai=True in CI).
Issues and PRs welcome.
git clone https://github.com/Yaniv2809/fixtureforge
cd fixtureforge
pip install -e ".[dev]"
PYTHONPATH=src python -m pytest tests/MIT © Yaniv2809
If FixtureForge saved you time, give it a ⭐ — it helps others find it.
