diff --git a/docs/plans/skill-format.mdx b/docs/plans/skill-format.mdx index 175ad76f74..656bc8360d 100644 --- a/docs/plans/skill-format.mdx +++ b/docs/plans/skill-format.mdx @@ -611,6 +611,7 @@ skills lane: gaia skill search "web search" [--json] [--hub-url URL] gaia skill install [--allow-experimental] [--force] [--yes] gaia skill remove +gaia skill lock [--check|--relock] [--json] gaia skill publish ./my-skill/ [--unsigned] [--audit-report FILE] [--dry-run] gaia skill keygen [--key-name NAME] [--force] gaia skill trust {list|add [--role publisher|amd]|remove } @@ -619,7 +620,18 @@ gaia skill trust {list|add [--role publisher|amd]|remove } `install` accepts a SemVer range and takes the highest published version satisfying it — the same rule agent `dependencies:` use. Installs are recorded in `~/.gaia/skills/skill-lock.json` with the requested range, the resolved version, -the artifact SHA-256, the signing key, and the tier actually installed at. +the artifact SHA-256, a digest of the installed files, the signing key, and the +tier actually installed at. + +`lock` reads that record back: `--check` (the default) reports version, tier, +content, and presence drift between the lock and the disk, exiting `4` when +anything differs; `--relock` re-records the current state after an intentional +change. A `community` / `verified` skill whose files drifted refuses to load and +cannot be relocked — those bytes are not the ones its signature covered. Nor can +one with no digest recorded yet (an entry from a pre-digest build): stamping the +current bytes there would assert a signature nothing checked, so those reinstall +instead. An unreadable lock refuses **every** user-root skill, local ones +included — it cannot say which are attested. **Still PROPOSED** — `update` (marketplace): @@ -1473,6 +1485,7 @@ Only the sandbox/run-time-enforcement and REST/UI layers are greenfield; the mar | Hub catalog lane (server side) | `skill` catalog `type`, `POST /publish/skill`, `skills/` R2 prefix, SKILL.md publish validation, audit gate | `workers/agent-hub/src/` | **Exists** (#2467) | | **Project-local `./.gaia/skills/` root** | — | **NOT FOUND** | **Deferred — PROPOSED** | | **Registry lock (`skill-lock.json`)** | `SkillLock`, `LockEntry` | `skills/lock.py` | **Exists** (#2467) | +| **Lock drift detection + relock** | `check_drift()`, `assert_no_fatal_drift()`, `relock()`, `gaia skill lock` | `skills/drift.py` | **Exists** (#2467) | --- diff --git a/docs/reference/cli.mdx b/docs/reference/cli.mdx index e90bca0c40..1978c1f5aa 100644 --- a/docs/reference/cli.mdx +++ b/docs/reference/cli.mdx @@ -2022,7 +2022,7 @@ Author and manage agent skills. A skill is a directory whose only required file gaia skill {list,info,create,import,export,migrate} [OPTIONS] # author locally gaia skill deltas [OPTIONS] # what an agent LEARNED gaia skill audit [OPTIONS] # pre-publish gate -gaia skill {search,install,remove,promote,publish,keygen,trust} [OPTIONS] # marketplace + trust +gaia skill {search,install,remove,lock,promote,publish,keygen,trust} [OPTIONS] # marketplace + trust ``` Skills are discovered from three roots, highest precedence first: @@ -2213,7 +2213,21 @@ Skills are discovered from three roots, highest precedence first: NAME VERSION TIER TOOLS DESCRIPTION web-research 1.2.0 community 1 Search the web and summarize results incident-review 0.3.1 experimental 0 Structure a postmortem from an incident log + + Catalog: hub, fetched just now + ``` + + The trailing line is the catalog's age. Offline still works — GAIA serves the + cached catalog rather than refusing — but it says how old that copy is, and + warns loudly once it passes a week: + + ``` + ⚠ This catalog was last refreshed 93 days ago — over 7 days old. The hub is + unreachable, so skills published since are missing and skills unpublished + since are still listed, including any whose security tier changed. ``` + + `--json` carries the same facts as `age_seconds`, `age_text`, and `stale`. @@ -2239,6 +2253,73 @@ Skills are discovered from three roots, highest precedence first: gate passes. + + ```bash + gaia skill lock [--check | --relock] [--json] + ``` + + | Option | Description | + |--------|-------------| + | `--check` | Report drift, change nothing (the default) | + | `--relock` | Re-record the current state as intended | + | `--json` | Emit JSON instead of text | + + Compares every skill in `~/.gaia/skills` against `skill-lock.json` — version, + the tier install enforced, and a digest of the files — and reports skills the + lock has lost track of plus skills it tracks that are gone. Exits `4` when + anything differs, `0` when the lock and the disk agree. + + ``` + ⚠ 2 difference(s) between ~/.gaia/skills and skill-lock.json (1 blocking, 1 advisory): + + web-research + ✗ [content] The files in ~/.gaia/skills/web-research no longer hash to the + digest recorded at install — something was added, removed, or edited. + → Reinstall the attested copy with 'gaia skill install web-research --force'. + my-notes + ⚠ [untracked] 'my-notes' is installed but skill-lock.json does not track it. + → Expected if you created, imported, or migrated it. + ``` + + Untracked skills are normal right after `create` / `import` / `migrate`; + `--relock` starts tracking them as `source: local`. + + + **After upgrading from a build that predates content digests**, skills + installed by the older build have nothing to compare against. They report as + *unverifiable*, not as changed — `gaia skill list` says so in those words and + still exits `0`: + + ``` + ⚠ 2 skill(s) in ~/.gaia/skills predate content digests, so skill-lock.json + cannot verify them: + ``` + + `gaia skill lock` still exits `4` here: it is an explicit verification verb, + and "could not verify" is not "verified clean". Don't gate CI on it without + running `--relock` first. Locally-authored and `experimental` skills are + recorded by that one command; a `community` / `verified` one needs + `gaia skill install --force` instead (see the warning below). + + A **corrupt or future-schema** `skill-lock.json` is different: it is reported + alongside the `gaia skill list` inventory (exit `4`), and every skill in + `~/.gaia/skills` refuses to load until the file is fixed or deleted — + including purely local ones. Nothing can tell an attested skill from a local + one while the lock is unreadable, so loading anyway would let corrupting one + file switch the check off. + + + + `--relock` **refuses** to re-record a `community` or `verified` skill whose + files changed, **or one that has no recorded digest at all**. Writing a digest + in either case would leave the lock asserting a publisher signature over bytes + that signature never covered — and the no-digest case is where every upgrade + from a pre-digest build starts, so it is the likeliest way to launder an + attestation, not the rarest. Reinstall it, or `gaia skill remove` then + `gaia skill import` to keep the edit at `experimental`. + + + ```bash gaia skill promote @@ -2340,8 +2421,8 @@ stamp is a provenance claim, not a runtime sandbox. Hub installs are tracked in `~/.gaia/skills/skill-lock.json`. Discovery answers *what is on disk*; the lock answers what was **requested**, what **resolved**, what -the artifact **hashed** to, who **signed** it, and which tier it was installed at -after the ceiling was applied: +the artifact and the installed files **hashed** to, who **signed** it, and which +tier it was installed at after the ceiling was applied: ```json { @@ -2356,6 +2437,7 @@ after the ceiling was applied: "hub_url": "https://hub.amd-gaia.ai", "artifact_sha256": "9f2c…", "artifact_filename": "web-research-1.2.0.zip", + "content_digest": "sha256:4b1d…", "claimed_tier": "community", "attested_tier": "community", "installed_tier": "community", @@ -2373,8 +2455,25 @@ after the ceiling was applied: } ``` -Only hub installs are tracked — a skill from `create` or `import` has no hub -provenance to record. +`gaia skill install` writes `source: "hub"` entries; a skill from `create` or +`import` has no hub provenance, so install never invents one. `gaia skill lock +--relock` may record it as `source: "local"`, which claims exactly the origin it +has. + +`content_digest` covers the **installed** directory (after the tier re-stamp), not +the downloaded archive. That is what makes post-install tampering detectable: +`gaia skill lock` compares against it, and a `community` / `verified` skill whose +files no longer match **refuses to load** — those bytes are not the ones the +signature covered, so the tier badge no longer means anything. An `experimental` +or locally-authored skill warns and keeps working; nothing attested it, and +editing a skill you wrote is the normal workflow. + + +`skill-lock.json` has no integrity protection of its own — anyone who can write +to it can rewrite a digest. It defends against tampering with the *skill +directory* (a swapped bundle, an edited manifest, a downgraded version), which +until now was entirely silent. + **Examples:** @@ -2415,6 +2514,12 @@ gaia skill install some-skill --allow-experimental gaia skill remove web-research ``` +```bash Verify nothing changed since install +gaia skill lock # exits 4 if anything drifted +gaia skill lock --json | jq '.drifts[] | select(.fatal)' +gaia skill lock --relock # accept your own edits +``` + ```bash Publish your own gaia skill keygen # once, per machine export GAIA_HUB_TOKEN=... diff --git a/docs/spec/agent-skills.mdx b/docs/spec/agent-skills.mdx index 99c056ad7f..a1a75d234b 100644 --- a/docs/spec/agent-skills.mdx +++ b/docs/spec/agent-skills.mdx @@ -246,7 +246,12 @@ The user root is **also** the registry-lock root: `gaia skill install` writes `~/.gaia/skills/skill-lock.json` beside the skills it installs ([#2467](https://github.com/amd/gaia/issues/2467)). The lock is provenance, not a fourth discovery root — discovery still reads the directory, and a skill created -or imported locally simply has no lock entry. +or imported locally has no lock entry until `gaia skill lock --relock` records one +as `source: local`. Loading a skill from the user root does check it against the +lock: a `community` / `verified` install whose files changed since is refused, +because those are not the bytes its signature covered. If the lock itself is +unreadable, **every** user-root skill is refused — including local ones — since +nothing can tell an attested skill from a local one at that point. ### Progressive disclosure @@ -691,7 +696,9 @@ immutability rules ([#2467](https://github.com/amd/gaia/issues/2467)). resolves a SemVer range against the catalog, verifies the artifact checksum and the bundle signature, enforces the tier ceiling, and records the outcome in `~/.gaia/skills/skill-lock.json` ([#2467](https://github.com/amd/gaia/issues/2467), - [#647](https://github.com/amd/gaia/issues/647)); publish the other direction with + [#647](https://github.com/amd/gaia/issues/647)) — including a digest of the + installed files, which `gaia skill lock` compares against later; publish the + other direction with `gaia skill publish`. Off-hub, `gaia skill import ` copies a skill in (always stamped `experimental` — no hub provenance to trust) and `gaia skill export ` produces the bundle. See diff --git a/src/gaia/agents/base/agent.py b/src/gaia/agents/base/agent.py index 605f65b404..5253106b59 100644 --- a/src/gaia/agents/base/agent.py +++ b/src/gaia/agents/base/agent.py @@ -2161,11 +2161,17 @@ def load_skill( Raises: SkillNotFoundError: no discovery root contains that skill. SkillValidationError: the manifest is invalid or contradicts - ``tools.py``. Nothing is registered. + ``tools.py``, or ``~/.gaia/skills/skill-lock.json`` is unreadable + — an unreadable lock cannot tell an attested skill from a local + one, so no user-root skill loads until it is fixed or deleted. + Nothing is registered. SkillPermissionError: the skill declares a local-capability permission (``filesystem``/``shell``/``database``/``desktop``/ ``env``), which this phase refuses rather than loading unenforced. + SkillDriftError: the skill was installed from the hub at + ``community`` / ``verified`` and its files no longer match + ``skill-lock.json`` — the signature covered different bytes. Example: class WebAgent(Agent): diff --git a/src/gaia/hub/catalog.py b/src/gaia/hub/catalog.py index 1df424aaad..37f33daae0 100644 --- a/src/gaia/hub/catalog.py +++ b/src/gaia/hub/catalog.py @@ -23,6 +23,11 @@ stays usable offline — and every offline path is flagged (`offline=True`) rather than hidden. Installing from the hub still fails loudly (you cannot pull an artifact from an unreachable hub). + +Offline is supported; pretending offline data is current is not. The disk cache +records when it was last refreshed, and every result carries ``age_seconds`` +plus a ``stale`` flag once :data:`CACHE_STALE_AFTER_SECONDS` has passed, so a +caller can say "3 months old" instead of rendering it as today's catalog. """ from __future__ import annotations @@ -46,6 +51,16 @@ # discover panel opens; 5 minutes keeps it fresh without hammering R2. CACHE_TTL_SECONDS = 300 +# How old the on-disk cache may get before serving it is worth a warning. The +# hub rewrites index.json on every publish, so a week without a successful fetch +# means new skills are invisible, unpublished ones still listed, and a +# security-tier change unseen. Offline still works — it just says how old it is. +CACHE_STALE_AFTER_SECONDS = 7 * 24 * 60 * 60 + +# When the disk cache was last refreshed from the network, stamped into the +# cached document. Underscore-prefixed so it cannot collide with a hub field. +CACHE_STAMP_KEY = "_gaia_cached_at" + # HTTP timeout for catalog fetches (seconds). Short — the catalog is small and # the offline cache covers a slow/absent network. _HTTP_TIMEOUT = 10 @@ -166,7 +181,10 @@ def clear_cache() -> None: def _write_disk_cache(cache_path: Path, data: Dict[str, Any]) -> None: try: cache_path.parent.mkdir(parents=True, exist_ok=True) - cache_path.write_text(json.dumps(data, indent=2), encoding="utf-8") + # Stamp a copy: the caller keeps the hub's document, and the in-memory + # cache never picks up a field the hub did not send. + stamped = {**data, CACHE_STAMP_KEY: _utc_now_iso()} + cache_path.write_text(json.dumps(stamped, indent=2), encoding="utf-8") except OSError as exc: # Cache write failure must not break a successful live fetch; log it. logger.warning("catalog: could not write cache %s: %s", cache_path, exc) @@ -182,6 +200,70 @@ def _read_disk_cache(cache_path: Path) -> Optional[Dict[str, Any]]: return None +def _utc_now_iso() -> str: + from datetime import datetime, timezone + + return datetime.now(timezone.utc).isoformat(timespec="seconds") + + +def _parse_stamp(value: Any) -> Optional[float]: + """Epoch seconds for an ISO-8601 stamp, or ``None`` if it is unusable.""" + from datetime import datetime, timezone + + if not isinstance(value, str) or not value: + return None + try: + parsed = datetime.fromisoformat(value) + except ValueError: + logger.warning("catalog: cache stamp %r is not an ISO-8601 time", value) + return None + if parsed.tzinfo is None: + parsed = parsed.replace(tzinfo=timezone.utc) + return parsed.timestamp() + + +def cache_age_seconds( + cache_path: Optional[Path] = None, + cached: Optional[Dict[str, Any]] = None, +) -> Optional[float]: + """How long ago the disk cache was last refreshed from the network. + + Reads the stamp this module writes. A cache written by an older GAIA has no + stamp, so its file mtime is used instead — a real measurement of the same + fact, not a guess, and the only alternative is claiming an unknown age is + fresh. ``None`` means there is no cache at all. + """ + path = Path(cache_path) if cache_path else default_cache_path() + document = cached if cached is not None else _read_disk_cache(path) + if document is None: + return None + + stamped = _parse_stamp(document.get(CACHE_STAMP_KEY)) + if stamped is not None: + return max(0.0, time.time() - stamped) + + try: + return max(0.0, time.time() - path.stat().st_mtime) + except OSError as exc: + logger.warning("catalog: could not stat cache %s: %s", path, exc) + return None + + +def describe_age(age_seconds: Optional[float]) -> str: + """``age_seconds`` as something a person reads ("3 days ago").""" + if age_seconds is None: + return "unknown" + if age_seconds < 90: + return "just now" + minutes = age_seconds / 60 + if minutes < 90: + return f"{round(minutes)} minutes ago" + hours = minutes / 60 + if hours < 36: + return f"{round(hours)} hours ago" + return f"{round(hours / 24)} days ago" + + # --------------------------------------------------------------------------- # Catalog fetch # --------------------------------------------------------------------------- @@ -195,6 +277,17 @@ class CatalogResult: offline: bool source: str # "memory" | "network" | "cache" generated_at: Optional[str] = None + #: Seconds since this data last came off the network. 0 for a live fetch, + #: ``None`` only when the age genuinely cannot be established. + age_seconds: Optional[float] = None + #: True once :data:`CACHE_STALE_AFTER_SECONDS` has passed. Offline still + #: works — the caller is told how old the answer is, not refused one. + stale: bool = False + + @property + def age_text(self) -> str: + """The age as a phrase, for anything user-facing.""" + return describe_age(self.age_seconds) def load_index( @@ -212,6 +305,10 @@ def load_index( 2. Live network fetch → refreshes both caches, ``offline=False``. 3. On network/parse failure, the on-disk cache → ``offline=True``. 4. If none of the above yield data, raises :class:`CatalogError`. + + Every result carries ``age_seconds`` and ``stale``. Serving a cache past + :data:`CACHE_STALE_AFTER_SECONDS` logs a warning and sets ``stale`` — going + offline is supported, presenting months-old data as current is not. """ base_url = (base_url or get_hub_base_url()).rstrip("/") fetcher = fetcher or fetch_bytes @@ -230,6 +327,9 @@ def load_index( offline=False, source="memory", generated_at=data.get("generated_at"), + # Bounded by CACHE_TTL_SECONDS, so never stale by construction. + age_seconds=now - _MEM.fetched_at, + stale=False, ) try: @@ -246,11 +346,26 @@ def load_index( f"available. Check your internet connection or GAIA_HUB_URL " f"(currently {base_url}). Original error: {exc}" ) from exc + age = cache_age_seconds(cache_path, cached=cached) + stale = age is not None and age >= CACHE_STALE_AFTER_SECONDS + if stale: + logger.warning( + "catalog: serving the offline cache %s, last refreshed %s (%s). " + "Newly published packages are missing and unpublished ones are " + "still listed. Reconnect and re-run to refresh, or check " + "GAIA_HUB_URL (currently %s).", + cache_path, + describe_age(age), + cached.get(CACHE_STAMP_KEY) or "no timestamp; using the file mtime", + base_url, + ) return CatalogResult( agents=_validate_index(cached), offline=True, source="cache", generated_at=cached.get("generated_at"), + age_seconds=age, + stale=stale, ) # Live fetch succeeded — refresh both caches. @@ -263,6 +378,8 @@ def load_index( offline=False, source="network", generated_at=data.get("generated_at"), + age_seconds=0.0, + stale=False, ) @@ -541,6 +658,11 @@ class UnifiedCatalog: skills: List[Dict[str, Any]] = field(default_factory=list) offline: bool = False generated_at: Optional[str] = None + #: Seconds since this data last came off the network (see + #: :class:`CatalogResult`). Surfaced so the UI can say how old the list is + #: instead of rendering a months-old catalog as the current one. + age_seconds: Optional[float] = None + stale: bool = False def to_dict(self) -> Dict[str, Any]: return { @@ -548,6 +670,9 @@ def to_dict(self) -> Dict[str, Any]: "skills": self.skills, "offline": self.offline, "generated_at": self.generated_at, + "age_seconds": self.age_seconds, + "age_text": describe_age(self.age_seconds), + "stale": self.stale, "total": len(self.agents), } @@ -576,6 +701,8 @@ def build_catalog( index_agents = result.agents offline = result.offline generated_at = result.generated_at + age_seconds = result.age_seconds + stale = result.stale except CatalogError as exc: # No remote catalog AND no cache: still show what's installed locally. logger.warning( @@ -585,6 +712,8 @@ def build_catalog( index_agents = [] offline = True generated_at = None + age_seconds = None + stale = False merged = merge_with_registry( index_agents, @@ -597,4 +726,6 @@ def build_catalog( skills=skill_entries(index_agents), offline=offline, generated_at=generated_at, + age_seconds=age_seconds, + stale=stale, ) diff --git a/src/gaia/skills/cli.py b/src/gaia/skills/cli.py index 883005c04a..78e2d2a543 100644 --- a/src/gaia/skills/cli.py +++ b/src/gaia/skills/cli.py @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MIT """ CLI for ``gaia skill -{list|info|create|import|export|migrate|audit|search|install|remove|publish|keygen|trust}``. +{list|info|create|import|export|migrate|audit|search|install|remove|lock|publish|keygen|trust}``. Three groups of verbs, all real: @@ -14,7 +14,8 @@ rejection. * **Marketplace** (#2467) — ``search`` / ``install`` / ``remove`` / ``publish``, plus the ``keygen`` / ``trust`` key management the tier ladder rests on. These - talk to the Agent Hub's skills lane. + talk to the Agent Hub's skills lane. ``lock`` reads ``skill-lock.json`` back + and reports (or re-records) drift between it and what is installed. Exit codes are shared by all: ``0`` ok, ``2`` usage, ``3`` not found, ``4`` invalid (a malformed skill, a refused install, a rejected publish). ``audit`` @@ -30,7 +31,7 @@ import tempfile import zipfile from pathlib import Path -from typing import Optional +from typing import TYPE_CHECKING, Optional from gaia.logger import get_logger from gaia.skills.errors import SkillError, SkillNotFoundError, SkillValidationError @@ -43,6 +44,7 @@ parse_skill_file, reset_security_tier, ) +from gaia.skills.lock import forget_skill from gaia.skills.manager import SkillManager from gaia.skills.migrate import ( VENDORS, @@ -56,6 +58,11 @@ from gaia.skills.signing import ROLE_AMD, ROLE_PUBLISHER from gaia.skills.tiers import LOWEST_TIER +if TYPE_CHECKING: # pragma: no cover - typing only + # Runtime imports of drift stay inside the two verbs that need it, to keep + # its cost off every other `gaia skill` invocation. + from gaia.skills.drift import RelockResult + log = get_logger(__name__) EXIT_OK = 0 @@ -328,6 +335,36 @@ def _add_marketplace_subparsers(sub: argparse._SubParsersAction) -> None: p_remove = sub.add_parser("remove", help="Remove an installed skill") p_remove.add_argument("name", help="Skill name to remove") + p_lock = sub.add_parser( + "lock", + help="Check installed skills against skill-lock.json, or re-record it", + description=( + "Compare every skill in ~/.gaia/skills against skill-lock.json: its " + "version, the security tier install enforced, and a digest of its " + "files. Reports skills the lock has lost track of and skills it " + "tracks that are gone. Exits " + f"{EXIT_INVALID} when anything differs, {EXIT_OK} when the lock and " + "the disk agree. Untracked skills are expected right after " + "'gaia skill create' / 'import' / 'migrate' — --relock records them." + ), + ) + lock_mode = p_lock.add_mutually_exclusive_group() + lock_mode.add_argument( + "--check", + action="store_true", + help="Report drift without changing anything (the default)", + ) + lock_mode.add_argument( + "--relock", + action="store_true", + help="Re-record the current state as intended. Refuses to re-record a " + "signature-backed skill whose files changed — that would launder an " + "attestation the new bytes never had.", + ) + p_lock.add_argument( + "--json", action="store_true", dest="as_json", help="Emit JSON instead of text" + ) + p_promote = sub.add_parser( "promote", help="Trust a CAPTURED skill's code after a clean security audit", @@ -428,6 +465,7 @@ def handle(args: argparse.Namespace) -> int: "search": _handle_search, "install": _handle_install, "remove": _handle_remove, + "lock": _handle_lock, # Trust step for captured skills (code inert until promoted) "promote": _handle_promote, "publish": _handle_publish, @@ -463,12 +501,27 @@ def _manager() -> SkillManager: def _handle_list(args: argparse.Namespace) -> int: + from gaia.skills.drift import DRIFT_UNRECORDED, DRIFT_UNTRACKED, check_drift + manager = _manager() skills = manager.list_skills() if args.root: skills = [s for s in skills if s.root == args.root] errors = manager.discovery_errors + # A damaged lock is reported beside the inventory, not instead of it — the + # skills are still there, and the user needs the list to act on the error. + drift = None + lock_error = "" + try: + drift = check_drift(manager.user_root) + except SkillValidationError as exc: + lock_error = str(exc) + + # An untracked skill is already a row in the table below; re-flagging it here + # would bury the drifts that are not otherwise visible. + notable = [d for d in drift.drifts if d.kind != DRIFT_UNTRACKED] if drift else [] + if getattr(args, "as_json", False): payload = { "roots": [ @@ -478,9 +531,11 @@ def _handle_list(args: argparse.Namespace) -> int: "skills": [_skill_summary(s) for s in skills], "shadowed": [_skill_summary(s) for s in manager.shadowed()], "errors": errors, + "drift": drift.to_dict() if drift else None, + "lock_error": lock_error, } print(json.dumps(payload, indent=2)) - return EXIT_INVALID if errors else EXIT_OK + return EXIT_INVALID if errors or lock_error else EXIT_OK if not skills: print("No skills found. Searched:") @@ -504,12 +559,39 @@ def _handle_list(args: argparse.Namespace) -> int: file=sys.stderr, ) + if lock_error: + print(f"\n❌ {lock_error}", file=sys.stderr) + + if notable: + # "Cannot be verified" and "does not match" are different findings; one + # message for both tells every upgrading user their files were edited. + unverifiable = [d for d in notable if d.kind == DRIFT_UNRECORDED] + differing = [d for d in notable if d.kind != DRIFT_UNRECORDED] + if unverifiable: + names = ", ".join(sorted({d.skill for d in unverifiable})) + print( + f"\n⚠ {len(unverifiable)} skill(s) predate content digests, so " + f"{drift.lock_file.name} cannot verify them ({names}). Record them " + "with 'gaia skill lock --relock'; it refuses signature-backed " + "skills — 'gaia skill lock --check' says what those need.", + file=sys.stderr, + ) + if differing: + blocking = sum(1 for d in differing if d.fatal) + names = ", ".join(sorted({d.skill for d in differing})) + print( + f"\n⚠ {len(differing)} skill(s) no longer match " + f"{drift.lock_file.name} ({names}" + f"{f'; {blocking} will refuse to load' if blocking else ''}). " + "Details: gaia skill lock --check", + file=sys.stderr, + ) + if errors: print(f"\n{len(errors)} skill folder(s) failed to load:", file=sys.stderr) for path, message in errors.items(): print(f" {path}: {message}", file=sys.stderr) - return EXIT_INVALID - return EXIT_OK + return EXIT_INVALID if errors or lock_error else EXIT_OK def _handle_info(args: argparse.Namespace) -> int: @@ -600,6 +682,7 @@ def _handle_create(args: argparse.Namespace) -> int: if target.exists() and args.force: shutil.rmtree(target) + forget_skill(parent, args.name) target.mkdir(parents=True) skill.write(target / SKILL_FILENAME) if args.with_tools: @@ -639,6 +722,10 @@ def _handle_import(args: argparse.Namespace) -> int: previous_tier = reset_security_tier(imported) imported.write(target / SKILL_FILENAME) + # These bytes are not the hub's any more, so its provenance must not describe + # them — otherwise the replaced skill reads as tampered-with hub content. + forget_skill(destination_root, name) + print(f"✅ Imported skill '{name}' into {target}") if previous_tier != "experimental": print( @@ -884,6 +971,9 @@ def _handle_search(args: argparse.Namespace) -> int: "query": args.query, "offline": found.offline, "generated_at": found.generated_at, + "age_seconds": found.age_seconds, + "age_text": found.age_text, + "stale": found.stale, "skills": results, }, indent=2, @@ -893,10 +983,22 @@ def _handle_search(args: argparse.Namespace) -> int: # Say so before the results, not after: a stale list read as current is how a # user ends up installing something that was unpublished. - if found.offline: + if found.stale: + from gaia.hub.catalog import CACHE_STALE_AFTER_SECONDS + + days = round(CACHE_STALE_AFTER_SECONDS / 86400) print( - f"⚠ The hub was unreachable — showing the offline catalog cache " - f"(generated {found.generated_at or 'unknown'}). It may be stale.", + f"⚠ This catalog was last refreshed {found.age_text} — over {days} " + "days old. The hub is unreachable, so skills published since are " + "missing and skills unpublished since are still listed, including " + "any whose security tier changed. Reconnect and re-run to refresh, " + "or check GAIA_HUB_URL.", + file=sys.stderr, + ) + elif found.offline: + print( + f"⚠ The hub was unreachable — showing the offline catalog cache from " + f"{found.age_text}. It may be stale.", file=sys.stderr, ) @@ -917,7 +1019,13 @@ def _handle_search(args: argparse.Namespace) -> int: f"{entry.get('id', '?'):<28} {entry.get('latest_version', '-'):<10} " f"{entry.get('security_tier', '-'):<13} {tools:<6} {description}" ) - print("\nInstall one with: gaia skill install ") + origin = ( + f"offline cache, last refreshed {found.age_text}" + if found.offline + else f"hub, fetched {found.age_text}" + ) + print(f"\nCatalog: {origin}") + print("Install one with: gaia skill install ") return EXIT_OK @@ -984,6 +1092,96 @@ def _handle_remove(args: argparse.Namespace) -> int: return EXIT_OK +def _handle_lock(args: argparse.Namespace) -> int: + """Check the user root against ``skill-lock.json``, or re-record it.""" + from gaia.skills.drift import DRIFT_UNRECORDED, DRIFT_UNTRACKED, check_drift, relock + + root = _manager().user_root + + if getattr(args, "relock", False): + return _render_relock(relock(root), root, as_json=args.as_json) + + report = check_drift(root) + + if getattr(args, "as_json", False): + print(json.dumps(report.to_dict(), indent=2)) + return EXIT_OK if report.clean else EXIT_INVALID + + if report.clean: + print(f"✅ {report.lock_file} matches what is installed in {root}") + return EXIT_OK + + # Untracked is excluded from the verdict, not from the report: it is the + # expected residue of create/import/migrate, and one of them must not turn + # "your skills predate digests" back into "your skills differ". + unrecorded = [d for d in report.drifts if d.kind == DRIFT_UNRECORDED] + notable = [d for d in report.drifts if d.kind != DRIFT_UNTRACKED] + if unrecorded and len(unrecorded) == len(notable): + untracked = len(report.drifts) - len(unrecorded) + also = f", plus {untracked} the lock does not track" if untracked else "" + print( + f"⚠ {len(unrecorded)} skill(s) in {root} predate content digests, so " + f"{report.lock_file.name} cannot verify them{also}:\n", + file=sys.stderr, + ) + print(report.render(), file=sys.stderr) + return EXIT_INVALID + + fatal, warnings = len(report.fatal), len(report.warnings) + print( + f"⚠ {len(report.drifts)} difference(s) between {root} and " + f"{report.lock_file.name} ({fatal} blocking, {warnings} advisory):\n", + file=sys.stderr, + ) + print(report.render(), file=sys.stderr) + if fatal: + print( + "\nThe blocking ones are signature-backed skills whose files changed, " + "so they will refuse to load until reinstalled — the tier they carry " + "was earned by different bytes.", + file=sys.stderr, + ) + print( + "\nRe-record the state you intended with: gaia skill lock --relock", + file=sys.stderr, + ) + return EXIT_INVALID + + +def _render_relock(result: "RelockResult", root: Path, *, as_json: bool) -> int: + if as_json: + print(json.dumps(result.to_dict(), indent=2)) + return EXIT_INVALID if result.refused else EXIT_OK + + if not result.changed and not result.refused: + print(f"✅ Nothing to re-record — {result.lock_file} already matches {root}") + return EXIT_OK + + if result.changed: + print(f"✅ Re-recorded {result.lock_file}") + if result.updated: + print(f" updated : {', '.join(result.updated)}") + if result.added: + print(f" now tracked (source: local) : {', '.join(result.added)}") + if result.removed: + print(f" dropped (no longer installed) : {', '.join(result.removed)}") + + if not result.refused: + return EXIT_OK + + names = ", ".join(sorted({d.skill for d in result.refused})) + print( + f"\n❌ Refused to re-record {names}: installed at a signature-backed tier " + "over bytes this lock cannot vouch for. Writing a digest now would leave " + "the lock asserting a publisher signature over content nothing verified.", + file=sys.stderr, + ) + for drift in result.refused: + print(f" ✗ [{drift.kind}] {drift.skill}: {drift.summary}", file=sys.stderr) + print(f" → {drift.remediation}", file=sys.stderr) + return EXIT_INVALID + + def _handle_publish(args: argparse.Namespace) -> int: from gaia.hub.publisher import get_hub_token from gaia.skills.publish import publish_skill diff --git a/src/gaia/skills/drift.py b/src/gaia/skills/drift.py new file mode 100644 index 0000000000..4d4c021eec --- /dev/null +++ b/src/gaia/skills/drift.py @@ -0,0 +1,658 @@ +# Copyright(C) 2025-2026 Advanced Micro Devices, Inc. All rights reserved. +# SPDX-License-Identifier: MIT +""" +Read ``skill-lock.json`` back: detect drift between it and the skills on disk. + +:mod:`gaia.skills.lock` records what ``gaia skill install`` resolved, verified, +and enforced. Nothing re-read it, which made the lock decoration: after install, +a skill directory can be swapped, downgraded, or hand-edited and the loader would +still hand its instructions to the model and import its ``tools.py`` — under the +tier badge the *original* bytes earned. This module closes that. + +Four things are compared, per skill in the user root: + +============= ===================================================== +``version`` ``SKILL.md``'s version vs. the resolved one +``tier`` ``SKILL.md``'s ``security_tier`` vs. the enforced one +``content`` the directory's digest vs. the one recorded at install +presence locked-but-gone, and on-disk-but-untracked +============= ===================================================== + +The digest is :func:`gaia.skills.audit.findings.content_digest` — the same +algorithm the audit engine and the hub Worker use, so a skill has exactly one +content identity across audit, publish, and install. + +**Drift is fatal for a signature-backed skill and a warning otherwise.** A +``community`` / ``verified`` install means a signature over those exact bytes was +verified and a permission ceiling applied to that exact manifest; different bytes +under the same badge defeat both, so :func:`assert_no_fatal_drift` refuses the +load. An ``experimental`` or locally-authored skill was never attested — the user +already accepted unverified code, and editing a skill you wrote is the normal +workflow — so those report and keep working. + +**What this does not defend against.** ``skill-lock.json`` is a plain file in the +user's own config directory with no integrity protection. An attacker who can +write to it can rewrite a digest as easily as a skill. The threat this addresses +is tampering with the *skill directory* — a swapped bundle, an edited manifest, a +downgraded version — which is the realistic case and, until now, entirely silent. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Optional + +from gaia.logger import get_logger +from gaia.skills.audit.findings import content_digest +from gaia.skills.errors import SkillDriftError, SkillError +from gaia.skills.format import SKILL_FILENAME, Skill, parse_skill_metadata +from gaia.skills.lock import ( + LOCK_FILENAME, + SOURCE_HUB, + SOURCE_LOCAL, + LockEntry, + SkillLock, + lock_path, +) +from gaia.skills.tiers import LOWEST_TIER, TIER_ORDER + +log = get_logger(__name__) + +#: ``SKILL.md`` declares a different version than the lock resolved. +DRIFT_VERSION = "version" +#: ``SKILL.md`` declares a different ``security_tier`` than install enforced. +DRIFT_TIER = "tier" +#: The directory's bytes no longer hash to the recorded digest. +DRIFT_CONTENT = "content" +#: The lock tracks a skill that is no longer in the user root. +DRIFT_MISSING = "missing" +#: A skill directory in the user root that the lock does not track. +DRIFT_UNTRACKED = "untracked" +#: A lock entry written before content digests were recorded — content cannot be +#: checked at all, which is not the same as "checked and clean". +DRIFT_UNRECORDED = "unrecorded" + +DRIFT_KINDS = ( + DRIFT_VERSION, + DRIFT_TIER, + DRIFT_CONTENT, + DRIFT_MISSING, + DRIFT_UNTRACKED, + DRIFT_UNRECORDED, +) + + +def _attested(entry: LockEntry) -> bool: + """Whether the lock says a signature earned this skill more than the floor. + + Only a hub install has an attestation to defeat: a ``local`` entry's tier is + the author's own claim, never something a signature or a ceiling enforced. + An unrecognized tier is treated as attested — a lock this build cannot read + is not a reason to relax the check. + """ + if entry.source != SOURCE_HUB: + return False + tier = entry.installed_tier or LOWEST_TIER + if tier not in TIER_ORDER: + return True + return TIER_ORDER.index(tier) > TIER_ORDER.index(LOWEST_TIER) + + +@dataclass(frozen=True) +class Drift: + """One difference between the lock and the disk.""" + + skill: str + kind: str + expected: str + actual: str + #: True when this drift must block loading (see the module docstring). + fatal: bool + #: What is wrong and what to do about it, in one sentence each. + summary: str + remediation: str + path: str = "" + + def to_dict(self) -> dict[str, Any]: + return { + "skill": self.skill, + "kind": self.kind, + "expected": self.expected, + "actual": self.actual, + "fatal": self.fatal, + "summary": self.summary, + "remediation": self.remediation, + "path": self.path, + } + + def __str__(self) -> str: + return f"{self.summary} {self.remediation}" + + +@dataclass(frozen=True) +class DriftReport: + """Every drift found under one skills root.""" + + root: Path + lock_file: Path + drifts: tuple[Drift, ...] = () + + @property + def clean(self) -> bool: + return not self.drifts + + @property + def fatal(self) -> tuple[Drift, ...]: + return tuple(d for d in self.drifts if d.fatal) + + @property + def warnings(self) -> tuple[Drift, ...]: + return tuple(d for d in self.drifts if not d.fatal) + + def for_skill(self, name: str) -> tuple[Drift, ...]: + return tuple(d for d in self.drifts if d.skill == name) + + def to_dict(self) -> dict[str, Any]: + return { + "root": str(self.root), + "lock_file": str(self.lock_file), + "clean": self.clean, + "fatal": len(self.fatal), + "warnings": len(self.warnings), + "drifts": [d.to_dict() for d in self.drifts], + } + + def render(self) -> str: + """A human-readable report; empty string when there is no drift.""" + if self.clean: + return "" + lines = [] + for name in sorted({d.skill for d in self.drifts}): + lines.append(f"{name}") + for drift in self.for_skill(name): + marker = "✗" if drift.fatal else "⚠" + lines.append(f" {marker} [{drift.kind}] {drift.summary}") + lines.append(f" → {drift.remediation}") + return "\n".join(lines) + + +# ---------------------------------------------------------------------- +# Detection +# ---------------------------------------------------------------------- + + +def _read_metadata(directory: Path) -> tuple[Optional[Skill], str]: + """Parse a skill directory's front matter. Returns ``(skill, error)``.""" + try: + return parse_skill_metadata(directory), "" + except SkillError as exc: + return None, str(exc) + + +def _version_drift(entry: LockEntry, skill: Skill, directory: Path) -> Optional[Drift]: + actual = skill.version or "" + if not entry.version or actual == entry.version: + return None + fatal = _attested(entry) + return Drift( + skill=entry.name, + kind=DRIFT_VERSION, + expected=entry.version, + actual=actual or "(unversioned)", + fatal=fatal, + summary=( + f"{LOCK_FILENAME} records version {entry.version}, but the installed " + f"{SKILL_FILENAME} declares {actual or '(unversioned)'}." + ), + remediation=_remediation(entry, fatal), + path=str(directory), + ) + + +def _tier_drift(entry: LockEntry, skill: Skill, directory: Path) -> Optional[Drift]: + """Compare the enforced tier against the one the manifest now claims. + + Hub entries only: ``installed_tier`` is the ``min(claimed, attested)`` + decision install made, so any disagreement means the manifest was rewritten + afterwards. A local entry has no enforced tier to disagree with — its tier + change shows up as content drift, which is what it is. + """ + if entry.source != SOURCE_HUB or not entry.installed_tier: + return None + actual = skill.security_tier + if actual == entry.installed_tier: + return None + + locked_rank = ( + TIER_ORDER.index(entry.installed_tier) + if entry.installed_tier in TIER_ORDER + else -1 + ) + actual_rank = TIER_ORDER.index(actual) if actual in TIER_ORDER else -1 + escalated = actual_rank > locked_rank + # An escalation is fatal from any baseline: it is a claim to trust that the + # install path examined and refused to grant. + fatal = escalated or _attested(entry) + direction = "above" if escalated else "below" + return Drift( + skill=entry.name, + kind=DRIFT_TIER, + expected=entry.installed_tier, + actual=actual, + fatal=fatal, + summary=( + f"The installed {SKILL_FILENAME} now claims security tier " + f"'{actual}' — {direction} the '{entry.installed_tier}' that install " + "enforced from its signature." + ), + remediation=_remediation(entry, fatal), + path=str(directory), + ) + + +def _content_drift(entry: LockEntry, directory: Path) -> Optional[Drift]: + if not entry.content_digest: + return Drift( + skill=entry.name, + kind=DRIFT_UNRECORDED, + expected="", + actual="", + fatal=False, + summary=( + f"{LOCK_FILENAME} has no content digest for '{entry.name}', so its " + "files cannot be checked for tampering." + ), + remediation=( + ( + "This entry predates digest recording. Reinstall it with " + f"'gaia skill install {entry.name} --force' to take the digest " + "from the hub's own verified bytes — 'gaia skill lock --relock' " + "refuses to stamp one over bytes no signature covered." + ) + if _attested(entry) + else ( + "This entry predates digest recording. Record it with " + "'gaia skill lock --relock'." + ) + ), + path=str(directory), + ) + + actual = content_digest(directory) + if actual == entry.content_digest: + return None + fatal = _attested(entry) + return Drift( + skill=entry.name, + kind=DRIFT_CONTENT, + expected=entry.content_digest, + actual=actual, + fatal=fatal, + summary=( + f"The files in {directory} no longer hash to the digest recorded at " + "install — something was added, removed, or edited." + ), + remediation=_remediation(entry, fatal), + path=str(directory), + ) + + +def _remediation(entry: LockEntry, fatal: bool) -> str: + """What to do about a drift — never naming an action the entry cannot take. + + A ``local`` skill was never on the hub, so telling its author to reinstall + from one is an instruction that fails. + """ + if fatal: + return ( + f"Reinstall the attested copy with 'gaia skill install {entry.name} " + f"--force'. If you meant to edit it, 'gaia skill remove {entry.name}' " + f"then 'gaia skill import ' re-stamps it '{LOWEST_TIER}' — a " + "modified skill cannot keep the tier its signature earned." + ) + undo = ( + f"reinstall with 'gaia skill install {entry.name} --force' or remove it " + f"with 'gaia skill remove {entry.name}'" + if entry.source == SOURCE_HUB + else f"inspect {entry.path or entry.name} and restore it from your own copy" + ) + return ( + "If you made this change, record it with 'gaia skill lock --relock'. If " + f"you did not, {undo}." + ) + + +def _would_launder(entry: LockEntry, drift: Drift) -> bool: + """Whether re-recording *drift* would put an attestation over unverified bytes. + + Fatal drift is the obvious case. A **missing** digest is the subtle one: the + lock never covered these bytes at all, so stamping the current ones asserts + the publisher's signature over content nothing checked. That is the state + every upgrade from a pre-digest build lands in, which makes it the likeliest + way to launder an attestation, not the rarest. + """ + if drift.fatal: + return True + return drift.kind == DRIFT_UNRECORDED and _attested(entry) + + +def _inspect(entry: LockEntry, directory: Path) -> list[Drift]: + """Every drift for one locked skill.""" + if not (directory / SKILL_FILENAME).is_file(): + return [ + Drift( + skill=entry.name, + kind=DRIFT_MISSING, + expected=str(directory), + actual="(absent)", + fatal=False, + summary=( + f"{LOCK_FILENAME} tracks '{entry.name}' {entry.version} at " + f"{directory}, but nothing is installed there." + ), + remediation=( + ( + f"Reinstall it with 'gaia skill install {entry.name}', or " + "drop the stale entry with 'gaia skill lock --relock'." + ) + if entry.source == SOURCE_HUB + else ( + "Restore the directory from your own copy, or drop the " + "stale entry with 'gaia skill lock --relock'." + ) + ), + path=str(directory), + ) + ] + + skill, parse_error = _read_metadata(directory) + if skill is None: + fatal = _attested(entry) + return [ + Drift( + skill=entry.name, + kind=DRIFT_CONTENT, + expected=entry.content_digest or entry.version, + actual="(unparseable)", + fatal=fatal, + summary=( + f"The installed {SKILL_FILENAME} for '{entry.name}' no longer " + f"parses: {parse_error}" + ), + remediation=_remediation(entry, fatal), + path=str(directory), + ) + ] + + found = [ + _version_drift(entry, skill, directory), + _tier_drift(entry, skill, directory), + _content_drift(entry, directory), + ] + return [d for d in found if d is not None] + + +def check_skill(skills_root: Path | str, skill: Skill) -> tuple[Drift, ...]: + """Drift for one already-parsed skill, or ``()`` when the lock skips it. + + Takes the parsed :class:`~gaia.skills.format.Skill` rather than a name so the + load path does not re-read ``SKILL.md`` it has already read. + """ + root = Path(skills_root) + directory = skill.directory + if directory is None: + return () + entry = SkillLock.load(root).get(skill.name) + if entry is None: + return () + + found = [ + _version_drift(entry, skill, directory), + _tier_drift(entry, skill, directory), + _content_drift(entry, directory), + ] + return tuple(d for d in found if d is not None) + + +def assert_no_fatal_drift(skills_root: Path | str, skill: Skill) -> None: + """Refuse a signature-backed skill whose bytes changed after install. + + Raises: + SkillDriftError: the lock records an attested install and the on-disk + copy no longer matches it. Non-fatal drift is logged, not raised. + """ + drifts = check_skill(skills_root, skill) + if not drifts: + return + + fatal = [d for d in drifts if d.fatal] + for drift in drifts: + if not drift.fatal: + log.warning("Skill '%s' drifted from the lock: %s", skill.name, drift) + if not fatal: + return + + detail = " ".join(d.summary for d in fatal) + raise SkillDriftError( + f"Refusing to load skill '{skill.name}': it no longer matches " + f"{LOCK_FILENAME}. {detail} It was installed at a signature-backed tier, " + "so these are not the bytes that were verified, and the tier badge no " + f"longer means anything. {fatal[0].remediation} Inspect the full picture " + "with 'gaia skill lock --check'." + ) + + +def check_drift(skills_root: Path | str) -> DriftReport: + """Compare every skill in *skills_root* against ``skill-lock.json``.""" + root = Path(skills_root) + lock = SkillLock.load(root) + drifts: list[Drift] = [] + + for name in sorted(lock.entries): + entry = lock.entries[name] + drifts.extend(_inspect(entry, root / name)) + + for directory in sorted(_skill_dirs(root)): + name = directory.name + if name in lock.entries: + continue + drifts.append( + Drift( + skill=name, + kind=DRIFT_UNTRACKED, + expected="(no lock entry)", + actual=str(directory), + fatal=False, + summary=( + f"'{name}' is installed at {directory} but {LOCK_FILENAME} does " + "not track it, so nothing can tell whether it has changed." + ), + remediation=( + "Expected if you created, imported, or migrated it. Start " + "tracking it with 'gaia skill lock --relock'; if you did not " + f"put it there, remove it with 'gaia skill remove {name}'." + ), + path=str(directory), + ) + ) + + report = DriftReport(root=root, lock_file=lock_path(root), drifts=tuple(drifts)) + log.info( + "Lock check over %s: %d drift(s) (%d fatal) across %d locked skill(s)", + root, + len(report.drifts), + len(report.fatal), + len(lock.entries), + ) + return report + + +def _skill_dirs(root: Path) -> list[Path]: + """Every directory under *root* that holds a ``SKILL.md``.""" + if not root.is_dir(): + return [] + return [ + entry + for entry in root.iterdir() + if entry.is_dir() and (entry / SKILL_FILENAME).is_file() + ] + + +# ---------------------------------------------------------------------- +# Relock +# ---------------------------------------------------------------------- + + +@dataclass +class RelockResult: + """What ``gaia skill lock --relock`` changed.""" + + lock_file: Path + #: Entries whose recorded state was updated to match disk. + updated: list[str] + #: Skills newly tracked as ``source: local``. + added: list[str] + #: Stale entries dropped because the skill is gone. + removed: list[str] + #: Entries left alone because relocking them would launder an attestation. + refused: list[Drift] + + @property + def changed(self) -> bool: + return bool(self.updated or self.added or self.removed) + + def to_dict(self) -> dict[str, Any]: + return { + "lock_file": str(self.lock_file), + "updated": list(self.updated), + "added": list(self.added), + "removed": list(self.removed), + "refused": [d.to_dict() for d in self.refused], + "changed": self.changed, + } + + +def relock(skills_root: Path | str) -> RelockResult: + """Re-record the current state of *skills_root* into ``skill-lock.json``. + + This is the explicit "yes, I meant to change that" step. It updates each + tracked skill's version, tier, and content digest to what is on disk, drops + entries whose skill is gone, and starts tracking untracked directories as + ``source: local``. + + It **refuses** to re-record a signature-backed entry whose content drifted, + or one that has no recorded digest at all. Writing a digest in either case + would leave the lock asserting a ``verified`` / ``community`` tier and a + publisher signature over bytes that signature never covered — laundering an + attestation is exactly what a lockfile exists to prevent. Those entries are + returned in :attr:`RelockResult.refused` with the two honest ways out + (reinstall, or remove and re-import at ``experimental``). + + Raises: + SkillValidationError: the existing lock is corrupt (from + :meth:`gaia.skills.lock.SkillLock.load`); relock will not start over + from scratch and silently discard provenance. + """ + root = Path(skills_root) + lock = SkillLock.load(root) + updated: list[str] = [] + added: list[str] = [] + removed: list[str] = [] + refused: list[Drift] = [] + + for name in sorted(lock.entries): + entry = lock.entries[name] + directory = root / name + + if not (directory / SKILL_FILENAME).is_file(): + lock.forget(name) + removed.append(name) + continue + + blocking = [d for d in _inspect(entry, directory) if _would_launder(entry, d)] + if blocking: + refused.extend(blocking) + continue + + skill, parse_error = _read_metadata(directory) + if skill is None: # pragma: no cover - a fatal parse error is refused above + raise SkillDriftError( + f"Cannot relock '{name}': its {SKILL_FILENAME} does not parse " + f"({parse_error}). Fix the manifest, or remove the skill with " + f"'gaia skill remove {name}'." + ) + + before = (entry.version, entry.installed_tier, entry.content_digest) + entry.version = skill.version or entry.version + entry.content_digest = content_digest(directory) + entry.permissions = list(skill.gaia.permissions) + entry.path = str(directory) + if entry.source != SOURCE_HUB: + # A local entry's tier is a record of the author's claim, not an + # enforcement decision, so it tracks the manifest. A hub entry's is + # the enforced tier — never overwritten from the file it governs. + entry.installed_tier = skill.security_tier + if before != (entry.version, entry.installed_tier, entry.content_digest): + updated.append(name) + + for directory in sorted(_skill_dirs(root)): + name = directory.name + if name in lock.entries: + continue + skill, parse_error = _read_metadata(directory) + if skill is None: + raise SkillDriftError( + f"Cannot relock '{name}': its {SKILL_FILENAME} does not parse " + f"({parse_error}). Fix the manifest at {directory}, or remove the " + "directory." + ) + lock.record( + LockEntry( + name=name, + version=skill.version or "", + requested="", + source=SOURCE_LOCAL, + content_digest=content_digest(directory), + claimed_tier=skill.security_tier, + installed_tier=skill.security_tier, + permissions=list(skill.gaia.permissions), + path=str(directory), + ) + ) + added.append(name) + + lock.save(lock_path(root)) + log.info( + "Relocked %s: %d updated, %d added, %d removed, %d refused", + root, + len(updated), + len(added), + len(removed), + len(refused), + ) + return RelockResult( + lock_file=lock_path(root), + updated=updated, + added=added, + removed=removed, + refused=refused, + ) + + +__all__ = [ + "DRIFT_CONTENT", + "DRIFT_KINDS", + "DRIFT_MISSING", + "DRIFT_TIER", + "DRIFT_UNRECORDED", + "DRIFT_UNTRACKED", + "DRIFT_VERSION", + "Drift", + "DriftReport", + "RelockResult", + "assert_no_fatal_drift", + "check_drift", + "check_skill", + "relock", +] diff --git a/src/gaia/skills/errors.py b/src/gaia/skills/errors.py index 80b16b4a7a..01191e5fe1 100644 --- a/src/gaia/skills/errors.py +++ b/src/gaia/skills/errors.py @@ -38,6 +38,16 @@ class SkillSetError(SkillError): """ +class SkillDriftError(SkillError): + """An installed skill no longer matches what ``skill-lock.json`` recorded. + + Raised only for a signature-backed install (``community`` / ``verified``): + those bytes were attested at install time, so serving different ones under + the same tier badge would make the signature meaningless. Unattested skills + warn instead — see :mod:`gaia.skills.drift`. + """ + + class SkillPermissionError(SkillError): """The skill declares a permission this phase cannot honor. diff --git a/src/gaia/skills/hub.py b/src/gaia/skills/hub.py index bc9be67132..ff90ba9f51 100644 --- a/src/gaia/skills/hub.py +++ b/src/gaia/skills/hub.py @@ -99,11 +99,26 @@ class SkillSearchResult: actionably different from a fresh one: a skill listed from cache may have been unpublished, and one published since will be missing. Presenting cached results as current is the quiet-wrong-answer failure mode. + + ``offline`` alone does not say *how* stale, though — a cache from ten minutes + ago and one from three months ago look the same to a caller. ``age_seconds`` + and ``stale`` answer that. """ entries: list[dict[str, Any]] offline: bool = False generated_at: Optional[str] = None + #: Seconds since the catalog behind these results came off the network. + age_seconds: Optional[float] = None + #: True past :data:`gaia.hub.catalog.CACHE_STALE_AFTER_SECONDS`. + stale: bool = False + + @property + def age_text(self) -> str: + """The catalog's age as a phrase ("3 days ago").""" + from gaia.hub.catalog import describe_age + + return describe_age(self.age_seconds) def __iter__(self): return iter(self.entries) @@ -181,13 +196,19 @@ def haystack(entry: dict[str, Any]) -> str: ) if index.offline: log.warning( - "Serving %d skill(s) from the offline catalog cache (generated %s) — " - "the hub was unreachable, so this list may be stale", + "Serving %d skill(s) from the offline catalog cache, last refreshed %s " + "(hub generated it %s) — the hub was unreachable, so this list may be " + "stale", len(matched), + index.age_text, index.generated_at or "unknown", ) return SkillSearchResult( - entries=matched, offline=index.offline, generated_at=index.generated_at + entries=matched, + offline=index.offline, + generated_at=index.generated_at, + age_seconds=index.age_seconds, + stale=index.stale, ) diff --git a/src/gaia/skills/install.py b/src/gaia/skills/install.py index e522cd6427..ba558caf62 100644 --- a/src/gaia/skills/install.py +++ b/src/gaia/skills/install.py @@ -26,7 +26,9 @@ 8. **Enforce the permission ceiling** for the effective tier, then prompt for dangerous grants (``community``) or require ``--allow-experimental``. 9. **Copy in and re-stamp** the effective tier into the installed ``SKILL.md``, - and record the whole decision in ``skill-lock.json``. + and record the whole decision in ``skill-lock.json`` — including a digest of + the bytes that landed, which is what lets :mod:`gaia.skills.drift` notice if + they change afterwards. Steps 4/9 reuse ``gaia skill import``'s materialize → parse → copy → re-stamp shape on purpose: one save path with one security behavior. Import always stamps @@ -45,6 +47,7 @@ from gaia.hub.catalog import Fetcher, get_hub_base_url from gaia.logger import get_logger +from gaia.skills.audit.findings import content_digest from gaia.skills.errors import SkillError, SkillNotFoundError, SkillValidationError from gaia.skills.format import ( SKILL_FILENAME, @@ -299,6 +302,10 @@ def install_skill( SKILL_FILENAME, ) + # Hashed after the re-stamp so the digest describes the bytes that landed, + # not the ones in the archive. `gaia skill lock` compares against this. + installed_digest = content_digest(target) + lock.record( LockEntry( name=name, @@ -311,6 +318,7 @@ def install_skill( hub_url=base_url or get_hub_base_url(), artifact_sha256=artifact.sha256, artifact_filename=artifact.filename, + content_digest=installed_digest, claimed_tier=claimed, attested_tier=attested, installed_tier=tier, @@ -539,10 +547,13 @@ def remove_skill(name: str, *, manager: Optional[SkillManager] = None) -> Remove def installed_provenance(manager: Optional[SkillManager] = None) -> dict[str, Any]: - """``{name: lock-entry-dict}`` for every hub-installed skill. + """``{name: lock-entry-dict}`` for every skill the lock tracks. Feeds ``gaia skill list --json`` and the (future) ``/api/skills`` router, so - both read provenance from the lock rather than re-deriving it. + both read provenance from the lock rather than re-deriving it. Each entry + carries its own ``source`` — ``hub`` for an install, ``local`` for a skill + ``gaia skill lock --relock`` started tracking — so a caller that only wants + hub provenance filters on it rather than assuming it. """ resolver = manager if manager is not None else SkillManager() lock = SkillLock.load(resolver.user_root) diff --git a/src/gaia/skills/lock.py b/src/gaia/skills/lock.py index 12601af08a..511b194103 100644 --- a/src/gaia/skills/lock.py +++ b/src/gaia/skills/lock.py @@ -25,6 +25,7 @@ "hub_url": "https://hub.amd-gaia.ai", "artifact_sha256": "…", "artifact_filename": "web-research-1.2.0.zip", + "content_digest": "sha256:…", "claimed_tier": "community", "installed_tier": "community", "attested_tier": "community", @@ -37,17 +38,24 @@ } } -Hub-installed skills and **captured** skills are tracked. A capture +``gaia skill install`` writes ``source: "hub"`` entries. A capture (``source: "captured"``) records where the bundle came from (``origin``) and the code-trust state (``captured`` / ``code_trusted``) that gates whether its ``tools.py`` may register — see :mod:`gaia.skills.capture`. A skill created with ``gaia skill create`` or copied in with ``gaia skill import`` has no provenance -to record, so inventing a lock entry for it would assert a source it does not -have. +of either kind, so neither path invents one; ``gaia skill lock --relock`` +(:mod:`gaia.skills.drift`) may record it as ``source: "local"``, which asserts +exactly the origin it has and nothing more. Only a ``hub`` entry's tier is an +enforcement decision; any other source's is the author's own claim. A missing lock file is an **empty** lock, not an error — that is a machine with no hub-installed skills. A *corrupt* one raises: silently starting over would drop the provenance of skills that are still installed. + +The lock is only useful if something reads it back: :mod:`gaia.skills.drift` +compares it against what is actually on disk, and +:meth:`gaia.skills.manager.SkillManager.load` refuses a signature-backed skill +whose bytes no longer match. """ from __future__ import annotations @@ -75,6 +83,12 @@ #: ``source`` value for a skill captured from pasted text, a URL, or a folder. SOURCE_CAPTURED = "captured" +#: ``source`` value for a skill that lives in the user root without hub +#: provenance — authored with ``gaia skill create``, copied in with +#: ``gaia skill import``, or converted by ``gaia skill migrate``. Recorded only +#: by ``gaia skill lock --relock``, so drift detection covers it too. +SOURCE_LOCAL = "local" + def _now() -> str: return datetime.now(timezone.utc).isoformat(timespec="seconds") @@ -92,6 +106,12 @@ class LockEntry: hub_url: str = "" artifact_sha256: str = "" artifact_filename: str = "" + #: ``sha256:`` over the *installed* directory + #: (:func:`gaia.skills.audit.findings.content_digest`), recorded after the + #: tier re-stamp so it describes the bytes that actually landed. This is what + #: makes post-install tampering detectable — ``artifact_sha256`` only covers + #: the downloaded archive, which nobody re-reads once it is unpacked. + content_digest: str = "" #: Tier the skill's own front matter claimed. claimed_tier: str = "" #: Highest tier its signature earned (see :func:`gaia.skills.signing.attested_tier`). @@ -227,3 +247,24 @@ def __contains__(self, name: object) -> bool: def lock_path(skills_root: Path) -> Path: """Path of the lock file for a skills root.""" return Path(skills_root) / LOCK_FILENAME + + +def forget_skill(skills_root: Path, name: str) -> bool: + """Drop *name* from the lock under *skills_root*; True if it was tracked. + + Every path that **replaces** a skill directory without hub provenance — + ``import``, ``create --force``, ``migrate --force`` — must call this. The + hub copy those bytes described is gone, so leaving its entry behind would + have the lock asserting a version, tier, and signature over a different + skill, which :mod:`gaia.skills.drift` would then (correctly) refuse to load. + + A root with no lock file is a no-op: nothing is created. + """ + if not lock_path(skills_root).is_file(): + return False + lock = SkillLock.load(skills_root) + if not lock.forget(name): + return False + lock.save() + log.info("Dropped lock entry for '%s': it was replaced by a local copy", name) + return True diff --git a/src/gaia/skills/manager.py b/src/gaia/skills/manager.py index a90feb1811..e76d08b22d 100644 --- a/src/gaia/skills/manager.py +++ b/src/gaia/skills/manager.py @@ -261,15 +261,28 @@ def resolve_path(self, name: str) -> Path: def load(self, name: str) -> Skill: """Fully parse the winning skill, body included (level 2). + A skill from the user root is checked against ``skill-lock.json`` here — + the single chokepoint every consumer (``Agent.load_skill``, ``gaia skill + info``) passes through — so a signature-backed install whose files + changed cannot be handed to a model under a tier it no longer earns. + Raises: SkillNotFoundError: no root contains a skill of that name. SkillValidationError: the skill exists but fails validation. + SkillDriftError: the skill was installed at ``community`` / + ``verified`` and no longer matches what the lock recorded. """ metadata = self.get(name) assert metadata.path is not None # discovery always sets it skill = parse_skill_file( metadata.path, root=metadata.root, read_only=metadata.read_only ) + if metadata.root == ROOT_USER: + # Deferred: keeps the marketplace modules off the import path of + # every agent that merely composes a bundled skill. + from gaia.skills.drift import assert_no_fatal_drift + + assert_no_fatal_drift(self.user_root, skill) log.debug( "Loaded skill '%s' from root '%s' (%s, tier=%s, %d tool(s))", skill.name, diff --git a/src/gaia/skills/migrate.py b/src/gaia/skills/migrate.py index 223c6f8cc4..309b7b98b3 100644 --- a/src/gaia/skills/migrate.py +++ b/src/gaia/skills/migrate.py @@ -59,6 +59,7 @@ reset_security_tier, split_frontmatter, ) +from gaia.skills.lock import forget_skill from gaia.skills.naming import skill_directory from gaia.skills.permissions import refuse_unbridged_permissions @@ -946,6 +947,9 @@ def install_migrated( "to replace it, or --name to install under a different name." ) shutil.rmtree(target) + # A migrated skill has no hub provenance; leaving the replaced install's + # lock entry behind would describe bytes that no longer exist. + forget_skill(Path(destination_root), skill.name) target.mkdir(parents=True) if copy_support_files and source_dir is not None: diff --git a/tests/unit/skills_helpers.py b/tests/unit/skills_helpers.py index 7544ee2d34..3ecbbaf9e3 100644 --- a/tests/unit/skills_helpers.py +++ b/tests/unit/skills_helpers.py @@ -261,6 +261,69 @@ def make_key(tmp_path: Path, name: str = "publisher"): return generate_key(tmp_path, name=name) +def make_marketplace(tmp_path: Path) -> "Marketplace": + """A cold marketplace: empty hub, empty skills root, empty trust store. + + Publish and install run for real against :class:`FakeHub`, so a lock entry + produced here carries the same digests, signature, and enforced tier a real + install would write. + """ + return Marketplace(tmp_path) + + +class Marketplace: + """Bundles the fake hub with an isolated skills root and its helpers.""" + + def __init__(self, tmp_path: Path) -> None: + self.tmp_path = Path(tmp_path) + self.hub = fake_hub(self.tmp_path) + self.skills_root = self.tmp_path / "gaia-home" / "skills" + self.manager = isolated_manager( + self.tmp_path, user_skills_root=self.skills_root + ) + self.audit = write_audit_report(self.tmp_path) + + def publish(self, source, **kwargs): + from gaia.skills.publish import publish_skill + + kwargs.setdefault("keys_root", self.skills_root) + kwargs.setdefault("audit_report", self.audit) + return publish_skill( + source, + token="test-token", + hub_url=self.hub.BASE_URL, + uploader=self.hub.accept_publish, + **kwargs, + ) + + def install(self, reference, **kwargs): + from gaia.skills.install import install_skill + + return install_skill( + reference, + manager=self.manager, + base_url=self.hub.BASE_URL, + fetcher=self.hub.fetcher, + **kwargs, + ) + + def trust(self, key, *, role="publisher", publisher="acme"): + import base64 + + from gaia.skills.signing import TrustStore + + store = TrustStore.load(self.skills_root) + store.add( + public_key_b64=base64.b64encode(key.public_bytes).decode("ascii"), + publisher=publisher, + role=role, + ) + store.save() + + def keygen(self, name: str = "publisher"): + return make_key(self.skills_root, name=name) + + def isolated_manager(tmp_path: Path, **kwargs): """A :class:`SkillManager` whose roots are all inside ``tmp_path``. diff --git a/tests/unit/test_hub_catalog.py b/tests/unit/test_hub_catalog.py index 3772a88af1..741308c301 100644 --- a/tests/unit/test_hub_catalog.py +++ b/tests/unit/test_hub_catalog.py @@ -500,3 +500,147 @@ def _boom(url): assert unified.offline is True assert [a["id"] for a in unified.agents] == ["demo"] assert [s["id"] for s in unified.skills] == ["web-research"] + + +# --------------------------------------------------------------------------- +# Offline-cache staleness (#2467 follow-up) +# +# The offline cache used to be able to serve arbitrarily old data with nothing +# saying so. Working offline is supported; presenting a months-old catalog as +# the current one is not. +# --------------------------------------------------------------------------- + + +def _stale_cache(tmp_path, *, age_days, agent_id="cached"): + """Write a disk cache stamped ``age_days`` in the past.""" + import datetime + + cache = tmp_path / "catalog-cache.json" + stamped_at = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta( + days=age_days + ) + document = _index(_entry(agent_id)) + document[catalog.CACHE_STAMP_KEY] = stamped_at.isoformat(timespec="seconds") + cache.write_text(json.dumps(document)) + return cache + + +def _offline(cache): + def failing_fetcher(url): + raise ConnectionError("no network") + + return load_index( + base_url="https://hub.test", + fetcher=failing_fetcher, + cache_path=cache, + force=True, + ) + + +def test_a_live_fetch_stamps_the_cache_with_its_refresh_time(tmp_path): + """Without the stamp there is nothing to measure age against.""" + cache = tmp_path / "catalog-cache.json" + + result = load_index( + base_url="https://hub.test", + fetcher=lambda url: json.dumps(_index(_entry("demo"))).encode(), + cache_path=cache, + force=True, + ) + + assert result.age_seconds == 0 + assert result.stale is False + assert catalog.CACHE_STAMP_KEY in json.loads(cache.read_text()) + + +def test_a_live_fetch_does_not_put_the_stamp_in_the_in_memory_catalog(tmp_path): + """The stamp is ours; it must not look like a field the hub sent.""" + load_index( + base_url="https://hub.test", + fetcher=lambda url: json.dumps(_index(_entry("demo"))).encode(), + cache_path=tmp_path / "catalog-cache.json", + force=True, + ) + + assert catalog.CACHE_STAMP_KEY not in (catalog._MEM.raw or {}) + + +def test_a_stale_offline_cache_is_flagged_and_warns(tmp_path, caplog): + cache = _stale_cache(tmp_path, age_days=90) + + with caplog.at_level("WARNING", logger="gaia.hub.catalog"): + result = _offline(cache) + + assert result.offline is True + assert result.stale is True + assert result.age_seconds > catalog.CACHE_STALE_AFTER_SECONDS + assert "90 days ago" in result.age_text + # Offline must still WORK — the point is that it says how old it is. + assert [a["id"] for a in result.agents] == ["cached"] + assert any("last refreshed" in r.getMessage() for r in caplog.records) + + +def test_a_fresh_offline_cache_is_not_flagged_stale(tmp_path, caplog): + cache = _stale_cache(tmp_path, age_days=1) + + with caplog.at_level("WARNING", logger="gaia.hub.catalog"): + result = _offline(cache) + + assert result.offline is True + assert result.stale is False + assert result.age_seconds < catalog.CACHE_STALE_AFTER_SECONDS + assert not any("last refreshed" in r.getMessage() for r in caplog.records) + + +def test_a_cache_just_over_the_threshold_is_stale(tmp_path): + """The boundary is where an off-by-one would hide the whole feature.""" + threshold_days = catalog.CACHE_STALE_AFTER_SECONDS / 86400 + + assert _offline(_stale_cache(tmp_path, age_days=threshold_days + 0.5)).stale is True + assert ( + _offline(_stale_cache(tmp_path, age_days=threshold_days - 0.5)).stale is False + ) + + +def test_an_unstamped_cache_falls_back_to_its_file_mtime(tmp_path): + """A cache written by an older GAIA must not read as fresh by default.""" + import os + import time + + cache = tmp_path / "catalog-cache.json" + cache.write_text(json.dumps(_index(_entry("legacy")))) + old = time.time() - catalog.CACHE_STALE_AFTER_SECONDS * 2 + os.utime(cache, (old, old)) + + result = _offline(cache) + + assert result.stale is True + assert result.age_seconds > catalog.CACHE_STALE_AFTER_SECONDS + + +def test_build_catalog_surfaces_the_cache_age_to_its_consumers(tmp_path): + """The UI renders this dict; if age never reaches it, nothing can show it.""" + unified = build_catalog( + _FakeReg([]), + base_url="https://hub.test", + fetcher=_raise_offline, + cache_path=_stale_cache(tmp_path, age_days=45), + force=True, + ) + + payload = unified.to_dict() + assert payload["stale"] is True + assert payload["age_text"] == "45 days ago" + assert payload["age_seconds"] > catalog.CACHE_STALE_AFTER_SECONDS + + +def _raise_offline(url): + raise ConnectionError("no network") + + +def test_describe_age_reads_like_a_person_wrote_it(): + assert catalog.describe_age(0) == "just now" + assert catalog.describe_age(60 * 30) == "30 minutes ago" + assert catalog.describe_age(3600 * 5) == "5 hours ago" + assert catalog.describe_age(86400 * 3) == "3 days ago" + assert catalog.describe_age(None) == "unknown" diff --git a/tests/unit/test_skills_cli.py b/tests/unit/test_skills_cli.py index a93b2ea356..eb3db8e092 100644 --- a/tests/unit/test_skills_cli.py +++ b/tests/unit/test_skills_cli.py @@ -491,6 +491,7 @@ def test_real_cli_help_lists_the_authoring_and_marketplace_verbs(tmp_path): "search", "install", "remove", + "lock", "publish", "keygen", "trust", @@ -505,3 +506,22 @@ def test_real_cli_missing_skill_exit_code(tmp_path): result = _real_cli("info", "ghost", home=tmp_path, cwd=tmp_path) assert result.returncode == 3 assert "No skill named 'ghost'" in result.stderr + + +def test_the_staleness_banner_a_user_reads_is_a_sentence(monkeypatch, capsys): + """``age_text`` is a phrase ("93 days ago"), so the banner has to fit it.""" + from gaia.skills import cli as skills_cli + from gaia.skills.hub import SkillSearchResult + + monkeypatch.setattr( + "gaia.skills.hub.search_skills", + lambda *a, **k: SkillSearchResult( + entries=[], offline=True, age_seconds=93 * 86400, stale=True + ), + ) + + skills_cli.handle( + argparse.Namespace(skill_action="search", query="", hub_url=None, as_json=False) + ) + + assert "was last refreshed 93 days ago — over 7 days old" in capsys.readouterr().err diff --git a/tests/unit/test_skills_drift.py b/tests/unit/test_skills_drift.py new file mode 100644 index 0000000000..6675528866 --- /dev/null +++ b/tests/unit/test_skills_drift.py @@ -0,0 +1,771 @@ +# Copyright(C) 2025-2026 Advanced Micro Devices, Inc. All rights reserved. +# SPDX-License-Identifier: MIT +""" +``skill-lock.json`` is read back: drift detection and relock (#2467 follow-up). + +The lock was written by ``gaia skill install`` and never re-read, so a skill +could be swapped, downgraded, or hand-edited after install and nothing noticed. +These tests are the counter-evidence, and each one is written to fail if the +check it covers is removed. + +Every lock entry here is produced by the **real** publish → install path against +the stand-in hub, so the digests, signature, and enforced tier are the ones a +genuine install writes — not a hand-built fixture that would keep passing if +install stopped recording them. +""" + +from __future__ import annotations + +import argparse + +import pytest + +from gaia.skills import cli as skills_cli +from gaia.skills.drift import ( + DRIFT_CONTENT, + DRIFT_MISSING, + DRIFT_TIER, + DRIFT_UNRECORDED, + DRIFT_UNTRACKED, + DRIFT_VERSION, + check_drift, + relock, +) +from gaia.skills.errors import SkillDriftError, SkillValidationError +from gaia.skills.lock import SOURCE_HUB, SOURCE_LOCAL, SkillLock +from tests.unit.skills_helpers import make_marketplace + +SKILL_BODY = """# Web Research + +1. Call `web-research/search_web` with the user's question. +2. Summarize the top results. +""" + + +def _write_source( + tmp_path, + *, + name="web-research", + version="1.0.0", + tier="community", + permissions=("network:read:*.brave.com",), +): + """Author a publishable skill source directory.""" + directory = tmp_path / "src" / name + directory.mkdir(parents=True, exist_ok=True) + lines = [ + "---", + f"name: {name}", + "description: Search the web for current information and summarize it.", + f"version: {version}", + "license: MIT", + "metadata:", + " gaia:", + f" security_tier: {tier}", + ] + if permissions: + lines.append(" permissions:") + lines.extend(f" - {p}" for p in permissions) + lines += ["---", "", SKILL_BODY] + (directory / "SKILL.md").write_text("\n".join(lines), encoding="utf-8") + return directory + + +@pytest.fixture +def marketplace(tmp_path): + return make_marketplace(tmp_path) + + +@pytest.fixture +def attested(marketplace, tmp_path): + """A ``community`` install: publisher-signed, signature verified at install.""" + key = marketplace.keygen() + marketplace.trust(key, role="publisher") + marketplace.publish(_write_source(tmp_path), publisher="acme") + result = marketplace.install("web-research") + assert result.installed_tier == "community" + return result + + +@pytest.fixture +def unattested(marketplace, tmp_path): + """An ``experimental`` install: published unsigned, opted into explicitly.""" + source = _write_source(tmp_path, name="scratch-skill", tier="experimental") + marketplace.publish(source, unsigned=True) + result = marketplace.install("scratch-skill", allow_experimental=True) + assert result.installed_tier == "experimental" + return result + + +def _retier(skill_dir, tier): + """Rewrite the installed manifest's declared security tier.""" + path = skill_dir / "SKILL.md" + text = path.read_text(encoding="utf-8") + original, _, rest = text.partition("security_tier: ") + current, _, tail = rest.partition("\n") + path.write_text(f"{original}security_tier: {tier}\n{tail}", encoding="utf-8") + assert current != tier, "the test must actually change the tier" + + +def _reversion(skill_dir, version): + path = skill_dir / "SKILL.md" + text = path.read_text(encoding="utf-8") + path.write_text(text.replace("version: 1.0.0", f"version: {version}"), "utf-8") + + +def _kinds(report): + return {d.kind for d in report.drifts} + + +# ---------------------------------------------------------------------- +# Install records the digest the whole feature rests on +# ---------------------------------------------------------------------- + + +def test_install_records_a_content_digest_of_the_bytes_that_landed( + marketplace, attested +): + """Without this field there is nothing to compare against later.""" + from gaia.skills.audit.findings import content_digest + + entry = SkillLock.load(marketplace.skills_root).get("web-research") + + assert entry is not None + assert entry.source == SOURCE_HUB + assert entry.content_digest.startswith("sha256:") + # The digest must cover the INSTALLED directory, not the downloaded archive. + assert entry.content_digest == content_digest(marketplace.skills_root / entry.name) + assert entry.content_digest != entry.artifact_sha256 + + +# ---------------------------------------------------------------------- +# No false positives +# ---------------------------------------------------------------------- + + +def test_a_clean_install_reports_no_drift(marketplace, attested): + report = check_drift(marketplace.skills_root) + + assert report.clean, report.render() + assert report.fatal == () + + +def test_two_clean_installs_report_no_drift(marketplace, tmp_path, attested): + """A second skill must not make the first look drifted.""" + marketplace.publish( + _write_source(tmp_path, name="note-taker", tier="experimental"), unsigned=True + ) + marketplace.install("note-taker", allow_experimental=True) + + assert check_drift(marketplace.skills_root).clean + + +def test_a_clean_install_still_loads(marketplace, attested): + """The load gate must not refuse a skill that has not changed.""" + marketplace.manager.reload() + + assert marketplace.manager.load("web-research").name == "web-research" + + +# ---------------------------------------------------------------------- +# One test per drift mode +# ---------------------------------------------------------------------- + + +def test_content_drift_is_detected_when_an_installed_file_is_edited( + marketplace, attested +): + (attested.path / "SKILL.md").write_text( + (attested.path / "SKILL.md").read_text(encoding="utf-8") + + "\n\n", + encoding="utf-8", + ) + + report = check_drift(marketplace.skills_root) + + assert DRIFT_CONTENT in _kinds(report) + assert not report.clean + + +def test_content_drift_is_detected_when_a_file_is_added(marketplace, attested): + """A dropped-in module is content the signature never covered.""" + (attested.path / "helper.py").write_text("import os\n", encoding="utf-8") + + assert DRIFT_CONTENT in _kinds(check_drift(marketplace.skills_root)) + + +def test_version_drift_is_detected_when_the_manifest_version_changes( + marketplace, attested +): + _reversion(attested.path, "9.9.9") + + report = check_drift(marketplace.skills_root) + version = next(d for d in report.drifts if d.kind == DRIFT_VERSION) + + assert version.expected == "1.0.0" + assert version.actual == "9.9.9" + + +def test_missing_drift_is_detected_when_the_directory_is_deleted(marketplace, attested): + import shutil + + shutil.rmtree(attested.path) + + report = check_drift(marketplace.skills_root) + missing = next(d for d in report.drifts if d.kind == DRIFT_MISSING) + + assert missing.skill == "web-research" + # Nothing can load, so this is a report-level finding, not a load blocker. + assert missing.fatal is False + + +def test_untracked_drift_is_detected_when_an_extra_skill_appears( + marketplace, tmp_path, attested +): + """A directory nobody installed is exactly what a lock exists to surface.""" + import shutil + + planted = _write_source(tmp_path, name="planted", tier="experimental") + shutil.copytree(planted, marketplace.skills_root / "planted") + + report = check_drift(marketplace.skills_root) + untracked = next(d for d in report.drifts if d.kind == DRIFT_UNTRACKED) + + assert untracked.skill == "planted" + + +def test_a_lock_entry_without_a_digest_reports_as_unverifiable(marketplace, attested): + """A pre-digest lock must not read as 'checked and clean'.""" + lock = SkillLock.load(marketplace.skills_root) + lock.get("web-research").content_digest = "" + lock.save() + + report = check_drift(marketplace.skills_root) + + assert DRIFT_UNRECORDED in _kinds(report) + assert not report.clean + + +# ---------------------------------------------------------------------- +# Fatal vs. warning +# ---------------------------------------------------------------------- + + +def test_an_attested_skill_refuses_to_load_after_content_drift(marketplace, attested): + """Drift under a signature-backed tier defeats the signature — hard stop.""" + (attested.path / "tools.py").write_text("import socket\n", encoding="utf-8") + marketplace.manager.reload() + + with pytest.raises(SkillDriftError) as caught: + marketplace.manager.load("web-research") + + message = str(caught.value) + assert "web-research" in message + assert "gaia skill install web-research --force" in message + + +def test_an_unattested_skill_warns_but_still_loads_after_content_drift( + marketplace, unattested, caplog +): + """Editing an experimental skill is the normal workflow, not an attack.""" + (unattested.path / "notes.md").write_text("scratch\n", encoding="utf-8") + marketplace.manager.reload() + + with caplog.at_level("WARNING", logger="gaia.skills.drift"): + skill = marketplace.manager.load("scratch-skill") + + assert skill.name == "scratch-skill" + assert any("drifted from the lock" in r.message for r in caplog.records) + + +def test_tier_escalation_is_fatal_even_from_the_experimental_floor( + marketplace, unattested +): + """Claiming 'verified' after install is a claim install examined and refused.""" + _retier(unattested.path, "verified") + marketplace.manager.reload() + + report = check_drift(marketplace.skills_root) + tier = next(d for d in report.drifts if d.kind == DRIFT_TIER) + assert tier.expected == "experimental" + assert tier.actual == "verified" + assert tier.fatal is True + + with pytest.raises(SkillDriftError): + marketplace.manager.load("scratch-skill") + + +def test_a_skill_outside_the_user_root_is_not_gated(marketplace, tmp_path, attested): + """The lock only describes the user root; a bundled skill has no entry there.""" + import shutil + + from tests.unit.skills_helpers import isolated_manager + + bundled = tmp_path / "agent-skills" + shutil.copytree(attested.path, bundled / "web-research") + (bundled / "web-research" / "SKILL.md").write_text( + (bundled / "web-research" / "SKILL.md").read_text(encoding="utf-8") + + "\nedit\n", + encoding="utf-8", + ) + manager = isolated_manager( + tmp_path, + user_skills_root=tmp_path / "empty-user-root", + agent_skill_dirs=[bundled], + ) + + assert manager.load("web-research").name == "web-research" + + +# ---------------------------------------------------------------------- +# Relock +# ---------------------------------------------------------------------- + + +def test_relock_records_an_intentional_edit_and_then_validates_clean( + marketplace, unattested +): + (unattested.path / "notes.md").write_text("scratch\n", encoding="utf-8") + assert not check_drift(marketplace.skills_root).clean + + result = relock(marketplace.skills_root) + + assert result.updated == ["scratch-skill"] + assert result.refused == [] + assert check_drift(marketplace.skills_root).clean + + +def test_relock_tracks_an_untracked_skill_as_local_and_then_validates_clean( + marketplace, tmp_path +): + import shutil + + shutil.copytree( + _write_source(tmp_path, name="hand-written", tier="experimental"), + marketplace.skills_root / "hand-written", + ) + + result = relock(marketplace.skills_root) + + assert result.added == ["hand-written"] + entry = SkillLock.load(marketplace.skills_root).get("hand-written") + # Never claim hub provenance for a skill that was never installed from one. + assert entry.source == SOURCE_LOCAL + assert entry.content_digest.startswith("sha256:") + assert check_drift(marketplace.skills_root).clean + + +def test_relock_drops_the_entry_for_a_skill_that_is_gone(marketplace, unattested): + import shutil + + shutil.rmtree(unattested.path) + + result = relock(marketplace.skills_root) + + assert result.removed == ["scratch-skill"] + assert "scratch-skill" not in SkillLock.load(marketplace.skills_root).entries + assert check_drift(marketplace.skills_root).clean + + +def test_relock_refuses_to_re_record_an_attested_skill_that_drifted( + marketplace, attested +): + """Rewriting the digest would leave a signature asserting bytes it never signed.""" + before = SkillLock.load(marketplace.skills_root).get("web-research").content_digest + (attested.path / "tools.py").write_text("import socket\n", encoding="utf-8") + + result = relock(marketplace.skills_root) + + assert [d.skill for d in result.refused] == ["web-research"] + assert "web-research" not in result.updated + after = SkillLock.load(marketplace.skills_root).get("web-research") + assert after.content_digest == before + # And the skill still refuses to load — relock did not launder it. + marketplace.manager.reload() + with pytest.raises(SkillDriftError): + marketplace.manager.load("web-research") + + +def test_relock_refuses_a_hub_entry_whose_manifest_changed_tier( + marketplace, unattested +): + """Changing the enforced tier by editing the file is never re-recordable.""" + _retier(unattested.path, "community") + + result = relock(marketplace.skills_root) + + assert [d.skill for d in result.refused] == ["scratch-skill"] + entry = SkillLock.load(marketplace.skills_root).get("scratch-skill") + assert entry.installed_tier == "experimental" + + +def test_relock_never_writes_a_hub_entrys_tier_from_the_manifest_it_governs( + marketplace, unattested +): + """A legacy hub entry with no recorded tier must not adopt the file's claim. + + ``installed_tier`` is what install *enforced* against the signature. Taking + it from ``SKILL.md`` would let anyone promote a hub skill by editing it and + running relock — the reachable case is an entry written before the tier was + recorded, where the drift check has no enforced tier to compare against. + """ + lock = SkillLock.load(marketplace.skills_root) + lock.get("scratch-skill").installed_tier = "" + lock.save() + _retier(unattested.path, "verified") + + relock(marketplace.skills_root) + + entry = SkillLock.load(marketplace.skills_root).get("scratch-skill") + assert entry.source == SOURCE_HUB + assert entry.installed_tier == "" + + +# ---------------------------------------------------------------------- +# Replacing an install locally must retire its provenance +# +# Otherwise 'import --force' over a hub skill leaves a lock entry describing +# bytes that are gone — and the replacement, being unattested, would then be +# refused as tampered-with hub content. +# ---------------------------------------------------------------------- + + +def test_importing_over_a_hub_install_retires_its_lock_entry( + marketplace, tmp_path, attested, monkeypatch +): + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + replacement = _write_source( + tmp_path / "replacement", name="web-research", tier="experimental" + ) + + exit_code = skills_cli.handle( + argparse.Namespace( + skill_action="import", + source=str(replacement), + name="web-research", + force=True, + ) + ) + + assert exit_code == skills_cli.EXIT_OK + assert SkillLock.load(marketplace.skills_root).get("web-research") is None + assert check_drift(marketplace.skills_root).fatal == () + marketplace.manager.reload() + assert marketplace.manager.load("web-research").security_tier == "experimental" + + +def test_creating_over_a_hub_install_retires_its_lock_entry( + marketplace, attested, monkeypatch +): + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + + exit_code = skills_cli.handle( + argparse.Namespace( + skill_action="create", + name="web-research", + directory=None, + description=None, + with_tools=False, + force=True, + ) + ) + + assert exit_code == skills_cli.EXIT_OK + assert SkillLock.load(marketplace.skills_root).get("web-research") is None + assert check_drift(marketplace.skills_root).fatal == () + + +# ---------------------------------------------------------------------- +# CLI +# ---------------------------------------------------------------------- + + +def _lock_args(**kwargs): + kwargs.setdefault("check", False) + kwargs.setdefault("relock", False) + kwargs.setdefault("as_json", False) + return argparse.Namespace(skill_action="lock", **kwargs) + + +def test_cli_skill_lock_exits_invalid_on_drift_and_ok_when_clean( + marketplace, unattested, monkeypatch, capsys +): + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + + assert skills_cli.handle(_lock_args()) == skills_cli.EXIT_OK + + (unattested.path / "notes.md").write_text("scratch\n", encoding="utf-8") + assert skills_cli.handle(_lock_args()) == skills_cli.EXIT_INVALID + assert "scratch-skill" in capsys.readouterr().err + + args = _lock_args() + args.relock = True + assert skills_cli.handle(args) == skills_cli.EXIT_OK + assert skills_cli.handle(_lock_args()) == skills_cli.EXIT_OK + + +def test_cli_skill_lock_relock_reports_a_refusal_as_a_failure( + marketplace, attested, monkeypatch, capsys +): + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + (attested.path / "tools.py").write_text("import socket\n", encoding="utf-8") + + args = _lock_args() + args.relock = True + + assert skills_cli.handle(args) == skills_cli.EXIT_INVALID + assert "Refused to re-record" in capsys.readouterr().err + + +def test_cli_skill_lock_json_carries_every_drift( + marketplace, unattested, monkeypatch, capsys +): + import json + + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + _reversion(unattested.path, "2.0.0") + + args = _lock_args() + args.as_json = True + exit_code = skills_cli.handle(args) + + payload = json.loads(capsys.readouterr().out) + assert exit_code == skills_cli.EXIT_INVALID + assert payload["clean"] is False + assert {d["kind"] for d in payload["drifts"]} >= {DRIFT_VERSION, DRIFT_CONTENT} + + +# ---------------------------------------------------------------------- +# What a user actually reads (#2929 review) +# ---------------------------------------------------------------------- + + +def _list_args(**kwargs): + kwargs.setdefault("root", None) + kwargs.setdefault("as_json", False) + return argparse.Namespace(skill_action="list", **kwargs) + + +def _drop_digest(root, name): + """Make an entry look like one written before digests were recorded.""" + lock = SkillLock.load(root) + lock.get(name).content_digest = "" + lock.save() + + +def test_relock_refuses_to_stamp_a_digest_onto_a_pre_digest_attested_entry( + marketplace, attested +): + """The upgrade state is the easiest place to launder an attestation. + + A ``community`` entry with no digest has never had its bytes checked. If + ``--relock`` records the current ones, whatever is on disk inherits the + publisher's signature — and the pre-digest state is exactly where every + upgrading user starts, so this is the likeliest path, not an exotic one. + """ + _drop_digest(marketplace.skills_root, "web-research") + (attested.path / "tools.py").write_text("import socket\n", encoding="utf-8") + + result = relock(marketplace.skills_root) + + assert result.updated == [] + assert [d.kind for d in result.refused] == [DRIFT_UNRECORDED] + assert ( + SkillLock.load(marketplace.skills_root).get("web-research").content_digest == "" + ) + assert "gaia skill install web-research --force" in result.refused[0].remediation + + +def test_relock_still_records_a_pre_digest_entry_that_nothing_attested( + marketplace, unattested +): + """The refusal must be scoped to attestation, not to every missing digest.""" + _drop_digest(marketplace.skills_root, "scratch-skill") + + result = relock(marketplace.skills_root) + + assert result.refused == [] + assert check_drift(marketplace.skills_root).clean + + +def test_cli_skill_list_calls_a_pre_digest_entry_unverifiable_not_changed( + marketplace, unattested, monkeypatch, capsys +): + """Every upgrading user hits this path; it must not read as tampering.""" + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + _drop_digest(marketplace.skills_root, "scratch-skill") + + assert skills_cli.handle(_list_args()) == skills_cli.EXIT_OK + + err = capsys.readouterr().err + assert "predate content digests" in err + assert "gaia skill lock --relock" in err + assert "no longer match" not in err + + +def test_cli_skill_lock_says_predate_rather_than_differ_for_pre_digest_entries( + marketplace, unattested, monkeypatch, capsys +): + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + _drop_digest(marketplace.skills_root, "scratch-skill") + + assert skills_cli.handle(_lock_args()) == skills_cli.EXIT_INVALID + + err = capsys.readouterr().err + assert "predate content digests" in err + assert "difference(s)" not in err + + +def test_an_untracked_skill_does_not_undo_the_predate_wording( + marketplace, tmp_path, unattested, monkeypatch, capsys +): + """Untracked is the normal residue of create/import — not a difference.""" + import shutil + + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + _drop_digest(marketplace.skills_root, "scratch-skill") + shutil.copytree( + _write_source(tmp_path, name="hand-written", tier="experimental"), + marketplace.skills_root / "hand-written", + ) + + assert skills_cli.handle(_lock_args()) == skills_cli.EXIT_INVALID + + err = capsys.readouterr().err + assert "predate content digests" in err + assert "1 the lock does not track" in err + assert "difference(s)" not in err + + +def test_cli_skill_list_reports_a_real_change_as_a_mismatch( + marketplace, attested, monkeypatch, capsys +): + """The wording split must not blunt the finding that matters.""" + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + (attested.path / "tools.py").write_text("import socket\n", encoding="utf-8") + marketplace.manager.reload() + + assert skills_cli.handle(_list_args()) == skills_cli.EXIT_OK + + err = capsys.readouterr().err + assert "no longer match" in err + assert "1 will refuse to load" in err + assert "predate content digests" not in err + + +def test_cli_skill_list_reports_a_damaged_lock_without_hiding_the_skills( + marketplace, attested, monkeypatch, capsys +): + """A truncated lock must not take the inventory down with it.""" + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + (marketplace.skills_root / "skill-lock.json").write_text("{trunca", "utf-8") + + assert skills_cli.handle(_list_args()) == skills_cli.EXIT_INVALID + + captured = capsys.readouterr() + assert "web-research" in captured.out + assert "not readable JSON" in captured.err + + +def test_cli_skill_list_json_carries_the_lock_error_instead_of_crashing( + marketplace, attested, monkeypatch, capsys +): + import json + + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + (marketplace.skills_root / "skill-lock.json").write_text("{trunca", "utf-8") + + exit_code = skills_cli.handle(_list_args(as_json=True)) + + payload = json.loads(capsys.readouterr().out) + assert exit_code == skills_cli.EXIT_INVALID + assert payload["drift"] is None + assert "not readable JSON" in payload["lock_error"] + assert [s["name"] for s in payload["skills"]] == ["web-research"] + + +@pytest.mark.parametrize("relock_flag", [False, True], ids=["check", "relock"]) +@pytest.mark.parametrize( + "lock_text,expected", + [ + ("{trunca", "not readable JSON"), + ('{"schema_version": 99, "skills": {}}', "is schema v99"), + ], + ids=["truncated", "future-schema"], +) +def test_cli_skill_lock_reports_a_damaged_lock_cleanly( + marketplace, attested, monkeypatch, capsys, relock_flag, lock_text, expected +): + """`gaia skill lock` must exit 4 with the message, never a traceback. + + Unlike ``list``, this verb has nothing to render alongside the failure, so + ``handle()``'s ``SkillValidationError`` arm is the whole error path. Pinned + here because narrowing that arm would turn every corrupt-lock run into a + stack trace, and only an end-to-end assertion catches that. + """ + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + (marketplace.skills_root / "skill-lock.json").write_text(lock_text, "utf-8") + + args = _lock_args() + args.relock = relock_flag + + assert skills_cli.handle(args) == skills_cli.EXIT_INVALID + assert expected in capsys.readouterr().err + + +def test_cli_skill_lock_json_does_not_emit_a_half_report_for_a_damaged_lock( + marketplace, attested, monkeypatch, capsys +): + """Nothing on stdout beats JSON describing a check that never ran.""" + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + (marketplace.skills_root / "skill-lock.json").write_text("{trunca", "utf-8") + + exit_code = skills_cli.handle(_lock_args(as_json=True)) + + captured = capsys.readouterr() + assert exit_code == skills_cli.EXIT_INVALID + assert captured.out == "" + assert "not readable JSON" in captured.err + + +def test_a_damaged_lock_still_refuses_to_load_a_user_root_skill(marketplace, attested): + """Corrupting one file must not switch the drift check off.""" + (marketplace.skills_root / "skill-lock.json").write_text("{trunca", "utf-8") + marketplace.manager.reload() + + with pytest.raises(SkillValidationError): + marketplace.manager.load("web-research") + + +def test_cli_relock_does_not_claim_success_when_everything_was_refused( + marketplace, attested, monkeypatch, capsys +): + monkeypatch.setattr(skills_cli, "_manager", lambda: marketplace.manager) + (attested.path / "tools.py").write_text("import socket\n", encoding="utf-8") + + args = _lock_args() + args.relock = True + assert skills_cli.handle(args) == skills_cli.EXIT_INVALID + + captured = capsys.readouterr() + assert "Re-recorded" not in captured.out + assert "Refused to re-record" in captured.err + + +def test_a_local_skills_remediation_never_tells_you_to_reinstall_from_the_hub( + marketplace, tmp_path +): + """An actionable error must name an action the skill can actually take.""" + import shutil + + shutil.copytree( + _write_source(tmp_path, name="hand-written", tier="experimental"), + marketplace.skills_root / "hand-written", + ) + relock(marketplace.skills_root) + (marketplace.skills_root / "hand-written" / "extra.md").write_text("x", "utf-8") + + drift = check_drift(marketplace.skills_root).for_skill("hand-written")[0] + + assert "gaia skill lock --relock" in drift.remediation + assert "gaia skill install" not in drift.remediation