Repository navigation
perf(staged): add the worktree while a transfer from Blox is still capturing - #1089
Conversation
…pturing Moving a project from a Blox workspace to this machine, each branch's local worktree was added only after the branch had been snapshotted, downloaded and landed, and the step shown as "Adding the worktree" also covered a cold capture of the new worktree's shell environment. The worktree needs only a commit to check out, and the clone already holds one nearby. Each leg now picks it before its capture starts: the head the inspection saw when the clone has it, else origin's copy of the branch, its upstream, or the base. It records the worktree in the undo list, then adds it detached at that commit, with hooks off, on a scoped thread that also starts the shell-environment capture. The snapshot, the download and the landing run on the leg's thread as before. Once the branch has landed, `git checkout <branch>` moves the worktree onto it, which rewrites only the files that differ and fires `post-checkout` once, and the finish runs as it did. The early worktree is an optimisation with a complete fallback. With no commit to start from, an early add or switch that fails, or a guess about Git LFS (from the inspection's counts) that the snapshot contradicts, the leg clears the early worktree and adds one on the branch at the same path, as before. A capture that fails or is cancelled returns its own error; the thread is always joined first and rollback removes the recorded worktree, so no cleanup path is new. Only transfers to local take this path. The prewarmed environment was captured at the start commit, so it is dropped after the switch when the two commits differ under `bin/` or in a root-level marker file the cache keys on, and the finish recaptures. The path's cached environment is also dropped before the prewarm, since anything cached there belongs to an earlier worktree. Progress phases are unchanged. Each leg now logs its early add, switch, environment wait and finish durations and whether it fell back. Tests cover the worktree being added while the snapshot runs, a snapshot failing after the early add, a head that moves after the inspection, the early add failing, the LFS guess being wrong, the undo list naming the worktree at the first snapshotting event, the start-commit choice, the invalidation rule, and the detached add and switch firing one hook. Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…o hooks A transfer from Blox adds each branch's worktree while the capture still runs and starts a shell-environment capture in it. Review of that change found four things to fix. Rollback could delete the worktree while that capture's login shell was still starting inside it, since the leg joined the add but not the capture. Whatever the shell wrote could stop the removal or bring the directory back. The shell-environment cache has no way to stop a capture, so it gains `wait_out_and_invalidate`: wait for the capture in flight for a directory, never starting one, then drop what is cached. The shared removal routine calls it first, which covers rollback, startup recovery and the fallback alike, and replaces the fallback's own wait. A cancel during the prewarm now takes up to one capture longer. A capture made stale by the switch is waited out the same way instead of being dropped while it runs, so no second shell starts beside it and none is left where a later removal cannot wait for it. A `git worktree add` killed part way leaves an entry locked with the reason `initializing`. A singly forced removal refuses it, a prune keeps it, and every later add at that path fails. The removal routine now forces twice, and unlocks before pruning when that fails too. `post-checkout` fired from the switch, so it was told the start commit was the previous head, and for a fully pushed branch got the same commit twice. A hook that sets up a new worktree on seeing the null id did nothing. Hooks are now off for the switch as well, and the hook is fired once afterwards by `git hook run` with what `git worktree add <path> <branch>` passes: the null id, the head and `1`. A hook that fails there fails the leg, as it fails `git worktree add`, rather than falling back to an add that would fire it again. `git hook run` needs git 2.36, so a leg whose git lacks it plans no early worktree and adds late as before. One difference remains: git exports `GIT_DIR` to a hook it fires inside a linked worktree, where `git worktree add` clears it. The path with no early worktree now also drops what is cached for its path before adding, as the early path does. Tests: a gate holds the early add until the snapshot has started and the snapshot until the add has finished, so the overlap is proven rather than only the missing dependency. Hook tests record the arguments on both paths and compare them with a worktree git added itself, cover the equal-commits case, a failing hook and a git without `git hook run`. A real `git worktree add` is killed in a smudge filter to reproduce the locked entry, with and without its directory, and removal plus a re-add at the same path are checked. The shell-environment calls, no-ops under test, are now recorded per watched directory with whether the path existed, which pins the wait before the removal on rollback and on the fallback, and the wait before a recapture. The cache method is tested against a real held shell. Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…checkout A transfer from Blox adds each branch's worktree early and starts a shell-environment capture in it, then fires `post-checkout` once the worktree is on its branch. Review of that found the capture ran before the hook. A hook that sets a new worktree up, by allowing its `.envrc`, trusting a tool-versions file or writing either, did so after the capture's shell had started, and the finish then used that capture, which stays cached for up to an hour. A worktree added late is first captured after `git worktree add` has fired the hook. The hook also ran beside the capture's login shell in the same directory. Once the worktree is on its branch, the leg now asks which `post-checkout` hooks git would run there. When one would, it waits out the early capture and drops it before firing the hook, so no shell of that capture is in the directory while the hook works and the finish captures after it: the order the late path has. A repo with no such hook keeps the capture. The check that the two commits differ in what a capture reads shares that one wait, now before the hook instead of after it. `git::post_checkout_hooks_for_worktree` answers the question the way `git hook run` will: the hook directory's file from `git rev-parse --git-path`, which follows `core.hooksPath` from the worktree, counted when it is there and executable, and git's own `git hook list` where it has one, which is where git also runs hooks named in config. What it cannot tell counts as a hook. One hook is exempt: the `post-checkout` that `git lfs install` writes, so Git LFS repos keep the head start. `git lfs post-checkout` only sets the write flag of lockable files. The hook is recognised by its three lines: `#!/bin/sh`, the guard that reports a missing `git-lfs` and exits 2, and `git lfs post-checkout "$@"`. Git LFS has reworded the report several times, so its text is free as long as it stays one double-quoted string with nothing the shell would run. Anything added or changed, or a second hook from config beside it, makes it any other hook. A hook manager that owns the machine's `core.hooksPath` and installs a `post-checkout` for every repo counts as a hook, since what it goes on to run cannot be known. Repos under one lose the head start on the environment unless they set a hook directory of their own. The doc comment on `run_post_checkout_as_for_a_new_worktree` named `GIT_DIR` as the one difference from a late add. It now states the second: a relative `core.hooksPath` is resolved from the new worktree, so the branch's copy of the hook runs where `git -C <clone> worktree add` runs the main checkout's, and a branch without that directory fires no hook, silently. The behaviour is unchanged. Each leg's log line says when the early capture was dropped, how long the wait took and why. Tests: with a hook, the capture is waited out and dropped while the hook has not yet run, and the finish's wait comes after it; firing the hook first fails that test. With an empty hook directory, with a hook git would ignore for not being executable, and with the stock Git LFS hook, the capture is kept. The detection is tested against a real linked worktree for each case, a relative `core.hooksPath` whose directory only one branch has, a hook named in config, the four texts Git LFS has shipped, edited copies of the stock hook, and what `git lfs install` writes on the machine running the tests. The shell-environment recorder can note whether a file existed at each call, which is how the hook is placed among them. Tests that look at those calls give the clone its own hook directory, so a machine-wide one does not decide them. Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…-checkout hook A transfer from Blox adds each branch's worktree early with the hook directory turned off, switches it onto the branch the same way, then fires `post-checkout` once with `git hook run`. Review of that found one defect and two things to tighten. `core.hooksPath=/dev/null` only turns the hook directory off. Git 2.54 and later also run hooks that config names (`hook.<name>.event` with `hook.<name>.command`), wherever the hook directory points. Such a `post-checkout` fired at the detached add, told of the start commit, again at the switch, told the worktree came from there, and a third time on request. A leg now asks the clone's effective config, with `git config --get-regexp` over the `hook.<name>.event` keys, before it plans an early worktree. When any of them is `post-checkout` it plans none and adds the worktree once the branch has landed, as before this branch, so git fires the hook once with the null id. An event counts even if a later empty value or `hook.<name>.enabled` takes it back, and on a git too old to run it. The check that decides whether the early capture of the shell environment must be dropped read `git hook list` and took any failure for "no hooks named in config": no hooks, a git without the subcommand, a config git could not parse, and a child that never ran all looked the same. It now uses the same config query, which every git answers. Only exit 1 with nothing printed is "none". Any other exit, a killed child, or a git that cannot be spawned counts as a hook, so the capture is dropped. `cli::run_output` returns the exit status with the captured environment, as `run_lite_output` does with the inherited one. A hook that may do anything was only looked for once the worktree was on its branch, so the early add always started a capture that such a hook then cost: a login shell, one of the four capture slots, and a wait before the hook. `git::post_checkout_hooks_to_come` now answers from the clone before the add. The default hook directory and an absolute `core.hooksPath` are the same from every worktree, so a hook there is already known, and the early add then starts no capture. The worktree is still added early. A relative `core.hooksPath` makes the hook the branch's own file, so that case alone is still decided once the worktree is on its branch, with the wait-out as before. The check on the branch stays in every case, and still waits out whatever capture is in flight before a hook fires. Each leg's log line says when no early capture was started. The clone's config is read as its main checkout reads it. A conditional include that only the new worktree matches is not seen before the add. Tests no longer depend on the machine's git config. A test-only guard, `OwnConfigOnly`, keyed by directory like the shell-environment recorder, makes the git that `git::cli` runs for repos under it read no global or system config. The early-worktree tests and the hook-detection tests hold one, so a hook manager's `core.hooksPath` or a hook named in the user's config no longer decides them, and the default hook directory is exercised. The per-test empty hook directory is gone. Tests: a clone with a config-named hook that logs every invocation gets its worktree added on the branch, starts nothing early, and logs one line with the null id; with the plan check removed that test fails. A hook in an absolute hook directory keeps the early add, starts no capture, and runs before the finish's. A hook committed on the branch under a relative `core.hooksPath` keeps the earlier order: prewarm, wait-out, hook, capture. The detection is tested from the worktree and from the clone for the default directory, an absolute one, a relative one and a config-named hook, and the two must agree where the directory is shared. The exit-status reading is tested over "none", other events, a taken-back event, exits 128, 129 and 3, an exit 1 with a complaint or with output, and a killed child. A repo whose config git cannot parse, and a path that is no repo, count as having a hook. Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c6e7df3a9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let mut args = vec!["diff", "--name-only", "--no-renames", start, head, "--"]; | ||
| for input in crate::shell_env::env_root_inputs() { | ||
| args.push(input); | ||
| } | ||
| match crate::git::cli_run(clone, &args) { | ||
| Ok(changed) => !changed.trim().is_empty(), |
There was a problem hiding this comment.
Invalidate early captures when indirect environment inputs change
When the remote branch changes a file sourced by an unchanged environment marker, this whitelist incorrectly retains the environment captured at the starting commit. For example, an unchanged .envrc containing . ./project-env.sh can export different tool paths or variables after project-env.sh changes, yet this diff is empty because that file is outside env_root_inputs(). The finish script and subsequent commands then use the old environment, potentially for the cache's one-hour TTL, whereas the previous late capture read the landed commit. Conservatively invalidate when commits differ unless all dependencies of the environment configuration are known.
Useful? React with 👍 / 👎.
Summary
When moving a project from a Blox workspace to this machine, each branch's local worktree was added only after the branch had been snapshotted, downloaded and landed, and the "Adding the worktree" step also covered a cold capture of the new worktree's shell environment. This PR adds the worktree early, while the transfer is still capturing, and then hardens that path against the problems review turned up.
git checkout <branch>moves the worktree onto it. Any failure, a missing start commit, or a wrong Git LFS guess falls back to the previous late add at the same path.wait_out_and_invalidate, and the shared worktree removal routine waits out any in-flight capture before deleting, so rollback cannot pull a directory out from under a starting login shell. Removal also handles theinitializinglock left by a killedgit worktree add(force twice, then unlock and prune).post-checkoutis fired once afterwards viagit hook runwith the null id, the head and1. A git older than 2.36 plans no early worktree.post-checkouthook would run on the branch, the early capture is waited out and dropped before the hook fires, so the finish captures after it. The stock Git LFS hook is exempt so LFS repos keep the head start.hook.<name>.event(git 2.54+) run regardless ofcore.hooksPath=/dev/null, so a clone with such apost-checkoutplans no early worktree. Hooks in the default or an absolute hook directory are detected from the clone before the add so no doomed capture is started; a relativecore.hooksPathis still decided once the worktree is on its branch.Progress phases are unchanged. Each leg logs its early add, switch, environment wait and finish durations, whether it fell back, and whether the early capture was skipped or dropped.
Tests
New tests in
orchestrator_tests/early_worktree.rs,git/post_checkout_hook/tests.rsandshell_env/wait_out_tests.rscover the overlap of the add with the snapshot (gated both ways), fallbacks, the start-commit choice, the invalidation rule, hook arguments compared against a realgit worktree add, a hook killed mid-add leaving a locked entry, the LFS hook texts, detection of default/absolute/relative/config-named hooks from the worktree and the clone, and the cache wait against a real held shell. A test-onlyOwnConfigOnlyguard keeps the machine's git config out of these tests.🤖 Generated with Claude Code