Skip to content

Make the CI pipeline work with Pester 6 - #1743

Open
Mike-Crowley wants to merge 1 commit into
dfinke:masterfrom
Mike-Crowley:fix-ci-pester5
Open

Mike-Crowley wants to merge 1 commit into
dfinke:masterfrom
Mike-Crowley:fix-ci-pester5

Conversation

@Mike-Crowley

@Mike-Crowley Mike-Crowley commented Jul 29, 2026 •

Copy link
Copy Markdown

Fixes #1744

Problem

Every Azure Pipelines job currently fails on every PR — including trivial ones — before running a single test. All six jobs (e.g. on #1721, #1723, #1729, #1741, #1742) stop in the "Install and Test" step with:

Invoke-Pester : A parameter cannot be found that matches parameter name 'OutputFile'.
At D:\a\1\s\CI\CI.ps1:113 char:34

The pipeline's "Update Pester" step installs the latest Pester (currently 6.x) on every agent, but CI/CI.ps1 still uses the Pester 4 invocation syntax: -OutputFile belongs to the legacy parameter set, which Pester 5 deprecated and Pester 6.0.0 removed, and the result object's TestResult property became Tests/Failed.

Changes

CI/CI.ps1 — when New-PesterConfiguration exists (Pester 5+), build a configuration object instead: TestResult.Enabled + OutputPath produce the same TestResultsPS<version>.xml NUnit file the PublishTestResults task consumes (the default output format is NUnit 2.5, matching the task's testResultsFormat: 'NUnit'), and Filter.Tag/Filter.ExcludeTag reproduce the TestImportOnly handling. Failed tests are listed from the v5 Failed collection. On older Pester the original code runs unchanged.

Five tests — with the invocation fixed, five tests still fail under Pester 6 because Set-ItResult -Pending no longer exists (deprecated in Pester 5, removed in 6.0.0) (Parameter set cannot be resolved...). These are all placeholders for known EPPlus 4.5 bugs (plus one "can't test passwords on PS6+" guard); they now use -Skipped, which Pester 4.7+ and 5+ both support:

  • Export-Excel.Tests.ps1, ExtraLongCmd.tests.ps1, First10Races.tests.ps1, Join-Worksheet.tests.ps1 ("Bug in EPPLus 4.5")
  • PasswordProtection.tests.ps1 ("Can't test passwords on V6 and later")

Compare-WorkSheet.tests.ps1 — the Windows PowerShell GridView test spawns a child powershell.exe -Command $cmdline; native argument passing mangles the embedded quotes, so the child's Import-Module fails whenever the repository path contains spaces and the test fails with Expected 'FF90EE90', but got $null. CI agents (D:\a\1\s) never see this, but local contributors with spaces in their clone path do. The child is now launched with -EncodedCommand, which sidesteps quoting entirely.

Verification

Ran ./CI/CI.ps1 -Test end to end (including CI/Install.ps1, against a sandboxed PSModulePath) with Pester 6.0.1:

Result
PowerShell 7.6 277 passed, 0 failed, 7 skipped
Windows PowerShell 5.1 287 total, 0 failures (NUnit XML)
./CI/CI.ps1 -TestImportOnly tag filter selects exactly the 1 import test

Also verified the failure branch behaves (a failing test produces the "Test failures:" listing and a non-zero exit), and that the produced TestResultsPS*.xml is valid NUnit 2.5 with one test-case per test.

This should let the pipeline actually validate the open PRs again.

🤖 Generated with Claude Code

Every Azure Pipelines job installs the latest Pester and then runs
CI/CI.ps1, which still used the Pester 4 invocation syntax, so every
job on every PR failed before running a single test:

  Invoke-Pester : A parameter cannot be found that matches parameter
  name 'OutputFile'.

CI.ps1 now builds a configuration object via New-PesterConfiguration
when it is available (Pester 5+), producing the same NUnit test-results
file and tag filtering as before, and falls back to the original
parameters on older Pester. Failed tests are listed from the v5 result
object's Failed collection.

With the invocation fixed, five tests still fail under Pester 5+
because Set-ItResult -Pending no longer exists; those known-EPPlus-bug
placeholders now use -Skipped, which both Pester 4.7+ and 5+ support.

Compare-WorkSheet's Windows PowerShell GridView test also now launches
its child process with -EncodedCommand: passing the command via
-Command mangled the embedded quotes, which broke the test whenever the
repository path contains spaces (CI agents were unaffected).

Verified by running ./CI/CI.ps1 -Test end to end with Pester 6.0.1 on
PowerShell 7 (277 passed, 0 failed) and Windows PowerShell 5.1
(287 total, 0 failures), plus -TestImportOnly (tag filter selects the
single import test), and confirming the NUnit XML matches what
PublishTestResults expects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the repository’s CI test runner and a handful of tests to be compatible with Pester 5+ (while keeping a Pester 4 fallback), unblocking Azure Pipelines from failing before tests execute.

Changes:

  • Updated CI/CI.ps1 to use New-PesterConfiguration / Invoke-Pester -Configuration when available (Pester 5+), retaining the legacy Pester 4 invocation otherwise.
  • Replaced Set-ItResult -Pending with Set-ItResult -Skipped in five placeholder/guard tests to avoid Pester 5+ parameter-set failures.
  • Fixed Windows PowerShell child process invocation in Compare-WorkSheet.tests.ps1 by switching to -EncodedCommand to avoid quote-mangling when paths contain spaces.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
CI/CI.ps1 Switches CI Pester invocation to configuration-object mode for Pester 5+ while preserving Pester 4 behavior.
tests/PasswordProtection.tests.ps1 Updates placeholder test outcome from Pending to Skipped for Pester 5+ compatibility.
tests/Join-Worksheet.tests.ps1 Updates placeholder test outcome from Pending to Skipped for Pester 5+ compatibility.
tests/First10Races.tests.ps1 Updates placeholder test outcome from Pending to Skipped for Pester 5+ compatibility.
tests/ExtraLongCmd.tests.ps1 Updates placeholder test outcome from Pending to Skipped for Pester 5+ compatibility.
tests/Export-Excel.Tests.ps1 Updates placeholder test outcome from Pending to Skipped for Pester 5+ compatibility.
tests/Compare-WorkSheet.tests.ps1 Uses powershell.exe -EncodedCommand to make module import robust when repo paths contain spaces.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@franklesniak

Copy link
Copy Markdown

Test results for this PR

To check this PR with both Pester 5.9.0 and Pester 6.2.0, I ran the full Pester suite on its head commit, 9a8334a, whose parent is master at 5387c06. The runs used Invoke-Pester on the __tests__ folder directly, not CI/CI.ps1. Here are the results:

Shell Pester Passed Failed Skipped NotRun
PowerShell 7.6.7 6.2.0 278 0 7 3
PowerShell 7.6.7 5.9.0 278 0 7 3
Windows PowerShell 5.1 6.2.0 280 2 6 3
Windows PowerShell 5.1 5.9.0 280 2 6 3

On master with Pester 6.2.0, every test that calls Set-ItResult -Pending fails. That's five tests on PowerShell 7. Windows PowerShell 5.1 has four of them, because the "Password Supported" test only exists on PowerShell 6 and later. With this PR, those tests are skipped instead. Pester 5.9.0 already reports them as skipped on master, so this PR doesn't change the Pester 5.9.0 results.

The two failures on Windows PowerShell 5.1 are in PasswordProtection.tests.ps1: "Was able to append when the password was included" and "Could read the file when the password was included." Both fail the same way on master on my machine, so this PR doesn't cause them. They also passed in the Windows PowerShell 5.1 job of this PR's own Azure build 1445, which reports 281 tests passed and 0 failed. Since they pass there, they shouldn't block this PR.

These two failures come from an EPPlus bug. After EPPlus opens an encrypted workbook, it leaves the file open, so a later save can't replace the file. I reported that bug in #1750.

Pester 5 compared with Pester 6

Issue #1744 says these problems come from Pester 5 and later, but they start with Pester 6. Invoke-Pester -OutputFile and Set-ItResult -Pending both still exist in Pester 5.7.1, 5.9.0, and 5.9.1, although Pester 5 marks -Pending as deprecated. Pester 6.0.0 removed both. Its release notes say it dropped the legacy parameter set of Invoke-Pester and the -Pending parameter of Set-ItResult. The changes in this PR work on both versions. It might help to say "Pester 6" instead of "Pester 5 and later" in the PR description and the commit message.

This version difference is why only the Azure pipeline fails. Each Azure job runs Install-Module -Name Pester -Force, which now installs Pester 6.2.0. The job then stops at the Invoke-Pester call, before any test runs. The GitHub Actions workflow uses the Pester that comes with each runner image instead. That's Pester 5.9.0 on Ubuntu 24.04, macOS 26 Arm64, and Windows Server 2025 with Visual Studio 2026. For example, the ci.yml run for #1749 passed on all three systems, with -Pending still in the tests.

I didn't test the Compare-WorkSheet.tests.ps1 change from a folder path that contains spaces.

@Mike-Crowley Mike-Crowley changed the title Make the CI pipeline work with Pester 5 and later Make the CI pipeline work with Pester 6 Oct 23, 2026
@Mike-Crowley

Copy link
Copy Markdown
Author

Thanks for running this on both versions, that's exactly the check I couldn't do from the pipeline. You're right about the versioning: Pester 5 only deprecated -Pending and the legacy parameter set, Pester 6.0.0 removed them, so I've retitled this PR and #1744 to say Pester 6. The two Windows PowerShell 5.1 password failures are the EPPlus file-handle bug you filed as #1750, which exists on master without this change. @dfinke this is ready whenever you have a moment; every Azure job still fails before the first test runs.

@dfinke

dfinke commented Oct 23, 2026

Copy link
Copy Markdown
Owner

thanks @Mike-Crowley @franklesniak

i will be turning off the Azure parts.
working with @franklesniak on a CVE he raised.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Azure Pipelines CI fails for every PR: CI.ps1 uses Pester 4 syntax but agents install Pester 6

4 participants