diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/Readme.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/Readme.md index 1b8630e85..78492af71 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/Readme.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/Readme.md @@ -49,7 +49,18 @@ Use **GitHub Codespaces with the Azure / Infra / Sovereign Cloud devcontainer** ### Set up before Challenge 1 1. Sign in with a **personal GitHub account**. Optionally set **Settings > Codespaces > Default idle timeout** at [github.com/settings/codespaces](https://github.com/settings/codespaces) **before creating the Codespace**. The default is 30 minutes; you can choose 5-240 minutes, subject to organization policy. The setting applies to new Codespaces. -2. Open [microsoft/MicroHack](https://github.com/microsoft/MicroHack), then select **Code > Codespaces > ... > New with options**. +2. Open the **repository home page**, [microsoft/MicroHack](https://github.com/microsoft/MicroHack), while signed in to GitHub: + + - Above the file list, select the green **Code** button, then the **Codespaces** tab. This is the repository's Code menu, not a menu inside VS Code. + + GitHub documentation example of the Code menu with the Codespaces tab selected + + - In the **top-right corner of the Codespaces tab**, select **...**, then **New with options**. Do not use the quick-create button: this repository has multiple devcontainers, and you need to choose the Sovereign Cloud one. + + GitHub documentation example showing the Codespaces three-dot menu and New with options + + These two navigation screenshots are examples from [GitHub's Codespaces documentation](https://docs.github.com/en/codespaces/developing-in-a-codespace/creating-a-codespace-for-a-repository#creating-a-codespace-for-a-repository); repository names and existing Codespaces may differ. If the **Codespaces** tab is missing, check that you are signed in. Alternatively, open [Create a codespace](https://github.com/codespaces/new), select **microsoft/MicroHack**, and continue with the options below. + 3. Select **Azure / Infra / Sovereign Cloud** and **2-core**, then **Create codespace**. Wait for setup to finish before using the terminal. ![Codespace creation options showing the Sovereign Cloud devcontainer and a 2-core machine](./img/codespaces-create.png) diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-01.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-01.md index 5310c3285..a20fb3887 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-01.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-01.md @@ -10,7 +10,7 @@ The goal of this exercise is to establish foundational sovereign cloud governanc ## Actions -- Create and assign Azure Policy controls to restrict deployments to EU sovereign regions (Norway East, Germany North, North Europe). +- Create and assign Azure Policy controls to restrict deployments to the lab-approved European regions (Norway East, Germany North, North Europe, West Europe). West Europe accommodates Azure Local management resources when LocalBox is registered there. - Enforce resource tagging requirements for data classification and compliance tracking. - Block public IP resource creation and evaluate storage public-network-access restrictions. Disabling public network access does not create or verify a private endpoint. - Assign least-privilege RBAC roles for the SovereignOps team. diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-06.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-06.md index 9d9359a4f..f9b33db59 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-06.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/challenges/challenge-06.md @@ -2,7 +2,7 @@ ## Goal -The goal of this challenge is to operate a sovereign hybrid cloud environment by combining Microsoft Sovereign Public Cloud and Sovereign Private Cloud components. You will work with Azure Local, simulated via Azure Arc Jumpstart LocalBox, and provision your own VM. You will use Azure Arc to manage that VM through Azure, review its security posture with Microsoft Defender for Cloud, and assess its OS updates with Azure Update Manager. +The goal of this challenge is to operate a sovereign hybrid cloud environment by combining Microsoft Sovereign Public Cloud and Sovereign Private Cloud components. You will work with Azure Local, simulated via Azure Arc Jumpstart LocalBox, and provision your own VM. You will use Azure Arc to manage that VM through Azure, review its security posture with Microsoft Defender for Cloud, and assess its OS updates with Azure Update Manager. You will also deploy a container application in your team's namespace on the shared AKS cluster and access it privately from your Sovereign Cloud Codespace. ## Scenario @@ -10,11 +10,26 @@ Your organization must run workloads in a sovereign cloud while still leveraging ## Actions +Before deploying, confirm that your Challenge 1 exercise policies are back in +**DoNotEnforce** and ask the facilitator to confirm that the shared LocalBox custom +location's Azure region is permitted in your assigned resource group. This region +can differ from the resource group's location. If a policy blocks creation, follow +the [location-policy troubleshooting steps](../walkthrough/challenge-06/solution-06.md#if-validation-or-deployment-is-blocked-by-a-location-policy); +do not disable organizer-managed policies or change allowlists yourself. + * Explore the LocalBox hybrid infrastructure in the Azure Portal -* Deploy a sample application to AKS on Azure Local * Deploy your own VM on Azure Local using Azure Arc VM management and verify that guest management is connected * Verify Microsoft Defender for Cloud coverage and review security recommendations for the VM you provisioned * Use Azure Update Manager to assess OS updates on the VM you provisioned +* Verify your Console-provided Microsoft Entra administrator-group membership and the existing Azure Arc Enabled Kubernetes Cluster User Role with the facilitator; portal workload visibility alone does not prove group membership +* Inspect the organizer-prepared `arcnetworking` extension (`microsoft.arcnetworking`) on `localbox-aks` in `rg-localbox-shared`: confirm **Succeeded**, healthy MetalLB workloads, and the existing **`aks-pool`** with **ARP** advertisement and organizer-reserved service VIPs. Do not install or configure shared networking +* Follow [Task 5 of the walkthrough](../walkthrough/challenge-06/solution-06.md#task-5-deploy-a-container-to-the-aks-cluster-deployed-on-azure-local) in **Bash**: run `az connectedk8s proxy` with Microsoft Entra authentication and an isolated kubeconfig, derive a unique namespace from your assigned resource group, and deploy `aks-local-sample-app.yaml` only in that namespace +* Verify successful rollout and the Service's assigned IP, then keep the proxy and namespace-scoped `kubectl port-forward service/aks-container-1 8080:80` running in separate terminals. Open port **8080** through the **Private** Codespaces Ports view (or `localhost:8080` when running directly on a local workstation). Never expose the API proxy port or use service-account/admin tokens; no RDP or static routes are needed +* Clean up only your team's application resources and, when no longer needed by teammates, your exercise namespace. Leave shared infrastructure unchanged + +The organizer prepares MetalLB with `resources/prepare-localbox.ps1`; the Console's LocalBox deployment hook alone is not sufficient. If extension health, the pool, or access prerequisites are missing, contact the facilitator rather than installing extensions, creating pools, or granting additional permissions. + +The default service VIP range is **`10.10.0.10-10.10.0.100`**, excluding nodes **`10.10.0.101-10.10.0.199`**, control-plane IP **`10.10.0.5`**, and gateway **`10.10.0.1`**. Confirm any organizer customization. Inspect the pool's **IPAddressPool** and **L2Advertisement** in `kube-system` read-only; participants do not create them. ## Success criteria @@ -22,7 +37,11 @@ Your organization must run workloads in a sovereign cloud while still leveraging * You have deployed your own VM on Azure Local via the Azure Portal and verified that guest management is Enabled (Connected) * You have verified Defender for Servers coverage for your VM and reviewed its available recommendations, or identified that its assessment is still pending * You have completed an Azure Update Manager assessment for your VM and reviewed the results, including when no updates are pending -* You have deployed a sample application to AKS on Azure Local +* You have verified the shared MetalLB extension, workloads, and reserved ARP IP pool without changing shared infrastructure +* You have deployed the sample application to your team's unique AKS namespace, verified rollout, and recorded a Service IP from the reserved pool +* You have opened the application through a Microsoft Entra Arc proxy and a private application port-forward, with the API proxy never exposed +* You understand that production clients use the MetalLB VIP through configured network routing; proxy plus port-forward access does **not** validate that load-balancer network path +* You have cleaned up only your team's application resources * You understand how Azure Arc provides a unified control plane for sovereign hybrid scenarios ## Learning resources @@ -34,3 +53,4 @@ Your organization must run workloads in a sovereign cloud while still leveraging * [Microsoft Defender for Cloud with Arc-enabled servers](https://learn.microsoft.com/azure/defender-for-cloud/quickstart-onboard-machines) * [Azure Update Manager overview](https://learn.microsoft.com/azure/update-manager/overview) * [Azure Arc Jumpstart - LocalBox](https://jumpstart.azure.com/azure_jumpstart_localbox) +* [Azure CLI Cluster Connect proxy reference](https://learn.microsoft.com/cli/azure/connectedk8s#az-connectedk8s-proxy) diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/deploy-localbox.ps1 b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/deploy-localbox.ps1 index a80199f84..97b928933 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/deploy-localbox.ps1 +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/deploy-localbox.ps1 @@ -24,6 +24,9 @@ The azure_arc branch, tag, or commit used for both Bicep and runtime artifacts. .PARAMETER AzureLocalResourceProviderObjectId Optional tenant-specific object ID of the Microsoft.AzureStackHCI enterprise application. The script resolves it through Microsoft Graph when omitted. +.PARAMETER AzureLocalInstanceLocation +Azure Local registration region, separate from the Azure host region. +Defaults to West Europe to align with the Challenge 1 location allowlist. .PARAMETER NoWait Submit the deployment without waiting for ARM completion. #> @@ -54,7 +57,7 @@ param( [string]$AzureLocalResourceProviderObjectId, [ValidateSet('australiaeast', 'southcentralus', 'eastus', 'westeurope', 'southeastasia', 'canadacentral', 'japaneast', 'centralindia')] - [string]$AzureLocalInstanceLocation = 'australiaeast', + [string]$AzureLocalInstanceLocation = 'westeurope', [switch]$NoWait ) diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/resource-providers.ps1 b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/resource-providers.ps1 index f8a79f8f9..66091904c 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/resource-providers.ps1 +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/resource-providers.ps1 @@ -85,6 +85,7 @@ $providers = @( # Kubernetes (for AKS Arc) "Microsoft.Kubernetes", "Microsoft.KubernetesConfiguration", + "Microsoft.KubernetesRuntime", "Microsoft.ContainerService", "Microsoft.ContainerInstance", "Microsoft.ContainerRegistry", diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/shared-deploy-lab.ps1 b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/shared-deploy-lab.ps1 index 2b67d3abf..65441eb90 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/shared-deploy-lab.ps1 +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/labautomation/shared-deploy-lab.ps1 @@ -213,7 +213,7 @@ else { -ResourceGroupName $localBoxResourceGroupName ` -Location $localBoxLocation ` -AzureLocalResourceProviderObjectId $azureLocalResourceProviderObjectIds[0] ` - -AzureLocalInstanceLocation 'australiaeast' ` + -AzureLocalInstanceLocation 'westeurope' ` -UseConsoleCredentials ` -NoWait @@ -246,6 +246,25 @@ else { $localBoxScope = "/subscriptions/$SubscriptionId/resourceGroups/$localBoxResourceGroupName" +for ($attempt = 1; $attempt -le 60; $attempt++) { + Update-MhhToken | Out-Null + $runtimeProvider = Get-AzResourceProvider -ProviderNamespace Microsoft.KubernetesRuntime -ErrorAction Stop + if ($runtimeProvider.RegistrationState -eq 'Registered') { break } + Start-Sleep -Seconds 10 +} +if ($runtimeProvider.RegistrationState -ne 'Registered') { + throw 'Microsoft.KubernetesRuntime registration did not complete. LocalBox MetalLB preparation is blocked.' +} +$runtimeObjectIds = @(& az ad sp list --filter "appId eq '087fca6e-4606-4d41-b3f6-5ebdf75b8b4c'" --query '[].id' --output tsv --only-show-errors) +if ($LASTEXITCODE -ne 0) { throw 'Unable to resolve the Microsoft.KubernetesRuntime service principal for MetalLB.' } +$runtimeObjectIds = @($runtimeObjectIds | Where-Object { -not [string]::IsNullOrWhiteSpace($_) }) +$runtimeObjectId = [guid]::Empty +if ($runtimeObjectIds.Count -ne 1 -or -not [guid]::TryParse($runtimeObjectIds[0], [ref]$runtimeObjectId) -or $runtimeObjectId -eq [guid]::Empty) { + throw 'Expected exactly one valid Microsoft.KubernetesRuntime service-principal object ID. Have the tenant administrator verify provider registration.' +} +Update-AzTag -ResourceId $localBoxScope -Operation Merge ` + -Tag @{ 'microhack-k8s-runtime-object-id' = $runtimeObjectId.ToString() } -ErrorAction Stop | Out-Null + foreach ($participantObjectId in $AllowedEntraUserIds) { foreach ($roleName in @('Reader', 'Azure Stack HCI VM Contributor')) { $roleAssignment = Get-AzRoleAssignment ` diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/hosted-events/readme.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/hosted-events/readme.md index 907d07924..389c5634f 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/hosted-events/readme.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/hosted-events/readme.md @@ -21,9 +21,59 @@ The shared hook submits `localbox-*` in `rg-localbox-shared`, then waits up to s 1. Inspect the Azure deployment, Client bootstrap logs and Azure Local/Arc status. Resolve failures before proceeding. 2. If the Client password is unknown, an authorized event lead/coach uses the Azure VM's **Help > Reset password** to set a known password for the `arcdemo` host account, then connects to `LocalBox-Client` through Bastion. Do not expose this password through participant credentials or job logs. A host password reset does not change the nested-node Windows credentials. See [LocalBox credentials](#localbox-credentials). 3. Copy **Lab Group ObjectId** from the Console's **Credentials** tab for the preparation prompt or `-AksAdminGroupObjectId`. Confirm that intended Azure lab identities/coaches are members; see the group dependency below. -4. Follow [LocalBox preparation](../localbox/readme.md), once per shared subscription, in elevated PowerShell 7. Azure authentication uses the Client VM's managed identity; preparation constructs the nested Windows credential locally from the installed configuration without printing it. An explicit `-NodeCredential` overrides that value after a nested-account rotation. +4. Follow [LocalBox preparation](../localbox/readme.md), once per shared subscription, in elevated PowerShell 7. It creates AKS and then installs/verifies the shared MetalLB extension and reserved ARP pool. Azure authentication uses the Client VM's managed identity; preparation constructs the nested Windows credential locally from the installed configuration without printing it. An explicit `-NodeCredential` overrides that value after a nested-account rotation. 5. Run [Pester health checks](../tests/readme.md) for LocalBox and every selected participant lab. A control-plane-only pass does not establish full readiness. 6. Complete the [participant VM readiness exercise](../localbox/manual-preparation.md#step-6-test-the-environment), including guest management, Defender and Update Manager. Subscription-wide paid-plan changes require the authorized owner and approved budget. +7. Verify an intended participant can use Entra-authenticated `az connectedk8s proxy` and a private application port-forward from their Codespace. Participants inspect MetalLB, not install it, and never need LocalBox-Client access. This validates the lab access path, not routing to the MetalLB service VIP. + +Shared setup registers `Microsoft.KubernetesRuntime`, waits for registration, +resolves its tenant service-principal object ID using the Console's existing +Graph-capable identity, and records it in the shared RG tag +`microhack-k8s-runtime-object-id`. Missing registration or identity resolution +fails setup explicitly. Organizer preparation reads that nonsecret tag; it does +not require Graph permissions on the Client managed identity. This does not move +AKS or MetalLB installation into the Console hook: they remain part of the +organizer's post-provisioning preparation. + +### LocalBox registration region and location policies + +There are three separate locations to check: the participant resource group's +metadata location, the Azure region hosting the LocalBox simulator, and the +Azure Local/custom-location registration region. The hosted +[shared hook](../../labautomation/shared-deploy-lab.ps1) explicitly passes +`AzureLocalInstanceLocation = westeurope`; the +[manual deployment](../manual-setup/localbox/deploy-localbox.ps1) defaults to +`westeurope`. Host-region fallback does not change that registration parameter. +The hosted deployer's default also matches West Europe. This changes registration +for fresh deployments, not the Azure host-region selection. Inspect the deployed +custom location's **JSON View** for its actual `location`. + +Azure Local VM management resources use the custom location's region even when +participants create them in a resource group whose location differs. Policy +permission on the shared LocalBox group therefore does not establish permission +to create the participant resources. Before the event, validate the +[Challenge 6 VM creation flow](../../walkthrough/challenge-06/solution-06.md#23-review-and-create) +in the designated test participant group, not only in the shared group. + +For a policy denial, collect the exact assignment/definition IDs and rejected +resource type/location. First check whether a participant left their Challenge 1 +location assignment or bonus initiative enforcing; those exercise assignments +should be **DoNotEnforce**. For an inherited or organizer-owned denial, ask the +policy owner to review the approved locations or an appropriately scoped +exception. Do not disable unrelated policies or automatically broaden the +subscription allowlist. The hosted control tags below do not override arbitrary +location-deny policies. + +[Microsoft's current Azure Local region list](https://learn.microsoft.com/azure/azure-local/concepts/system-requirements-23h2#azure-requirements) +includes West Europe as its only European region for hyperconverged deployments. +The current Challenge 1 exercise allowlists include West Europe, but older +assignments may still use the original three-region list. This does not prove +that West Europe is allowed by the event subscription's inherited policies, and +it does not permit Australia East. Existing Azure assignments are not updated +automatically. Earlier hosted test environments registered in Australia East +are not relocated by this change; validate the next event using a fresh deployment +from the updated content. Existing-environment cleanup is a separate organizer +action, not part of the registration-region change. ### Hosted MCAPS control-tag initiative diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/manual-preparation.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/manual-preparation.md index a1eb3308a..2f03c3462 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/manual-preparation.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/manual-preparation.md @@ -97,6 +97,7 @@ Before participants begin, follow the [Challenge 6 walkthrough](../../walkthroug 3. Confirm that the same VM appears in Defender for Cloud **Inventory**, verify Defender for Servers coverage and onboarding, and review its assessment status. Allow time for recommendations to populate; an empty list is not proof of completed assessment 4. Locate that VM in Azure Update Manager **Resources -> Machines**, run **Check for updates**, and verify a successful assessment and its timestamp. Zero pending updates is a valid result; a pending or failed assessment is not 5. Confirm that neither exercise requires selecting shared cluster nodes, the LocalBox host, Arc Resource Bridge, or another participant's VM, or changing subscription-level settings +6. Complete the organizer [AKS and MetalLB preparation](readme.md#metallb-preparation), then use the same participant identity to inspect MetalLB, deploy the sample into a unique team namespace, and access it through `az connectedk8s proxy` plus a private `kubectl port-forward`. Follow the current Challenge 6 walkthrough; no Client VM credentials or Windows routes should be needed by the attendee. Verify the Service receives a reserved VIP, but record that port-forwarding does not validate routing through that VIP. Resolve connectivity, extension provisioning, or permission failures before the event. Keep any permission changes limited to the operation and resource scope actually required. Do not install patches or restart shared resources during this readiness test. diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/readme.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/readme.md index 1a0516568..249bde74f 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/readme.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/localbox/readme.md @@ -9,9 +9,10 @@ Run [prepare-localbox.ps1](../prepare-localbox.ps1) **inside LocalBox-Client**, - Healthy Azure Local nodes, storage, Arc Resource Bridge, `jumpstart` custom location and `hybridaksextension`. - The Client VM's system-assigned managed identity and its existing LocalBox resource-group permissions. It needs `Microsoft.Authorization/roleAssignments/read` at the AKS connected-cluster scope and, if the group lacks the proxy role, `Microsoft.Authorization/roleAssignments/write` there. Contributor alone cannot create role assignments. An authorized access administrator must arrange this permission or preassign the group role before preparation; the script never elevates its own identity. It uses a temporary isolated CLI login and restores the caller's profile. No subscription-wide or Graph permissions are added. - An existing Entra AKS admin-group object ID. Hosted events use **Lab Group ObjectId** from the Console's **Credentials** tab, published by [shared automation](../hosted-events/readme.md#console-group-dependency). Group membership is not validated through Graph by this script. +- `Microsoft.KubernetesRuntime` registered, and its tenant-specific service-principal **object ID** available for MetalLB. Updated Console setup stores this nonsecret ID in the shared resource group's `microhack-k8s-runtime-object-id` tag. Manual organizers supply `-KubernetesRuntimeObjectId`; see [MetalLB preparation](#metallb-preparation) below. The Client managed identity needs write access to cluster extensions and KubernetesRuntime load balancers at the AKS connected-cluster scope; do not grant participants those permissions for installation. - The installed Jumpstart configuration referenced by `$env:LocalBoxConfigFile` (or `-ConfigPath`), containing `SDNDomainFQDN` and `SDNAdminPassword`. Preparation constructs the nested administrator credential locally without printing it. Supply `-NodeCredential` to override it if the nested password/account has changed. Azure managed identity cannot authenticate Windows PowerShell Direct inside nested nodes. - `Microsoft.EdgeMarketplace` registered by the subscription owner. The Marketplace image workflow also requires the Azure Connected Machine Resource Manager role for the `Microsoft.AzureStackHCI` resource-provider identity on the image resource group; ask the deployment owner to verify this prerequisite. No automatic privilege escalation is attempted. -- Verified DHCP exclusions/static reservations for both pools, sufficient backing `V:` capacity and event budget. Dynamic disks do not add physical backing capacity. +- Verified DHCP exclusions/static reservations for both node/VM pools **and the AKS service VIP range**, sufficient backing `V:` capacity and event budget. Dynamic disks do not add physical backing capacity. For the standard Jumpstart configuration, the nested-node administrator is `jumpstart\Administrator`. The Client VM's `arcdemo` account is not the nested-node administrator. Use the nested domain's credentials (adjust the domain if customized), not a password reset solely on the Client VM. @@ -44,13 +45,65 @@ If the Console-owned group is not available yet, `-SkipAks` explicitly prepares | VM image | `2025-datacenter-azure-edition-smalldisk-01`, matching the walkthrough screenshot; Marketplace Windows Server 2025 Azure Edition smalldisk, explicitly on `UserStorage1` | | VM network | `localbox-vm-lnet-vlan200`, VLAN 200, `192.168.200.0/24`, pool `.10-.199` after reservation review | | AKS network | `localbox-aks-lnet-vlan110`, VLAN 110, `10.10.0.0/24`, nodes `.101-.199` | -| AKS control plane / service reservation | `10.10.0.5`; `.10-.100` reserved for future service VIPs; this script does not install a load balancer | +| AKS control plane / service reservation | `10.10.0.5`; `.10-.100` reserved for MetalLB service VIPs, separate from nodes `.101-.199` | | AKS cluster | `localbox-aks`, one control-plane node and three Linux workers; `Standard_A4_v2` defaults, subject to installed-version capacity/CLI validation | | AKS proxy access | Azure Arc Enabled Kubernetes Cluster User Role for `AksAdminGroupObjectId`, scoped to the connected-cluster resource; matching direct or inherited grants are reused | +| MetalLB | `arcnetworking` Arc extension (`microsoft.arcnetworking`), ARP pool `aks-pool`, addresses from `AKSVIPStartIP` through `AKSVIPEndIP` (standard config: `10.10.0.10-10.10.0.100`) | | Output | `C:\LocalBox\sovereign-localbox.json`, nonsecret IDs and expected configuration for health tests | Topology defaults come from the installed Jumpstart config, except the VM pool bounds and configurable resource names/sizes. The network/broadcast addresses, gateways and Jumpstart infrastructure reservations must not be allocated. `-AddressReservationsConfirmed` is your explicit confirmation that DHCP/static reservations were reviewed; the script cannot safely infer all DHCP leases from Azure. It does not change routers or DHCP configuration. +## MetalLB preparation + +MetalLB is installed automatically **after AKS is ready**, by this organizer +preparation script, not by participants. The Console shared hook still deploys +LocalBox only; it registers the Runtime provider and supplies the object-ID tag +needed for this later step. No Client VM access is granted to attendees. + +The script uses ARM APIs for the extension and load-balancer pool, without +adding CLI extensions or querying Microsoft Graph as the Client managed identity. +For manual setup, an authorized organizer can resolve the provider identity from +their own authenticated session after provider registration: + +```powershell +az ad sp list --filter "appId eq '087fca6e-4606-4d41-b3f6-5ebdf75b8b4c'" --query '[].id' --output tsv +``` + +There must be exactly one result. Use that **object ID**, not the application ID +in the filter, as `-KubernetesRuntimeObjectId` when running preparation on the +Client. If the lookup is denied or empty, ask the tenant administrator; do not +grant Graph permissions to the Client identity. + +The VIP reservation is checked against the AKS subnet, gateway, control-plane +address and node pool. Do not use `10.10.0.150-10.10.0.160` from older screenshots: +it overlaps the node pool. The address-reservation confirmation includes DHCP and +other static consumers of the service VIPs. + +Matching extensions and pools are reused. Conflicting identities, address ranges, +advertisement modes, service selectors, additional pools or differently named +MetalLB extensions stop preparation for organizer review, rather than replacing +shared configuration. Failed operations are reported; no rollback deletes AKS or +an already installed extension. `-SkipAks` also skips MetalLB; `-WhatIf` reports +the planned extension and pool without querying a not-yet-created cluster or +changing Azure resources. + +The manifest records `MetalLb` extension/pool IDs and expected configuration. +Rerun preparation to update older manifests before health validation. Control-plane +checks verify provisioned resources; Full checks additionally verify the Kubernetes +address pool/L2 advertisement and system workloads. Neither proves client access +through the load-balancer VIP. + +Attendees follow [Challenge 6](../../walkthrough/challenge-06/solution-06.md) to +inspect MetalLB, deploy their own namespace, and use Entra-authenticated +`az connectedk8s proxy` plus `kubectl port-forward`. This lab path does not require +Client VM credentials or static routes. Real-world clients use the MetalLB address +through the site's configured network routing; proxy/port-forward access is not +a load-balancer connectivity test. + +Reference: [MetalLB on AKS on Azure Local](https://learn.microsoft.com/azure/aks/aksarc/deploy-load-balancer-cli). + +## Preparation execution details + Azure Local may append generated suffixes to its `UserStorage1` and `UserStorage2` resource names. The script resolves the actual storage-container ID and verifies its custom location; ambiguous names fail rather than selecting the first match. On Windows, the script invokes Azure CLI through its bundled Python executable rather than `az.cmd`. This preserves arguments such as `ConvergedSwitch(compute_management)` that the batch wrapper otherwise interprets as command syntax. Keep the standard Azure CLI installation layout intact. diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/manual-setup/subscription-preparations/1-resource-providers.ps1 b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/manual-setup/subscription-preparations/1-resource-providers.ps1 index 1d9010da6..66732ae13 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/manual-setup/subscription-preparations/1-resource-providers.ps1 +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/manual-setup/subscription-preparations/1-resource-providers.ps1 @@ -114,6 +114,7 @@ $providers = @( # Kubernetes (for AKS Arc) "Microsoft.Kubernetes", "Microsoft.KubernetesConfiguration", + "Microsoft.KubernetesRuntime", "Microsoft.ContainerService", "Microsoft.ContainerInstance", "Microsoft.ContainerRegistry", diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/prepare-localbox.ps1 b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/prepare-localbox.ps1 index 051563230..49023b5e2 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/prepare-localbox.ps1 +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/prepare-localbox.ps1 @@ -11,6 +11,9 @@ versions, including during WhatIf. Azure, Hyper-V and storage changes remain simulated during WhatIf. AKS preparation grants its Entra admin group the cluster-scoped Azure Arc Enabled Kubernetes Cluster User Role for proxy access, reusing applicable existing grants. +It also installs the MetalLB Arc extension and an ARP pool from the configured +service VIP reservation. The Runtime provider object ID comes from the organizer, +not a Graph lookup using the Client managed identity. Dot-source this file to load its functions without running provisioning. #> [CmdletBinding(SupportsShouldProcess)] @@ -19,6 +22,7 @@ param( [string]$ResourceGroupName = $env:resourceGroup, [string]$ConfigPath = $env:LocalBoxConfigFile, [string]$AksAdminGroupObjectId, + [string]$KubernetesRuntimeObjectId, [switch]$SkipAks, [pscredential]$NodeCredential, [string]$VmSwitchName, @@ -192,6 +196,12 @@ function Get-LocalBoxResource { if ($Id -match '/providers/Microsoft\.HybridContainerService/provisionedClusterInstances/') { $arguments += @('--api-version', '2024-01-01') } + elseif ($Id -match '/providers/Microsoft\.KubernetesConfiguration/extensions/') { + $arguments += @('--api-version', '2023-05-01') + } + elseif ($Id -match '/providers/Microsoft\.KubernetesRuntime/loadBalancers/') { + $arguments += @('--api-version', '2024-08-01') + } Invoke-LocalBoxAz $arguments } @@ -266,6 +276,102 @@ function Sync-LocalBoxResource { } } +function Get-LocalBoxMetalLbPlan { + param([Parameter(Mandatory)][hashtable]$Configuration, [Parameter(Mandatory)][string]$ClusterId) + if ($ClusterId -notmatch '^/subscriptions/[^/]+/resourceGroups/[^/]+/providers/Microsoft\.Kubernetes/connectedClusters/[^/]+$') { + throw 'MetalLB must target an Azure Arc connected-cluster resource.' + } + Test-LocalBoxAddressPool $Configuration.AKSIPPrefix $Configuration.AKSVIPStartIP $Configuration.AKSVIPEndIP ` + $Configuration.AKSGWIP @($Configuration.AKSControlPlaneIP) ` + -ReservedRanges @(@{ Start = $Configuration.AKSNodeStartIP; End = $Configuration.AKSNodeEndIP }) + return @{ + ExtensionId = "$ClusterId/providers/Microsoft.KubernetesConfiguration/extensions/arcnetworking" + PoolId = "$ClusterId/providers/Microsoft.KubernetesRuntime/loadBalancers/aks-pool" + PoolExpected = @{ properties = @{ + addresses = @("$($Configuration.AKSVIPStartIP)-$($Configuration.AKSVIPEndIP)") + advertiseMode = 'ARP' + } } + } +} + +function Get-LocalBoxArmCollection { + param([Parameter(Mandatory)][string]$Url) + do { + $page = Invoke-LocalBoxAz @('rest', '--method', 'get', '--url', $Url) + if ($null -eq $page -or -not $page.Contains('value') -or $page.value -isnot [array]) { + throw "Invalid ARM collection response for $Url; existing resources cannot be determined." + } + $page.value + $Url = $page.nextLink + if ($Url -and -not $Url.StartsWith('https://management.azure.com/', [StringComparison]::OrdinalIgnoreCase)) { + throw 'Unexpected ARM pagination URL.' + } + } while ($Url) +} + +function Sync-LocalBoxMetalLb { + [CmdletBinding(SupportsShouldProcess)] + param( + [Parameter(Mandatory)][string]$ClusterId, + [Parameter(Mandatory)][hashtable]$Plan, + [Parameter(Mandatory)][string]$RuntimeObjectId, + [int]$TimeoutSeconds = 3600 + ) + $objectId = [guid]::Empty + if (-not [guid]::TryParse($RuntimeObjectId, [ref]$objectId) -or $objectId -eq [guid]::Empty -or + $objectId -eq [guid]'087fca6e-4606-4d41-b3f6-5ebdf75b8b4c') { + throw 'Supply the tenant-specific Microsoft.KubernetesRuntime service-principal object ID, not its application ID.' + } + $extensionExpected = @{ + identity = @{ type = 'SystemAssigned' } + properties = @{ + extensionType = 'microsoft.arcnetworking' + configurationSettings = @{ k8sRuntimeFpaObjectId = $objectId.ToString() } + autoUpgradeMinorVersion = $true + } + } + if ($WhatIfPreference) { + $null = $PSCmdlet.ShouldProcess($Plan.ExtensionId, 'Install or verify MetalLB Arc extension') + $null = $PSCmdlet.ShouldProcess($Plan.PoolId, 'Create or verify reserved ARP address pool') + return + } + $provider = Invoke-LocalBoxAz @('provider', 'show', '--namespace', 'Microsoft.KubernetesRuntime') + if ($provider.registrationState -ne 'Registered') { + throw 'Ask the subscription owner to register Microsoft.KubernetesRuntime before MetalLB preparation.' + } + $resources = @( + @{ Id = $Plan.ExtensionId; ApiVersion = '2023-05-01'; Expected = $extensionExpected } + @{ Id = $Plan.PoolId; ApiVersion = '2024-08-01'; Expected = $Plan.PoolExpected } + ) + # The Runtime API needs the extension before it can manage in-cluster pools. + foreach ($entry in $resources) { + $collectionId = $entry.Id.Substring(0, $entry.Id.LastIndexOf('/')) + $existing = @(Get-LocalBoxArmCollection "https://management.azure.com${collectionId}?api-version=$($entry.ApiVersion)") + $others = @($existing | Where-Object { $_.id -ine $entry.Id }) + if ($entry.Id -eq $Plan.ExtensionId -and @($others | Where-Object { $_.properties.extensionType -ieq 'microsoft.arcnetworking' }).Count) { + throw 'A differently named MetalLB extension exists. Ask the organizer to review it; no second extension will be installed.' + } + if ($entry.Id -eq $Plan.PoolId -and $others.Count) { + throw 'An additional MetalLB address pool exists. Have the organizer review address overlap and service selection before continuing; existing pools are not changed.' + } + if (-not @($existing | Where-Object id -ieq $entry.Id).Count) { + if (-not $PSCmdlet.ShouldProcess($entry.Id, 'Create shared MetalLB resource')) { + throw 'MetalLB creation was declined; preparation is incomplete.' + } + $body = $entry.Expected | ConvertTo-Json -Depth 10 -Compress + Invoke-LocalBoxAz @('rest', '--method', 'put', '--url', + "https://management.azure.com$($entry.Id)?api-version=$($entry.ApiVersion)", '--body', $body) ` + -TimeoutSeconds $TimeoutSeconds | Out-Null + } + $actual = Wait-LocalBoxResource $entry.Id -TimeoutSeconds $TimeoutSeconds + Assert-LocalBoxProperties $actual $entry.Expected + if ($entry.Id -eq $Plan.PoolId -and $actual.properties.serviceSelector.Count -gt 0) { + throw 'The MetalLB pool has a service selector; organizer review is required.' + } + } + Write-Host "MetalLB extension and ARP pool are provisioned: $($Plan.PoolExpected.properties.addresses -join ', '). Run Full health checks to verify in-cluster readiness." +} + function Sync-LocalBoxAksProxyRole { [CmdletBinding(SupportsShouldProcess)] param([Parameter(Mandatory)][string]$ClusterId, [Parameter(Mandatory)][string]$GroupObjectId) @@ -486,7 +592,7 @@ function Invoke-LocalBoxPreparation { Test-LocalBoxAddressPool $config.AKSIPPrefix $config.AKSControlPlaneIP $config.AKSControlPlaneIP $config.AKSGWIP -ReservedRanges @($vipRange) if ($config.vmVLAN -eq $config.AKSVLAN -or $config.vmIpPrefix -eq $config.AKSIPPrefix) { throw 'VMs and AKS must use separate VLANs and subnets.' } if (-not $Settings.AddressReservationsConfirmed) { - throw 'Verify BOTH pools against DHCP leases/exclusions and static reservations, then pass -AddressReservationsConfirmed. The script does not alter DHCP/router configuration.' + throw 'Verify BOTH VM/node pools and the AKS service VIP reservation against DHCP leases/exclusions and static reservations, then pass -AddressReservationsConfirmed. The script does not alter DHCP/router configuration.' } $Settings.NodeCredential = Resolve-LocalBoxNodeCredential -Configuration $config -Credential $Settings.NodeCredential $previousConfig = $env:AZURE_CONFIG_DIR @@ -512,6 +618,19 @@ function Invoke-LocalBoxPreparation { return } $scope = "/subscriptions/$($Settings.SubscriptionId)/resourceGroups/$($Settings.ResourceGroupName)" + $metalLb = $null + if (-not $Settings.SkipAks) { + $metalLb = Get-LocalBoxMetalLbPlan -Configuration $config -ClusterId "$scope/providers/Microsoft.Kubernetes/connectedClusters/$($Settings.AksClusterName)" + if (-not $Settings.KubernetesRuntimeObjectId) { + $group = Invoke-LocalBoxAz @('group', 'show', '--subscription', $Settings.SubscriptionId, '--name', $Settings.ResourceGroupName) + $Settings.KubernetesRuntimeObjectId = $group.tags.'microhack-k8s-runtime-object-id' + } + $runtimeId = [guid]::Empty + if (-not [guid]::TryParse($Settings.KubernetesRuntimeObjectId, [ref]$runtimeId) -or $runtimeId -eq [guid]::Empty -or + $runtimeId -eq [guid]'087fca6e-4606-4d41-b3f6-5ebdf75b8b4c') { + throw 'Supply -KubernetesRuntimeObjectId from the subscription organizer, or rerun the updated Console shared setup to publish its resource-group tag. The Client does not query Graph.' + } + } $resources = @(Invoke-LocalBoxAz @('resource', 'list', '--resource-group', $Settings.ResourceGroupName)) $cluster = @($resources | Where-Object { $_.type -ieq 'Microsoft.AzureStackHCI/clusters' -and $_.name -eq $config.ClusterName }) if ($cluster.Count -ne 1) { throw 'Azure Local cluster not found; complete Jumpstart provisioning first.' } @@ -615,6 +734,8 @@ function Invoke-LocalBoxPreparation { } if (-not $Settings.SkipAks) { Sync-LocalBoxAksProxyRole -ClusterId $aksId -GroupObjectId $groupId -WhatIf:$WhatIfPreference + Sync-LocalBoxMetalLb -ClusterId $aksId -Plan $metalLb -RuntimeObjectId $Settings.KubernetesRuntimeObjectId ` + -TimeoutSeconds $timeout -WhatIf:$WhatIfPreference } $manifest = @{ SubscriptionId = $Settings.SubscriptionId; ResourceGroupName = $Settings.ResourceGroupName; CustomLocationId = $custom.id @@ -624,6 +745,7 @@ function Invoke-LocalBoxPreparation { NodeNames = @($config.NodeHostConfig.Hostname); StorageSizeGB = $Settings.StorageSizeGB; PreparedAt = [datetime]::UtcNow.ToString('o') FullHealthVerified = $false AksPreparationSkipped = [bool]$Settings.SkipAks + MetalLb = $metalLb } if ($PSCmdlet.ShouldProcess($Settings.ManifestPath, 'Write nonsecret preparation manifest')) { $manifest | ConvertTo-Json -Depth 30 | Set-Content -LiteralPath $Settings.ManifestPath -Encoding utf8 @@ -644,7 +766,8 @@ function Invoke-LocalBoxPreparation { if ($MyInvocation.InvocationName -ne '.') { $settings = @{ SubscriptionId = $SubscriptionId; ResourceGroupName = $ResourceGroupName; ConfigPath = $ConfigPath - AksAdminGroupObjectId = $AksAdminGroupObjectId; SkipAks = [bool]$SkipAks; NodeCredential = $NodeCredential; VmSwitchName = $VmSwitchName + AksAdminGroupObjectId = $AksAdminGroupObjectId; KubernetesRuntimeObjectId = $KubernetesRuntimeObjectId + SkipAks = [bool]$SkipAks; NodeCredential = $NodeCredential; VmSwitchName = $VmSwitchName ImageName = $ImageName; ImageVersion = $ImageVersion; VmPoolStart = $VmPoolStart; VmPoolEnd = $VmPoolEnd AksClusterName = $AksClusterName; KubernetesVersion = $KubernetesVersion; NodeVmSize = $NodeVmSize ControlPlaneVmSize = $ControlPlaneVmSize; NodeCount = $NodeCount; StorageSizeGB = $StorageSizeGB diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/localbox.health.tests.ps1 b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/localbox.health.tests.ps1 index 65001da27..beeb7d1be 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/localbox.health.tests.ps1 +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/localbox.health.tests.ps1 @@ -8,7 +8,7 @@ BeforeAll { Describe 'Shared LocalBox control plane' { It 'has all mandatory manifest fields' { $LocalBox.AksPreparationSkipped | Should -Not -BeTrue -Because 'a partial preparation is not full LocalBox readiness' - foreach ($field in @('SubscriptionId', 'ResourceGroupName', 'ClusterId', 'BridgeId', 'ExtensionId', 'CustomLocationId', 'StorageId', 'ImageId', 'AksId', 'AksInstanceId', 'AksAdminGroupObjectId', 'ImageExpected', 'AksExpected', 'NodeNames', 'StorageSizeGB')) { + foreach ($field in @('SubscriptionId', 'ResourceGroupName', 'ClusterId', 'BridgeId', 'ExtensionId', 'CustomLocationId', 'StorageId', 'ImageId', 'AksId', 'AksInstanceId', 'AksAdminGroupObjectId', 'ImageExpected', 'AksExpected', 'NodeNames', 'StorageSizeGB', 'MetalLb')) { $LocalBox[$field] | Should -Not -BeNullOrEmpty -Because "$field is part of the expected inventory" } @($LocalBox.Networks).Count | Should -Be 2 @@ -61,6 +61,15 @@ Describe 'Shared LocalBox control plane' { foreach ($resource in $selected) { Wait-LocalBoxResource $resource.id -TimeoutSeconds $TimeoutSeconds | Should -Not -BeNullOrEmpty } } } + It 'has the organizer-provisioned MetalLB extension and matching ARP address pool' { + $LocalBox.MetalLb.ExtensionId | Should -Be "$($LocalBox.AksId)/providers/Microsoft.KubernetesConfiguration/extensions/arcnetworking" + $LocalBox.MetalLb.PoolId | Should -Be "$($LocalBox.AksId)/providers/Microsoft.KubernetesRuntime/loadBalancers/aks-pool" + $extension = Wait-LocalBoxResource $LocalBox.MetalLb.ExtensionId -TimeoutSeconds $TimeoutSeconds + $extension.properties.extensionType | Should -Be 'microsoft.arcnetworking' + $pool = Wait-LocalBoxResource $LocalBox.MetalLb.PoolId -TimeoutSeconds $TimeoutSeconds + Assert-LocalBoxProperties $pool $LocalBox.MetalLb.PoolExpected + [int]$pool.properties.serviceSelector.Count | Should -Be 0 + } } Describe 'LocalBox runtime health' -Skip:($Mode -ne 'Full') { @@ -101,4 +110,15 @@ Describe 'LocalBox runtime health' -Skip:($Mode -ne 'Full') { } $result | Should -Be 200 } + It 'has an in-cluster MetalLB address pool and L2 advertisement' { + Wait-SovereignCheck -TimeoutSeconds $TimeoutSeconds -Check { + $pool = Invoke-SovereignKubectl $LocalBox.Kubeconfig @('get', 'ipaddresspools.metallb.io', 'aks-pool', '-n', 'kube-system') + Assert-LocalBoxProperties $pool.spec @{ addresses = $LocalBox.MetalLb.PoolExpected.properties.addresses } + $pool.spec.autoAssign | Should -Not -BeFalse + $advertisements = Invoke-SovereignKubectl $LocalBox.Kubeconfig @('get', 'l2advertisements.metallb.io', '-n', 'kube-system') + @($advertisements.items | Where-Object { + $_.spec.ipAddressPools -contains 'aks-pool' + }).Count | Should -BeGreaterThan 0 + } + } } \ No newline at end of file diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/prepare-localbox.tests.ps1 b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/prepare-localbox.tests.ps1 index 5de1cfaf6..d24e3b9ff 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/prepare-localbox.tests.ps1 +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/prepare-localbox.tests.ps1 @@ -698,6 +698,47 @@ Describe 'Console LocalBox credential isolation' { } } +Describe 'LocalBox registration region contract' { + It 'defaults to West Europe independently of the host region' -TestCases @( + @{ Path = 'labautomation/deploy-localbox.ps1' } + @{ Path = 'resources/manual-setup/localbox/deploy-localbox.ps1' } + ) { + param($Path) + $errors = $null + $ast = [System.Management.Automation.Language.Parser]::ParseFile( + "$PSScriptRoot/../../$Path", [ref]$null, [ref]$errors) + $errors.Count | Should -Be 0 + $registration = $ast.ParamBlock.Parameters | Where-Object { $_.Name.VariablePath.UserPath -eq 'AzureLocalInstanceLocation' } + $registration.DefaultValue.SafeGetValue() | Should -Be 'westeurope' + $hostRegion = $ast.ParamBlock.Parameters | Where-Object { $_.Name.VariablePath.UserPath -eq 'Location' } + $hostRegion.DefaultValue.SafeGetValue() | Should -Be 'swedencentral' + $ast.Extent.Text | Should -Match 'azureLocalInstanceLocation\s*=\s*@\{\s*value\s*=\s*\$AzureLocalInstanceLocation\s*\}' + $ast.Extent.Text | Should -Match 'location\s*=\s*@\{\s*value\s*=\s*\$Location\s*\}' + } + It 'uses West Europe registration while preserving shared host-region selection' { + $errors = $null + $ast = [System.Management.Automation.Language.Parser]::ParseFile( + "$PSScriptRoot/../../labautomation/shared-deploy-lab.ps1", [ref]$null, [ref]$errors) + $errors.Count | Should -Be 0 + $ast.Extent.Text | Should -Match "-AzureLocalInstanceLocation 'westeurope'" + $ast.Extent.Text | Should -Match '-Location \$localBoxLocation' + $ast.Extent.Text | Should -Not -Match "-AzureLocalInstanceLocation 'australiaeast'" + } + It 'includes the registration region in all four Challenge 1 policy parameter lists' { + $guide = Get-Content "$PSScriptRoot/../../walkthrough/challenge-01/solution-01.md" -Raw + $lists = [regex]::Matches($guide, '"listOfAllowedLocations"\s*:\s*\{\s*"value"\s*:\s*(\[[^\]]*\])') + $lists.Count | Should -Be 4 + foreach ($list in $lists) { + $regions = @($list.Groups[1].Value | ConvertFrom-Json) + $regions.Count | Should -Be 4 + foreach ($region in @('norwayeast', 'germanynorth', 'northeurope', 'westeurope')) { + $regions | Should -Contain $region + } + $regions | Should -Not -Contain 'australiaeast' + } + } +} + Describe 'Console LocalBox password selection' { BeforeAll { function New-MhhStablePassword { [CmdletBinding()] param($Purpose, $Length) } @@ -847,6 +888,14 @@ Describe 'LocalBox input validation' { } Describe 'AKS Local resource API selection' { + It 'pins the MetalLB extension and Runtime pool API versions' -ForEach @( + @{ Suffix = 'Microsoft.KubernetesConfiguration/extensions/arcnetworking'; Version = '2023-05-01' } + @{ Suffix = 'Microsoft.KubernetesRuntime/loadBalancers/aks-pool'; Version = '2024-08-01' } + ) { + Mock Invoke-LocalBoxAz { @{} } + Get-LocalBoxResource "/subscriptions/test/resourceGroups/localbox/providers/Microsoft.Kubernetes/connectedClusters/localbox-aks/providers/$Suffix" + Should -Invoke Invoke-LocalBoxAz -Times 1 -Exactly -ParameterFilter { $Arguments -contains '--api-version' -and $Arguments -contains $Version } + } It 'specifies the API version for nested AKS Local node pools' { Mock Invoke-LocalBoxAz { @{} } $identifier = '/subscriptions/test/resourceGroups/localbox/providers/Microsoft.Kubernetes/connectedClusters/localbox-aks/providers/Microsoft.HybridContainerService/provisionedClusterInstances/default/agentPools/nodepool1' @@ -957,18 +1006,251 @@ Describe 'AKS Local Arc proxy RBAC' { $ast = [Management.Automation.Language.Parser]::ParseFile("$PSScriptRoot/../prepare-localbox.ps1", [ref]$null, [ref]$null) $guard = $ast.Find({ param($node) $node -is [Management.Automation.Language.IfStatementAst] -and - $node.Clauses[0].Item1.Extent.Text -eq '-not $Settings.SkipAks' + $node.Clauses[0].Item1.Extent.Text -eq '-not $Settings.SkipAks' -and + $node.Extent.Text -match 'Sync-LocalBoxAksProxyRole' }, $true) $guard | Should -Not -BeNullOrEmpty $reconcile = [scriptblock]::Create($guard.Extent.Text) $aksId = $clusterId - $Settings = @{ SkipAks = $true } + $Settings = @{ SkipAks = $true; KubernetesRuntimeObjectId = $groupId } + $metalLb = @{} Mock Sync-LocalBoxAksProxyRole {} + Mock Sync-LocalBoxMetalLb {} & $reconcile Should -Invoke Sync-LocalBoxAksProxyRole -Times 0 + Should -Invoke Sync-LocalBoxMetalLb -Times 0 $Settings.SkipAks = $false & $reconcile Should -Invoke Sync-LocalBoxAksProxyRole -Times 1 -Exactly -ParameterFilter { $ClusterId -eq $aksId -and $GroupObjectId -eq $groupId } + Should -Invoke Sync-LocalBoxMetalLb -Times 1 -Exactly -ParameterFilter { $ClusterId -eq $aksId -and $RuntimeObjectId -eq $groupId } + } +} + +Describe 'LocalBox MetalLB preparation' { + BeforeEach { + $clusterId = '/subscriptions/test/resourceGroups/localbox/providers/Microsoft.Kubernetes/connectedClusters/localbox-aks' + $runtimeId = '11111111-2222-3333-4444-555555555555' + $configuration = @{ + AKSIPPrefix = '10.10.0.0/24'; AKSVIPStartIP = '10.10.0.10'; AKSVIPEndIP = '10.10.0.100' + AKSGWIP = '10.10.0.1'; AKSControlPlaneIP = '10.10.0.5' + AKSNodeStartIP = '10.10.0.101'; AKSNodeEndIP = '10.10.0.199' + } + $plan = Get-LocalBoxMetalLbPlan $configuration $clusterId + $script:metalLbResources = @{} + $script:metalLbCalls = [Collections.Generic.List[object]]::new() + Mock Invoke-LocalBoxAz { + $script:metalLbCalls.Add($Arguments) + if ($Arguments[0] -eq 'provider') { return @{ registrationState = 'Registered' } } + $url = $Arguments[4] + if ($Arguments[2] -eq 'get') { + $collection = ($url -split '\?')[0].Replace('https://management.azure.com', '') + return @{ value = @($script:metalLbResources.Values | Where-Object { $_.id.StartsWith("$collection/") }) } + } + if ($Arguments[2] -eq 'put') { + $id = ($url -split '\?')[0].Replace('https://management.azure.com', '') + $resource = $Arguments[6] | ConvertFrom-Json -AsHashtable + $resource.id = $id + $resource.properties.provisioningState = 'Succeeded' + $script:metalLbResources[$id] = $resource + return $resource + } + throw "Unexpected command: $Arguments" + } + Mock Get-LocalBoxResource { $script:metalLbResources[$Id] } + Mock Start-Sleep {} + } + It 'uses the complete reserved VIP range without overlapping nodes or control plane' { + $plan.PoolExpected.properties.addresses | Should -Be @('10.10.0.10-10.10.0.100') + $plan.PoolExpected.properties.advertiseMode | Should -Be 'ARP' + $plan.ExtensionId | Should -Be "$clusterId/providers/Microsoft.KubernetesConfiguration/extensions/arcnetworking" + } + It 'rejects invalid VIP reservation before Azure writes' -ForEach @( + @{ Fault = 'nodes'; Start = '10.10.0.150'; End = '10.10.0.160' } + @{ Fault = 'control-plane'; Start = '10.10.0.5'; End = '10.10.0.50' } + @{ Fault = 'gateway'; Start = '10.10.0.1'; End = '10.10.0.50' } + @{ Fault = 'outside'; Start = '10.11.0.10'; End = '10.11.0.100' } + ) { + $configuration.AKSVIPStartIP = $Start + $configuration.AKSVIPEndIP = $End + { Get-LocalBoxMetalLbPlan $configuration $clusterId } | Should -Throw + Should -Invoke Invoke-LocalBoxAz -Times 0 + } + It 'creates extension before pool and reuses both unchanged on a second run without Graph' { + Sync-LocalBoxMetalLb $clusterId $plan $runtimeId + $puts = @($script:metalLbCalls | Where-Object { $_[2] -eq 'put' }) + $puts.Count | Should -Be 2 + $puts[0][4] | Should -Match '/extensions/arcnetworking\?api-version=2023-05-01$' + $puts[1][4] | Should -Match '/loadBalancers/aks-pool\?api-version=2024-08-01$' + $script:metalLbResources[$plan.ExtensionId].properties.configurationSettings.k8sRuntimeFpaObjectId | Should -Be $runtimeId + Sync-LocalBoxMetalLb $clusterId $plan $runtimeId + @($script:metalLbCalls | Where-Object { $_[2] -eq 'put' }).Count | Should -Be 2 + Should -Invoke Invoke-LocalBoxAz -Times 0 -ParameterFilter { $Arguments[0] -in @('ad', 'role', 'login') } + } + It 'does not query a nonexistent cluster or create resources during WhatIf' { + Sync-LocalBoxMetalLb $clusterId $plan $runtimeId -WhatIf + Should -Invoke Invoke-LocalBoxAz -Times 0 + } + It 'rejects missing or invalid Runtime object IDs including the application ID' -ForEach @( + @{ Value = 'not-a-guid' } + @{ Value = '00000000-0000-0000-0000-000000000000' } + @{ Value = '087fca6e-4606-4d41-b3f6-5ebdf75b8b4c' } + @{ Value = '{087fca6e-4606-4d41-b3f6-5ebdf75b8b4c}' } + ) { + { Sync-LocalBoxMetalLb $clusterId $plan $Value } | Should -Throw '*object ID*' + Should -Invoke Invoke-LocalBoxAz -Times 0 + } + It 'rejects an unregistered Runtime provider without registering it as the Client' { + Mock Invoke-LocalBoxAz { @{ registrationState = 'Registering' } } -ParameterFilter { $Arguments[0] -eq 'provider' } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw '*subscription owner*' + Should -Invoke Invoke-LocalBoxAz -Times 0 -ParameterFilter { $Arguments[0] -eq 'rest' } + } + It 'rejects the application ID before storage or AKS orchestration' { + $ast = [Management.Automation.Language.Parser]::ParseFile("$PSScriptRoot/../prepare-localbox.ps1", [ref]$null, [ref]$null) + $guard = $ast.Find({ param($node) + $node -is [Management.Automation.Language.IfStatementAst] -and + $node.Clauses[0].Item1.Extent.Text -eq '-not $Settings.SkipAks' -and + $node.Extent.Text -match 'Get-LocalBoxMetalLbPlan' + }, $true) + $config = $configuration + $scope = '/subscriptions/test/resourceGroups/localbox' + $Settings = @{ SkipAks = $false; AksClusterName = 'localbox-aks'; KubernetesRuntimeObjectId = '087fca6e-4606-4d41-b3f6-5ebdf75b8b4c' } + { & ([scriptblock]::Create($guard.Extent.Text)) } | Should -Throw '*KubernetesRuntimeObjectId*' + Should -Invoke Invoke-LocalBoxAz -Times 0 + } + It 'rejects an unexpected selector returned immediately after pool creation' { + Mock Get-LocalBoxResource { + $resource = $script:metalLbResources[$Id] + if ($Id -eq $plan.PoolId) { $resource.properties.serviceSelector = @{ team = 'unexpected' } } + return $resource + } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw '*service selector*' + Should -Invoke Invoke-LocalBoxAz -Times 0 -ParameterFilter { $Arguments -contains 'delete' } + } + It 'propagates discovery failure rather than assuming the extension is missing' { + Mock Invoke-LocalBoxAz { throw 'AuthorizationFailed' } -ParameterFilter { $Arguments[0] -eq 'rest' } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw '*AuthorizationFailed*' + Should -Invoke Invoke-LocalBoxAz -Times 0 -ParameterFilter { $Arguments[2] -eq 'put' } + } + It 'rejects malformed discovery output rather than recreating resources' { + Mock Invoke-LocalBoxAz { @{} } -ParameterFilter { $Arguments[0] -eq 'rest' } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw '*Invalid ARM collection*' + Should -Invoke Invoke-LocalBoxAz -Times 0 -ParameterFilter { $Arguments[2] -eq 'put' } + } + It 'follows ARM pagination to discover an existing extension' { + $script:pages = 0 + Mock Invoke-LocalBoxAz { + $script:pages++ + if ($script:pages -eq 1) { return @{ value = @(); nextLink = 'https://management.azure.com/next' } } + @{ value = @(@{ id = 'found-on-page-two' }) } + } + @(Get-LocalBoxArmCollection 'https://management.azure.com/first')[0].id | Should -Be 'found-on-page-two' + Should -Invoke Invoke-LocalBoxAz -Times 2 -Exactly + } + It 'does not install a second differently named MetalLB extension' { + $id = "$clusterId/providers/Microsoft.KubernetesConfiguration/extensions/other" + $script:metalLbResources[$id] = @{ id = $id; properties = @{ extensionType = 'microsoft.arcnetworking' } } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw '*differently named*' + Should -Invoke Invoke-LocalBoxAz -Times 0 -ParameterFilter { $Arguments[2] -eq 'put' } + } + It 'does not silently replace a conflicting pool or identity' -ForEach @( + @{ Fault = 'pool' } + @{ Fault = 'identity' } + @{ Fault = 'selector' } + @{ Fault = 'extra-pool' } + ) { + Sync-LocalBoxMetalLb $clusterId $plan $runtimeId + switch ($Fault) { + 'pool' { $script:metalLbResources[$plan.PoolId].properties.addresses = @('10.10.0.150-10.10.0.160') } + 'identity' { $script:metalLbResources[$plan.ExtensionId].properties.configurationSettings.k8sRuntimeFpaObjectId = 'wrong' } + 'selector' { $script:metalLbResources[$plan.PoolId].properties.serviceSelector = @{ app = 'other' } } + 'extra-pool' { + $id = "$clusterId/providers/Microsoft.KubernetesRuntime/loadBalancers/other" + $script:metalLbResources[$id] = @{ id = $id; properties = @{ addresses = @('10.10.0.150-10.10.0.160') } } + } + } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw + @($script:metalLbCalls | Where-Object { $_[2] -eq 'put' }).Count | Should -Be 2 + } + It 'stops before pool creation when extension provisioning fails' { + Mock Get-LocalBoxResource { @{ properties = @{ provisioningState = 'Failed' } } } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw '*Failed*' + @($script:metalLbCalls | Where-Object { $_[2] -eq 'put' }).Count | Should -Be 1 + } + It 'retains the extension after pool failure rather than deleting shared resources' { + Mock Invoke-LocalBoxAz { throw 'Pool creation failed' } -ParameterFilter { $Arguments[2] -eq 'put' -and $Arguments[4] -match '/loadBalancers/' } + { Sync-LocalBoxMetalLb $clusterId $plan $runtimeId } | Should -Throw '*Pool creation failed*' + $script:metalLbResources.ContainsKey($plan.ExtensionId) | Should -BeTrue + Should -Invoke Invoke-LocalBoxAz -Times 0 -ParameterFilter { $Arguments -contains 'delete' } + } + It 'keeps Runtime registration and Graph resolution in organizer setup, not the Client' { + foreach ($path in @('../../labautomation/resource-providers.ps1', '../manual-setup/subscription-preparations/1-resource-providers.ps1')) { + Get-Content "$PSScriptRoot/$path" -Raw | Should -Match '"Microsoft.KubernetesRuntime"' + } + $shared = Get-Content "$PSScriptRoot/../../labautomation/shared-deploy-lab.ps1" -Raw + $shared | Should -Match "appId eq '087fca6e-4606-4d41-b3f6-5ebdf75b8b4c'" + $shared | Should -Match 'microhack-k8s-runtime-object-id.*\$runtimeObjectId\.ToString' + $preparation = ${function:Invoke-LocalBoxPreparation}.ToString() + $preparation | Should -Match "group.tags.'microhack-k8s-runtime-object-id'" + $preparation | Should -Not -Match "'ad', 'sp'|az ad" + $preparation | Should -Match 'MetalLb = \$metalLb' + } +} + +Describe 'MetalLB health evidence' { + BeforeAll { + $ast = [Management.Automation.Language.Parser]::ParseFile("$PSScriptRoot/localbox.health.tests.ps1", [ref]$null, [ref]$null) + $checks = @{} + foreach ($name in @('has the organizer-provisioned MetalLB extension and matching ARP address pool', 'has an in-cluster MetalLB address pool and L2 advertisement')) { + $command = $ast.Find({ param($node) + $node -is [Management.Automation.Language.CommandAst] -and $node.GetCommandName() -eq 'It' -and + $node.CommandElements[1].Value -eq $name + }, $true) + $checks[$name] = $command.CommandElements[-1].ScriptBlock.GetScriptBlock() + } + $controlPlaneCheck = $checks['has the organizer-provisioned MetalLB extension and matching ARP address pool'] + $runtimeCheck = $checks['has an in-cluster MetalLB address pool and L2 advertisement'] + } + BeforeEach { + $clusterId = '/subscriptions/test/resourceGroups/localbox/providers/Microsoft.Kubernetes/connectedClusters/localbox-aks' + $expected = @{ properties = @{ addresses = @('10.10.0.10-10.10.0.100'); advertiseMode = 'ARP' } } + $LocalBox = @{ AksId = $clusterId; Kubeconfig = 'mock'; MetalLb = @{ + ExtensionId = "$clusterId/providers/Microsoft.KubernetesConfiguration/extensions/arcnetworking" + PoolId = "$clusterId/providers/Microsoft.KubernetesRuntime/loadBalancers/aks-pool" + PoolExpected = $expected + } } + $script:healthPool = @{ spec = @{ addresses = @('10.10.0.10-10.10.0.100'); autoAssign = $true } } + $script:healthAds = @{ items = @(@{ spec = @{ ipAddressPools = @('aks-pool') } }) } + Mock Wait-SovereignCheck { & $Check } + Mock Wait-LocalBoxResource { + if ($Id -like '*/extensions/*') { return @{ properties = @{ extensionType = 'microsoft.arcnetworking' } } } + return $expected + } + Mock Invoke-SovereignKubectl { + if ($Arguments[1] -eq 'ipaddresspools.metallb.io') { return $script:healthPool } + return $script:healthAds + } + } + It 'accepts matching provisioned configuration and in-cluster pool advertisement' { + { & $controlPlaneCheck; & $runtimeCheck } | Should -Not -Throw + } + It 'rejects missing or wrong in-cluster evidence ' -ForEach @( + @{ Fault = 'pool' } + @{ Fault = 'address' } + @{ Fault = 'advertisement' } + @{ Fault = 'auto-assign' } + ) { + switch ($Fault) { + 'pool' { $script:healthPool = @{} } + 'address' { $script:healthPool.spec.addresses = @('10.10.0.150-10.10.0.160') } + 'advertisement' { $script:healthAds.items = @() } + 'auto-assign' { $script:healthPool.spec.autoAssign = $false } + } + { & $runtimeCheck } | Should -Throw + } + It 'rejects control-plane configuration drift' { + $expected.properties.advertiseMode = 'BGP' + $LocalBox.MetalLb.PoolExpected = @{ properties = @{ addresses = @('10.10.0.10-10.10.0.100'); advertiseMode = 'ARP' } } + { & $controlPlaneCheck } | Should -Throw '*Conflicting*' } } diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/readme.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/readme.md index 3c6da2534..466b347a5 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/readme.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/tests/readme.md @@ -16,6 +16,29 @@ From this directory, run `Invoke-Pester ./visual-attestation.tests.ps1 -Output D A fresh Codespace smoke test is still required to validate the devcontainer build, Docker daemon, `confcom` policy generation and Azure deployment together; the mocked tests do not establish end-to-end Codespaces support. +## MetalLB preparation checks + +MetalLB preparation tests are included in `prepare-localbox.tests.ps1`. Run +`Invoke-Pester ./prepare-localbox.tests.ps1 -FullName 'LocalBox MetalLB preparation*','MetalLB health evidence*' -Output Detailed` +for the focused offline checks. They cover reserved VIP validation, idempotent +extension/pool installation, WhatIf, discovery failures, conflicting configuration +and failure propagation without deleting shared resources. Mocked health checks +also reject missing or mismatched address pools and L2 advertisements. The live LocalBox +suite now requires the `MetalLb` manifest fields; rerun preparation for older +manifests. Full checks also inspect the Kubernetes IPAddressPool and L2Advertisement. +They do not replace the participant proxy/port-forward test or a separately +authorized test of real network access through the load-balancer address. + +## LocalBox registration-region checks + +The LocalBox registration-region contract is checked offline with +`Invoke-Pester ./prepare-localbox.tests.ps1 -FullName 'LocalBox registration region contract*' -Output Detailed`. +It verifies that both deployment entry points default registration to West Europe, +the shared hook passes that region independently of host-region selection, and +all four Challenge 1 policy parameter examples include it. These checks make no +Azure changes; a fresh Console deployment and participant VM creation remain +required to validate regional service availability and effective inherited policies. + ## Hosted tag-policy unit tests From this directory, run `Invoke-Pester ./hosted-tag-policy.tests.ps1 -Output Detailed`. These offline tests mock all Azure and Console commands. They cover hosted-only wiring, subscription targeting, tag-policy propagation, both tag values on resources and resource groups, preservation of unrelated tags, bounded failure, and temporary probe cleanup. Compile the initiative separately with `az bicep build --file ../../labautomation/infra/hosted-tag-policy.bicep --stdout`. diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-01/solution-01.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-01/solution-01.md index a51ed02f7..08cca1006 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-01/solution-01.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-01/solution-01.md @@ -17,7 +17,7 @@ Please ensure that you successfully verified the [General prerequisites](https:/ You are a cloud architect at a European organization that must comply with data sovereignty requirements. Your workloads contain sensitive data that must remain within EU sovereign regions. Additionally, regulatory requirements mandate: -- **Geographic restrictions**: All resources must be deployed only in approved sovereign regions (Norway East, Germany North, North Europe) +- **Geographic restrictions**: All resources must be deployed only in the lab-approved European regions (Norway East, Germany North, North Europe, West Europe) - **Data classification**: All resources must be tagged with appropriate data classification labels - **Network isolation**: Sensitive resources must not expose public IP addresses - **Access control**: Only authorized teams should have access, following least-privilege principles @@ -152,8 +152,13 @@ az policy definition show --name e56962a6-4747-49cd-b67b-bf8b01975c4c \ - Norway East - Germany North - North Europe + - West Europe 7. Click **Review + create** and then **Create** +West Europe is included to accommodate Azure Local management resources in Challenge 6 when the shared LocalBox is registered there, reducing location-policy conflicts if enforcement is accidentally left enabled. It does not authorize Australia East or relocate an existing LocalBox registration. Keep **Do not enforce** and your **own resource-group scope** as instructed; other exercise policies can still block later challenges if left enforcing. + +If you already created your own exercise assignment using the earlier three-region list, use **Edit assignment > Parameters** to add **West Europe**, retain **Do not enforce**, then **Review + save > Save**. Update your resource-group location assignment and bonus initiative values below too, if you created them. Do not edit inherited or organizer-managed assignments. + ### Step 3 - option B: Assign the Policy Using Azure CLI Alternatively, use Azure CLI for automation: @@ -172,16 +177,18 @@ az policy assignment create \ --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP" \ --policy "$POLICY_DEFINITION_ID" \ --enforcement-mode DoNotEnforce \ - --params '{"listOfAllowedLocations":{"value":["norwayeast","germanynorth", "northeurope"]}}' + --params '{"listOfAllowedLocations":{"value":["norwayeast","germanynorth","northeurope","westeurope"]}}' ``` ![image](./img/cloud-shell3.jpg) +Older screenshots may show only three regions; use all four locations in the current instructions. + ### Step 4: Also Restrict Resource Group Locations ⚠️ **Important**: The "Allowed locations" policy applies to resources, but resource groups have their own location metadata. The separate **"Allowed locations for resource groups"** policy evaluates that metadata. At your resource-group scope, this is a compliance demonstration for your existing group; it does not restrict the creation of other resource groups. Subscription-wide enforcement belongs to the organizer, not participants. -**Using Azure Portal:** In **Policy > Assignments > Assign policy**, choose **Allowed locations for resource groups** (definition ID ending `e765b5de-1225-4ba3-bd56-1ac6695af988`). Set scope to your resource group, assignment name to `Lab User-0024 - Restrict Resource Groups to Sovereign Regions` (use your number), and enforcement to **Do not enforce**. Under **Parameters**, select Norway East, Germany North, and North Europe, then **Review + create > Create**. +**Using Azure Portal:** In **Policy > Assignments > Assign policy**, choose **Allowed locations for resource groups** (definition ID ending `e765b5de-1225-4ba3-bd56-1ac6695af988`). Set scope to your resource group, assignment name to `Lab User-0024 - Restrict Resource Groups to Sovereign Regions` (use your number), and enforcement to **Do not enforce**. Under **Parameters**, select Norway East, Germany North, North Europe, and West Europe, then **Review + create > Create**. **Using Azure CLI:** @@ -195,7 +202,7 @@ az policy assignment create \ --policy "$RG_POLICY_DEFINITION_ID" \ --params '{ "listOfAllowedLocations": { - "value": ["norwayeast", "germanynorth", "northeurope"] + "value": ["norwayeast", "germanynorth", "northeurope", "westeurope"] } }' \ --enforcement-mode DoNotEnforce @@ -399,8 +406,8 @@ az policy assignment create \ | Policy shown on the Policies tab | Values on the Policy parameters tab | |---|---| -| Allowed locations | Norway East, Germany North, North Europe | -| Allowed locations for resource groups | Norway East, Germany North, North Europe | +| Allowed locations | Norway East, Germany North, North Europe, West Europe | +| Allowed locations for resource groups | Norway East, Germany North, North Europe, West Europe | | Require a tag and its value on resources | Tag name: `DataClassification`; Tag value: `Sovereign` | | Not allowed resource types | `Microsoft.Network/publicIPAddresses` | @@ -417,7 +424,8 @@ az policy assignment create \ "value": [ "norwayeast", "germanynorth", - "northeurope" + "northeurope", + "westeurope" ] } } @@ -429,7 +437,8 @@ az policy assignment create \ "value": [ "norwayeast", "germanynorth", - "northeurope" + "northeurope", + "westeurope" ] } } diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/manifests/portal-quickstart.yaml b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/manifests/aks-local-sample-app.yaml similarity index 100% rename from 03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/manifests/portal-quickstart.yaml rename to 03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/manifests/aks-local-sample-app.yaml diff --git a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/solution-06.md b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/solution-06.md index 16dc6fa2b..2a315bafd 100644 --- a/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/solution-06.md +++ b/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/solution-06.md @@ -12,7 +12,7 @@ This challenge uses **Azure Arc Jumpstart LocalBox** to simulate an Azure Local Please ensure that you successfully verified the [General prerequisites](https://github.com/microsoft/MicroHack/blob/main/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/Readme.md#general-prerequisites) before continuing with this challenge. -Keep your [Sovereign Cloud Codespace](https://github.com/microsoft/MicroHack/blob/main/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/Readme.md#recommended-environment-github-codespaces) for lab work, but complete the core tasks below in the Azure portal. No local Windows installation or remote desktop connection is required. +Use your [Sovereign Cloud Codespace](https://github.com/microsoft/MicroHack/blob/main/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/Readme.md#recommended-environment-github-codespaces) for the Bash commands in Task 5 and the Azure portal for Tasks 1–4. No local Windows installation or remote desktop connection is required. **Additional requirements for this challenge:** @@ -22,6 +22,11 @@ Keep your [Sovereign Cloud Codespace](https://github.com/microsoft/MicroHack/blo - **Security Reader** access at subscription scope to review Defender for Cloud coverage and recommendations - Defender for Servers enabled by the organizer on the lab subscription, using an approved plan and the required Defender for Endpoint integration - A guest network with a valid IP address, working DNS, and outbound access to the required Azure Arc, Defender, and configured Windows update-source endpoints +- For Task 5, the shared `localbox-aks` cluster in `rg-localbox-shared`, with Microsoft Entra authentication, Kubernetes RBAC, and organizer-prepared MetalLB networking +- Membership in the cluster's configured Microsoft Entra administrator group, supplied through the workshop Console, and the **Azure Arc Enabled Kubernetes Cluster User Role** already assigned by the organizer's preparer for Cluster Connect. These are separate prerequisites; ask the facilitator to verify either if access fails + +> [!IMPORTANT] +> Complete Challenge 1's [Preparing for Next Challenges](../challenge-01/solution-01.md#preparing-for-next-challenges): your exercise policy assignments, including **Allowed locations** and any bonus initiative, must be in **DoNotEnforce**. This does not override inherited or organizer-managed policies. The organizer must also confirm that the shared LocalBox **custom location's Azure region** is permitted for resources created in your participant resource group. > [!NOTE] > LocalBox is typically deployed by the workshop facilitator due to resource requirements and deployment time. See the [LocalBox deployment and readiness guide](https://github.com/microsoft/MicroHack/blob/main/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/resources/demo-vm-creator/README.md). For a personal subscription, an authorized owner must also enable Defender for Servers before this challenge and review the plan's charges. Students in hosted labs should not change subscription-level Defender plans. @@ -98,6 +103,11 @@ LocalBox runs as a nested lab environment hosted in Azure. In a production sover Verify that **Custom location** points to the shared LocalBox environment and **Virtual machine kind** is **Azure Local**. Leave **Storage path** set to **Choose automatically** unless the facilitator instructs otherwise. +> [!NOTE] +> The selected custom location determines the Azure region used for the VM's management resources; it need not match your resource group's location or the Azure region hosting the LocalBox simulator. Fresh hosted deployments and the manual deployment default to **West Europe**. Earlier hosted test deployments used **Australia East**. Inspect the actual custom location in your event rather than assuming the latest default has changed an existing environment. +> +> In production, the workload runs on the Azure Local hardware, while Azure stores management data in the registration region. In this lab, that hardware is simulated inside Azure-hosted LocalBox. Registration in Australia East does **not** mean that the nested VM's compute has moved there, but the management-data location still matters for sovereignty and Azure Policy. See [Azure Local regions](https://learn.microsoft.com/azure/azure-local/concepts/system-requirements-23h2#azure-requirements) and [Azure Local data handling](https://learn.microsoft.com/azure/azure-local/faq#does-my-data-stored-on-azure-local-get-sent-to-the-cloud). + The creation screenshots use `labuser24-vm-01`, while the validation and management screenshots use `labuser23-vm-01`. These are examples from different participants; create and use only your own VM throughout the challenge. 5. **Image**: Select the available gallery image **2025-datacenter-azure-edition-smalldisk-01** (Windows Server 2025) @@ -148,6 +158,22 @@ Click **Next** twice ![Azure Arc VM Review and create page showing two virtual processors, 4096 MB memory, guest management enabled, and one network interface](./images/localbox_09.png) +#### If validation or deployment is blocked by a location policy + +Do not change the resource group, select another team's custom location, or disable policies at subscription scope to bypass the error. + +1. Expand **Error details** on **Review + create**. If the deployment was submitted, open your resource group's **Deployments**, select the failed deployment, and open the failed operation's details. Also check **Activity log** if needed. +2. Record the innermost error code, rejected resource name/type and requested location, **policy assignment ID**, **policy definition ID**, and correlation ID. For an initiative, also record the policy definition reference ID if present. Share only the relevant error details with the facilitator, not deployment parameters, passwords, TAPs or tokens. +3. For `RequestDisallowedByPolicy`, open **Policy > Assignments** and locate the assignment identified by the error. Inspect its **Scope**, **Policy enforcement** and **Parameters**. Check the actual ID, not just the friendly name: an initiative or inherited assignment may enforce a second location restriction. + - **Your own Challenge 1 exercise assignment:** restore **Do not enforce** using the [Challenge 1 instructions](../challenge-01/solution-01.md#preparing-for-next-challenges), including your bonus initiative if applicable. + - **Organizer-managed or inherited assignment:** stop and ask the facilitator to review the required metadata region and the approved policy configuration. A resource-group assignment cannot relax a deny inherited from subscription or management-group scope. + - **Different error code or no policy identifiers:** retain the exact error for the facilitator. A message mentioning a region is not by itself proof that the Challenge 1 policy caused the failure. +4. After the authorized correction has propagated, retry validation in your assigned resource group. Check for partially created resources before retrying a submitted deployment; do not delete shared LocalBox resources. + +The current Challenge 1 exercise allowlist includes West Europe to match fresh LocalBox deployments; older assignments may still have only three regions. This does not permit earlier test deployments registered in Australia East. Do not broaden organizer-managed or inherited allowlists without the policy owner's approval. The updated registration-region default applies to fresh deployments, not existing custom locations. The hosted `SecurityControl=Ignore` and `CostControl=Ignore` tags are **not** general Azure Policy exemptions. + +Reference: [Resolve RequestDisallowedByPolicy errors](https://learn.microsoft.com/azure/azure-resource-manager/troubleshooting/error-policy-requestdisallowedbypolicy). + ### 2.4 Validate VM Deployment and Guest Management 1. Click **Go to resource** when the deployment is finished @@ -299,193 +325,205 @@ This exercise stops at assessment. Leave periodic assessment unchanged; the **En ## Task 5: Deploy a container to the AKS cluster deployed on Azure Local -### Task 5.1: Verify the AKS Cluster - -The deployed AKS cluster should be setup as **Microsoft Entra authentication with Kubernetes RBAC** in the *Authentication and Authorization* setting - -*Look toward the bottom of the screenshot* - -AKS cluster access configuration using Microsoft Entra authentication and Kubernetes RBAC - -### Task 5.2: Verify Microsoft Entra Authentication - -After the deployment completes: - -1. Open the AKS cluster in the Azure portal. -2. Go to **Kubernetes resources > Workloads**. - -Kubernetes workloads option in the Azure portal - -The cluster system workloads should be visible. - -System workloads running on the AKS cluster - -Seeing these workloads without being asked for a bearer token confirms that: - -- The cluster is configured for Microsoft Entra authentication with Kubernetes RBAC. -- The signed-in user is a member of the assigned `AKSAdmin` group. -- The user has permission to view Kubernetes resources through the Azure portal. - -> **Important:** If the portal requests a bearer token, verify the cluster authentication configuration and confirm that the signed-in user is a direct member of the assigned Microsoft Entra administrator group. - -### Task 5.3 Configure the Load Balancer - -Before deploying the application, configure MetalLB. MetalLB assigns an IP address to Kubernetes Services of type `LoadBalancer`. - -### Task 5.4: Install the MetalLB Extension +**Deploy only your team's application; inspect shared infrastructure without changing it.** The organizer runs `resources/prepare-localbox.ps1` to prepare the AKS cluster and MetalLB after LocalBox deployment. The Console's LocalBox deployment hook alone does not prepare MetalLB. Attendees must not install extensions, create address pools, change routes, or change shared RBAC. -1. Open the AKS cluster. -2. Go to **Settings > Networking**. +### Task 5.1: Verify Cluster Access and Authentication -AKS cluster Networking page requesting installation of the MetalLB extension +1. In the Azure portal, open **`localbox-aks`** in **`rg-localbox-shared`**. +2. Confirm that its authentication configuration is **Microsoft Entra authentication with Kubernetes RBAC**. Have the facilitator confirm your Console-provided membership in the configured administrator group; the historical screenshot's group name is only an example. +3. Open **Kubernetes resources > Workloads** and inspect the system workloads. -3. Select **Install extension**. -4. Select **MetalLB for Azure Arc-enabled Kubernetes**. -5. Wait for the `arcnetworking` extension to report **Succeeded**. +The following older screenshot illustrates the authentication setting at the bottom. It is **not** an instruction to create or reconfigure the shared cluster. Its cluster name, region, and group can differ from your workshop. Click any screenshot to view it full-size. -MetalLB extension installation in the Azure portal +Historical cluster form illustrating Microsoft Entra authentication with Kubernetes RBAC -### Task 5.5: Create the MetalLB Address Pool +Kubernetes workloads option in the Azure portal -After the extension installation completes, return to **Settings > Networking**. +Example system workload readiness in the Azure portal -AKS Networking page after installing the MetalLB extension +Successful portal workload access proves only that the current identity can perform that operation. It does **not** independently prove administrator-group membership or that CLI access is ready. -Select **Add** to create a load-balancer address pool. - -MetalLB address-pool configuration - -Use the following settings: - -| Setting | Value | -|---|---| -| Name | A lowercase name, such as `aks-pool` | -| Advertisement mode | **ARP** | -| IP address range | `10.10.0.150-10.10.0.160` | -| Service selector | Leave blank | - -> **Warning:** The screenshot was captured with `10.10.0.150-10.10.0.160`. It is advisable to not use an overlapping range with the the AKS node allocation pool. +> [!IMPORTANT] +> Use your workshop Microsoft Entra identity throughout. If the portal requests a bearer token or the CLI reports `Unauthorized`/`Forbidden`, stop and ask the facilitator to verify the configured group, your membership, the existing **Azure Arc Enabled Kubernetes Cluster User Role**, and Cluster Connect readiness. After membership changes, sign in again to refresh your session. Do not create a service account token, retrieve admin credentials, or grant yourself additional roles. -### Task 5.6: Deploy the Application +### Task 5.2: Inspect Organizer-Prepared MetalLB -After MetalLB is ready, deploy the application YAML. +MetalLB assigns service virtual IP addresses (VIPs) to Kubernetes Services of type `LoadBalancer`. Before deploying: -1. Download or open the [`portal-quickstart.yaml`](./manifests/portal-quickstart.yaml) manifest. -2. Go to **Kubernetes resources**. -3. Select **Create from YAML**. +1. Open **Settings > Extensions** and inspect **`arcnetworking`**, extension type **`microsoft.arcnetworking`**. Its provisioning state must be **Succeeded**. +2. Under **Kubernetes resources > Workloads**, inspect the MetalLB controller and speaker workloads. Check that controller replicas and speaker DaemonSet pods are ready, with no persistent pending or crashing pods. Extension provisioning success alone is not proof of healthy workloads. +3. Open **Settings > Networking** and inspect the existing **`aks-pool`**. Confirm successful provisioning, **ARP** advertisement, and the IP range reserved by the organizer for service VIPs. Record that range; it must exclude the AKS node allocation pool, control-plane IP, gateway, and any other allocated addresses. +The default LocalBox service VIP reservation is **`10.10.0.10-10.10.0.100`**, separate from the node allocation pool **`10.10.0.101-10.10.0.199`**, control-plane IP **`10.10.0.5`**, and gateway **`10.10.0.1`**. The preparer uses the deployment configuration's exact reservation, so confirm the actual range if the organizer customized it. The older screenshots' `10.10.0.150` address overlaps the default node range and must not be reused as a service VIP. +The organizer-created ARP pool is represented in Kubernetes by an **IPAddressPool** and **L2Advertisement** in **`kube-system`**. After starting the proxy in Task 5.3, inspect these read-only resources as shown in Task 5.4; do not create or edit them. See the [official AKS enabled by Azure Arc load-balancer documentation](https://learn.microsoft.com/azure/aks/aksarc/deploy-load-balancer-cli) for background only, not participant setup instructions. Provider registration and managed-identity preparation belong to the organizer; participants need no Microsoft Graph permissions and must not run the linked installation commands. -Create from YAML option for the AKS cluster +If the extension, healthy workloads, or pool are missing, stop and contact the facilitator. Do not select **Install extension**, **Add**, **Delete**, or **Uninstall extension**. No participant-side MetalLB configuration is needed. -YAML editor in the Azure portal +### Task 5.3: Start the Microsoft Entra Cluster Connect Proxy — Terminal 1 -4. Paste the contents of `portal_quickstart.yaml` into the editor. -5. Select **Apply**. +Open a **Bash** terminal in your Sovereign Cloud Codespace. This assumes you are already signed in to Azure CLI with your workshop identity and have selected the workshop subscription. Retrieve the tenant and subscription IDs from the current CLI context, then enter your **assigned participant resource group**, not `rg-localbox-shared`, when prompted. Teammates sharing an assigned resource group share one namespace; coordinate deployment and cleanup. -The portal first performs a dry run and then creates the Kubernetes resources. +```bash +TENANT_ID=$(az account show --query tenantId --output tsv) || exit 1 +SUBSCRIPTION_ID=$(az account show --query id --output tsv) || exit 1 +read -r -p "Your assigned participant resource group: " PARTICIPANT_RG +if [[ -z "$TENANT_ID" || -z "$SUBSCRIPTION_ID" || -z "$PARTICIPANT_RG" || + "${PARTICIPANT_RG,,}" == "rg-localbox-shared" ]]; then + echo "Verify your current Azure CLI context and use your own assigned resource group." + exit 1 +fi +printf 'Current tenant: %s\nCurrent subscription: %s\n' "$TENANT_ID" "$SUBSCRIPTION_ID" +az group show --subscription "$SUBSCRIPTION_ID" --name "$PARTICIPANT_RG" \ + --query '{name:name,id:id}' --output table || exit 1 +``` -Successful Kubernetes YAML deployment in the Azure portal +Confirm the displayed tenant, subscription, and resource group match your workshop before continuing. The following derives a valid, bounded namespace from your resource group, adding a stable hash of the subscription and group to distinguish names that normalize alike. It saves **only nonsecret settings**, outside the repository, for the other terminals. Never commit or share your kubeconfig or authentication cache. + +```bash +RG_SLUG=$(printf '%s' "${PARTICIPANT_RG,,}" | + sed -E 's/[^a-z0-9]+/-/g; s/^-+//; s/-+$//' | cut -c1-40) +RG_HASH=$(printf '%s/%s' "${SUBSCRIPTION_ID,,}" "${PARTICIPANT_RG,,}" | + sha256sum | cut -c1-12) +TEAM_NAMESPACE="team-${RG_SLUG:-rg}-${RG_HASH}" +AKS_CONTEXT="sovereign-${TEAM_NAMESPACE}" +AKS_KUBECONFIG="$HOME/.kube/${AKS_CONTEXT}.config" +MANIFEST="$(git rev-parse --show-toplevel)/03-Azure/01-03-Infrastructure/01_Sovereign_Cloud/walkthrough/challenge-06/manifests/aks-local-sample-app.yaml" +umask 077 +mkdir -p "$HOME/.config/microhack" "$HOME/.kube" +printf 'export %s=%q\n' \ + SUBSCRIPTION_ID "$SUBSCRIPTION_ID" PARTICIPANT_RG "$PARTICIPANT_RG" \ + TEAM_NAMESPACE "$TEAM_NAMESPACE" AKS_CONTEXT "$AKS_CONTEXT" \ + AKS_KUBECONFIG "$AKS_KUBECONFIG" MANIFEST "$MANIFEST" \ + > "$HOME/.config/microhack/challenge06.env" +source "$HOME/.config/microhack/challenge06.env" +printf 'Team namespace: %s\n' "$TEAM_NAMESPACE" +az connectedk8s proxy \ + --subscription "$SUBSCRIPTION_ID" \ + --resource-group rg-localbox-shared --name localbox-aks \ + --file "$AKS_KUBECONFIG" --kube-context "$AKS_CONTEXT" --port 47011 +``` -The deployment should complete quickly. Select the application workload to verify that the pod is running. +Wait for the proxy to report that it is listening and complete any Microsoft Entra sign-in prompt. **Leave Terminal 1 running** for all following steps. This command writes an isolated kubeconfig rather than replacing your normal configuration; `--kube-context` names its context. See the official [`az connectedk8s proxy` reference and examples](https://learn.microsoft.com/cli/azure/connectedk8s#az-connectedk8s-proxy). -Running application workload on the AKS cluster +> [!CAUTION] +> Port **47011** is the Kubernetes API proxy, not your application. Never publish or forward this port through Codespaces Ports; if it is automatically detected, stop forwarding it in that view (leave the proxy process running). Never make the API proxy public. Do not pass `--token` or use service-account/admin credentials. -### Task 5.7: Find the Application IP Address +### Task 5.4: Deploy in Your Team Namespace — Terminal 2 -The application Service receives an external IP address from the MetalLB address pool. +Open a **second Bash terminal** in the same Codespace. Environment variables do not automatically carry between terminals, so source the nonsecret settings again. Every Kubernetes operation below explicitly selects the isolated kubeconfig and context. -1. Go to **Kubernetes resources > Services and ingresses**. -2. Open the application Service. -3. Record its **External IP**. +```bash +source "$HOME/.config/microhack/challenge06.env" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + cluster-info +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace kube-system get ipaddresspools.metallb.io,l2advertisements.metallb.io -o yaml +``` -Kubernetes Service showing its MetalLB external IP address +Confirm that the `aks-pool` IPAddressPool contains the organizer-reserved range and that an L2Advertisement covers that pool (an advertisement without an `ipAddressPools` restriction can cover all pools). Missing resources are a readiness issue for the facilitator, not an instruction to install or configure them. + +Discover the actual MetalLB controller Deployment and speaker DaemonSet names from their running configuration rather than assuming names or labels: + +```bash +source "$HOME/.config/microhack/challenge06.env" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace kube-system get deployments,daemonsets \ + -o 'custom-columns=KIND:.kind,NAME:.metadata.name,IMAGES:.spec.template.spec.containers[*].image' +read -r -p "MetalLB controller Deployment name from the list: " METALLB_DEPLOYMENT +read -r -p "MetalLB speaker DaemonSet name from the list: " METALLB_DAEMONSET +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace kube-system wait --for=condition=Available \ + "deployment/${METALLB_DEPLOYMENT:?Enter the listed MetalLB Deployment name}" --timeout=180s +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace kube-system rollout status \ + "deployment/$METALLB_DEPLOYMENT" --timeout=180s +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace kube-system rollout status \ + "daemonset/${METALLB_DAEMONSET:?Enter the listed MetalLB DaemonSet name}" --timeout=180s +``` -> **Note:** The screenshot shows the previously assigned address `10.10.0.150`. With the corrected address pool, the Service will normally receive an address beginning at `10.10.0.50`. Always use the External IP currently displayed for the Service. +Identify the MetalLB entries using their names and container images; ask the facilitator if uncertain. These read-only checks wait for controller availability, a completed Deployment rollout, and an available, updated speaker DaemonSet. They do not restart or modify workloads. If a workload is missing or a wait times out, stop and report the output rather than repairing shared configuration. Continue only after these checks and the pool inspection pass. -### Task 5.8: Configure Routing on LocalBox-Client +Now create your team's namespace: -Before connecting to the application, ensure that `LocalBox-Client` has a route to the AKS network `10.10.0.0/24`. +```bash +source "$HOME/.config/microhack/challenge06.env" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + create namespace "$TEAM_NAMESPACE" +``` -#### Review the Azure NIC Effective Routes +If namespace creation reports `AlreadyExists`, proceed **only** if it is your team's namespace from an earlier attempt. Stop on authentication or authorization errors rather than switching identities or contexts. -Open the network interface assigned to `LocalBox-Client` and review **Effective routes**. +The supplied [`aks-local-sample-app.yaml`](./manifests/aks-local-sample-app.yaml) creates a ConfigMap, Deployment, and `LoadBalancer` Service named `aks-container-1` (the ConfigMap is `aks-container-1-content`). The manifest has no hard-coded namespace; `--namespace` below keeps these resources out of `default` and other teams' namespaces. -Effective routes for the LocalBox-Client Azure network interface +```bash +source "$HOME/.config/microhack/challenge06.env" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace "$TEAM_NAMESPACE" apply -f "$MANIFEST" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace "$TEAM_NAMESPACE" rollout status deployment/aks-container-1 --timeout=180s +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace "$TEAM_NAMESPACE" get pods,service -l app.kubernetes.io/name=aks-container-1 +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace "$TEAM_NAMESPACE" get service aks-container-1 --watch +``` -The Azure NIC effective routes show that the Azure fabric does not provide a usable route to `10.10.0.0/24`. The required route must therefore be configured inside the Windows guest operating system. +Wait for successful rollout, ready application pods, and an **EXTERNAL-IP** from the organizer-confirmed `aks-pool` range. Record the assigned IP; do not assume the first address in the pool. Press **Ctrl+C in Terminal 2** to stop only the Service watch once the address appears. If it remains `` or is outside the reserved range, report it to the facilitator; do not create or edit pools. -#### Connect to LocalBox-Client +You can also inspect **Workloads** and **Services and ingresses** in the portal, filtering to **your team namespace**. These older screenshots show where to inspect a workload and its Service; their `default` namespace and `10.10.0.150` address are historical examples, **not values to reuse**. The workload screenshot shows a ready pod while its Deployment summary is still updating; rely on your own successful rollout and current readiness. -1. Connect to the `LocalBox-Client` VM using RDP. -2. If required, enable just-in-time access first. -3. Verify that you are using the current public IP address assigned to the VM. +Historical application workload view; inspect your team namespace and current rollout -> **Note:** If you have forgotten the deployment username, check the Bicep parameter file used for the LocalBox deployment. Do not include passwords or secrets in this documentation or commit them to the repository. +Historical Services view illustrating the External IP column, not the current pool address -#### Review the Windows Route Table +### Task 5.5: Forward and Open the Application — Terminals 2 and 3 -Open Command Prompt and run: +Keep the API proxy running in **Terminal 1**. In **Terminal 2**, start the application forward: -```console -route print +```bash +source "$HOME/.config/microhack/challenge06.env" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace "$TEAM_NAMESPACE" port-forward \ + service/aks-container-1 8080:80 --address 127.0.0.1 ``` -Windows route table before adding the AKS network route - -The route table does not initially contain a route for the AKS network `10.10.0.0/24`. +Wait for `Forwarding from 127.0.0.1:8080` and **leave Terminal 2 running too**. Use a **third Bash terminal** for the HTTP check, not either occupied terminal: -#### Add the Static Route - -Open PowerShell as an administrator and first verify the internal LocalBox adapter and router: - -```powershell -Get-NetIPAddress -AddressFamily IPv4 | - Where-Object IPAddress -eq '192.168.1.20' - -Test-NetConnection 192.168.1.1 +```bash +curl --fail --show-error http://127.0.0.1:8080/ ``` -#### Add the route through the LocalBox router: +In Codespaces, open the **Ports** view, forward **8080** if it was not detected, and confirm its visibility is **Private** before selecting **Open in Browser**. Sign in to GitHub if prompted. Open that private application URL, not port 47011 and not the MetalLB VIP. -```powershell -$adapter = Get-NetIPAddress -AddressFamily IPv4 | - Where-Object IPAddress -eq '192.168.1.20' +If running this same workflow directly on a local workstation with Bash, Azure CLI, and `kubectl`, instead open **http://localhost:8080/** on that workstation. A Codespace's `localhost` is not your workstation's `localhost`; use its private Ports URL when working in Codespaces. -New-NetRoute ` - -DestinationPrefix '10.10.0.0/24' ` - -InterfaceIndex $adapter.InterfaceIndex ` - -NextHop '192.168.1.1' ` - -RouteMetric 10 -``` +> [!IMPORTANT] +> Production clients reach a MetalLB VIP through configured network routing and the advertised service network. This lab's Arc proxy plus `kubectl port-forward` tunnels to an application pod through the Kubernetes API. A working page verifies application access through that tunnel; it **does not validate the load-balancer network path**, ARP reachability, or routing to the VIP. No RDP session or static-route change is needed. -PowerShell command adding the static route to the AKS network +If the forward stops after a pod restart, confirm the rollout again and restart it in Terminal 2. If either terminal is closed, restart the proxy first and then the port-forward, sourcing `challenge06.env` in each replacement terminal. -#### Verify the new route: +### Task 5.6: Clean Up Only Your Team's Application -```powershell -Get-NetRoute -DestinationPrefix '10.10.0.0/24' -``` +After recording your results and coordinating with teammates, press **Ctrl+C in Terminal 2** to stop the application forward. Keep Terminal 1's proxy running while deleting **only** the application resources in your team namespace: -Test connectivity to the External IP currently assigned to the application: - -```powershell -Test-NetConnection 10.10.0.50 -Port 80 +```bash +source "$HOME/.config/microhack/challenge06.env" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace "$TEAM_NAMESPACE" delete -f "$MANIFEST" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + --namespace "$TEAM_NAMESPACE" get deployment,service,configmap,pods ``` -> **Note:** Replace `10.10.0.50` if the Service displays a different External IP. - -### Task 5.9: Validate the Application - -Open a browser on `LocalBox-Client` and connect to: +The application Deployment, Service, ConfigMap, and pods should be gone (the namespace's automatically supplied `kube-root-ca.crt` ConfigMap can remain). Delete the namespace only if your team created it exclusively for this exercise and all teammates are finished: -```text -http:// +```bash +source "$HOME/.config/microhack/challenge06.env" +kubectl --kubeconfig "$AKS_KUBECONFIG" --context "$AKS_CONTEXT" \ + delete namespace "$TEAM_NAMESPACE" ``` -The application web page should load. - -Containerized web application reached from LocalBox-Client +Do not delete shared resources, system namespaces, the AKS cluster, MetalLB, its IP pool, or another team's resources. Finally, stop the proxy with **Ctrl+C in Terminal 1** and remove the application forwarding entry from the Codespaces Ports view. Retain your Task 2 VM until its assessments and workshop work are complete; any VM cleanup must target only your own VM, not shared LocalBox infrastructure. --- @@ -512,13 +550,13 @@ After completing this challenge, you should understand: - Azure Update Manager assesses updates on that same VM without changing shared infrastructure ✅ **Deploying a container to your AKS cluster** -- Validated AKS on Azure Local with Microsoft Entra authentication and Kubernetes RBAC -- Installed the MetalLB extension -- Created a MetalLB address pool using ARP advertisement -- Deployed a containerized application from YAML -- Assigned the application an external IP through a Kubernetes `LoadBalancer` Service -- Added the required Windows route on `LocalBox-Client` -- Connected to the application from a browser +- Verified Microsoft Entra access prerequisites and connected through the Arc proxy +- Inspected organizer-prepared MetalLB extension health and the ARP address pool without changing them +- Deployed a containerized application from YAML in a unique team namespace +- Recorded the Kubernetes `LoadBalancer` Service's assigned external IP +- Opened the application through a private Codespaces port-forward, without RDP or route changes +- Distinguished tunneled application access from production load-balancer network-path validation +- Cleaned up only your team's application resources ### 6.2 Real-World Applications @@ -550,6 +588,10 @@ Before completing this challenge, verify: - [ ] You have deployed your own VM on Azure Local and verified that guest management is Enabled (Connected) - [ ] You have verified Defender for Servers coverage for your VM and reviewed its available recommendations, or recorded that its assessment is still pending - [ ] You have completed an Azure Update Manager assessment for your VM and reviewed its timestamp and results, including when no updates are pending +- [ ] You verified the organizer-prepared `arcnetworking` extension is Succeeded, MetalLB workloads are healthy, and `aks-pool` has the reserved ARP VIP range +- [ ] Your application rolled out successfully in your team's namespace and received an IP from that pool +- [ ] You opened the application using a Microsoft Entra Arc proxy and a private application port-forward, never exposing the API proxy +- [ ] You understand why this tunnel does not validate the production load-balancer network path and cleaned up only your team's application - [ ] You understand how Azure Arc provides a unified control plane for sovereign hybrid scenarios ---