Skip to content

vmprobe: apply relabelings of the scrape class - #2738

Open
spicysharkco wants to merge 1 commit into
VictoriaMetrics:masterfrom
spicysharkco:fix/vmprobe-scrape-class-relabelings
Open

spicysharkco wants to merge 1 commit into
VictoriaMetrics:masterfrom
spicysharkco:fix/vmprobe-scrape-class-relabelings

Conversation

@spicysharkco

Copy link
Copy Markdown

A VMAgent or VMSingle defines a scrape class with relabelConfigs and metricRelabelConfigs, for example env=prod in relabelConfigs and a keep of probe_.* in metricRelabelConfigs. A VMProbe sets spec.scrapeClass to that class, or the class is the default one.

The rendered scrape job of the probe has neither list. It has only what the probe sets itself. The VMServiceScrape, VMPodScrape, VMNodeScrape, VMStaticScrape and VMScrapeConfig jobs get both lists from the same class, so the same class behaves differently on a probe. The authentication settings of the class do reach the probe.

Cause: generateProbeConfig merges only the auth of the class (probe.go#L65-L68). The siblings also call mergeEndpointRelabelingsWithScrapeClass, for example nodescrape.go#L32. The probe renders spec.MetricRelabelConfigs at probe.go#L210 and has no list of its own for relabelConfigs.

Fix: the class metricRelabelConfigs go in front of the probe's own, as for the other scrape resources. The class relabelConfigs go first in the probe's relabel_configs, in front of the relabelingConfigs of the targets, the same order as for the other scrape resources since #2003. They see the target address in __address__ and the __meta_* labels; __param_target, instance and the prober address are set after them. The ProbeScrapeRelabelTemplate and the namespace enforcement still come last. No API type changes. I added a docs/CHANGELOG.md line under tip.

The new case "scrapeClass relabelings and metric relabelings" in Test_generateProbeConfig renders a static probe with a class that sets both lists, a target relabeling of the same label env, and its own metricRelabelConfigs.

go test -count=1 -v -run '^Test_generateProbeConfig$' ./internal/controller/operator/factory/vmscrapes/

Before the fix the test fails: the class env entry and the probe_.* keep are missing from the output. After the fix it passes, and the other 9 tests in the package pass. Putting back either changed spot (the metric merge, the relabel loop) fails the test on its own missing lines.

Overlap checked: searched issues and PRs for scrapeClass VMProbe, scrapeClass metricRelabelConfigs, scrape class probe relabel, vmprobe scrapeClass relabel, scrapeClass relabelConfigs ignored, vmprobe relabel and the open PR list for probe and scrapeClass. No issue or open PR covers it. #1997 and #2003 set the order (class before the rules of the resource). release-0.68 has the same auth-only block in probe.go; this PR does not touch it.

@spicysharkco
spicysharkco force-pushed the fix/vmprobe-scrape-class-relabelings branch from 1ff1591 to eda8e08 Compare October 17, 2026 16:17

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found and verified against the latest diff

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/controller/operator/factory/vmscrapes/probe.go">

<violation number="1" location="internal/controller/operator/factory/vmscrapes/probe.go:69">
P2: `generateProbeConfig` mutates `sc.Spec.MetricRelabelConfigs`, so repeated generation of the same `VMProbe` prefixes the class rules again. Keep the merged list local and pass that list to `addMetricRelabelingsTo` instead of modifying the input spec.</violation>
</file>

Reply to a comment to ask cubic a question or push back. It learns from your replies.
Click Fix with cubic on a comment, or reply @cubic-dev-ai fix this, and cubic writes the fix.

Re-trigger cubic

scrapeClass := getScrapeClass(spec.ScrapeClassName, sp)
if scrapeClass != nil {
mergeEndpointAuthWithScrapeClass(&spec.EndpointAuth, scrapeClass)
spec.MetricRelabelConfigs = slices.Concat(scrapeClass.MetricRelabelConfigs, spec.MetricRelabelConfigs)

@cubic-dev-ai cubic-dev-ai Bot Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: generateProbeConfig mutates sc.Spec.MetricRelabelConfigs, so repeated generation of the same VMProbe prefixes the class rules again. Keep the merged list local and pass that list to addMetricRelabelingsTo instead of modifying the input spec.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At internal/controller/operator/factory/vmscrapes/probe.go, line 69:

<comment>`generateProbeConfig` mutates `sc.Spec.MetricRelabelConfigs`, so repeated generation of the same `VMProbe` prefixes the class rules again. Keep the merged list local and pass that list to `addMetricRelabelingsTo` instead of modifying the input spec.</comment>

<file context>
@@ -65,6 +66,7 @@ func generateProbeConfig(
 	scrapeClass := getScrapeClass(spec.ScrapeClassName, sp)
 	if scrapeClass != nil {
 		mergeEndpointAuthWithScrapeClass(&spec.EndpointAuth, scrapeClass)
+		spec.MetricRelabelConfigs = slices.Concat(scrapeClass.MetricRelabelConfigs, spec.MetricRelabelConfigs)
 	}
 
</file context>
Fix with cubic

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each config generation lists the probes again (selectProbes reads a new VMProbeList) and calls generateProbeConfig once per probe, so the class rules are added once. The other scrape objects merge the class lists into their spec the same way (mergeEndpointRelabelingsWithScrapeClass), and this function already writes the path and module defaults into the spec, so I kept that pattern.

A VMProbe that references a scrape class got only the authentication
settings of the class. The relabelConfigs and metricRelabelConfigs of
the class were never added to the scrape config, while VMServiceScrape,
VMPodScrape, VMNodeScrape, VMStaticScrape and VMScrapeConfig apply them.

Add the class relabelConfigs in front of the relabelings of the targets,
and put the class metricRelabelConfigs in front of the ones set on the
VMProbe.
@spicysharkco
spicysharkco force-pushed the fix/vmprobe-scrape-class-relabelings branch from eda8e08 to c590220 Compare October 19, 2026 18:33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants