Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions articles/tools/observability/configuration.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ See <<reference#ui-state-size,UI State Size>>.

|`vaadin.observability.client`
|`true`
|Browser-side timing collected from the client.
|Browser-side timing, errors, and connection state collected from the client.

|`vaadin.observability.resync`
|`true`
Expand Down Expand Up @@ -222,7 +222,7 @@ The feature toggles map directly to the built-in meters and spans:
Off by default.

|`navigation`
|Server-side navigation timing, tagged by route and outcome.
|Server-side navigation timing, tagged by route, outcome, and error.

|`requests`
|Server-side request and RPC timing.
Expand All @@ -236,7 +236,7 @@ Also gates the data query insights.
Also decorates the session error handler, which is what makes the failures Flow routes there countable and attributable to a component.

|`client`
|Browser-observed timing -- bootstrap, navigation, Web Vitals, and client errors.
|Browser-observed signals -- bootstrap, navigation, and Web Vitals timing, client errors, and connection-state transitions with their downtime.

|`resync`
|A counter of UIDL message resends and client-requested resynchronizations, tagged by type.
Expand Down
2 changes: 1 addition & 1 deletion articles/tools/observability/customization.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Observability Kit records everything it measures into your application's Microme
Your own instrumentation uses the *same* registry and the *same* Observation API, so any meters and spans you add sit right alongside the built-in ones and export through whatever backend you already configured -- Prometheus, OTLP, Zipkin, and so on.
There's nothing kit-specific to learn: if you know Micrometer, you know how to extend the kit.

Three ways exists for adding custom instrumentation, listed here from the most declarative to the most explicit:
You can add custom instrumentation in three ways, listed here from the most declarative to the most explicit:

[unordered]
<<#annotation-based,Annotation-based>>:: Add [annotationname]`@Timed`, [annotationname]`@Counted`, or [annotationname]`@Observed` to a method and let an aspect do the work.
Expand Down
22 changes: 22 additions & 0 deletions articles/tools/observability/getting-started.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ If you're upgrading an existing project, see the <<migrating-to-v5#,Migrating to
* Spring Boot 4, for the `observability-kit-starter`.
Plain-Spring and standalone setups are also supported; see <<#other-setups,Other Setups>>.

GraalVM native images are supported out of the box: the kit ships the native-image metadata its resources need.


== Add the Dependency

Expand Down Expand Up @@ -210,6 +212,26 @@ With this overload the kit registers no handlers of its own, so the registry nee
Keep a reference to both registries -- for example in a static holder -- if you want to use them for <<customization#,custom instrumentation>>.


== Troubleshooting

No `vaadin.*` meters appear at all::
In development mode the kit validates your commercial license at startup.
If validation fails, the application keeps running but the kit registers no instrumentation -- it logs a warning rather than failing the build or the deployment.
Check the application log; the <<insights#,insights endpoint>> also reports `instrumentation: inactive` in this state.

`vaadin.errors` is missing::
The counter is registered lazily, on the first error it counts.
An application that hasn't failed yet publishes no `vaadin.errors` series -- absence means zero, not a broken setup.

The names look different in the backend::
Backends render Micrometer's dotted names in their own conventions, and Prometheus also renames the `.created` counters to `vaadin_sessions_total` and `vaadin_ui_total`.
See the naming note on the <<reference#,Reference>> page.

Percentile queries return no data::
Timers publish no histogram buckets by default; a `histogram_quantile()` query or a p95 panel silently comes up empty until you enable them.
See <<integrations#percentiles,Percentiles and Histogram Buckets>>.


== Next Steps

* <<configuration#,Configuration>> -- turn features on or off and tune the kit.
Expand Down
32 changes: 32 additions & 0 deletions articles/tools/observability/insights.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ Requires `vaadin.observability.errors` (on by default), which supplies the failu
Slow interactions:: An invocation that succeeded but took longer than the 1000 ms UX budget.
Requires `vaadin.observability.requests` (on by default), which supplies the timing path.
Beyond roughly a second a user stops feeling that they're operating directly on the UI, so the budget is absolute rather than relative to a historical baseline.
It's also fixed: there is no property to change it.

=== Data Provider Queries

Expand Down Expand Up @@ -68,13 +69,18 @@ management.endpoints.web.exposure.include=vaadin

Add it to whatever else you already expose, for example `prometheus,vaadin`.

Exposure is all a standard setup needs, since Actuator endpoints allow access by default.
If your application restricts endpoint access globally with [propertyname]`management.endpoints.access.default`, restore it for this endpoint with `management.endpoint.vaadin.access=unrestricted`.

[IMPORTANT]
====
The payload describes what users did and what broke, so treat the endpoint as privileged.
Secure it as you would any other Actuator endpoint -- put it behind authentication, or bind the management port to an internal interface with `management.server.port` and `management.server.address`.
====

The endpoint is part of the Spring Boot starter and needs Actuator on the classpath.
The payload is also available in-process: inject the [classname]`VaadinObservabilityEndpoint` bean and call [methodname]`section("observability")` -- for example to feed an admin view or an AI agent without going through HTTP.

In plain-Spring and standalone deployments the collectors still run, and you can read the buffers yourself through [methodname]`ObservabilityKit.getRecentInteractions()` and [methodname]`ObservabilityKit.getRecentQueries()`, passing both to an [classname]`InsightsService` to render the same payload.


Expand Down Expand Up @@ -290,6 +296,32 @@ This adds the exception message (truncated to 200 characters), the top five stac
An exception message is free-form text and can carry a whole payload, which is exactly why it's opt-in.


[[ai-agents]]
== Fixing Insights with an AI Agent

The payload is designed to be handed to a coding agent: every field an agent needs is machine-readable and versioned by [propertyname]`schemaVersion`.
[propertyname]`applicationFrame` names the class, file, and line to open; [propertyname]`replay` lists the steps that reproduce the problem; and [propertyname]`suggestion` states a starting hypothesis grounded in the evidence.

Fetch the payload and hand it to an agent that has the codebase checked out:

[source,terminal]
----
curl -s http://localhost:8080/actuator/vaadin/observability | claude -p \
"These are insights from the running application: user interactions \
that failed or blew the UX budget. For each insight, open the \
applicationFrame, verify the problem against the replay steps and the \
evidence, and propose a fix."
----

The example pipes the payload into the Claude Code CLI, but any agent that can read files and apply edits works the same way: paste the JSON into the conversation, or let the agent fetch the endpoint itself.

Three practicalities:

- Point the agent at the same revision that produced the insights; otherwise the line number in [propertyname]`applicationFrame` may have drifted.
- The default payload already carries what an agent needs -- the route, component, event, exception type, and application frame -- while withholding exception messages and session IDs, so it's safe to forward to an external tool without enabling <<#detail,detail collection>> first.
- A query insight carries no [propertyname]`suggestion` or [propertyname]`applicationFrame`, so tell the agent to find the data provider that the named component is given on the named route.


== Configuration

[cols="2,1,3"]
Expand Down
44 changes: 44 additions & 0 deletions articles/tools/observability/integrations/grafana.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,47 @@ Select a metric from the [guilabel]*Metric* drop-down -- for example `vaadin_ses

Open the Explore view and select [guilabel]*Loki* as the data source.
Under [guilabel]*Labels*, configure a label for `service_name` with your application's service name, then click the refresh button in the top-right to search for logs.


== Example Dashboard Queries

The following PromQL expressions cover the most useful panels for a Vaadin dashboard.
Remember that Prometheus renders the dotted meter names in its own convention -- `vaadin.request.duration` becomes `vaadin_request_duration_seconds`, and the `.created` counters become `vaadin_sessions_total` and `vaadin_ui_total`.

[cols="1,2"]
|===
|Panel |Query

|Active sessions
|`sum(vaadin_sessions_active)`

|Active UIs (browser tabs)
|`sum(vaadin_ui_active)`

|Request rate
|`sum(rate(vaadin_request_duration_seconds_count[$__rate_interval]))`

|Server errors
|`sum(rate(vaadin_errors_total[$__rate_interval]))`

|Request latency, p95
|`histogram_quantile(0.95, sum by (le) (rate(vaadin_request_duration_seconds_bucket[$__rate_interval])))`

|RPC latency by type, p95
|`histogram_quantile(0.95, sum by (le, type) (rate(vaadin_rpc_duration_seconds_bucket[$__rate_interval])))`

|Navigation rate by route
|`sum by (route) (rate(vaadin_navigation_seconds_count[$__rate_interval]))`

|Session lock wait, p95
|`histogram_quantile(0.95, sum by (le) (rate(vaadin_session_lock_wait_seconds_bucket[$__rate_interval])))`

|Retained UI state against the heap
|`sum(vaadin_ui_state_size_bytes)` charted next to `sum(jvm_memory_used_bytes{area="heap"})` and `sum(jvm_memory_max_bytes{area="heap"})`
|===

The percentile queries read `_bucket` series, which the kit's timers don't publish by default -- enable them per meter first; see <<index#percentiles,Percentiles and Histogram Buckets>>.
The UI state panel needs the opt-in `vaadin.observability.ui-state` feature, and a byte figure appears only when [propertyname]`vaadin.observability.ui-state-bytes-per-node` is set; see <<../reference#ui-state-size,UI State Size>>.

A complete dashboard built from these queries ships with the https://github.com/vaadin/use-cases/tree/main/observability[Vaadin observability use-case application]: a provisioned Grafana dashboard ([filename]`grafana/dashboards/vaadin.json`) together with a Docker Compose stack that runs Prometheus and Grafana against the application.
Import the JSON into your own Grafana as a starting point, or run the stack to see the panels over live data.
28 changes: 28 additions & 0 deletions articles/tools/observability/integrations/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,34 @@ While testing an integration, set it to `1.0` so every request produces a span.
Lower it again for production.


[[percentiles]]
== Percentiles and Histogram Buckets

The kit's timers export their count, sum, and max only by default.
That's enough for rates and averages, but a percentile query -- a `histogram_quantile()` expression, a Grafana p95 panel, a latency alert -- needs histogram buckets, and Micrometer publishes those only when asked.
A percentile query over a bucket-less timer doesn't fail; it silently returns no data.

Enable buckets for the timers you build percentiles on:

.`application.properties`
[source,properties]
----
management.metrics.distribution.percentiles-histogram.vaadin.request.duration=true
management.metrics.distribution.percentiles-histogram.vaadin.rpc.duration=true
management.metrics.distribution.slo.vaadin.request.duration=25ms,50ms,100ms,250ms,500ms,1s,2s
management.metrics.distribution.slo.vaadin.rpc.duration=25ms,50ms,100ms,250ms,500ms,1s,2s
----

The [propertyname]`slo` lines are optional: they add explicit bucket boundaries at your service-level objectives, the durations you alert on.
The `1s` boundary matches the UX budget that <<../insights#,interaction insights>> use, so dashboard percentiles and insights draw their line at the same place.

The property keys match meters by dotted prefix, so `management.metrics.distribution.percentiles-histogram.vaadin=true` covers every `vaadin.*` timer at once.
Prefer enabling buckets per meter, though: each bucketed timer publishes tens of additional series per tag combination, so a blanket switch multiplies the metric volume for timers you never query by percentile.

One meter is an exception: `vaadin.db.fetch.rows` publishes pre-computed p95 and p99 percentiles out of the box -- in Prometheus, as `quantile` series -- so you can alert on runaway result sets without any bucket configuration.
Pre-computed percentiles are calculated per instance and per tag value; they can be read directly but not re-aggregated across instances the way histogram buckets can.


== Local Development & Testing

For development-time testing on your own machine, use one of these setups:
Expand Down
2 changes: 1 addition & 1 deletion articles/tools/observability/integrations/prometheus.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -90,5 +90,5 @@ If you don't know the metric name, click the globe icon next to [guibutton]*Exec
To view a result as a graph, click the [guilabel]*Graph* tab.

Prometheus renders Micrometer's dotted meter names in its own convention.
For example, the `vaadin.request.duration` timer appears as `vaadin_request_duration_seconds` with `_count`, `_sum`, and bucket suffixes.
For example, the `vaadin.request.duration` timer appears as `vaadin_request_duration_seconds` with `_count`, `_sum`, and `_max` suffixes -- plus a parallel `vaadin_request_duration_active_seconds` long-task family when tracing is on, and `_bucket` series once you opt into <<index#percentiles,histogram buckets>>.
See the <<../reference#,Reference>> page for the full list of meters.
15 changes: 15 additions & 0 deletions articles/tools/observability/migrating-to-v5.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,19 @@ Actuator registers JVM and process metrics out of the box.

For the complete, current list of meters, see the <<reference#,Reference>> page.

.Queries Break Structurally, Not Only by Name
[IMPORTANT]
====
Three export-level changes affect existing queries and alerts beyond the renames above:

* The version 4 agent exported OpenTelemetry histograms with buckets.
Version 5 timers publish count, sum, and max only, so a `histogram_quantile()` query or a latency alert finds no `_bucket` series until you enable them per meter -- and it fails silently, returning no data rather than an error.
See <<integrations#percentiles,Percentiles and Histogram Buckets>>.
* Prometheus strips the reserved `_created` suffix, so `vaadin.sessions.created` and `vaadin.ui.created` appear as `vaadin_sessions_total` and `vaadin_ui_total`.
* With tracing on, each observed timer also publishes a parallel long-task timer (`_active` series in Prometheus), which roughly doubles the series count for those meters.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_active is explained only here; reference.adoc:24 and prometheus.adoc:93 enumerate suffixes without it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f2ea140. The _active_seconds family is now also mentioned in the reference naming note and on the Prometheus page, where the suffixes are listed.

Account for it when estimating metric volume.
====

.Tracing Data Is Shaped Differently
[NOTE]
Version 4 produced a rich span tree with many Vaadin-specific attributes, some of them per element synchronization.
Expand All @@ -344,6 +357,8 @@ Because the kit no longer exports telemetry itself, integration with backends no

* *Metrics* are exported by adding the corresponding `micrometer-registry-*` dependency (Prometheus, OTLP, Graphite, and others) and exposing it through Actuator.
* *Traces* are exported by adding a Micrometer tracing bridge -- for example OpenTelemetry (`micrometer-tracing-bridge-otel`) or Zipkin (`micrometer-tracing-bridge-brave`) -- as you would for any Micrometer-instrumented application.
* *Logs* are no longer exported by the kit at all.
The version 4 agent could ship them over OTLP; in version 5, route them through your logging pipeline instead -- see the vendor pages under <<integrations#,Integrations>> for the options.

Vendors such as Prometheus, Grafana, Datadog, and New Relic are all reachable this way, either through a native Micrometer registry or through an OTLP bridge to an OpenTelemetry collector.
See the <<integrations#,Integrations>> page for vendor-specific setup.
Expand Down
Loading
Loading