Skip to content

Segment benchmarks by core/support - #2984

Merged
howardchung merged 4 commits into
odota:masterfrom
geracosta:benchmark-roles
Oct 3, 2026
Merged

howardchung merged 4 commits into
odota:masterfrom
geracosta:benchmark-roles

Conversation

@geracosta

Copy link
Copy Markdown
Contributor

Implements #2980, with the core/support split you okayed there. Cores and supports get their own benchmark sets within each bracket, since a support Mirana and a core Mirana shouldn't share a distribution.

Write side: updateBenchmarks ranks each team by gold_per_min and calls the top 3 cores and the rest supports, then zadds the sampled value into the role sets next to the existing bracket keys. That's the split I measured in #2980: exact 1-5 from api data alone is only 58%, but core vs support is 99.4% (924/930 players over the same 100 pro matches from the #1590 study). Only computed where there's a bracket, so turbo and matches without avg_rank_tier are untouched, and teams that don't have 5 players are skipped.

Following the key layout from ff39364, the role is its own slot rather than reusing an existing one: benchmarks:{epoch}:{metric}:{heroId}:{turbo}:{bracket}:{role}. Existing keys come out byte-identical, so this only doubles the bracket sets and nothing collides.

Read side: GET /benchmarks?hero_id=X&bracket=5&role=core. Validated against core/support, and 400 if role comes without a bracket, since the role only segments the bracketed sets. Omitting both behaves exactly as today.

Same caveat as #2972: the sets start filling on deploy, so role queries will be sparse until the current epoch completes. Nothing to migrate.

Comment thread svc/util/queries.ts Outdated
const suffix = bracket ? `:${bracket}` : "";
// Bracketed sets add the 1-8 rank bracket after the global/turbo slot, and
// core/support after that. The role only segments the bracketed sets.
const suffix = bracket ? (role ? `:${bracket}:${role}` : `:${bracket}`) : "";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer appending bracket and role if they exist to the empty string, rather than nesting like this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. folded #2985 in here too since it's the same line, rather than landing two changes that would conflict.

suffix is built by appending now, and the bracket and role are independent slots, so either works on its own: ...:<hero>::5, ...:<hero>::core, ...:<hero>::5:core. no empty padding needed, since a bracket is 1-8 and a role is text they can't be confused in the same slot.

one thing that came out of making them independent: roles were only being computed for matches that had a bracket, which would have made ?role=core quietly cover fewer matches than the global set. it's every non-turbo match now, same population as global.

Comment thread svc/util/queries.ts Outdated
// Bracketed sets add the 1-8 rank bracket after the global/turbo slot
const suffix = bracket ? `:${bracket}` : "";
// The bracket and the role are independent slots after the global/turbo one,
// so either can be used on its own: ...:<hero>::5, ...:<hero>::core

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this comment correct? The 5 and the core should be in different slots, right?

so it could be:
::5:
::5:core
:::core

I think this is different from the implementation below, where it's storing a key that can have either bracket or role in the same slot. It might be less confusing to always have the same number of slots, with empty data for the "undefined" case.

I would keep the original keys as with no suffix just for backward compatibility but for all the new filter options, always include the same number of colons/segments

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right. i read your earlier note about appending as being about the key layout too, so i'd collapsed the bracket and the role into one slot.

fixed slots now, with the unfiltered key left alone:

...:<hero>:          no filters, unchanged
...:<hero>::5:       bracket
...:<hero>:::core    role
...:<hero>::5:core   both

every filtered key carries both segments with an empty one for the filter not in use, so they all have the same number of colons.

this also changes the bracket-only key from ...:<hero>::5 to ...:<hero>::5:, so the bracket sets from #2972 that are live right now stop being read once this ships. they refill within one retention epoch and the read already falls back to the current epoch, so it's the same sparse window as when brackets first shipped. just didn't want it to look like only the new role sets start empty.

Comment thread svc/util/queries.ts
// Bracketed sets add the 1-8 rank bracket after the global/turbo slot
const suffix = bracket ? `:${bracket}` : "";
// Filtered sets always carry both slots, empty for the ones not in use:
// ...:<hero>::5: ...:<hero>:::core ...:<hero>::5:core

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's an extra colon in this comment?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's right, but only because the base key already ends in the empty turbo slot, which the comment didn't show. spelled the slots out in de4a1c1 so it reads without that context:

// Key slots are benchmarks:<epoch>:<metric>:<hero>:<turbo>:<bracket>:<role>

The examples only add up once you know the base key already ends in the
empty turbo slot, so spell out every slot.
@howardchung
howardchung merged commit be73c2b into odota:master Oct 3, 2026
1 check passed
@geracosta
geracosta deleted the benchmark-roles branch October 3, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants