Repository navigation
Skip decoding huge member signatures in the tree view's type system - #4356
Open
christophwille wants to merge 1 commit into
Open
christophwille wants to merge 1 commit into
christophwille wants to merge 1 commit into
Conversation
The assembly tree decodes a field's type on the UI thread to render its label. A signature that expands exponentially (#2064: a nested type referenced through a derived generic class, 29 MB of blob) blocked the UI for about 20 s and allocated 3 GB. The blob length is known before decoding, so an opt-in size limit is enough and no replacement for the System.Reflection.Metadata decoder is needed. Only the type system behind the tree sets the option; the decompiler keeps decoding in full. Assisted-by: Claude:claude-fable-5-2:Claude Code
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the tree view hang in #2064. It does not close the issue on its own: the IL view of such a type is still slow (see "Known limits").
Problem
Browsing to a field whose type signature is exponentially large freezes the UI. The test uses the sample from Daniel's second comment on the issue:
A nested type cannot be referenced through a derived class in metadata, so every
Y.step expands toX<Y, Y, Y, Y, Y>.Ywith the previous type substituted five times. The signature grows 5x per level.The assembly tree reads the field node's label on the UI thread, and that decodes the full field type into the tree's uncached type system. Measured on current master (Debug build, the tree's type-system options
Default | Uncached | KeepModifiers):field.TypeThe cost is the decode itself; rendering the label afterwards takes milliseconds.
Change
TypeSystemOptions.LimitSignatureSize, not part ofDefault.MetadataField,MetadataMethodandMetadataPropertydo not decode a signature blob larger than 16 KB (MetadataModule.MaxLimitedSignatureSize). The check throwsBadImageFormatExceptioninside the existingtryblocks, so the existing fallback applies: the type isSpecialType.UnknownTypeand, for methods and properties, the parameter list is empty.LoadedAssembly.GetTypeSystemOrNulloverloads set the option. That is the type system behind the assembly tree, search and the analyzers.ilspycmddo not set the option and decode in full, as before.In the tree the field now shows with an unknown type instead of freezing the window. Selecting the type still decompiles it completely.
Options considered
1. Blob-size limit behind an opt-in type-system option (this PR). The length of a signature blob is known before any decoding starts, so the limit is a constant-time check and needs no decoder of our own. It sits in the three places where member signatures are decoded and reuses the fallback that already exists for malformed signatures, which keeps the change to about 30 lines of product code. Because it lives in the type system, everything that reads members from the tree's type system is covered at once, not only the label. The cost is a new public
TypeSystemOptionsmember, and that consumers of that type system see an unknown type where a real (if absurd) one exists.2. Guard in the UI only (
Language.EntityToString). This would leave the decompiler library untouched: the label code would look up the entity's signature size and print a placeholder instead of asking for the type. The drawback is that the label is only one of several readers. Search results, tooltips and the analyzers readIField.Typeand friends from the same type system, and each would need the same guard, or stay exposed to the same multi-second, multi-gigabyte decode. A guard that has to be repeated at every call site is easy to miss the next time someone adds a reader.3. The same limit, always on. Dropping the option would be the least code and would also protect the decompiler and disassembler from such signatures. But the assembly is valid, and today
ilspycmddecompiles the 8-level sample correctly in about 3 s. An unconditional limit would make the decompiler emit an unknown type for a field it can currently decompile, which is a regression in output for the sake of a UI problem. Daniel's comment on the issue draws the same line: a small limit for the tree view, none for the decompiler.4. Replace the System.Reflection.Metadata signature decoder with our own. This is what Daniel suggested on the issue, and it has real advantages: decoding could be aborted midway on any budget (size, depth, node count), and
ApplyAttributeTypeVisitorcould be merged into the decoder, saving allocations for every signature, not just pathological ones. It is also a large change to a central piece of the type system, with every element type and every provider callback to re-implement and re-test. For the purpose of not decoding a huge signature it is not needed, because the blob length already tells us the size up front. It remains a reasonable project on its own merits.5. Make decoding cheaper (interning or caching in
TypeProvider). Identical sub-signatures could be shared so that the 5^n instantiations collapse to n distinct objects, which would cut the memory use dramatically. It does not change the fact that the decoder walks all 29 MB of blob and calls the provider for every node, and the visitor that runs after decoding walks the result again, so the time stays proportional to the blob size. It would also add a cache with its own lifetime questions to a type system that is deliberately uncached for the tree. It reduces the constant but does not remove the freeze. This option was evaluated by reading the code only; no prototype was built.Test
SignatureSizeLimitTests.HugeFieldSignaturecompiles the sample above (TestCases/IsolatedDecompilation/HugeGenericSignature.cs) and asserts that the field's type isUnknownwith the option and a fully decoded class with five type arguments without it. There is no timing assertion. The fixture is excluded from the test assembly's own compilation, otherwise the 5.9 MB signature would end up inICSharpCode.Decompiler.Tests.dll.The test was red with only the enum member in place, then green with the guard.
Results on Windows:
ICSharpCode.Decompiler.Tests,TypeSystemnamespace: 191 of 191 passed.ILSpy.Tests: shards 1 to 3 without failures; shard 4 run withoutSwitchListAndDeleteAllTests(it hangs on my machine independent of this change, see Keep the Ctrl+A delete test at the four assemblies it opens #4355): 474 of 474 passed.ilspycmdC# output for typeYof the 8-level sample is byte-identical before and after.Not run locally: the full decompiler suite (Pretty, Correctness and the other file-based runners), and a manual check in the running application.
Known limits
The 16 KB limit is a pick, not a tuned value. The reasoning behind it: ordinary signatures are tens to a few hundred bytes, and decoding costs roughly 0.6 microseconds per byte in the uncached type system (derived from the table above), so 16 KB is about 10 ms, short enough for the UI thread. In the sample, 4 nesting levels (9.4 KB) still decode and 5 levels (47 KB) do not. It has not been checked against a corpus of real assemblies, so it is unknown how large the largest legitimate signatures in the wild are. Heavily generic or generated code is where a legitimate signature would most plausibly cross the line. If that happens, the effect is an unknown type in the tree label, not a wrong decompilation, and the constant is a single place to change.
Analyzers and search see
UnknownTypefor such members. They run on the same type system as the tree, so for a member over the limit they get an unknown type and, for methods and properties, an empty parameter list. An analyzer that matches on a member's declared type or parameter types through the type system can therefore miss such a member, and search results render it with an unknown type. Analyzers that scan the raw signature blob themselves (for example the "type used by" analyzer, which has its own allocation-free decoder) are unaffected. Before this change the same operations paid the multi-second, multi-gigabyte decode for these members. I have not gone through every analyzer to list exactly which ones are affected.The IL view of such a type remains slow. Disassembling
Ytook 57 s for the 8-level sample (ilspycmd -il, Debug build) and produced 14 MB of text. The disassembler does not go through the type system: it decodes the signature into a tree of closures first and only then writes text, so the text view's output length limit can only trigger after the whole signature has been walked, and cancellation is only checked between members. In the application this runs on a background thread, so the window stays responsive, but the tab is busy for that long and the work cannot be cancelled mid-signature. Fixing it needs a size check or cancellation inside the disassembler's signature writing, which is a separate change. The C# view is not affected in the same way (about 3 s for the same type).Assisted-by: Claude:claude-fable-5-2:Claude Code
🤖 Generated with Claude Code