Repository navigation
Conversation
`LinkInlineProcessor.getText` scanned forward from each `[` to find its matching `]`. The inline treeprocessor calls it for every `[` in the text, so when the brackets never close each call scanned to the end of the text and conversion became quadratic in the number of brackets. Find the closing bracket for every index in a single pass and cache the result for the current text. Matches and returned text are unchanged.
| ) | ||
|
|
||
| def test_many_unclosed_square_brackets(self): | ||
| # Each unclosed bracket used to rescan the rest of the text. |
There was a problem hiding this comment.
The spell checker seems to not like the word "rescan." It should probably be added to .spell-dict.
There was a problem hiding this comment.
Added rescan to .spell-dict in 57f3021. It only appears in that test comment, so that should be the only word the spell checker was flagging from this PR.
|
I am going to let @facelessuser review this as he is more familiar with the affected code. |
|
I will note that I've seen the review request and will try to get to it sometime soon. |
|
Sorted the conflict now that #1676 is in. It was only |
| return ''.join(text), index, bracket_count == 0 | ||
| # Scanning forward from every `[` is quadratic when the brackets do not | ||
| # close, so find the closing bracket for every index of `data` at once. | ||
| if getattr(self, '_bracket_data', None) is not data: |
There was a problem hiding this comment.
Why not declare this in __init__ and defaul it to None so we don't have to check if it exists every time. It should just always exist.
There was a problem hiding this comment.
For clarity, _bracket_data should be initialized in __init__.
`LinkInlineProcessor.getLink` scans forward from the `(` of a link target
to pair the brackets. The treeprocessor calls it for every `[text](` in
the text, and when the parentheses never close each call scans to the end,
so conversion time grows with the square of the number of unclosed `(`.
For example `markdown.markdown('[a](' * n)` on CPython 3.12:
'[a](' x4000 (16000 bytes): 2.27s -> 0.015s
'[a](x ' x4000 (24000 bytes): 3.56s -> 0.018s
'[a](<u>' x4000 (28000 bytes): 4.11s -> 0.064s
When no quote appears before the matching `)` the scan is a plain bracket
match, so resolve it from a table of closing positions built in a single
pass per text and only fall back to the character-by-character loop when a
quote may start a title (the quote/title inputs were already linear). The
new `getLink` was compared with the old loop at every index of all 1.1M
strings up to length 6 over the 10 characters `[ ] ( ) ' " < > a` and
space, and of 2M longer random strings; the returned tuples were identical
in every case.
This is the `getLink` companion to the `getText` fix in Python-Markdown#1664.
Description
LinkInlineProcessor.getText(used by the link, image and reference processors) scans forward from a[until the brackets balance. The treeprocessor calls it for every[in the text, and when the brackets never close each call scans to the end, so conversion time grows with the square of the number of unclosed brackets. Timings formarkdown.markdown(s * n)on CPython 3.11:This now finds the closing bracket for every index in one pass and caches the result for the current text, so repeated calls on the same text are a list lookup. I compared the new
getTextwith the old loop on every index of about 30,000 random strings made of[,],aand((about 400,000 calls) and the results were identical. The new tests pass with and without the change.test_many_unclosed_square_bracketstakes about 27 seconds without the fix and finishes instantly with it.getLinkhas a similar rescan for unclosed(but its quote/title handling is more involved, so I left it out of this change.AI Assistance Disclosure
No AI tools were used in preparing this PR.
AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.
AI tools were use to assist with the following:
Tool used: Claude Code (Claude Opus 5.5). wrote the patch, tests, changelog entry, commit message and this description. I reviewed and checked all of it before submitting.
Checklist