JIT: fix start positions with an offset limit and in the prefix scan (2/3, stacked on #1004) - #1005
Open
shivneelmistry wants to merge 2 commits into
Open
shivneelmistry wants to merge 2 commits into
shivneelmistry wants to merge 2 commits into
Conversation
Four fixes found by differential testing of the JIT against the interpreter: - JIT: a variable-length lookbehind no longer moves STR_END to the lookbehind point. Assertions inside the lookbehind (\b, \B, $, lookaheads) now see the real end of the subject, and each branch is checked to end exactly at the lookbehind point instead. This makes the STR_END restore added for OP_ASSERTBACK_NA backtracking in PCRE2Project#912 unnecessary; its test still passes. Example: /y(?<!.{1,2}\b)b/ on "eyb" matched with the interpreter only. - JIT: (*THEN) in a branch of a variable-length lookbehind retried the next start position instead of moving to the next alternative. Example: /(?<=a?(*THEN)(?<=e)|x)B/ on "eBx" matched with JIT only. - JIT: (*THEN) in a standalone positive assertion nested inside a negative assertion escaped to the outer assertion, contrary to pcre2pattern ("The effect of (*THEN) is not allowed to escape beyond an assertion"). Example: /(?!(?=(*THEN)(*F))?)/ on "x" matched with JIT only. - Compile: a conditional group without a "no" branch inside a lookbehind was given the length of its "yes" branch only, so the lookbehind was treated as fixed length. Both matchers could miss matches, and the JIT could return a match that ends before it starts. Example: /(*naplb:(?(?=x)a))/ at offset 2 of "ae" returned (2,1) from the JIT. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two JIT bugs where a match attempt started at the wrong place:
- With PCRE2_USE_OFFSET_LIMIT, the bumpalong loop checked the limit
before advancing, so a multi-unit UTF character or a skipped CRLF
could move the next attempt past the limit.
Example: /\b/utf,use_offset_limit on "\x{300}\x{4e2d}\x{1f600}1" with
offset_limit=6 matched at offset 9 with JIT only.
- scan_prefix() kept the repeat count from an OP_TYPEEXACT whose type it
could not handle and applied it to the next alternative. The computed
prefix was then wrong and fast-forwarding skipped real matches (and it
tripped an SLJIT_ASSERT in debug builds).
Example: /(?:xyzw\R{4}|abc)/ did not match "abc" with JIT.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 24, 2026
Member
|
I can't really contribute anything to the review of this one. I do however have infinite free Astra tokens, so I can throw some compute at it. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1004. Only the last commit is new; review ed406ce. I'll rebase once #1004 is merged.
This is the second of three stacked PRs from JIT vs. interpreter differential fuzzing:
(*THEN)in assertionsBoth bugs let the JIT try matches at the wrong start positions. Both give wrong results in normal (non-partial) matching.
Fixes
Offset limit bumpalong.
mainloop_entry()checkedSTR_PTR < limitbefore advancing one character. In UTF mode, or when a CRLF is skipped, that advance is more than one code unit, so the next attempt could start past the limit. The limit is now checked after the advance, in the same place for all paths. On failure it jumps to the normal "no more start positions" exit, which keeps a soft partial match found earlier./\b/utf,use_offset_limit\x{300}\x{4e2d}\x{1f600}1\=offset_limit=6/(?=X)/newline=crlf,use_offset_limit,no_start_optimize\r\nX\=offset_limit=1Prefix scan with a stale repeat count. In
scan_prefix(),OP_TYPEEXACTsetsrepeatbefore the type is examined. If the type is not supported (for example\R,\X,\h), the branch is abandoned, butrepeatwas not reset before the next alternative was taken off the stack. The next alternative's first character was then repeated into later positions of the prefix, and fast-forwarding skipped real matches. For a class this also trippedSLJIT_ASSERT(last == TRUE && repeat == 1)in debug builds./(?:xyzw\R{4}|abc)/abcabc/(?:wxyz\h{2}|abcd)/abcdabcd/(Z|[ab])\R{4}/Tests
testinput2cases. The unpatched JIT gets 3 of them wrong; the other two guard the debug assertion and a\Xvariant.testinput10(8-bit) andtestinput12(16/32-bit) get the UTF offset-limit cases. Code-unit offsets differ by width, so the expected results differ too.pcre2_jit_test.c.The whole stack was validated as described in #1004.
🤖 Generated with Claude Code