Skip to content

PE: Separate GNU EH-frame hints from unwind entries - #789

Open
zardus wants to merge 1 commit into
masterfrom
feature/pe-eh-frame-function-boundaries
Open

zardus wants to merge 1 commit into
masterfrom
feature/pe-eh-frame-function-boundaries

Conversation

@zardus

@zardus zardus commented Aug 26, 2026 •

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

CLE does not read a PE's GNU .eh_frame at all, and it labels native exception-directory records EH_FRAME, which is the name angr reserves for something else.

Both halves reproduce on binaries already on angr/binaries master, so this needs nothing unmerged to see:

import cle
# a 32-bit MinGW PE with a .eh_frame section and no exception directory
o = cle.Loader("binaries/tests/i386/windows/known_patterns_wdk_ksud.exe", auto_load_libs=False).main_object
print(len(o.function_hints))          # master: 0        this branch: 81, all EH_FRAME

o = cle.Loader("binaries/tests/x86_64/windows/7995a0325b446c462bdb6ae10b692eee2ecadd8e888e9d7729befe4412007afb",
               auto_load_libs=False).main_object
print({h.source for h in o.function_hints})   # master: {EH_FRAME}   this branch: {EXCEPTION_DIRECTORY}

The .eh_frame of known_patterns_wdk_ksud.exe is not visible to a grep of its section headers: the name is nine characters, a header field is eight, so it is stored as /4 and resolved through the COFF string table.

The new fixture on angr/binaries#196 covers the case those two cannot: an FDE whose start address CFGFast has already assigned to an enclosing function. In it the wrapper at 0x401006 tail-jumps to an unnamed six-byte function at 0x40100a whose FDE is the only record that it is a function at all.

Root cause

The PE backend never parsed GNU .eh_frame. Its only function hints came from the native unwind directory, so downstream angr could not distinguish the precise GNU FDE start from lower-confidence native unwind entries.

Fix

Parse GNU PE FDEs as EH_FRAME hints and classify native unwind records as EXCEPTION_DIRECTORY. This gives angr the source distinction needed to preserve an occupied authoritative boundary.

This has a merge order: it must not land ahead of angr's pull request 6948. With this change and without that one, separating the two CLE hint sources turns all 4,219 exception-directory records of tests/x86_64/windows/7995a0325b446c462bdb6ae10b692eee2ecadd8e888e9d7729befe4412007afb into named function hints, and angr's existing assertion that 0x140032fd3 is not recovered fails. Land angr 6948 first, or land the two together. EXCEPTION_DIRECTORY is 4 rather than 3 because 3 is MACHO_FUNCTION_STARTS, which cle#810 added to this same class.

Testing

test_gnu_eh_frame_function_hints asserts FunctionHint(0x40100a, 6, EH_FRAME) on the angr/binaries#196 fixture, and test_exception_directory_function_hint_source preserves all 4,219 native hints as EXCEPTION_DIRECTORY on a binary already on binaries master. The coordinated angr regression preserves a separate function and decompiles return 42;. Validation: #789 (comment)

sync: angr/angr#6948
sync: angr/binaries#196

session: sharpen

@zardus

zardus commented Aug 26, 2026 •

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 9de1a5eee379be5900c4036db4722e35b5e12245 against baseline 997d4322678fe5876d9f83e0dd34e1052813b9db.

  • Chain heads: binaries c6d5b50df8706ffd6bbf78dd0c90ff97becb919c; angr 1b99ccbe452ed556e79177c41e624093f26ad8ed; CLE 89ae8c133b4c9c2ba2921429d3504ac2a0624407
  • Head transition: previous validation head 1ada2051049d8ed1595cab00b528934cf6aaf2f3 was rebased from parent 46a37333f4f59b0facf8774ee743ebc4cc074e9b onto a4fb8003198229d33c84df6a82f749729232fd31; the current PR remains one commit, Load GNU PE EH frame function hints
  • Blob identity: cle/backends/backend.py and cle/backends/pe/pe.py match the previous validation head; tests/test_pe.py changed from blob 397f6ad48410cd480c70c2b144f2f4d0fcec2626 to 5e41d37657501e23c9db0917b8f03ae7340f055d
  • Dependency state: binaries#196 is at c6d5b50df8706ffd6bbf78dd0c90ff97becb919c on base 3bfd1de09dd357deafbe29b4075aa55a9379ce50; its four PR fixture/provenance blobs match the prior e3e781a7d7193ca00182ab781a69b41c73e6ee68 head, and the current tree includes tests/java/interface_default.jar
  • Hosted refresh: exact-head CI run 33158448076 completed successfully on attempt 1; Build, Pyodide, macOS, Windows, decompiler snapshot, lint, typecheck, result publication, and all ten ci / Test shards passed
  • External statuses: docs/readthedocs.org:cle and pre-commit.ci - pr passed
  • Regression: python -m pytest tests/test_pe.py::TestPEBackend::test_gnu_eh_frame_function_hints tests/test_pe.py::TestPEBackend::test_exception_directory_function_hint_source remains the focused CLE coverage for the GNU PE FDE hint and native exception-directory source split
  • Fixture provenance: authored BSD-2-Clause source and deterministic GCC 15.3.0/Binutils 2.46 build; source SHA-256 1b2246525aca4cddbb510b4836116cf4aa78d8d4722ff4cb58caf7e029dbd99b; PE SHA-256 1cd9ae0b00d1c4cde4cd95579cf9aae14104d9f99031aab77b0116e4e9aa7c8a

Merge order: binaries#196, then the backward-compatible angr#6948 consumer, then CLE#789. Merging CLE first would expose its new exception-directory source to older angr versions.

Re-keyed 2026-08-29 to head bb0d2e9660093abcac45f5ea9f5f12ad714e26c8. Re-keyed from
89ae8c133b4c9c2ba2921429d3504ac2a0624407 after a rebase onto cle master
eac0e5540516b9199dd6a91933e80dc774ea3eac, which the branch needed once cle#800 merged.
The bullets above were measured at 89ae8c13, and they still describe this patch:
git range-diff a4fb8003..89ae8c13 eac0e554..bb0d2e96 marks the single commit ! rather
than =, and the whole of that is hunk offsets plus one context line master added next to
this branch's from io import BytesIO in cle/backends/pe/pe.py. Filtering both diffs to
the added and removed lines and comparing them, the branch's own payload is byte-identical
in all three files — cle/backends/backend.py 1 line, cle/backends/pe/pe.py 51,
tests/test_pe.py 28.

CI at this head is not green yet, and one leg is predicted to go red for a reason that
is not this branch's.
(Superseded; see the 2026-08-30 note below.) Read 2026-08-29T21:03Z on run
33270239532: Test macos-15 and
Test windows-2022 passed, Test (Pyodide) and ci / Build are queued, and the ten
ci / Test shards, ci / Lint, ci / Typecheck and the snapshot job have not been
scheduled yet — angr Actions is running a deep backlog.

When the ci / Test shards do run they are expected to fail
tests/analyses/decompiler/test_block_simplifier.py with
CLEFileNotFoundError on binaries/tests/i386/deep_sp_chain. Those shards run angr's
suite against this cle, and resolve_refs.py checks angr/binaries out at exactly the
revision this description pins — refs/pull/196/head, c7383da5957d77aef1cb63dec9e61d0d95098b72 —
rather than merging it with master. That tip is one commit behind binaries master, and the
commit it is missing, d4ffa2f0, is the one that added tests/i386/deep_sp_chain (binaries#215,
2026-08-29T19:12Z). angr master gained the test that loads it earlier the same day, in
angr#7032 at 14:39Z. So the fixture the pin is missing is one angr needs and nothing in
this pull request touches.

The repair for that belongs on the angr/binaries#196 branch — rebase it onto binaries
master so the pin carries d4ffa2f0 — not here. Nothing on this branch should be weakened
to make that leg green.

What was red earlier, since the record did not say. Two runs at the previous head
1ada2051 failed, both since superseded:

  • Run 32921788227 (2026-08-26):
    Test macos-15 failed with cle.errors.CLEFileNotFoundError: Could not find file .../binaries/tests/x86/windows/eh-frame-occupied-start.exe — this branch's own new
    fixture, which lives on angr/binaries#196 and is not on binaries master. Test windows-2022 and the ten shards were cancelled in the same run.
  • Run 33047263640 (2026-08-27):
    ci / Test (2) and ci / Test (3) failed on angr decompiler assertions, among them
    assert not cfg.kb.functions.contains_addr(0x140032FD3) in
    tests/analyses/decompiler/test_structurer.py — the merge-order interaction with
    angr#6948 that the description already describes.
  • Run 33158448076 (2026-08-28), at
    head 89ae8c13, passed all 18 named jobs including Test (Pyodide), Test windows-2022 and Test macos-15. That is the "Hosted refresh" bullet above, and it is
    the last complete run this branch has.

2026-08-30: that run finished green, and the deep_sp_chain failure it predicts is
dormant rather than gone.
Run
33270239532 succeeded on attempt 2,
which started 20:41:46Z and finished 23:36:39Z. Read at 2026-08-30T12:12Z, head bb0d2e96
carries 18 check runs, all success, plus docs/readthedocs.org:cle and
pre-commit.ci - pr.

Attempt 1 is what the paragraph above was watching, and it failed on a different missing
fixture. It started 19:11:53Z, ten minutes before angr/binaries#196 gained the tip it then
held, c7383da5957d77aef1cb63dec9e61d0d95098b72, at 19:21:57Z, so it resolved the tip before
that one, and ci / Test (4) died on CLEFileNotFoundError: Could not find file .../binaries/tests/riscv64/uefi/HighMemDxe.efi in cle's own
tests/test_pe.py::TestPEBackend::test_uefi_image_is_not_windows. deep_sp_chain appears
nowhere in that log. Attempt 2 resolved c7383da5, which does carry HighMemDxe.efi, and
passed; that branch has since moved again, as below.

The deep_sp_chain prediction did not fire for an unrelated reason: the ten ci / Test
shards run angr at refs/pull/6948/head ff19310a48ec0b9b31b54e911a8e1946fbac5283, and
tests/analyses/decompiler/test_block_simplifier.py does not exist at that revision. It
reaches angr master in ab6b5d6e4570f51537eb526f7b0d5cd1761b521f (angr#7032,
2026-08-29T14:39Z), which ff19310a predates. The test that loads the fixture is simply not
in the tree those shards build.

What that leaves is a standing requirement rather than a fact about today: once the angr tree
these shards resolve does contain test_block_simplifier.py — because angr#6948 is rebased
onto current master, or because it merges and the sync: line goes inert so angr comes from
master — the angr/binaries#196 pin has to contain d4ffa2f0, or that test loads a fixture
that is not there.

Measured at 2026-08-30T13:25Z, it does. angr/binaries#196 is at
ebb7fe0ff74943b68528fbb9f9d6ba5a61d17994, whose only parent is binaries master's tip
45819e52; it is one commit ahead of master and behind by none, d4ffa2f0 is an ancestor of
it, and tests/i386/deep_sp_chain is in its tree. Nothing further is needed there. The
rebase from c7383da5 added files and changed none: both fixtures the figures below were
measured against, tests/x86/windows/eh-frame-occupied-start.exe and the x86-64 PE, are the
same blobs at both revisions.

This paragraph is timestamped because it has been wrong twice, in opposite directions, while
that branch moved under it. The first draft said the pin carried the fixture when it did not,
taken from the base.sha the pulls API reports — a field that records where the base branch
pointer stood at the last synchronisation, not what the head is built on. The second said the
branch had never been rebased, and it was rebased while that draft was being reviewed. A
claim about an object somebody else is moving has to carry the time it was taken, or be
re-derived at publication.

The c6d5b50df8706ffd6bbf78dd0c90ff97becb919c in the bullets above was that pin's tip when
the bullets were measured; it moved to c7383da5 and then to ebb7fe0f. Those bullets, and the
33158448076 hosted-refresh row among them, describe 89ae8c13 and are kept as the
measurement history rather than as statements about the head this record is keyed to.

Re-keyed 2026-08-30 to head 2e72aa81902dd3c9f576e9922cf7af213db5f84f, and this time the
patch itself changed.
FunctionHintSource.EXCEPTION_DIRECTORY moved from 3 to 4, because
CLE 754 adds FUNCTION_STARTS = 3 to the same class at the same line. Union-merging cle
master 929991db's class with each branch shows what the old value cost: with
EXCEPTION_DIRECTORY = 3 the merged class has five names and four distinct values, so
EXCEPTION_DIRECTORY == FUNCTION_STARTS is true and a consumer filtering on either source
also takes the other's hints; with 4 it has five names and five values. Nothing else moved —
git range-diff eac0e554..bb0d2e96 eac0e554..2e72aa81 reports one changed line, and diffing
the two branch patches gives exactly one -/+ pair plus the blob index line.

Measured at 2e72aa81 in the workspace venv (Python 3.12.13, pytest 9.1.1), with this branch
on PYTHONPATH and angr/binaries checked out at refs/pull/196/head c7383da5:

  • python -m pytest tests/test_pe.py — 17 passed
  • python -m pytest tests — 246 passed, 9 skipped
  • pre-commit run --all-files — every hook passed and no file was rewritten
  • merge-base lint and type comparison over the three changed files — no regression

The same two suites at bb0d2e96 in the same environment give the same counts, which is the
expected result: every use of this constant in CLE, in angr master and in the two consumer
pull requests is by name, so no test can observe the value moving. The one literal anywhere
in the set is in angr#6948's tests/analyses/cfg/test_cfgfast.py, which patches
EXCEPTION_DIRECTORY to 3 with create=True to emulate this change against a CLE that
predates it; its hasattr guard means it never binds once this branch lands, but it should
move to 4 so it does not emulate a value this class no longer uses.

Not run locally: the angr suite that ci / Test runs against this CLE, and the macOS,
Windows and Pyodide legs. Live corpus sweeps pin the workspace checkouts, so entering the
workspace shell would reinstall editables underneath them; hosted CI at the new head covers
those and is the result to read.


Re-keyed 2026-09-02 to head 60e49e9f8676a4ce60e59395b20d0fb4c9522b31 against baseline 2f7657fda2a657ec76a201d5c63245c6262f60b7. The previous key was head 2e72aa81902dd3c9f576e9922cf7af213db5f84f on baseline eac0e5540516b9199dd6a91933e80dc774ea3eac; the branch was rebased onto current cle master and the patch did not move. git range-diff eac0e554..2e72aa81 2f7657fd..60e49e9f marks the single commit =, and the diff against the merge base is byte-identical on both sides, so the figures measured at 2e72aa81 describe this head unchanged.

CI at this head is green. Run 33582200386, event pull_request, attempt 1, conclusion success: 18 check runs -- ci / Build, ci / Lint, ci / Typecheck, ci / Decompiler Snapshot Testing (0), ci / Publish Unit Tests Results, the ten ci / Test shards, Test (Pyodide), Test macos-15 and Test windows-2022 -- all success, plus the docs/readthedocs.org:cle and pre-commit.ci - pr statuses. Read 2026-09-02. The standing deep_sp_chain requirement recorded above is met at this head rather than merely dormant: this run's ci / Build resolved angr at refs/pull/6948/head 026f563c5125eb48dfa9c4604fa6150432d8d0c5, which does carry tests/analyses/decompiler/test_block_simplifier.py, and angr/binaries at refs/pull/196/head ebb7fe0ff74943b68528fbb9f9d6ba5a61d17994, which does carry tests/i386/deep_sp_chain. The test the paragraph above says is absent is present, it runs, and the ten shards pass.

Merge order is unchanged: angr/binaries#196, then angr/angr#6948, then this pull request.


Re-keyed 2026-09-05 to head 53b85965a0e971b368b43761faff78f1bd480cb5, on baseline 0e77ade3c39a3cee05f65051e57955675e1ac21b. The branch was CONFLICTING after cle#810 merged.

The conflict was FunctionHintSource: #810 added MACHO_FUNCTION_STARTS = 3 where this branch adds EXCEPTION_DIRECTORY = 4. Both are kept, the five members hold five distinct values, and nothing else conflicted. Between the old baseline 2f7657fd and this one, master touched cle/backends/pe/pe.py exactly once, in b6ff025b (#808), which adds one line registering the Go pclntab; gopclntab.py never touches function_hints, and the 4,219 figure below is unchanged at this head.

The rebase left the patch where it was — 80 added and removed lines on each side, identical — and then one further change was made on top of it, so this is not a bare replay: _get_section_name, which this branch extracts out of _register_sections, had lost the comment master keeps there explaining the /NNN string-table indirection. The comment is restored and the helper gained a docstring. Comments only; ruff check and ruff format --diff are clean.

Two corrections to the record above.

  • The paragraph about angr#6948's test_cfgfast.py says it "patches EXCEPTION_DIRECTORY to 3 with create=True" and "should move to 4". At #6948's head 026f563c there is no such literal; the test uses hasattr/getattr. Nothing there needs to move.
  • The description used to explain the value 4 by pointing at the then-open cle#754. That is no longer why: 3 belongs to MACHO_FUNCTION_STARTS, which cle#810 merged. cle#754 is superseded by Fix Mach-O section metadata and function discovery #810 and is being closed.

Public reproducers, added because the record only ever cited a fixture on an unmerged pull request. Both files are on angr/binaries master:

  • tests/i386/windows/known_patterns_wdk_ksud.exe — 32-bit MinGW PE with a .eh_frame section and no exception directory. len(main_object.function_hints) is 0 on 0e77ade3 and 81 on this head, every one EH_FRAME, all inside .text (0x401000–0x402b84), all distinct, first two (0x401000, 1) and (0x401010, 260). Its CFG does not change — 295 functions under both trees, identical address sets — so it demonstrates the loader defect and no downstream win.
  • tests/x86_64/windows/7995a03…12007afb — 4,219 hints under both trees, labelled EH_FRAME on master and EXCEPTION_DIRECTORY here.

The merge order, measured rather than asserted. With angr master and CFGFast(normalize=False) on that second binary, only the cle tree differing: 4,414 functions on cle master against 6,181 on this head, and 2,272 of the 4,219 hint addresses recovered against 3,715. That is what cfg_base.py:1093/1106 and cfg_fast.py:1750 versus 2394-2397 predict — the relabel moves those records off the occupancy-filtered path onto the unconditional one — and it is why tests/analyses/decompiler/test_structurer.py:351-352, which asserts 0x140032FD3 and 0x140032D1B are not recovered, fails without angr#6948.

python -m pytest tests/test_pe.py at this head: 16 passed, 1 failed. The failure is test_gnu_eh_frame_function_hints, whose fixture is on angr/binaries#196 and not on binaries master; hosted CI resolves it from the sync: line and does not see this.

Hosted CI is re-running at this head; run 33582200386 above describes 60e49e9f and is superseded by whatever this one reports.


Re-keyed 2026-09-30 to head 9de1a5eee379be5900c4036db4722e35b5e12245, on baseline 997d4322678fe5876d9f83e0dd34e1052813b9db. The branch was CONFLICTING after fifteen commits landed on cle master. It was rebased, and then one thing about the patch changed, below.

The rebase. One hunk conflicted, in _register_sections. #835 now passes image_size and file_size when it builds a PESection, where this branch had lifted the section-name resolution out into _get_section_name. Both are kept: the helper supplies the name and #835's two extra arguments go with it. Nothing else conflicted; cle/backends/backend.py and tests/test_pe.py merged on their own. FunctionHintSource.EXCEPTION_DIRECTORY is still 4, because master's class holds 0 through 3.

The PE .eh_frame walk now uses master's own parser, and that is a real change to this branch. #839 landed parse_fde_ranges in cle/backends/elf/eh_frame.py during the rebase window, for the reason its docstring gives: "pyelftools decodes every CFI instruction, which is too slow to run on every loaded ELF object." This branch was walking PE FDEs with exactly that pyelftools path, on every PE that carries an .eh_frame. It now calls parse_fde_ranges, and the five io/elftools imports it needed are gone.

That also closes a hole this branch had. except (DWARFError, ELFParseError, ValueError) did not name what pyelftools actually raises. Flip one byte of tests/i386/windows/known_patterns_wdk_ksud.exe — file offset 0x2a09, 0x7a to 0x01, inside the first CIE's augmentation string — and the old code raised AssertionError: Unhandled augmentation string: b'\x01R' from elftools/dwarf/callframe.py:319 straight out of cle.Loader(), losing the whole load rather than just the hints. With parse_fde_ranges that file loads. Over 800 corrupted copies, 33 exceptions escaped the old clause (16 AssertionError, 15 KeyError, 2 RecursionError) and 0 escape the new one, which declares a single EhFrameParseError.

No real object was affected either way: across 1,030 objects carrying an .eh_frame section in a private corpus, the two walkers return identical FDE sets with zero disagreements, and the new one is about 41 times faster over that set.

Measured at this head, in a nix build of the branch against angr/binaries master merged with angr/binaries#196:

  • tests/i386/windows/known_patterns_wdk_ksud.exe — 81 EH_FRAME hints, 81 distinct addresses, first two (0x401000, 1) and (0x401010, 260), every one inside .text. Identical to what the pyelftools walk gave. The .text bound quoted further up, 0x402b84, reads 0x402c00 at this baseline because PE: Map a section over its raw size when that is larger than its virtual size #835 maps a section over its raw size; the hints are unchanged.
  • tests/x86_64/windows/7995a03…12007afb — 4,219 hints, every one EXCEPTION_DIRECTORY.

pytest tests/ at this head: 284 passed, 9 skipped, 1 failed. The failure is tests/test_macho.py::test_relocatable_object_no_symtab, which #843 added to master. It raises CLEFileNotFoundError on binaries/tests/x86_64/relocatable_object_no_symtab.macho, a fixture that is on angr/binaries#235 and not on binaries master, so it fails the same way on the baseline with this branch absent. Both of this branch's own tests pass, and pre-commit run --all-files is clean.

@angr-bot

Copy link
Copy Markdown
Member

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_789

@zardus zardus closed this Aug 26, 2026
@zardus zardus reopened this Aug 26, 2026
@zardus

zardus commented Aug 26, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Dependency-aware CI is now green on the unchanged head 1ada2051049d8ed1595cab00b528934cf6aaf2f3.

The earlier run's macOS/Windows jobs used the pull_request payload captured before angr/binaries#196 was added to this PR body, so rerunning that historical event continued to select binaries master. Closing and reopening the PR created a fresh event without changing the head. Run 32967835356 resolved the current dependency declarations and passed all 17 jobs, including macOS, Windows, build, lint, typecheck, snapshots, and all ten test shards.

GitHub now reports the PR MERGEABLE / CLEAN. The older failed/cancelled attempt remains in the historical check rollup but is superseded by the fresh all-green run.

@zardus zardus closed this Aug 27, 2026
@zardus zardus reopened this Aug 27, 2026
@ltfish

ltfish commented Aug 28, 2026

Copy link
Copy Markdown
Member

Dear agent, can you please tell me what exact behaviors on which binary you are trying to fix, the intended behaviors of cle and angr, and how the current implementation in cle fails?

@zardus

zardus commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

The reproducer is the stripped 32-bit MinGW PE tests/x86/windows/eh-frame-occupied-start.exe from angr/binaries#196. Its entry at 0x401000 calls a wrapper at 0x401006; that wrapper tail-jumps to the six-byte function at 0x40100a, which returns 42. There are no symbols for that target. The binary's GNU .eh_frame section contains the single FDE that identifies 0x40100a as a function start with size 6.

CLE's intended behavior is to expose that FDE as FunctionHint(0x40100a, 6, EH_FRAME), while exposing native PE exception-directory unwind records as EXCEPTION_DIRECTORY hints. Those sources need separate identities because they have different confidence downstream.

angr's intended behavior is to treat a GNU FDE start as authoritative even when CFGFast has already assigned those bytes to an enclosing function. On the baseline, CFGFast finds the block at 0x40100a but assigns it to the wrapper function at 0x401006; there is no function at 0x40100a. With the hint, reconstruction preserves a separate function at 0x40100a, and the decompiler recovers return 42;. Native exception-directory entries remain lower-confidence hints and do not split already occupied ranges.

Current CLE never parses GNU .eh_frame sections in PE files. Its only PE function hints come from the native exception directory, and it labels those as EH_FRAME. Therefore this binary produces no hint for 0x40100a, and angr cannot distinguish a precise GNU FDE boundary from the much larger native unwind-table population. This PR parses the PE .eh_frame FDE and separates the two sources; the coordinated angr change consumes that distinction.

The focused CLE regression is python -m pytest tests/test_pe.py::TestPEBackend::test_gnu_eh_frame_function_hints tests/test_pe.py::TestPEBackend::test_exception_directory_function_hint_source. The end-to-end CFG/decompiler behavior is covered by the coordinated angr regression.

@zardus
zardus force-pushed the feature/pe-eh-frame-function-boundaries branch from 1ada205 to 89ae8c1 Compare August 28, 2026 09:13
@zardus

zardus commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Full function-hint state for the unnamed target at 0x40100a in tests/x86/windows/eh-frame-occupied-start.exe, before and after this change.

Before — PE GNU FDEs are not parsed and native unwind records use the wrong source identity:

CLE baseline
GNU EH_FRAME hints at 0x40100a: []
native exception-directory hint source: EH_FRAME
angr function at 0x40100a: absent; block owned by wrapper 0x401006

After — the GNU FDE and native records are distinct:

with this change
GNU EH_FRAME hints: [(0x40100a, 6)]
native exception-directory hints: 4219, all EXCEPTION_DIRECTORY
coordinated angr result: separate function at 0x40100a
decompiled body: return 42;

@zardus

zardus commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Rebased onto cle master eac0e554; the branch was CONFLICTING after #800 merged. New head bb0d2e96.

The conflict was the import block of cle/backends/pe/pe.py: #800 added from typing import Any where this branch adds from io import BytesIO. Both are kept; nothing else conflicted.

The change itself did not move: comparing the branch's own diff before and after the rebase with context discarded gives a byte-identical set of added and removed lines.

Validation at the new head: pytest tests/test_pe.py — 17 passed; the whole pytest tests suite — 246 passed, 9 skipped. Both were run against an angr/binaries checkout with this branch's referenced fixture pull request merged into binaries master, since the fixture is not on binaries master yet. The merge-base lint and type comparison that ci / Lint and ci / Typecheck apply to changed files reports no regression at the new base.

I also added the merge order to the description, which stated the dependency only as a sync: line: this must not land ahead of the angr side, because separating the two hint sources without it turns every exception-directory record into a named function hint and breaks an assertion angr already makes.

@zardus
zardus force-pushed the feature/pe-eh-frame-function-boundaries branch 2 times, most recently from bb0d2e9 to 2e72aa8 Compare August 30, 2026 19:02
@zardus

zardus commented Aug 30, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

FunctionHintSource.EXCEPTION_DIRECTORY is now 4 rather than 3. CLE 754 adds
FUNCTION_STARTS = 3 to the same class at the same line and is open alongside this one;
with both at 3, a merge that keeps both lines gives one value two names, the two sources
compare equal, and neither the tests nor the type checker sees it. With 4 the two can land
in either order.

New head 2e72aa81902dd3c9f576e9922cf7af213db5f84f; the rest of the branch is unchanged and
the validation record is re-keyed.

@zardus
zardus force-pushed the feature/pe-eh-frame-function-boundaries branch 2 times, most recently from 60e49e9 to 53b8596 Compare September 5, 2026 21:32
@zardus

zardus commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

@ltfish — I answered this in August at #issuecomment-5448139941, but the answer leaned on a binary that only exists on the unmerged angr/binaries#196, so you could not run any of it. Here it is again on two files that are on angr/binaries master today.

The binary. tests/i386/windows/known_patterns_wdk_ksud.exe — a 32-bit MinGW PE with a .eh_frame section and no exception directory.

What cle does now, and what it should do:

import cle
o = cle.Loader("binaries/tests/i386/windows/known_patterns_wdk_ksud.exe", auto_load_libs=False).main_object
print(len(o.function_hints))
# cle master 0e77ade3: 0
# this branch:         81, every one FunctionHintSource.EH_FRAME

All 81 lie inside .text (0x401000–0x402b84) and all are distinct, the first two being (0x401000, 1) and (0x401010, 260). cle emits none of them today because the PE backend never parses .eh_frame — it is an ELF path only. Worth saying because it is why nobody noticed: the section name is nine characters and a PE section header holds eight, so it is stored as /4 and resolved through the COFF string table. Across the 106 PEs tracked in angr/binaries, a grep of the section-header names finds .eh_frame zero times.

The second half, the relabel. On tests/x86_64/windows/7995a0325b446c462bdb6ae10b692eee2ecadd8e888e9d7729befe4412007afb, also on binaries master, cle master reports 4,219 function hints and labels every one EH_FRAME. They are not .eh_frame records; they are RUNTIME_FUNCTION entries from the native exception directory. This branch labels them EXCEPTION_DIRECTORY and leaves EH_FRAME for actual FDEs.

Why that name matters to angr, rather than being cosmetic. cfg_base.py:1093 routes hints whose source is EH_FRAME into _function_addresses_from_eh_frame; line 1106 routes every other source into _function_addr_and_names_from_hints. Those two go to different places in CFGFast: the second becomes an unconditional initial starting point at cfg_fast.py:1750, while the first is drained last and skipped when the address is already occupied (cfg_fast.py:2394-2397). So today's label puts native unwind starts behind the occupancy filter and gives a real GNU FDE nowhere to be.

Measured on that x86-64 PE, angr master, CFGFast(normalize=False), only the cle tree differing:

cle master this branch
functions recovered 4,414 6,181
of the 4,219 hint addresses, recovered as functions 2,272 3,715

1,824 functions added, 57 removed. That is a large behaviour change and it is the reason the description says this must not land ahead of angr#6948 — without it, the two addresses angr master asserts are not recovered, 0x140032fd3 and 0x140032d1b, both become unconditional seeds and the existing assertion fails. If you would rather the relabel not carry that consequence, that is the thing to push back on, and I would rather hear it now than after it lands.

Where the unmerged fixture still earns its place. Neither public binary shows the case the change was originally written for: an FDE whose start CFGFast has already given to an enclosing function. On known_patterns_wdk_ksud.exe the CFG does not change at all — 295 functions under both trees, identical address sets — because all 81 FDE addresses are already recovered from its symbol table. So it proves the loader defect and not a downstream win, and I am not claiming one for it. eh-frame-occupied-start.exe on angr/binaries#196 is the one that exercises the occupied-boundary path.

@zardus
zardus force-pushed the feature/pe-eh-frame-function-boundaries branch from 53b8596 to 9de1a5e Compare September 30, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants