Stop loading a COFF symbol table that is not numbered for the image by zardus · Pull Request #832 · angr/cle

zardus · 2026-09-08T05:15:15Z

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

cle.Loader raises IndexError on a PE whose retained COFF symbol table names a
section the image does not have.

  File "cle/backends/pe/pe.py", line 166, in __init__
    coff_symbol_types = self._load_symbols_from_coff_header()
  File "cle/backends/pe/pe.py", line 1235, in _load_symbols_from_coff_header
    rva = self._pe.sections[section - 1].VirtualAddress + value
IndexError: list index out of range

The two images it was measured on are 32-bit Machine=0x14c PEs with four
sections whose symbol table names sections 5 and 6. Both come from a
corpus whose redistribution is prohibited, so there is no committed binary; the
regression is a unit test and Testing says what else was measured.

Root cause

The section number in a symbol record is read out of the file as a signed short
and used as an index:

if section > 0 and type_ in type_to_symbol_type and VALID_SYMBOL_NAME_RE.fullmatch(name):
    rva = self._pe.sections[section - 1].VirtualAddress + value

section > 0 rejects the reserved values -- IMAGE_SYM_UNDEFINED is 0,
ABSOLUTE is -1, DEBUG is -2 -- and nothing checks the other end.

Bounding the index and skipping that record stops the crash and leaves a worse
problem behind. On both of these images the table is not numbered for the image
at all:

It is not one stray record. 206 of image A's 379 records name a missing
section, and 208 of image B's 415.
The records that survive are mis-attributed. In both, _mainCRTStartup
carries Value = 0x11cb, which is AddressOfEntryPoint exactly; adding
.text's 0x1000 puts the entry symbol at 0x21cb.

So skipping only the out-of-range records trades one loud IndexError for a
hundred quiet symbols: 104 and 108 of them, of which 14 and 13 fall outside
[min_addr, max_addr].

That is two images, not a law about the format. What makes the disposition safe
regardless is below.

Fix

Treat it the way this function already treats a symbol table that runs past the
end of the file, a few lines above -- warn, and load no symbols from it:

WARNING | cle.backends.pe.pe | PE symbol table names section 5 of 4; not loading symbols from it

Nothing that loads today can lose a symbol by this. The new branch needs
section > 0 and section > len(sections), which is exactly the state that
raises IndexError on master. An image with one stray out-of-range record in an
otherwise sound table would be a reason to prefer a bare bounds check -- but such
an image does not load today either, so there is no input whose symbols get
worse.

The two images load with 38 and 36 symbols, all inside the image, from their
import tables. Symbols are collected and added after the walk so that a table
condemned part-way through leaves none behind, and _handle_exports gets the
same empty mapping the existing guard already hands it.

Testing

tests/test_pe_coff_symbols.py::test_coff_symbols_are_dropped_when_a_section_number_is_out_of_range
drives _load_symbols_from_coff_header with three records, the middle one naming
a section the fixture does not have, and asserts both that the mapping is empty
and that the symbol built before it was not kept. It fails on master with the same IndexError out of pe.py:1235 and passes
here; the file's two existing tests pass on both.
It adds no binary: it packs symbol records against the same fake _make_pe
already in that file.

There is no fixture because there is no object to make one from.
angr/binaries at fc07821c tracks 106 PEs, 23 with a COFF symbol table and 22
with it in bounds. Not one of the 22 has a single record, whether or not it would
reach the indexing, that names a section past the section count. Ten of them name
their own last section and the other twelve stop short of it, the widest being a
16-section image whose table never names a section above 3.

Truncating a tracked fixture cannot produce one. Cutting a file's tail cannot
raise the section number written in a symbol record, and over the 23, truncated
at 1,582 lengths between them, it never lowered the section count either: every
truncation still entered _load_symbols_from_coff_header, and of the 129 that
got past the file-extent guard and walked 556,365 records between them, not one
saw a section count different from the untruncated file's.

A wider survey of 183,459 distinct PE objects finds 28,911 carrying a retained,
in-bounds COFF symbol table -- 1,411 Wine, 561 MSYS2, 30 Windows release
binaries, 12 decbench, 9 from the copy of angr/binaries this corpus carries at 718e154d, 8 from a
real-language matrix, 26,880 generated -- and 0 whose records reaching the
indexing name a section past the count. Most of that corpus is not public, so this is
evidence rather than a survey you can re-run.

Nothing else moves. Every MZ file under angr/binaries loaded on master
0e77ade3 and on this branch, comparing section list, entry point, is_dotnet
and the full sorted symbol list of (name, RVA, type, is_export):

106 PE objects, 41,106 symbols, 0 differing records

The same harness unchanged over the two failing images reports 2 of 2 records
differing, so an empty diff is a result and not a broken instrument.

Validation: #832 (comment)

🤖 Generated with Claude Code

session: sharpen

PE._load_symbols_from_coff_header walks the COFF symbol table some toolchains still leave in a linked image and turns each symbol into an RVA by adding its value to the start of the section its record names. The section number came out of the file and nothing checked it against the number of sections the image has, so a symbol naming a section that is not there raised IndexError and the load failed. Bounding the index and skipping that one record is not enough. On both images this was measured on, a table that names a missing section does so wholesale and its remaining numbers do not describe this image either: over half of all records name a missing section, the values are already image RVAs rather than section offsets, and adding a section address to them puts the entry-point symbol 0x1000 past the entry point. Skipping only the out-of-range records loads a hundred symbols of which a dozen fall outside the image. So treat it the way this function already treats a symbol table that runs past the end of the file: warn, load no symbols from it, and leave the image's imports to speak for it. Symbols are collected and added after the walk so a table condemned part-way through leaves none behind. Nothing that loads today reaches the new branch, because reaching it is exactly the state that raises IndexError. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

zardus · 2026-09-08T05:15:24Z

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

The two images, on master, on a bare bounds check, and on this branch

Both objects come from a corpus whose redistribution is prohibited, so the probe
reports what each file's header declares rather than naming it. The three arms
are cle at 0e77ade3, the same branch with the smaller fix -- bound the index,
skip the record -- and this branch. They differ only in
cle/backends/pe/pe.py.

Before

cle at 0e77ade

cle: master 0e77ade3

image A: Machine=0x14c sections=4 records=379 entry RVA=0x11cb
  206 records name a section past the section count; 86 records pass the load guard, 20 of those name a missing section
  12 distinct names, all section 5, storage classes [2, 3] (EXTERNAL, STATIC): $hadvapi, $hcrtdll, $hkernel, $huser32, $hwsock3, $tadvapi, $tcrtdll, $tkernel, $tuser32, $twsock3, fthunk, hname
    File "cle/backends/pe/pe.py", line 166, in __init__
      coff_symbol_types = self._load_symbols_from_coff_header()
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "cle/backends/pe/pe.py", line 1235, in _load_symbols_from_coff_header
      rva = self._pe.sections[section - 1].VirtualAddress + value
            ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  IndexError: list index out of range

image B: Machine=0x14c sections=4 records=415 entry RVA=0x11cb
  208 records name a section past the section count; 96 records pass the load guard, 24 of those name a missing section
  14 distinct names, all section 5, storage classes [2, 3] (EXTERNAL, STATIC): $hadvapi, $hcrtdll, $hkernel, $huser32, $hwinine, $hwsock3, $tadvapi, $tcrtdll, $tkernel, $tuser32, $twinine, $twsock3, fthunk, hname
    File "cle/backends/pe/pe.py", line 166, in __init__
      coff_symbol_types = self._load_symbols_from_coff_header()
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "cle/backends/pe/pe.py", line 1235, in _load_symbols_from_coff_header
      rva = self._pe.sections[section - 1].VirtualAddress + value
            ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  IndexError: list index out of range

The bound-and-skip variant, for comparison

the smaller fix: bound the index, skip the record

cle: the bound-and-skip variant

image A: Machine=0x14c sections=4 records=379 entry RVA=0x11cb
  206 records name a section past the section count; 86 records pass the load guard, 20 of those name a missing section
  12 distinct names, all section 5, storage classes [2, 3] (EXTERNAL, STATIC): $hadvapi, $hcrtdll, $hkernel, $huser32, $hwsock3, $tadvapi, $tcrtdll, $tkernel, $tuser32, $twsock3, fthunk, hname
  cle.Loader(...) -> 4 sections, 104 symbols, 14 outside [min_addr, max_addr], entry 0x4011cb

image B: Machine=0x14c sections=4 records=415 entry RVA=0x11cb
  208 records name a section past the section count; 96 records pass the load guard, 24 of those name a missing section
  14 distinct names, all section 5, storage classes [2, 3] (EXTERNAL, STATIC): $hadvapi, $hcrtdll, $hkernel, $huser32, $hwinine, $hwsock3, $tadvapi, $tcrtdll, $tkernel, $tuser32, $twinine, $twsock3, fthunk, hname
  cle.Loader(...) -> 4 sections, 108 symbols, 13 outside [min_addr, max_addr], entry 0x4011cb

PE symbol table names section 5 of 4; not loading symbols from it
PE symbol table names section 5 of 4; not loading symbols from it

After

this branch

cle: this branch

image A: Machine=0x14c sections=4 records=379 entry RVA=0x11cb
  206 records name a section past the section count; 86 records pass the load guard, 20 of those name a missing section
  12 distinct names, all section 5, storage classes [2, 3] (EXTERNAL, STATIC): $hadvapi, $hcrtdll, $hkernel, $huser32, $hwsock3, $tadvapi, $tcrtdll, $tkernel, $tuser32, $twsock3, fthunk, hname
  cle.Loader(...) -> 4 sections, 38 symbols, 0 outside [min_addr, max_addr], entry 0x4011cb

image B: Machine=0x14c sections=4 records=415 entry RVA=0x11cb
  208 records name a section past the section count; 96 records pass the load guard, 24 of those name a missing section
  14 distinct names, all section 5, storage classes [2, 3] (EXTERNAL, STATIC): $hadvapi, $hcrtdll, $hkernel, $huser32, $hwinine, $hwsock3, $tadvapi, $tcrtdll, $tkernel, $tuser32, $twinine, $twsock3, fthunk, hname
  cle.Loader(...) -> 4 sections, 36 symbols, 0 outside [min_addr, max_addr], entry 0x4011cb

The two PE symbol table names section 5 of 4 lines at the foot of the
bound-and-skip block are stderr from the arm after it: they are what this branch
prints on both images, and the capture interleaved them ahead of its stdout.

hname and fthunk are dlltool's .idata$4 / .idata$5 markers and appear in
10 of the 22 tracked PEs whose COFF symbol table is in bounds. The $t<dll> and
$h<dll> labels beside them appear in none of those 22, which is the point:
these tables are unlike every symbol table angr/binaries holds.

The middle arm is why a bounds check is not the fix. It loads, and produces 104
and 108 symbols of which 14 and 13 sit outside the image, because the values in
these tables are already RVAs and the code adds a section address on top. This
branch produces 38 and 36, all inside the image, from the import tables.

Nothing else moves

Every MZ file under angr/binaries was loaded on master 0e77ade3 and on this
branch. The record per object is the section list, the entry point, is_dotnet,
and the full sorted symbol list of (name, RVA, type, is_export). The second
line is the same harness, unchanged, over the two failing images -- so an empty
diff is a result rather than a broken instrument:

angr/binaries: 106 PE objects, 41106 symbols before / 41106 after, 0 differing records
calibration: 2 PE objects, 0 symbols before / 74 after, 2 differing records
    image A | IndexError: list index out of range -> ok nsym=38
    image B | IndexError: list index out of range -> ok nsym=36

Why the regression is a unit test and not a fixture

Of angr/binaries' 106 PEs at fc07821c, 23 carry a COFF symbol table and 22
have it in bounds. Not one of the 22 has a single record, whether or not it would
reach the indexing, that names a section past the section count. Ten name their
own last section; the other twelve stop short of it, the widest being a
16-section image whose table never names a section above 3. The 23rd is
tests/x86/windows/packed_pe32.exe, whose table runs past the end of the file --
the case the existing guard already covers.

Truncating a tracked fixture cannot produce one, for a reason that does not
depend on the corpus: cutting a file's tail cannot raise the section number
written in a symbol record. The 23 were truncated at 1,582 lengths between them,
about 69 apiece, and it did not lower the section count either:

targets: 23   truncation lengths tried: 1582
entered _load_symbols_from_coff_header: 1582
past the file-extent guard: 129   records walked: 556365
of those 129, truncations with a section count unlike the untruncated file: 0
out-of-range section numbers produced: 0

A wider survey finds none either: 183,459 distinct PE objects, 28,911 of them
carrying a retained, in-bounds COFF symbol table, and 0 in which a record
reaching the indexing names a section past the count. The survey counts records
that pass the same three-way guard the loader applies, which is the population
that can crash it. Of the collections below only decbench is a published
dataset; the rest are corpora we hold, so this is evidence rather than something
you can re-run.

corpus	distinct	in-bounds COFF symtab
decbench	12	12
Rust/Go cross-compilation matrix	5,987	0
mixed real-world Windows	5,733	2,019
generated loader/UEFI matrix	61,135	19,968
generated Windows matrix	110,592	6,912
total	183,459	28,911

The 2,031 non-generated ones are 1,411 Wine PEs, 561 from MSYS2, 30 Windows
release binaries, 12 decbench, 9 from angr/binaries and 8 from a real-language
matrix. The 9 is not the 22 above: this corpus carries its own copy of
angr/binaries at 718e154d and counts by content hash, so the two figures are
of different trees. One further
object -- packed_pe32.exe again -- carries a symbol table running past the end
of the file, the case the existing guard already covers, counted apart from the
28,911.

zardus · 2026-09-08T05:15:25Z

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 286bd1521b5b943f7dce726b636835a45cb11d65, one commit on
0e77ade3c39a3cee05f65051e57955675e1ac21b. Working tree clean, tree
a6e7b8d0a860ba359506ec6fd3a3780e8506048f. File sha256:
cle/backends/pe/pe.py b6f6d5fecb89277d179ecd436b7e32950f7b28a80b6160a9623262aae5281f83,
tests/test_pe_coff_symbols.py f960eedcc5d360eec013e3652bad9681258f743a8874cf423a61fffaff22f65d.

Local gate

All fourteen suites ran; none was skipped. Twelve pass. The two that fail are the
machine, not this change, and both are named below.

suite	result
workspace checks	58 OK, 177 OK (3 skipped), 663 with 1 failure
test inputs in angr/binaries	pass, 6 checkouts
tests import from a package	pass, 7 checkouts
pre-commit hooks	pass, every configured hook in six checkouts
feature builds, tests and tears down	fail, on its nested worktree-cleanliness check
mono pipeline	139 tests, OK, 2 skipped
archinfo	39 passed, 34 subtests
pypcode	46 passed, 187 subtests
pyvex	67 passed
pysoot	pass, as the pysoot-check derivation
cle	262 passed, 9 skipped
angr (Python)	2,917 passed, 47 skipped, 2 xfailed, 286 subtests, 39m24s
angr (Rust)	pass, as the rustylib-check derivation
angr-management (GUI)	710 passed

The two failures. Neither touches cle, and both were checked rather than
assumed.

workspace fails one assertion of 663: test_sweepctl_stop.py:136 requires a
worker that ignores SIGTERM to be killed within 45 seconds and measured 70.25.
The gate ran beside several others and a 31-worker corpus sweep, at a load
average of 50 on 20 cores. It is a timing bound against a loaded box.

feature-build builds a throwaway feature, runs a nested gate in it and checks
the workspace is unchanged afterwards. Its nested cle suite passed (261 passed,
9 skipped); what it caught was other agents' live edits to files under
.agents/skills/ while it ran. A dozen agents share this checkout.

Neither failure involves cle/backends/pe/ or the test this change adds. cle's
own suite is 262 passed, 9 skipped, and angr's is 2,917 passed with 47 skipped
and 2 xfailed, both against the branch build.

The gate builds every package from the worktrees and runs the tests against the
store build, so the cle under test is this branch's:
features/pe-coff-secnum/repos/cle 286bd1521 feature/pe-coff-secnum clean.

The hosted Lint and Typecheck jobs, predicted locally

CI scores each changed file with pylint and counts pyright errors in it, and fails
the file if either gets worse than the merge base. Run against 0e77ade3c:

Lint       cle/backends/pe/pe.py:          10.00 -> 10.00
           tests/test_pe_coff_symbols.py:  10.00 -> 10.00
Typecheck  cle/backends/pe/pe.py:          errors 52 -> 52
           tests/test_pe_coff_symbols.py:  errors 0 -> 0

The hosted jobs that collect the new test are angr-ci (which cle's ci.yml
delegates to angr/ci-settings/.github/workflows/angr-ci.yml@master), Test windows-2022 and Test macos-15, which run pytest -n auto, and Test (Pyodide), which runs pytest tests. It is fixture-free, so all of them
collect it.

The regression fails on master for the reason claimed

Same test file, two store builds, the branch's and master's:

branch  3 passed
master  1 failed, 2 passed
        test_coff_symbols_are_dropped_when_a_section_number_is_out_of_range
        IndexError: list index out of range  at cle/backends/pe/pe.py:1235

Truncating a tracked fixture cannot produce this shape

Every tracked PE with a COFF symbol table, truncated at 1,582 lengths between them,
loaded through cle.Loader:

targets: 23   truncation lengths tried: 1582   loader accepted: 1582
entered _load_symbols_from_coff_header: 1582
past the file-extent guard: 129   records walked: 556365
of those 129, truncations with a section count unlike the untruncated file: 0
margin (highest section number named - section count): {-13: 4, -12: 4, -11: 8, -10: 32, 0: 81}
out-of-range section numbers produced: 0

The margin is never positive, and the reason is structural rather than a property of
this corpus: cutting a file's tail cannot raise the section number written in a
symbol record.

Nothing already tracked changes

Every MZ file under angr/binaries at fc07821c, loaded on master 0e77ade3 and on
this branch, comparing section list, entry point, is_dotnet and the full sorted
symbol list of (name, RVA, type, is_export):

angr/binaries: 106 PE objects, 41,106 symbols before / 41,106 after, 0 differing records
calibration:   2 PE objects, 0 symbols before / 74 after, 2 differing records

The second line is the same harness, unchanged, over the two objects that fail today,
so an empty diff is a result rather than a broken instrument.

angr-bot · 2026-09-08T05:41:49Z

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_832

zardus mentioned this pull request Sep 25, 2026

Load the PE COFF symbol tests from fixtures instead of packing them #846

Closed

Navigation Menu

Stop loading a COFF symbol table that is not numbered for the image - #832

Stop loading a COFF symbol table that is not numbered for the image#832
zardus wants to merge 1 commit into
masterfrom
feature/pe-coff-secnum

zardus commented Sep 8, 2026 •

edited

Loading

Uh oh!

zardus commented Sep 8, 2026 •

edited

Loading

Uh oh!

zardus commented Sep 8, 2026

Uh oh!

angr-bot commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

zardus commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Root cause

Fix

Testing

Uh oh!

zardus commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The two images, on master, on a bare bounds check, and on this branch

Nothing else moves

Why the regression is a unit test and not a fixture

Uh oh!

zardus commented Sep 8, 2026

Local gate

The hosted Lint and Typecheck jobs, predicted locally

The regression fails on master for the reason claimed

Truncating a tracked fixture cannot produce this shape

Nothing already tracked changes

Uh oh!

angr-bot commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

zardus commented Sep 8, 2026 •

edited

Loading

zardus commented Sep 8, 2026 •

edited

Loading