Repository navigation
Conversation
MetaELF.__init__ computes the RELRO level before ELF.__init__ gets to run, and
pyelftools reads the section header table to answer it: iter_segments() builds a
DynamicSegment for PT_DYNAMIC and that constructor walks iter_sections(), while
get_section_by_name(".dynamic") builds a name map over the whole table. On a file
whose e_shoff points past the end, pyelftools' ELFParseError escapes
Loader.__init__ and the whole load is lost -- including when
discard_section_headers is set, which is documented for section headers that are
corrupt or malicious.
ELF.__init__ already recovers from a section header table it cannot walk, by
reloading the file with the section header fields zeroed so that pyelftools reads
it from the program headers alone. Guarding the RELRO probe lets that recovery
happen. RELRO is a property CLE reports rather than one the load needs, and the
level is reported as none because the parse can fail inside the PT_GNU_RELRO test
itself, which leaves "no RELRO" and "cannot tell" indistinguishable; only
Relro.FULL changes any behaviour, and FULL is what needs the dynamic table.
tests/test_truncated_section_headers.py loads
tests/armel/fauxware.truncated.elf, which is fauxware cut at the end of its last
PT_LOAD.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Full load result for Before — pyelftools' cle master 997d432After — with this change |
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Validation record for head The two arms are store builds of the same worktree through the same nixpkgs pin -- baseline Dataset evidence. The affected sample is a VirusShare object that cannot be shared: SHA-256
How rare the shape is in the corpus, measured rather than assumed. 13 catalog rows across every dataset on this machine declare Public reproducer. The fixture raises through the other of the two section reads, which is worth saying because it decides the shape of the fix. Its traceback on the baseline is Regression, tests. Regression, public corpus. Every path Regression, private corpus. All 78 objects of the VirusShare shard the cited sample belongs to, same command and same row format, each with its own declared recipe. Exactly one row changed -- the cited sample, Workspace gate. One run of the full local gate over this branch and its sibling checkouts, at this head, in the head environment above, with
Lint and type, against the merge base, which the workspace gate never runs. This is where an earlier head of this branch failed, and it is worth saying because the local gate cannot see it: writing the guard as Conflicts. CI prediction, written before the first check ran. This pull request will be red on four checks, and none of them is this change. Expect The reason is a merge order this repository cannot express. Caveats, one line each:
|
|
Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_856 |
2 participants
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS
Problem
An ELF whose
e_shoffpoints past the end of the file loses its whole load, with pyelftools' exception escapingLoader.__init__. On the newtests/armel/fauxware.truncated.elf:main_opts={"discard_section_headers": True}raises exactly the same thing, even though that option is documented onELFas "Do not parse section headers. Use this if they are corrupted or malicious."Root cause
MetaELF.__init__runs beforeELF.__init__'s own body. It builds a plainELFFileand asks_get_relrofor the RELRO level, and two of that function's lines read the section header table through pyelftools:elf.iter_segments()builds aDynamicSegmentforPT_DYNAMIC, andDynamicSegment.__init__callselffile.iter_sections()looking for theDynamicSectionat the same file offset;elf.get_section_by_name(".dynamic")builds the_section_name_mapcached property, which enumerates every section.ELFFile._get_section_headerraises as soon as a section header starts paststream_len, so either line is fatal on a truncated file. Which one fires depends on the file: one with aPT_DYNAMICsegment dies on the first, one without it on the second.ELF.__init__is already written for this file. Immediately aftersuper().__init__()it probes the table itself, and when the walk raises it installs aPatchedStreamthat zeroese_shoff,e_shentsize,e_shnumande_shstrndxso pyelftools rereads the file from the program headers alone. Neither that nordiscard_section_headersis reachable, because the RELRO probe got there first.2a8adaa8built that fallback in 2018 and fixed this exact bug inextract_sonamein the same commit, by reading the program headers instead -- that is #113, where the diagnosis was that this is "supposed to be a pre-analysis stage that does extremely minor introspection".4a7e4f7aput a section read back into the same stage in 2021 (#278), this time for RELRO.Fix
Guard the RELRO probe, so a section header table pyelftools cannot parse reaches the recovery written for it instead of ending the load.
The level is reported as
Relro.NONE. The parse can fail inside thePT_GNU_RELROtest itself, so "no RELRO" and "cannot tell" are not distinguishable here, and nothing reads the difference: the only consumer of the attribute anywhere in CLE or angr isMetaELF._block_references_addr, which compares againstRelro.FULL, andFULLis exactly the answer that needs the dynamic table we could not reach.The guard is at the call site rather than inside
_get_relro, on purpose. #113's own conclusion -- "Once we get out of here and into the main loading phase we should opt out of using sections" -- is the right long-term shape, and the open #815 is already moving_get_relro's first line onto the program headers. But switching to segments does not settle this, because the segment route raises too: that is what the open #731 is about,iter_tags()on aPT_DYNAMICsegment with no string table. The invariant worth holding is that this probe cannot be fatal whatever it reads, and that belongs where the probe is called. #815 and #731 narrow when the guard fires; neither is needed for it.Testing
tests/test_truncated_section_headers.pyloads the new fixture, which istests/armel/fauxwarecut at0xf04 + 0x140, the end of its lastPT_LOAD's file bytes: every byte the program headers call loadable is present, and the section header table, at0x11b0, is gone. Both tests fail on the merge base with theELFParseErrorabove and pass with the change, and the untruncatedtests/armel/fauxwareloads identically on both sides, with its 30 sections andRelro.PARTIAL.The defect was found on a VirusShare sample that cannot be shared, SHA-256
7cb8df1ec5e810bf42b3cd9fca3c28a2117f696cd800dc8620f34fdf622448bd, an ARMET_EXECwithe_shoff1293496 in a 1010694-byte file and aload-errorin a corpus sweep with the same exception. On the base it loads nothing andangr.Projectraises out of its constructor; with this change it loads from its program headers andCFGFastrecovers 3079 functions over 14207 nodes. The regression set is the 947 other objects measured the same way on both sides -- every\x7fELFpathangr/binariestracks undertests/, the other 77 objects of that sample's shard, and the other 12 objects in any corpus here whose header declares the samee_shoff-past-the-end malformation -- and not one of their rows changed. Every per-set count is in the validation record.Commands, from the
cleworktree withangr/binariesbeside it andOBJECTthe unavailable sample:The fixture is new because nothing already tracked has the shape: of the 858 paths under
tests/whose first four bytes are\x7fELFat9eb02bd1032e1d96e729df2739f9a10a337bd154, zero have a section header whose offset falls past the end of the file. Merge angr/binaries#246 first. Master is separately red ontests/test_macho.py::test_relocatable_object_no_symtab, whose fixture sits in another openangr/binariespull request; the validation record names it, the four checks that costs, and why only one such reference can be resolved per build. Validation: #856 (comment)sync: angr/binaries#246
🤖 Generated with Claude Code
session: sharpen