Fix ClemoryView.backers clamping a backer to the wrong slice

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

ClemoryView.backers() hands out the wrong slice, at the wrong address,
whenever a parent backer runs past either end of the view's window. On
binaries/tests/x86_64/fauxware, loaded with auto_load_libs=False:

>>> [(hex(s), len(b)) for s, b in memory.backers()]
[('0x400000', 2676), ('0x600e28', 568), ('0x700000', 49)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400000, 0x400100).backers()]
[('0x0', 2420)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400010, 0x400100).backers()]
[('-0x10', 2404)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x600e30, 0x600e40, offset=0x10).backers()]
[('0x8', 536)]

A view covering 256 addresses reports 2420 bytes of them. A view whose window
starts 0x10 into a parent backer reports that backer at a negative address. A
view with an offset of its own reports a 16-address window as 536 bytes at
0x8, which is below the view's own _offset, so view.load(0x8, 536) raises
KeyError on the very backer the view just handed out.

Root cause

Both halves of the clamp are wrong.

if taddr + len(backer) - 1 >= self._endoffset:
    clamp_end = len(backer) - self._endoffset + taddr

self._endoffset - taddr is the offset inside the backer where the window ends,
and that is what clamp_end should be. The expression is len(backer) minus
that. The two coincide only when len(backer) is exactly twice
self._endoffset - taddr, which is a coincidence and not a case. The else arm
below it, taken when the backer does not overrun the window, is already right.

yield taddr, view[clamp_start:clamp_end]

taddr is where the parent backer starts in the view's address space. The slice
starts clamp_start bytes later than that, so the address is short by exactly
the amount the low end was clamped by. Where taddr is itself negative, because
the parent backer begins below self._rebase, so is the address the view
reports.

Fix

Take the intersection of the backer and the window in the backer's own offsets,
and yield the address the slice really begins at.

>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400000, 0x400100).backers()]
[('0x0', 256)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400010, 0x400100).backers()]
[('0x0', 240)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x600e30, 0x600e40, offset=0x10).backers()]
[('0x10', 16)]

The two branches above it are untouched, so a backer that lies wholly inside the
window is still yielded as itself rather than as a memoryview, and one wholly
outside is still skipped.

Testing

test_clemory_view_backers_are_clamped_to_the_window loads fauxware and asserts
the address and the length of what two differently clamped windows hand out
against the window's own bounds, and their bytes against what the parent
Clemory returns for the same range. It fails on master at the first assertion,
with (0, 2420) where (0, 256) is expected.

Validation: #823 (comment)

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head fc290898c632df4c96756f0614901506ec8376f1 against baseline 0e77ade3c39a3cee05f65051e57955675e1ac21b.

• Regression: pytest tests/test_clemory.py::test_clemory_view_backers_are_clamped_to_the_window — fails on baseline at assert [(0, 2420)] == [(0, 256)], passes on head
• Assertion audit: each of the four assertions was made to fail by a deliberate mutation of the head — dropping the high clamp gives (0, 2676), dropping the low clamp gives (4080, 256), yielding taddr instead of the slice's own address gives (4080, 240), and slicing from 0 or from clamp_start + 1 leaves the addresses and lengths right and the bytes wrong
• Full suite: pytest tests — head 262 passed / 9 skipped / 0 failed; baseline 261 passed / 9 skipped / 0 failed
• Lint/type: run-ci-diff-checks.py, which reproduces angr/ci-settings lint.py and typecheck.py — pylint cle/memory.py 10.00 -> 10.00, tests/test_clemory.py 5.12 -> 5.70; pyright errors cle/memory.py 0 -> 0, tests/test_clemory.py 9 -> 9; exit 0
• Hooks: pre-commit run --files cle/memory.py tests/test_clemory.py — exit 0, no file rewritten
• Push guards: check-test-inputs.py and check-stale-pins.py at the head — both exit 0

Caller survey:

• nothing constructs a ClemoryView. grep -rn 'ClemoryView(' over cle at the baseline returns only the class statement; git grep -n 'ClemoryView' over every file at angr master 87411a719c96ddc3f3954590b1833b1789e19697 and at angr-management master aa843e5c10645ba361620768d59642a460805e80 exits 1 with no output
• so the effect of this change is on cle's public API and on nothing in the three repositories today. That is the same footing as Fix ClemoryView.__setitem__ dropping the write #816, Return False from ClemoryView.__contains__ instead of raising #819 and Fix ClemoryView.find translating its bounds twice #820, which fix three other methods of this class

Merge state, measured on the committed head against every open pull request that touches cle/memory.py — 718, 721, 809, 816, 818, 819, 820, 821 — and against the two sibling candidates published beside this one: cle/memory.py auto-merges against all ten, including 819, which changes the method directly above this one, and 820, which changes find in the same class. tests/test_clemory.py conflicts against 721, 818 and both siblings. Positive control: merge-tree on 819 against 820 exits 1.

Caveats: the workspace-wide gate was not run — this machine's native libraries are pinned by running corpus sweeps and rebuilding them is forbidden — so validation is the cle suite, the two CI diff checks and the survey above. No corpus A/B: with no caller in any of the three repositories, a sweep has nothing to move.

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

What ClemoryView.backers() hands out for three windows over
binaries/tests/x86_64/fauxware, loaded with auto_load_libs=False, before and
after this change. The parent's own backers are
[('0x400000', 2676), ('0x600e28', 568), ('0x700000', 49)], so each of these
windows lies inside one parent backer and has to be clamped out of it.

Before — every window reports a length that is not its own, and two report
an address the view cannot load from:

angr/cle master 0e77ade

>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400000, 0x400100).backers()]
[('0x0', 2420)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400010, 0x400100).backers()]
[('-0x10', 2404)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x600e30, 0x600e40, offset=0x10).backers()]
[('0x8', 536)]
>>> view = cle.ClemoryView(memory, 0x600e30, 0x600e40, offset=0x10)
>>> [view.load(s, len(b)) for s, b in view.backers()]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "cle/memory.py", line 589, in load
    raise KeyError(addr)
KeyError: 8

After — each window reports its own length, at an address inside itself, and
the bytes read back through the view:

with this change

>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400000, 0x400100).backers()]
[('0x0', 256)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x400010, 0x400100).backers()]
[('0x0', 240)]
>>> [(hex(s), len(b)) for s, b in cle.ClemoryView(memory, 0x600e30, 0x600e40, offset=0x10).backers()]
[('0x10', 16)]
>>> view = cle.ClemoryView(memory, 0x600e30, 0x600e40, offset=0x10)
>>> [bytes(b) == view.load(s, len(b)) for s, b in view.backers()]
[True]

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_823