read_state_raw TypeError: certificate field decoded as str instead of bytes

[HavenCTO]

read_state_raw assumes decoded_obj["certificate"] is CBOR-encoded bytes, but some gateways return it as a hex str. This causes cbor2.loads(str) to fail with TypeError: string indices must be integers, not 'str'.

ic-py/ic/agent.py

Line 239 in 0e58fd1

def read_state_raw(self, canister_id, paths):

and

ic-py/ic/agent.py

Line 262 in 0e58fd1

async def read_state_raw_async(self, canister_id, paths):

Fix: Normalize the certificate value before the inner cbor2.loads:

cert_raw = decoded_obj["certificate"]
if isinstance(cert_raw, str):
    cert_raw = bytes.fromhex(cert_raw)
cert_dict = cbor2.loads(cert_raw)

[HavenCTO]

The fix proposed in the initial issue only covers one variant. There are actually two:

1. Variant A: The entire response is a CBOR-encoded hex string. cbor2.loads(ret) returns a str, so d["certificate"] throws TypeError: string indices must be integers, not 'str'.
2. Variant B: The response is a CBOR map, but d["certificate"] is a hex str instead of bytes. This is what the initial issue's fix addresses.

Also, cbor2 can return frozendict instead of dict, so isinstance(d, dict) checks will fail.

Proposed fix:
Check for str first (Variant A), then treat everything else as a mapping (Variant B). This handles both variants and avoids the frozendict issue.

d = cbor2.loads(ret)
if isinstance(d, str):
    cert_raw = bytes.fromhex(d)
else:
    cert_raw = d["certificate"]
    if isinstance(cert_raw, str):
        cert_raw = bytes.fromhex(cert_raw)
return cbor2.loads(cert_raw)

[HavenCTO]

After a monkey patch I'm getting...

Error: CBORDecodeEOF

cbor2.CBORDecodeEOF: premature end of stream (expected to read at least 14 bytes, got 0 instead)

This indicates the gateway returned an empty or incomplete certificate (b''), not a string. This typically happens when the gateway times out or drops the connection while reading the subnet state.

Relevant Stack Trace Flow:

encrypt_step.py calls _encrypt_with_haven_aol
haven_aol_local.py calls _ibe_encrypt_aes_key -> get_vetkd_public_key_b64
haven_aol_icp.py calls canister.getVetKDPublicKey()
ic/agent.py calls update_raw -> poll -> request_status_raw -> read_state_raw
haven_aol_icpy_patch.py calls _decode_certificate(ret)
cbor2.loads(cert_raw) fails with CBORDecodeEOF

I will need to do more research to see if this is my issue, which it may as well be.

[eliezhao]

The Python agent that is currently maintained and kept in sync with the latest ICP features is icp-py-core. Please use icp-py-core

@HavenCTO