Summary
A malformed LLM extraction response can be converted into an empty mapping by parse_json_result, after which synchronous /product/add reports success with data: [] and no memories are stored.
Related: #1355, fixed by #1977. This appears to be a different failure path: that fix corrected the fallback key in SimpleStructMemReader; this report concerns a parser failure returning {} without raising, so an exception-based fallback is not invoked.
Version and environment
- Upstream
main inspected: a7367d07e55db61099f7b4e2c1108bc5831a24f3 (also our checkout base).
- Self-hosted Python server, Python 3.11 on macOS.
- LLM:
glm-5.3-flash, through an OpenAI-compatible Z.AI endpoint.
- Neo4j Community 5.26.6, Qdrant 1.15.3, local Ollama embeddings.
/product/add with async_mode: "sync", explicitly created test cube.
Scope of reproduction: the parser failure below was reproduced from the unmodified upstream function at the pinned commit. The end-to-end API observation came from a local checkout with unrelated keyword-search/feedback and debug-logging changes; we have not reproduced the entire HTTP path on a pristine deployment. At the time of the failure, mem_reader/utils.py was unmodified. The isolated parser reproduction needs no LLM, database or server port.
Minimal deterministic reproduction
Run against the unmodified parser at the commit above, in an environment with MemOS dependencies installed:
from memos.mem_reader.utils import parse_json_result
valid = '{"memory list": [{"value": "synthetic evidence"}]}'
malformed = '{"memory list": [{"value": "synthetic evidence"}],}'
assert parse_json_result(valid) == {
"memory list": [{"value": "synthetic evidence"}]
}
assert parse_json_result(malformed) == {}The trailing comma is invalid JSON. The concern is not that strict parsing rejects it; the concern is that rejection becomes indistinguishable from a successful empty extraction downstream.
Observed HTTP behavior
In a bounded document-ingestion test:
- Created a new cube and submitted the document through synchronous
/product/add.
- The extraction provider returned
finish_reason: "stop", with a trailing comma before the final closing brace. This was not a reported output truncation.
- The parser logged
Failed to decode JSON: Expecting property name enclosed in double quotes and returned {}.
- The downstream log reported
No add/update items prepared.
- The API returned:
{"code": 200, "message": "Memory added successfully", "data": []}
/product/get_all and subsequent searches in that cube returned zero memories. Later chat requests had no supporting context.
The failed extraction consumed 13,545 provider-reported tokens. We did not retry it inside that frozen evaluation. No source document, private logs, credentials or user identifiers are included here.
Relevant code
Expected behavior
Please distinguish these outcomes in the extraction/add flow:
- Extraction failed because the model output could not be parsed or validated.
- Extraction succeeded and legitimately found no memories.
- Memories were successfully persisted.
A valid empty extraction should remain supported. An invalid extraction should not be represented solely as “Memory added successfully” with an empty list. A structured diagnostic or propagated extraction failure would let callers handle this without assuming persistence succeeded.
Local mitigation and validation limits
We tested a narrow local mitigation that removes commas immediately before } or ] only outside quoted strings, reparses with json.loads, and emits an explicit repair warning. It does not try to invent missing values or repair arbitrary malformed output.
- Four regression tests cover repair reporting, punctuation/escaped quotes inside strings, unchanged valid JSON, and rejection of a missing value.
- Two of these tests failed before the patch; all four passed afterward.
- The selected complete
tests/mem_reader run passed: 75 passed, with three warnings; formatting/lint passed in the validation copy.
- Replaying the original malformed final response locally recovered five memory objects, with no model calls or database writes.
- A later live run on the patched service stored five memories and supported both searches. However, its fresh model response was already valid JSON, so that live success is not causal evidence that the repair branch handled the original failure.
This mitigation does not solve the general empty-success ambiguity, and syntactic validity does not establish semantic fidelity. We are reporting the failure mode rather than proposing that every empty extraction must be treated as an error.
Summary
A malformed LLM extraction response can be converted into an empty mapping by
parse_json_result, after which synchronous/product/addreports success withdata: []and no memories are stored.Related: #1355, fixed by #1977. This appears to be a different failure path: that fix corrected the fallback key in
SimpleStructMemReader; this report concerns a parser failure returning{}without raising, so an exception-based fallback is not invoked.Version and environment
maininspected:a7367d07e55db61099f7b4e2c1108bc5831a24f3(also our checkout base).glm-5.3-flash, through an OpenAI-compatible Z.AI endpoint./product/addwithasync_mode: "sync", explicitly created test cube.Scope of reproduction: the parser failure below was reproduced from the unmodified upstream function at the pinned commit. The end-to-end API observation came from a local checkout with unrelated keyword-search/feedback and debug-logging changes; we have not reproduced the entire HTTP path on a pristine deployment. At the time of the failure,
mem_reader/utils.pywas unmodified. The isolated parser reproduction needs no LLM, database or server port.Minimal deterministic reproduction
Run against the unmodified parser at the commit above, in an environment with MemOS dependencies installed:
The trailing comma is invalid JSON. The concern is not that strict parsing rejects it; the concern is that rejection becomes indistinguishable from a successful empty extraction downstream.
Observed HTTP behavior
In a bounded document-ingestion test:
/product/add.finish_reason: "stop", with a trailing comma before the final closing brace. This was not a reported output truncation.Failed to decode JSON: Expecting property name enclosed in double quotesand returned{}.No add/update items prepared.{"code": 200, "message": "Memory added successfully", "data": []}/product/get_alland subsequent searches in that cube returned zero memories. Later chat requests had no supporting context.The failed extraction consumed 13,545 provider-reported tokens. We did not retry it inside that frozen evaluation. No source document, private logs, credentials or user identifiers are included here.
Relevant code
Expected behavior
Please distinguish these outcomes in the extraction/add flow:
A valid empty extraction should remain supported. An invalid extraction should not be represented solely as “Memory added successfully” with an empty list. A structured diagnostic or propagated extraction failure would let callers handle this without assuming persistence succeeded.
Local mitigation and validation limits
We tested a narrow local mitigation that removes commas immediately before
}or]only outside quoted strings, reparses withjson.loads, and emits an explicit repair warning. It does not try to invent missing values or repair arbitrary malformed output.tests/mem_readerrun passed: 75 passed, with three warnings; formatting/lint passed in the validation copy.This mitigation does not solve the general empty-success ambiguity, and syntactic validity does not establish semantic fidelity. We are reporting the failure mode rather than proposing that every empty extraction must be treated as an error.