Issues

• eval_open() scores open questions with string answers as Correct when the gold string only appears inside a longer token (MgS/MgSO4, 1/64/1/640, No/none)

  MMMU-Benchmark/MMMU#91 · Filter by author shaurya416 opened on Sep 30, 2026

• MMMU-Pro: under-specified finance question validation_Finance_5 — missing share-exchange ratio

  MMMU-Benchmark/MMMU#90 · Filter by author AISTALK opened on Sep 26, 2026

• [MMMU-Pro] test_Architecture_and_Engineering_492: table inconsistent with answer options

  MMMU-Benchmark/MMMU#89 · Filter by author AISTALK opened on Sep 25, 2026

• MMMU-Pro: ambiguous chemistry question test_Chemistry_240 — both MgCl2 and CaCl2 at twice the molarity are valid

  MMMU-Benchmark/MMMU#88 · Filter by author AISTALK opened on Sep 25, 2026

• Optional EvalPort export for MMMU's per-category evaluation_result

  MMMU-Benchmark/MMMU#87 · Filter by author adhabnr-ux opened on Sep 22, 2026
  2