Skip to content

ci: check the diagram bot's Mermaid parses before it posts - #25849

Merged
mshabarov merged 1 commit into
mainfrom
fix-diagram-syntax
Sep 21, 2026
Merged

mshabarov merged 1 commit into
mainfrom
fix-diagram-syntax

Conversation

@mshabarov

Copy link
Copy Markdown
Contributor

Follow-up to #25373

DX/docs · .github/workflows · reviewers reading a Diagram Bot comment

Background — Diagram Bot. A pull request whose change is about
structure, flow or ordering gets one Mermaid figure posted as a comment
by an agentic workflow. GitHub renders that block in the reader's own
browser, so a block with a syntax error becomes a red error box where
the picture should be.

Eight of the thirty-four figures the bot has posted so far do not
render. Each one breaks on a Mermaid rule the bot's own instructions
never stated, so the reviewer gets an error box and the bot looks
broken. It now parses its figure before posting, fixes what does not
parse, and stays silent when it cannot.

Risks:

  • ✅ no product code, no public API, no change for applications

Context. The instructions asked for quotes around node labels that
hold punctuation and said nothing about edge labels, backticks or
semicolons — which is where all eight failures are. The lock file is
unchanged on purpose: the prompt is pulled in at run time with
runtime-import, and the frontmatter did not change.

  • Added .github/scripts/validate-mermaid.mjs, which parses a figure
    with the same Mermaid version GitHub renders comments with and names
    the line at fault. It takes a .mmd file, a .md file whose fenced
    mermaid blocks it extracts, or the figure on stdin.
    • Mermaid and jsdom are installed from npm on first use and cached
      under the runner's temp directory. With no network the script still
      applies its own rules and exits 2, so an unreachable registry never
      passes a broken figure as sound.
    • Those rules cover the three failures below, plus %%{init}%%, raw
      HTML, quoted free text in a sequence diagram and a classDef that
      sets fill: without color:. Over the thirty-four figures posted
      so far they flag exactly the eight that do not parse, and none of
      the twenty-six that do.
  • Added a step to diagram-bot.md that runs the validator before the
    comment goes out, and records noop instead of posting when the
    figure still does not parse after three attempts.
  • Rewrote the syntax rules in diagram-bot.md around what actually
    broke: quote edge labels as well as node labels, never open a label
    with a backtick, and keep semicolons out of sequenceDiagram message
    and note text.
  • Documented the validator in .github/workflows/README.md.

Eight of the thirty-four figures posted so far render as an error box
instead of a picture. All eight break on a Mermaid rule the bot's
instructions did not state.
@mshabarov

mshabarov commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor Author

Type of change

  • Internal change

How to test

  1. Run the validator over the bot's own instructions, whose two example
    figures are real Mermaid: node .github/scripts/validate-mermaid.mjs .github/workflows/diagram-bot.md
  2. It prints OK for both blocks and exits 0.
  3. Save a figure that is known to be broken — the one from feat(devloop): start apps through the server plugin their build runs #25794 is the
    easiest — and run the validator over it:
cat > /tmp/broken.mmd <<'MERMAID'
sequenceDiagram
    participant A as One
    participant B as Two
    Note over A,B: runs in the build JVM; agents travel via MAVEN_OPTS
MERMAID
node .github/scripts/validate-mermaid.mjs /tmp/broken.mmd
  1. It exits 1 and prints both the parse error and the rule:
  PARSE  /tmp/broken.mmd
       Parse error on line 4: ...ravel via MAVEN_OPTS
       line 4: Note over A,B: runs in the build JVM; agents travel via MAVEN_OPTS
  lint   /tmp/broken.mmd: line 4, a semicolon ends the statement; use a comma or a second Note

Note

diagram-bot.lock.yml is not regenerated. The prompt reaches the agent
through {{#runtime-import .github/workflows/diagram-bot.md}}, so the
body of the workflow takes effect without recompiling, and the
frontmatter is untouched.

The eight figures that do not render

Every Diagram Bot comment posted since the bot was turned on was pulled
back and parsed with Mermaid 11, the version GitHub renders comments
with. Thirty-four figures, eight of them broken:

Cause Pull requests
Parentheses in a label that is not quoted, most often an edge label such as -->|classpathForced() sees no move| #25444, #25533, #25642, #25669
A node label opening with a backtick, from wrapping @PWA in backticks inside the quotes, which starts a Mermaid markdown string #25512, #25547
A semicolon in sequenceDiagram message or note text, which ends the statement #25704, #25794
Test coverage

There is no automated test for the script itself. It was checked against
the whole corpus above: it reports the same verdict as the Mermaid parser
on all thirty-four figures, and its rules alone — the path taken when npm
is unreachable — find every one of the eight without flagging any of the
twenty-six that render. Mermaid's own node shapes (A(("circle")),
A[[sub]], A[(db)], A{{hex}}) were checked separately, since a naive
parenthesis rule would flag them.

@mshabarov
mshabarov marked this pull request as ready for review September 21, 2026 12:49
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Test Results

 1 455 files   1 539 suites   1h 33m 9s ⏱️
11 942 tests 11 874 ✅ 68 💤 0 ❌
12 257 runs  12 189 ✅ 68 💤 0 ❌

Results for commit 1560715.

@mshabarov
mshabarov merged commit c04e86d into main Sep 21, 2026
52 checks passed
@mshabarov
mshabarov deleted the fix-diagram-syntax branch September 21, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant