Support new build report format, as reported by `bob`

[bsiegert]

This was the first example of such a bulk build report: https://mail-index.netbsd.org/pkgsrc-bulk/2026/04/02/msg028397.html

Note the header in particular:

URL: https://reports.pkgci.org/SmartOS/upstream/trunk/20260401T195428Z/report.html

Total:      29741  Platform:    SmartOS 20241212T000748Z/x86_64
Succeeded:   2161  Compiler:    gcc-14.3.0
Failed:      2446  Repository:  https://github.com/NetBSD/pkgsrc
UpToDate:   20533  Duration:    6h 22m 13s
Masked:      4601

• link is called "URL", not machine-readable report. It also goes to report.html, not the parseable output. Not sure what to do there?
• dual column format means a bit more complexity when parsing

[jperkin]

Note that one of the improvements with bob's reporting is that it includes this file:

https://reports.pkgci.org/SmartOS/upstream/trunk/20260401T195428Z/variables.json

If there are any other fields you want me to add to this I am happy to do so. It should make it much easier to parse and easily add new fields in the future without breaking things.

[jperkin]

Also note that there's a bug currently where the revision is missing from the email report, and the branch is missing from variables.json, I'll fix those.

[jperkin]

Alternatively I could add arbitrary email headers that bulktracker parses with links to the machine report and the variables file.

[jperkin]

I was trying to move away from xz completely given its terrible security track record, and the fact it's ridiculously slow compared to zstd for similar compression sizes. I don't think it's a good choice at all in 2026, but I can add it back if adding zstd support is too much of a blocker for now. I already added pkg_summary.zst support to pkgin and dropped pkg_summary.xz and pkg_summary.bz2 generation from bob.

[bsiegert]

Adding zstd support should be easy.

As for the machine-readable URL: can you put the URL to that into variables.json, maybe?

[bsiegert]

Some more breadcrumbs:

• the most common zstd decompressor: https://pkg.go.dev/github.com/klauspost/compress/zstd
• seekable implementation: https://pkg.go.dev/github.com/SaveTheRbtz/zstd-seekable-format-go

[jperkin]

Sure, I think what I'll do is add a report section and then fill it with useful information from the report (URLs, package counts, etc). If you have any other suggestions for variables.json then now is the time to make any changes, as while we can add new things at any point I want to set any existing variables in stone to avoid future churn.

[bsiegert]

Sounds great!

Regarding the variables: you have the revision there, but would it be possible to put the branch? Whether you are building HEAD or pkgsrc-2026Q1 is an important signal.

[jperkin]

Yep, that's currently a bug, it should be there, and is for builds performed with a full git repository. The problem here is that build was done via Jenkins which checks out a specific revision, and so git doesn't know exactly which branch it built from. I already knew about that and have an override in the config file which specifies the branch, but forgot to hook it up to the variables file.

[bsiegert]

One last thing: could you put the counts (OK, failed, prefailed, etc.) into the JSON file too?

[jperkin]

The latest build just published to pkgsrc-bulk@ should include all of these changes. Let me know what you think.

Unfortunately the git sync is broken so this is basically just a rebuild of what was there before.