Skip to content

test: build grcli-published bundle test fixtures for compatibility matrix #886

Description

@marcusburghardt

Context

Spike #854 proposed a 7-case compatibility-test matrix for validating
complyctl's consumption of bundles published by grcli. The cross-repo
integration tests (tests/cross-repo/) currently test against
complytime-providers, not against grcli-published bundles.

Problem

There is no test infrastructure that validates complyctl against bundles
produced by the upstream grcli toolchain. As the Gemara bundle format
evolves, complyctl needs regression coverage for bundle consumption
compatibility.

Proposed Compatibility Matrix

Test Case Bundle Source Verification Expected Behavior
Flat single-artifact bundle grcli Pack keyless get syncs, scan resolves graph
Multi-artifact bundle (policy + catalog + guidance) grcli Pack keyless Full DependencyGraph resolved
Bundle with embedded imports grcli Pack + Assemble keyless Imports resolved from cached OCI Layout
Bundle with external mapping refs Future (Smart Bundling) keyless TBD (depends on grc-store-clientkit#2)
Private registry bundle grcli to private registry keyed (PEM) Auth via docker cred helpers
Legacy split-layer artifact pre-v0.10 publisher optional Backward compat via DetectManifestShape
Version-skewed bundle grcli on go-gemara v0.9.2 keyless Unpack on v0.10.0 succeeds

Implementation Notes

  • Requires a test fixture pipeline using grcli (or bundle.Pack() directly)
    to produce artifacts in a test registry (e.g., zot, as used in acceptance tests)
  • The "external mapping refs" case is blocked on grc-store-clientkit#2
    (Smart Bundling spec)
  • Consider extending the existing acceptance test infrastructure
    (tests/acceptance/) rather than building a separate stack

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    Priority

    Low

    Effort

    Low

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions