Add support for the lj2k and zstd compression modes. by QuantaDude · Pull Request #5495 · AcademySoftwareFoundation/OpenImageIO

QuantaDude · 2026-09-24T16:59:59Z

Description

lj2k and zstd compression mapping in the OpenEXR C++ and OpenEXR Core input paths.
lj2k and zstd compression selection in the OpenEXR output path.
LJ2K quality handling with the OpenEXR-supported range of 1 to 150, defaulting to 110.
ZSTD compression-level handling with the OpenEXR-supported range of 1 to 22, defaulting to 5.
Regression coverage for both compression modes in the openexr-compression tests, including the OpenEXR Core path.
Reference EXR files and expected output for the new test cases.
Bumped OpenEXR version in src/build-scripts/build_openexr from v3.4.15 to v3.5.0
Updated the compression attributes documentation in src/doc/builtinplugins.md

Tests

abhirup@abhirup-artix …/OpenImageIO on  openexr-3.5-compression via △ v4.4.2 via 🐍 v3.14.7 
 &  cmake -S . -B build \
            -DCMAKE_BUILD_TYPE=Release \
            -DCMAKE_INSTALL_PREFIX="$PWD/dist" \
            -DOIIO_BUILD_TESTS=ON \
            -DBUILD_TESTING=ON \
            -DOIIO_DOWNLOAD_MISSING_TESTDATA=ON \
            -DUSE_OPENCV=OFF \
            -DUSE_QT=OFF \
            -DUSE_FFMPEG=OFF \
            -DUSE_PYTHON=ON \
            -DUSE_OPENVDB=OFF \
            -DUSE_PTEX=OFF \
            -DUSE_DCMTK=OFF \
            -DUSE_LIBUHDR=OFF
  
  abhirup@abhirup-artix …/OpenImageIO on  openexr-3.5-compression via △ v4.4.2 via 🐍 v3.14.7 
 &   ctest --test-dir build -R '^openexr-compression' --output-on-failure

output

abhirup@abhirup-artix …/OpenImageIO on  openexr-3.5-compression via △ v4.4.2 via 🐍 v3.14.7 took 16s 
 &  ctest --test-dir build -R '^openexr-compression' --output-on-failure
Test project /home/abhirup/OSS/OpenImageIO/build
    Start 152: openexr-compression
1/2 Test #152: openexr-compression ..............   Passed    3.11 sec
    Start 166: openexr-compression.core
2/2 Test #166: openexr-compression.core .........   Passed    3.09 sec

100% tests passed out of 2

Total Test time (real) =   6.21 sec

Checklist:

I have read the guidelines on contributions and code review procedures.
I have read the Policy on AI Coding Assistants
and if I used AI coding assistants, I have an Assisted-by: TOOL / MODEL
line in the pull request description above. => I have only used ChatGPT for researching, clarifying a few things.
I have updated the documentation if my PR adds features or changes
behavior.
I am sure that this PR's changes are tested in the testsuite.
I have run and passed the testsuite in CI before submitting the
PR, by pushing the changes to my fork and seeing that the automated CI
passed there. (Exceptions: If most tests pass and you can't figure out why
the remaining ones fail, it's ok to submit the PR and ask for help. Or if
any failures seem entirely unrelated to your change; sometimes things break
on the GitHub runners.)
My code follows the prevailing code style of this project and I
fixed any problems reported by the clang-format CI test.
If I added or modified a public C++ API call, I have also amended the
corresponding Python bindings. If altering ImageBufAlgo functions, I also
exposed the new functionality as oiiotool options. => No new APIs, just added two new compression modes for OpenEXR.

linux-foundation-easycla · 2026-09-24T17:00:07Z

The committers listed above are authorized under a signed CLA.

✅ login: QuantaDude / name: Abhirup Bhattacharyya (67ad861)

lgritz · 2026-09-24T19:42:13Z

+    set (_openexr_new_compression "1")
+else ()
+    set (_openexr_new_compression "0")


This name is likely to seem dated in two years when zstd isn't "new" and something else implemented in 2027 is added.

my bad, I'll name it more specific.

lgritz · 2026-09-24T19:44:00Z

                    URL http://github.com/AcademySoftwareFoundation/openexr-images)
-    # For OpenEXR >= 3.1, be sure to test with the core option on
+
+    if (OpenEXR_VERSION VERSION_GREATER_EQUAL 3.1.10)


Why 3.1.10? Isn't this a feature of 3.5?

The diff seems messy, I set a conditional variable in the cmake checking if the openEXR version is 3.5.0 or above and then I simply passed it as a env variable to the compression tests. I had to duplicate this existing if block two times.
Anyway, now I've a better way to deal with this using set_property()

Please check the newer commit, I now append the OIIO_OPENEXR_LJ2K_ZSTD_SUPPORT environment variable to the openexr-compression test when openexr version is at least 3.5.0.

lgritz · 2026-09-24T19:44:16Z

+    endif ()
+
    if (OpenEXR_VERSION VERSION_GREATER_EQUAL 3.1)
+        # For OpenEXR >= 3.5 we need to add the new compression tests, with the core library


Comment doesn't match the if. Which version are we talking about here?

lgritz · 2026-09-24T19:44:30Z

+    # For OpenEXR >= 3.5
+    if (OpenEXR_VERSION VERSION_GREATER_EQUAL 3.1.10)
+        oiio_add_tests (openexr-compression


Comment doesn't match the condition

QuantaDude

I improved testing.cmake, changed the env variable, simplified how it's set and bumped OpenEXR_BUILD_VERSION in build_OpenEXR.cmake.

I have also created a new reference out-legacy.txt file for the test to pass on older OpenEXR versions.

Does openexr_ver in ci.yml need to be set to v3.5.0 as well?

QuantaDude · 2026-09-25T02:24:44Z

                    URL http://github.com/AcademySoftwareFoundation/openexr-images)
-    # For OpenEXR >= 3.1, be sure to test with the core option on
+
+    if (OpenEXR_VERSION VERSION_GREATER_EQUAL 3.1.10)


Please check the newer commit, I now append the OIIO_OPENEXR_LJ2K_ZSTD_SUPPORT environment variable to the openexr-compression test when openexr version is at least 3.5.0.

lgritz · 2026-09-25T03:30:43Z

Does openexr_ver in ci.yml need to be set to v3.5.0 as well?

It should be for the "latest releases" jobs, but NOT for any of the other jobs, which are intentionally sampling building against a variety of OpenEXR versions.

QuantaDude · 2026-09-25T04:32:08Z

@lgritz I updated the git commit hash for the local, latest builds for OpenEXR and fixed clang-format CI errors.

Not sure why the openexr-compression test is failing, it passes on my machine with openexr 3.14.15

lgritz · 2026-09-25T06:24:39Z

     - float
     - aperture
-   * - ``compression``
+  * - ``compression``


This line seems to have changed for no reason

lgritz · 2026-09-25T23:17:06Z

Some of the CI failures you're getting revealed a separate problem with OpenEXR and zstd and static libraries. I have an OIIO-side fix here: #5498

And an OpenEXR side here: AcademySoftwareFoundation/openexr#2673

- `lj2k` and `zstd` compression mapping in the OpenEXR C++ and OpenEXR Core input paths. - `lj2k` and `zstd` compression selection in the OpenEXR output path. - LJ2K quality handling with the OpenEXR-supported range of `1` to `150`, defaulting to `110`. - ZSTD compression-level handling with the OpenEXR-supported range of `1` to `22`, defaulting to `5`. - Regression coverage for both compression modes in the `openexr-compression` tests, including the OpenEXR Core path. - Reference EXR files and expected output for the new test cases. - Bumped OpenEXR version in `src/build-scripts/build_openexr` from `v3.4.15` to `v3.5.0` - Updated the compression attributes documentation in `src/doc/builtinplugins.md` Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

Raise the local build OpenEXR default to 3.5.0 in build_OpenEXR.cmake, matching the version already bumped in the build shell script. Rename the OIIO_OPENEXR_NEW_COMPRESSION environment variable to OIIO_OPENEXR_LJ2K_ZSTD_SUPPORT, old env variable would have read oddly once something else lands in a future OpenEXR release. The lj2k/zstd env var is now added via set_property(TEST ... APPEND PROPERTY ENVIRONMENT ...) once OpenEXR_VERSION >= 3.5.0, after both the base and .core test variants already exist, rather than being threaded through ENVIRONMENT arguments on a separate set of oiio_add_tests() calls. Add testsuite/openexr-compression/ref/out-legacy.txt so the test still passes on OpenEXR < 3.5.0, where lj2k/zstd aren't compiled in. Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

…x docs indent - Added a guard in testing.cmake before setting the `OIIO_OPENEXR_LJ2K_ZSTD_SUPPORT` env variable property for the `openexr-compression` test. To fix openImageIO from not being able to configure and build when the `BUILD_TESTING`, `OIIO_BUILD_TESTS` flag options were not set to `ON`. Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

lgritz · 2026-09-26T02:56:13Z

It's looking like the lj2k.exr is such a close match that it, too, is getting mixed up with zip.exr and leading to failures because it's nondeterministic which of the two matching files it will find first.

Hmmm... Maybe you can intentionally choose a lower quality "lj2k" compression to force it to be distinct from the lossless compression? (And to prove to ourselves that there is not some bug that is forcing lj2k to always be equivalent to htj2k and not actually be lossy?)

lgritz · 2026-09-26T02:57:55Z

Oh, also, we should be testing htj2k as well as lj2k, obviously.

QuantaDude · 2026-09-26T04:29:18Z

@lgritz It's only failing in the bleeding edge build. I checked the bleeding edge's CI (which build on OpenEXR's main branch) CI's build log says openexr is version 4.0.0.
It's giving garbage values:

 Comparing "out-lj2k.exr" and "ref/out-lj2k.exr"
-128 x 96, 3 channels
-  Mean error = nan
-  RMS error = nan
-  Peak SNR = nan
-  Max error  = inf @ (71, 95, R)  values are -1.3589859e-05, 0.027938843, 0.15246582 vs -nan, 6.341934e-05, 0.00015449524
-  12118 pixels (98.6%) over 0.016
-  12263 pixels (99.8%) over 0.008
-FAILURE

I'll commit again with a new exr reference.

lgritz · 2026-09-26T04:36:43Z

Max error = inf @ (71, 95, R) values are -1.3589859e-05, 0.027938843, 0.15246582 vs -nan, 6.341934e-05, 0.00015449524

Well, that's not good -- a NaN is creeping in somewhere! Maybe a problem in OpenEXR main?

QuantaDude · 2026-09-26T05:09:20Z

yes, I can confirm. The main branch's LJ2K compression is producing corrupted output.

lgritz · 2026-09-26T05:46:33Z

D'oh! We were so close to having this wrapped up!

QuantaDude · 2026-09-26T06:14:55Z

@lgritz I opened an OpenEXR issue regarding this.

lgritz · 2026-09-26T06:21:53Z

Hopefully, with a straightforward repro case, somebody can fix the problem in OpenEXR right away.

In the mean time, would you like to add a htj2k to this test as well?

Added HTJ2K compression tests when OpenEXR 3.4.0 or newer is available, and add expected output for the `htj2k32` and `htj2k256` compression modes. Created a new reference file `out-exr3.1.txt` for idiff when OpenEXR version is older than `3.4.0`. Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

lgritz reviewed Sep 24, 2026

View reviewed changes

Comment thread src/build-scripts/build_openexr.bash

QuantaDude commented Sep 25, 2026

View reviewed changes

lgritz reviewed Sep 25, 2026

View reviewed changes

Comment thread src/openexr.imageio/exrinput_c.cpp

lgritz reviewed Sep 25, 2026

View reviewed changes

Comment thread testsuite/openexr-compression/ref/out-exr3.4.txt

lgritz reviewed Sep 25, 2026

View reviewed changes

Comment thread testsuite/openexr-compression/ref/out-zstd.exr Outdated

QuantaDude added 6 commits September 26, 2026 07:30

Gate LJ2K and ZSTD compression tests on OpenEXR 3.5

cd79121

Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

build: fix OpenEXR commit pin and clang-format violations

95f1cb0

Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

ci: bump OpenEXR to v3.5.0 for latest releases

982fa10

Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

QuantaDude force-pushed the openexr-3.5-compression branch from be1d0db to dcc7927 Compare September 26, 2026 02:07

tests: remove reference out-zstd.exr, and update reference out.txt

1c63df7

Signed-off-by: Abhirup Bhattacharyya <abhirup27022001@outlook.com>

Conversation

QuantaDude commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tests

Checklist:

Uh oh!

linux-foundation-easycla Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

QuantaDude left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lgritz commented Sep 25, 2026

Uh oh!

QuantaDude commented Sep 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lgritz commented Sep 25, 2026

Uh oh!

lgritz commented Sep 26, 2026

Uh oh!

lgritz commented Sep 26, 2026

Uh oh!

QuantaDude commented Sep 26, 2026

Uh oh!

lgritz commented Sep 26, 2026

Uh oh!

QuantaDude commented Sep 26, 2026

Uh oh!

lgritz commented Sep 26, 2026

Uh oh!

QuantaDude commented Sep 26, 2026

Uh oh!

lgritz commented Sep 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

QuantaDude commented Sep 24, 2026 •

edited

Loading

linux-foundation-easycla Bot commented Sep 24, 2026 •

edited

Loading