[clad] Bump clad to version 2.5 - #23479
Conversation
libLLVMTableGen.a depends on Support, and LLVMExports says so: the imported target carries an INTERFACE_LINK_LIBRARIES of LLVMSupport. clad never saw it, because the library is named as a path from find_library and a path carries no interface. LLVM_LINK_COMPONENTS then puts Support on the line first and the lookup appends TableGen after it, so every reference the archive makes into Support -- FoldingSetBase, ToolOutputFile, CleanupInstaller -- goes unresolved. ROOT hit it building clad beside its own llvm-project (root-project/root#23479). Ask LLVM how it was built instead. Where the components are archives, naming TableGen among them is all it takes: cmake reads the interface and orders Support behind it. Where there is a libLLVM to fold them into, that library carries no TableGenMain, so the found file is still what gets named -- and nothing is lost, because a shared library leaves no undefined symbol for an archive to satisfy. The file rather than the imported target on that second path, deliberately: the target would bring its include directories too, and on a Homebrew prefix that puts /usr/local ahead of the SDK's own headers. $<LINK_ONLY:> does not hold them back. No row saw this, and the reason is the second half of the change. ubu24-cling-llvm22 takes the llvm-root recipe, the configuration ROOT builds, so it is the one row with no libLLVM to fall back on and the one whose linker is as strict. It named cladPlugin and cladDifferentiator as its targets and so never built the tool at all. It builds it now, and with the fix reverted it fails there exactly as it does for ROOT.
libLLVMTableGen.a depends on Support, and LLVMExports says so: the imported target carries an INTERFACE_LINK_LIBRARIES of LLVMSupport. clad never saw it, because the library is named as a path from find_library and a path carries no interface. LLVM_LINK_COMPONENTS then puts Support on the line first and the lookup appends TableGen after it, so every reference the archive makes into Support -- FoldingSetBase, ToolOutputFile, CleanupInstaller -- goes unresolved. ROOT hit it building clad beside its own llvm-project (root-project/root#23479). Ask LLVM how it was built instead. Where the components are archives, naming TableGen among them is all it takes: cmake reads the interface and orders Support behind it. Where there is a libLLVM to fold them into, that library carries no TableGenMain, so the found file is still what gets named -- and nothing is lost, because a shared library leaves no undefined symbol for an archive to satisfy. The file rather than the imported target on that second path, deliberately: the target would bring its include directories too, and on a Homebrew prefix that puts /usr/local ahead of the SDK's own headers. $<LINK_ONLY:> does not hold them back. No row saw this, and the reason is the second half of the change. ubu24-cling-llvm22 takes the llvm-root recipe, the configuration ROOT builds, so it is the one row with no libLLVM to fall back on and the one whose linker is as strict. It named cladPlugin and cladDifferentiator as its targets and so never built the tool at all. It builds it now, and with the fix reverted it fails there exactly as it does for ROOT.
Test Results 22 files 22 suites 3d 8h 52m 25s ⏱️ For more details on these failures, see this check. Results for commit 4238354. ♻️ This comment has been updated with latest results. |
|
@guitargeek, can you take a look and possibly run your magic set of benchmarks? |
|
Yes @vgvassilev, I ran the Higgs likelihood benchmarks from CMS and ATLAS and there is no difference in gradient generation and runtime between Clad 2.4 and |
Can you share again your recipe, I wanted to check if the new optional analysis makes any difference. |
|
Sure! If you want to do the studies yourself, just clone rootbench, go in this directory and run the three commands explained in the I added the instructions some weeks ago so anyone can easily reproduce the ATLAS Higgs plot. For the CMS plot I have no public instructions yet, but the likelihood code structure is very similar anyway. |
Clad 2.5 is compatible with clang-14 through clang-23, having dropped clang-11 through clang-13 and ported to LLVM 23. The release makes the derivatives clad writes readable: generated code carries real source locations, so a debugger steps through a derivative and a diagnostic points into it, and -fgenerated-source-dir writes it out as a file. The analyses clad runs are described in one table that drives the switches (-fenable-analysis=/-fdisable-analysis=, and clad::opts::enable_* per request), and -Rclad-analysis=<name> reports what an analysis left in the derivative and what it was looking for; the command-line options are a table too, with a suggestion for a misspelled one and -fclad-porting-hints naming the custom derivatives a translation unit is missing. clad::immediate_mode is gone: clad decides from the call site whether a derivative is needed while the program compiles, in every mode. Reverse mode gains #pragma omp parallel, hessians assembled from hessian-vector products, std::vector construction/resize pullbacks, per-call state handed from a reverse_forw to its pullback through clad::pullback_state, and counted loops whose trip count is recomputed rather than counted, with a broadcast read's adjoint summed in a register. CUDA builtins are passive and a broadcast adjoint is summed per thread. Fifty issues are closed, among them crashes on lambdas with loops, early-return regressions, wrong gradients for compound indices and loop-carried divisors, and hessian failures on const pointers and std::max. Static builds no longer need the LLVM_LINK_LLVM_DYLIB workaround for TableGen, and the generated tables are rendered at build time. For details, please refer to the release notes.
4238354 to
8e9453b
Compare
guitargeek
left a comment
There was a problem hiding this comment.
Fantastic, thanks for the big update!
Build failures are unrelated.
Clad 2.5 is compatible with clang-14 through clang-23, having dropped clang-11 through clang-13 and ported to LLVM 23. The release makes the derivatives clad writes readable: generated code carries real source locations, so a debugger steps through a derivative and a diagnostic points into it, and -fgenerated-source-dir writes it out as a file. The analyses clad runs are described in one table that drives the switches (-fenable-analysis=/-fdisable-analysis=, and clad::opts::enable_* per request), and -Rclad-analysis= reports what an analysis left in the derivative and
what it was looking for; the command-line options are a table too, with a suggestion for a misspelled one and -fclad-porting-hints naming the custom derivatives a translation unit is missing. clad::immediate_mode is gone: clad decides from the call site whether a derivative is needed while the program compiles, in every mode. Reverse mode gains #pragma omp parallel, hessians assembled from hessian-vector products, std::vector construction/resize
pullbacks, per-call state handed from a reverse_forw to its pullback through clad::pullback_state, and counted loops whose trip count is recomputed rather than counted, with a broadcast read's adjoint summed in a register. CUDA builtins are passive and a broadcast adjoint is summed per thread. Fifty issues are closed, among them crashes on lambdas with loops, early-return regressions, wrong gradients for compound indices and loop-carried divisors, and hessian failures on const pointers and std::max. Static builds no longer need the LLVM_LINK_LLVM_DYLIB workaround for TableGen, and the generated tables are rendered at build time. For details, please refer to the release notes.