Regression in nightly: llvmcall with integer pointers is deprecated, use an actual pointer type instead

[fingolfin]

In a few packages I've seen tests with Julia nightly failing with this error.

E.g. in this log for GenericCharacterTables.jl:

Failed to precompile GenericCharacterTables [fdcc7804-6c20-4e83-8118-baad6b7d05b7] to "/home/runner/.julia/compiled/v1.12/GenericCharacterTables/jl_mWdjxN".
Internal error: encountered unexpected error during compilation of foreach:
ErrorException("llvmcall with integer pointers is deprecated, use an actual pointer type instead.")
ijl_error at /cache/build/tester-amdci5-13/julialang/julia-master/src/rtutils.c:41
emit_llvmcall at /cache/build/tester-amdci5-13/julialang/julia-master/src/ccall.cpp:998
emit_intrinsic at /cache/build/tester-amdci5-13/julialang/julia-master/src/intrinsics.cpp:1282
emit_call at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:5344
emit_expr at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:6345
emit_ssaval_assign at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:5883
emit_stmtpos at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:6161
emit_function at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:9410
jl_emit_code at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:9743
jl_emit_codeinst at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:9827
jl_compile_workqueue at /cache/build/tester-amdci5-13/julialang/julia-master/src/codegen.cpp:9941
jl_create_native_impl at /cache/build/tester-amdci5-13/julialang/julia-master/src/aotcompile.cpp:413
jl_precompile_ at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile_utils.c:265
jl_precompile_worklist at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile_utils.c:320 [inlined]
ijl_create_system_image at /cache/build/tester-amdci5-13/julialang/julia-master/src/staticdata.c:2987
ijl_write_compiler_output at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile.c:144
ijl_atexit_hook at /cache/build/tester-amdci5-13/julialang/julia-master/src/init.c:281
jl_repl_entrypoint at /cache/build/tester-amdci5-13/julialang/julia-master/src/jlapi.c:1060
main at /cache/build/tester-amdci5-13/julialang/julia-master/cli/loader_exe.c:58
unknown function (ip: 0x7f8315e29d8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4010b8)

[3523] signal 11 (1): Segmentation fault
in expression starting at none:0
_ZN4llvm11ConstantInt3getERNS_11LLVMContextERKNS_5APIntE at /opt/hostedtoolcache/julia/nightly/x64/bin/../lib/julia/libLLVM-17jl.so (unknown line)
_ZN4llvm11ConstantInt3getEPNS_11IntegerTypeEmb at /opt/hostedtoolcache/julia/nightly/x64/bin/../lib/julia/libLLVM-17jl.so (unknown line)
literal_static_pointer_val at /cache/build/tester-amdci5-13/julialang/julia-master/src/jitlayers.h:292 [inlined]
jl_create_native_impl at /cache/build/tester-amdci5-13/julialang/julia-master/src/aotcompile.cpp:427
jl_precompile_ at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile_utils.c:265
jl_precompile_worklist at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile_utils.c:320 [inlined]
ijl_create_system_image at /cache/build/tester-amdci5-13/julialang/julia-master/src/staticdata.c:2987
ijl_write_compiler_output at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile.c:144
ijl_atexit_hook at /cache/build/tester-amdci5-13/julialang/julia-master/src/init.c:281
jl_repl_entrypoint at /cache/build/tester-amdci5-13/julialang/julia-master/src/jlapi.c:1060
main at /cache/build/tester-amdci5-13/julialang/julia-master/cli/loader_exe.c:58
unknown function (ip: 0x7f8315e29d8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4010b8)
Allocations: 484227 (Pool: 484217; Big: 10); GC: 4

[3523] signal 11 (1): Segmentation fault
in expression starting at none:0
_ZN4llvm11ConstantInt3getERNS_11LLVMContextERKNS_5APIntE at /opt/hostedtoolcache/julia/nightly/x64/bin/../lib/julia/libLLVM-17jl.so (unknown line)
_ZN4llvm11ConstantInt3getEPNS_11IntegerTypeEmb at /opt/hostedtoolcache/julia/nightly/x64/bin/../lib/julia/libLLVM-17jl.so (unknown line)
literal_static_pointer_val at /cache/build/tester-amdci5-13/julialang/julia-master/src/jitlayers.h:292 [inlined]
jl_create_native_impl at /cache/build/tester-amdci5-13/julialang/julia-master/src/aotcompile.cpp:427
jl_precompile_ at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile_utils.c:265
jl_precompile_worklist at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile_utils.c:320 [inlined]
ijl_create_system_image at /cache/build/tester-amdci5-13/julialang/julia-master/src/staticdata.c:2987
ijl_write_compiler_output at /cache/build/tester-amdci5-13/julialang/julia-master/src/precompile.c:144
ijl_atexit_hook at /cache/build/tester-amdci5-13/julialang/julia-master/src/init.c:281
jl_repl_entrypoint at /cache/build/tester-amdci5-13/julialang/julia-master/src/jlapi.c:1060
main at /cache/build/tester-amdci5-13/julialang/julia-master/cli/loader_exe.c:58
unknown function (ip: 0x7f8315e29d8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4010b8)
Allocations: 484227 (Pool: 484217; Big: 10); GC: 4
ERROR: LoadError: Failed to precompile ThreadingUtilities [8290d209-cae3-49c0-8002-c8c24d57dab5] to "/home/runner/.julia/compiled/v1.12/ThreadingUtilities/jl_X0ieU2".
Stacktrace:
...

This has been going on for a few weeks there at least. I'll see if I can reproduce this locally, and if so, will try to bisect. At this point it looks to me as if this is a problem inside Julia, but of course it might be a problem in the package or some of its dependencies (several of which use ccall extensively, so who knows, perhaps something got corrupted). But I have no good idea how to debug this other than "try to find out in which Julia commit this started to behave differently" and hoping this will give a clue.

But maybe someone else has an idea what might cause this or how one might go about debugging it...

[lgoettgens]

I think this is somewhere in a JuliaSIMD repo, so I reported it in JuliaSIMD/ThreadingUtilities.jl#51. But these packages seem to no longer be maintained, so I won't hope for a fix.

But after all, a deprecation should not throw an Error Exception as I this case.

[lgoettgens]

Some breadcrumbs: the error message was added in #53687.
#53916 seems to be somehow related as well.

[giordano]

My understanding is that packages doing explicit llvmcalls need to adapt, because this is an upstream change in llvm.

[sgaure]

The test log reveals that it's run with --depwarn=error, so any package which hasn't updated its llvmcalls to use pointers will cause this error message.

Generated LLVM IR now uses actual pointer types instead of passing pointers as integers. This affects llvmcall: Inline LLVM IR should be updated to use i8* or ptr instead of i32 or i64, and remove unneeded ptrtoint/inttoptr conversions. For compatibility, IR with integer pointers is still supported, but generates a deprecation warning. (https://docs.julialang.org/en/v1.12-dev/NEWS/)

[maleadt]

This is not a regression, but an intentional change. Citing the linked PR:

Changing codegen like this is a breaking change for llvmcall users, but we don't promise any stability there. Also, with the switch to LLVM 17 where typed pointers have been removed, all llvmcall snippets will have to be updated anyway, so this seems like a good time to make that change.

Basically, llvmcall snippets have to be updated to use ptr instead of i64 (and remove the unneeded inttoptr/ptrtoint conversions). I have actually put code in place to help with the transition, however, the tests you refer to are explicitly asking for deprecation errors: https://github.com/oscar-system/GenericCharacterTables.jl/blob/a0d6c98708744613315373b7996eb67e894db43f/.github/workflows/CI.yml#L54-L56

In any case, llvmcall is an explicitly unstable API, so you should expect to have to regularly update its uses. LLVM doesn't offer compatibility of textual IR, so certain things break every time we bump LLVM.