precompiling cfunction calls

[stevengj]

I noticed in JuliaPy/PyCall.jl#167 that, even after running Base.compile, some cfunction calls in my __init__ function were taking a significant amount of time (0.8 seconds for 6 functions).

I could work around this by adding some explicit precompile lines in order to ensure that these were compiled, but this raises three problems:

• Calls to cfunction in __init__ should really cause the corresponding functions to be precompiled automatically.
• Even after precompiling, cfunction was quite expensive: 0.25 seconds for 6 functions.
• Calling precompile causes the functions to be compiled at the wrong time — before __init__ is called rather than after. Since my __init__ initializes some global const variables that are referenced in the precompiled functions, this means that type inference does not work properly when precompiling (since Julia does not know the types of those globals yet).

cc: @vtjnash

[vtjnash]

eventually, we need to disable the speed boost provided by calling global const from __init__ since it causes potentially serious bugs in type inference / constant propagation: (e2d842a#diff-c3408de56388e517a8975797c90d9448R385). although we'll likely need to implement the some alternative around the same time

[JeffBezanson]

One thing that would be neat, and should be entirely safe, is to automatically do precompile(__init__, ()) as part of compile.

[vtjnash]

i'm pretty sure you already thought of that before, and implemented it (

julia/src/toplevel.c

Line 85 in 8294d4f

if (f) jl_get_specialization(f, (jl_tupletype_t*)jl_typeof(jl_emptytuple));

)

[stevengj]

Yes, precompile(__init__, ()) made no difference, so it seems like jl_get_specialization is not specializing the function in the cfunction calls within __init__ (or any other function) unless you precompile them explicitly..

[stevengj]

Unfortunately, this doesn't seem to be completely fixed. When I remove the manual precompilation from PyCall/src/PyCall.jl, the load time goes from 1.35s to 1.68s on my machine.

[vtjnash]

we pre-infer __init__, but i'm not sure if we pre-compile it

[stevengj]

@vtjnash, adding precompile(__init__, ()) makes no difference, so that's not the problem.