You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@warn"Layer with Float32 parameters got Float64 input.
I think that this is the correct default behavior. However, in order to find out where the problem is coming from throwing an error to produce a stacktrace would be very helpful.
Possible Implementation
There could be a Preference or global flag that allows switching errors instead of warning for wrong precision inputs. This would also be made consistent with CUDA.allowscalar.
The text was updated successfully, but these errors were encountered:
Maybe it should literally be the same switch as CUDA.allowscalar, to avoid introducing ever more functions you have to know about? It's not exactly the same meaning, but it's likely to be used at the same time.
That function belongs to GPUArraysCore, which Flux doesn't directly load right now, but NNlib does.
When you say "same switch" do you mean defining something like Flux.throw_sanity_check_errors() which flips the requested precision switch in the OP as well as turning off CUDA.allowscalar? Or do you mean if CUDA.allowscalar is false, then we throw errors for precision? If it's the latter, then that seems wrong to me since scalar indexing and floating point precision are totally unrelated.
I also think it makes sense to a function for enabling each of these checks separately. But having a Flux.enable_all_sanity_check_errors() on top of that is good for discoverability (also since the docstring of that can also point to the relevant sub-functions).
Motivation and description
The warning about wrong precision is very helpful to point at potential performance issues
Flux.jl/src/layers/stateless.jl
Line 60 in 2f19e68
I think that this is the correct default behavior. However, in order to find out where the problem is coming from throwing an error to produce a stacktrace would be very helpful.
Possible Implementation
There could be a Preference or global flag that allows switching errors instead of warning for wrong precision inputs. This would also be made consistent with
CUDA.allowscalar
.The text was updated successfully, but these errors were encountered: