You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I know this could be workaround by not using Tuple, but I guess in most cases this is not a bottleneck and it will can be generated by other things (since it's just a type conversion) so #4 might not be helpful in such case. I think in practice, we could white list some functions that will use dynamic realloc but usually not performance/memory bottleneck. I think we did this in Alloc.jl before: https://github.com/FluxML/Alloc.jl/pull/1/files#diff-c678e3660b372f14911e30988c8632ceR45
This will also partly workaround #2 I think and for copy and similar I found it could be worth to handle them explicitly since there might be some performance issue (might be related to #2 ).
The text was updated successfully, but these errors were encountered:
Glad to see this is implemented with Cassette, I never had time to actually finish it.
I'm trying this with Zygote on my own machine learning models and hit some error, e.g
This is because there is a part of the backward rules make use of
Tuple(::Array)
to convert gradient of aTuple
back, which looks likeI know this could be workaround by not using
Tuple
, but I guess in most cases this is not a bottleneck and it will can be generated by other things (since it's just a type conversion) so #4 might not be helpful in such case. I think in practice, we could white list some functions that will use dynamic realloc but usually not performance/memory bottleneck. I think we did this in Alloc.jl before: https://github.com/FluxML/Alloc.jl/pull/1/files#diff-c678e3660b372f14911e30988c8632ceR45This will also partly workaround #2 I think and for
copy
andsimilar
I found it could be worth to handle them explicitly since there might be some performance issue (might be related to #2 ).The text was updated successfully, but these errors were encountered: