Path to 1.0

[dahong67]

Issue to track things that should be done before we tag 1.0:

• Use autodiff to compute derivative of loss by default #5
• Allow user to specify loss function using LossFunctions.jl #10
• Change API to use AbstractLoss types and handle custom case with new type #12
• Setup more systematic handling of constraints #24
• Setup support for multiple algorithms and add ALS #21
• Add the remaining losses #15
• Remove old method signature #23
• Improve module/file organization #27
• Make init one of the arguments #48
• Add grad_U, grad_λ!, and grad_λ functions
• Tests for all supported loss functions in LossFunctions.jl
• Use StableRNGs in testing/benchmarking - remove use of Random.seed!
• Make sure boolean/integer-valued tensors are handled correctly (useful for binary/count data)
• Check LBFGS options against what we used in TensorToolbox
• Faster MTTKRPS #17
• Support CUDA arrays #36
• Add ADAM method
• Support SparseTensors.jl
• Check feature-parity with our MATLAB implementation in TensorToolbox
• Add precompilation block
• Better tests (e.g., reference test against TensorToolbox, test recovery, etc.)
• Implement additional methods for CPD #16
• Complete the documentation (introduction to CP and GCP, examples, tutorial) #7
• Properly handle AbstractArrays including OffsetArrays
• Create CONTRIBUTING.md (e.g., like https://github.com/Ferrite-FEM/Ferrite.jl/blob/master/CONTRIBUTING.md#documentation)
• Use Base.Fix in place of anonymous functions (ref: Create Base.Fix as general Fix1/Fix2 for partially-applied functions JuliaLang/julia#54653)
• [] Mark relevant api public: https://julialang.org/blog/2024/10/julia-1.11-highlights/#new_public_keyword

Other ideas:

• more careful consideration of types for init
• reduce allocs by adding buffer arg to mttkrps! and grad_U!
• consider moving default functions to corresponding modules, e.g., default_constraints(...) -> GCPConstraints.default(...)
• consider renaming losses to drop "Loss" suffix
• Support computation of non-Float64 decompositions (GPUArrays? Units?)
• Improvements to benchmarking mentioned in Add Benchmarks #30 (comment)
• Audit/optimize memory usage (avoid unnecessary allocations, etc.)
• Create / use AbstractCPD type and consider other representations
• Create abstract TensorFactorization type and define an interface
• Better / more complete implementation of getindex for CPD
• See if there are cases where it is faster to compute the MTTKRP column by column (to avoid forming full Khatri-Rao products which could be expensive when the rank is large)
• Improve docstrings for mttkrp and khatrirao, add more discussion / explanations / teaching (what exactly do they compute? what are some of the techniques used to make the implementations efficient?)
• Add tests for mttkrp and khatrirao specifically
• Have multiple algorithms for mttkrp and khatrirao (including old/simple/straightforward/naive implementations) and possibly put them in a separate TensorKernels module
• Better to store factor matrices by component rather than by mode?
• Better error messages!
• Implement iterative algorithms as iterators, see: https://iterativesolvers.julialinearalgebra.org/stable/iterators/
• Better default initialization - should probably depend on the loss/constraints/algorithm
• Use https://github.com/ericphanson/Asciicast.jl to make docs more animated
• convert methods (cf: https://github.com/JuliaLang/julia/blob/0e6d7972c62eb1dedaf00ea9b966a0060e2effb8/stdlib/LinearAlgebra/src/factorization.jl#L101-L104)
• faster version of Array (split into product of two khatri-rao's)
• more memory-efficient version of permutecomps! defined in terms of swaps, analogous to: https://github.com/JuliaLang/julia/blob/99d1d6723d72d43074b5d2765e63d08439989e8e/base/combinatorics.jl#L66-L95
• variant of permutecomps or sortcomps that uses a reference tensor as in: http://tensorly.org/stable/modules/generated/tensorly.cp_tensor.cp_permute_factors.html
• SubCPD type that provides a view into a CPD. Note: the CPD type actually already supports "views" in a sense since the λandUfields can themselves beSubArray`s, but this could be a bit confusing for new users. Ref: https://github.com/JuliaLang/julia/blob/48d4fd48430af58502699fdf3504b90589df3852/base/subarray.jl
• add getλ and getU functions. Ref: https://docs.julialang.org/en/v1/manual/style-guide/#Prefer-exported-methods-over-direct-field-access
• extension for itensors (https://github.com/ITensor/ITensors.jl)