Releases, starting with 9/2/2021, are listed with the most recent release at the top.
Fixed Bugs:
#715 How to implement the following code
API Changes:
Add functional normalizations
Added torch.utils.tensorboard.SummaryWriter. Support for scalars only.
Fixed Bugs:
#694 torch.log10() computes torch.log()
#691 torch.autograd.backward()
#686 torch.nn.functional.Dropout() doesn't have the training argument.
API Changes:
Add repeat_interleave()
Add torch.broadcast_shapes()
Added meshgrid, mT, mH, and H
Added additional distributions.
Add dct and mu-law to torchaudio
Added torchvision -- sigmoid_focal_loss()
Update the arguments of dropout() in Tacotron2
Add static function for all(), any(), tile(), repeat_interleave().
Add an implementation of the ReduceLROnPlateau learning rate scheduler.
Breaking Changes:
This release contains a breaking change__ related to torch.tensor() and torch.from_array() which were not adhering to the semantics of the Pytorch equivalents (torch.from_numpy() in the case of torch.from_array()).
With this change, there will be a number of different APIs to create a tensor form a .NET array. The most significant difference between them is whether the underlying storage is shared, or whether a copy is made. Depending on the size of the input array, copying can take orders of magnitude more time in creation than sharing storage, which is done in constant time (a few μs).
The resulting tensors may be reshaped, but not resized.
// Never copy:
public static Tensor from_array(Array input)
// Copy only if dtype or device arguments require it:
public static Tensor frombuffer(Array input, ScalarType dtype, long count = -1, long offset = 0, bool requiresGrad = false, Device? device = null)
public static Tensor as_tensor(Array input, ScalarType? dtype = null, Device? device = null)
public static Tensor as_tensor(Tensor input, ScalarType? dtype = null, Device? device = null)
// Always copy:
public static Tensor as_tensor(IList<<VARIOUS TYPES>> input, ScalarType? dtype = null, Device? device = null)
public static Tensor tensor(<<VARIOUS TYPES>> input, torch.Device? device = null, bool requiresGrad = false)Fixed Bugs:
#670 Better align methods for creating tensors from .NET arrays with Pytorch APIs. This is the breaking change mentioned earlier.
#679 The default value of onesided or torch.istft() is not aligned with PyTorch
API Changes:
Added torch.nn.init.trunc_normal_
Added index_add, index_copy, index_fill
Added torch.frombuffer()
Added torch.fft.hfft2, hfftn, ihfft2, ihfftn
Adding SequentialLR to the collection of LR schedulers.
Add 'training' flag to functional dropout methods.
Add missing functions to torchaudio.functional
Adding TestOfAttribute to unit tests
This release is made shortly after 0.97.0, since it adresses a serious performance issue when creating large tensors from .NET arrays.
Fixed Bugs:
#670 Tensor allocation insanely slow for from_array()
API Changes:
RNN, LSTM, GRU support PackedSequence
Add element-wise comparison methods of torch class.
Fix clamp and (non)quantile method declarations
Implementing isnan()
Added torchaudio.models.Tacotron2()
Fixed Bugs:
#653:Tensor.to(Tensor) doesn't change dtype of Tensor.
API Changes:
Add ability to load and save TorchScript modules created using Pytorch
Add torch.utils.rnn
Add torchvision.io
Add Tensor.trace() and torch.trace() (unrelated to torch.jit.trace)
Add Tensor.var and Tensor.var_mean
Add torchaudio.datasets.SPEECHCOMMANDS
Add torchaudio.Resample()
Breaking Changes:
This release contains a fix to inadvertent breaking changes in 0.96.7, related to Tensor.str(). This fix is itself breaking, in that it breaks any code that relies on the order of arguments to str() introduced in 0.96.7. However, since the pre-0.96.7 argument order makes more sense, we're taking this hit now rather than keeping the inconvenient order in 0.96.7.
Fixed Bugs:
#618 TorchSharp.Modules.Normal.sample() Expected all tensors [...]
#621 torch.roll missing
#629 Missing dependency in 0.96.7 calling TorchSharp.torchvision.datasets.MNIST
#632 gaussian_nll_loss doesn't work on GPU
API Changes:
Add torchaudio.datasets.YESNO().
Added torch.from_array() API to create a tensor from an arbitry-dimension .NET array.
Added torch.tensor() overloads for most common dimensions of .NET arrays: ndim = [1,2,3,4]
Added the most significant API additions from Pytorch 1.11.
Added juliastr() and npstr().
Added two torchaudio APIs.
Added 'decimals' argument to Tensor.round()
Changed tensor.str() to undo the breaking change in 0.96.7
Added torch.std_mean()
Dependency Changes:
This version integrates with the libtorch 1.11.0 backend. API updates to follow.
API Changes:
Strong name signing of the TorchSharp library to allow loading it in .NET Framework strongly name signed apps.
Added the 'META' device type, which can be used to examine the affect of shape from tensor operations without actually doing any computations.
Added a few methods from the torch.nn.utils namespace.
Add torch.stft() and torch.istft()
Fixed Bugs:
#567 pad missing the choice to fill at start or end
API Changes:
#587 Added the Storage classes, and Tensor.storage()
Added torchvision.models.resnet***() factories
Added torchvision.models.alexnet() factory
Added torchvision.models.vgg*() factories
Added 'skip' list for loading and saving weights.
Added torchvision.models.interception_v3() factory
Added torchvision.models.googlenet() factory
Fixed Bugs:
#582 unbind missing
#592 GRU and Input and hidden tensors are not at the same device,[...]
Fixed Module.Dispose() and Sequential.Dispose() (no issue filed)
Same-day release. The previous release was made without propert testing of the ToString() improvements in a notebook context. It turned out that when the standard Windows line-terminator "\r\n" is used in a VS Code notebook, an extra blank line is created.
This release fixes that by allowing the caller of ToString() to pass in the line terminator string that should be used when formatting the string. This is easily done in the notebook.
In this release, the big change is support for .NET FX 4.7.2 and later.
There are no breaking changes that we are aware of, but see the comment on API Changes below -- backporting code to .NET 4.7 or 4.8, which were not previously supported, may lead to errors in code that uses tensor indexing.
API Changes:
Due to the unavailability of System.Range in .NET FX 4.7, indexing of tensors using the [a..b] syntax is not available. In its place, we have added support for using tuples as index expressions, with the same semantics, except that the "from end" unary operator ^ of the C# range syntax is not available. The tuple syntax is also available for versions of .NET that do support System.Range
A second piece of new functionality was to integrate @dayo05's work on DataLoader into the Examples. A couple of MNIST and CIFAR data sets are now found in torchvision.datasets
A Numpy-style version of ToString() was added to the existing Julia-style, and the argument to the verbose ToString() was changed from 'Boolean' to an enumeration.
A number of the "bugs" listed below represent missing APIs.
Fixed Bugs:
#519 Multiprocessing dataloader support
#529 pin_memory missing
#545 Implement FractionalMaxPool{23}d
#554 Implement MaxUnpool{123}d
#555 Implement LPPool{12}d
#556 Implement missing activation modules
#559 Implement miscellaneous missing layers.
#564 torch.Tensor.tolist
#566 Implicit conversion of scalars to tensors
#576 load_state_dict functionality
API Changes:
NOTE: This release contains breaking changes.
The APIs to create optimizers all take 'parameters()' as well as 'named_parameters()' now.
Support for parameter groups in most optimizers.
Support for parameter groups in LR schedulers.
Fixed Bugs:
#495 Add support for OptimizerParamGroup
#509 Tensor.conj() not implemented
#515 what's reason for making register_module internal?
#516 AdamW bug on v0.96.0
#521 Can't set Tensor slice using indexing
#525 LSTM's forward function not work with null hidden and cell state
#532 Why does storing module layers in arrays break the learning process?
NOT RELEASED
API Changes:
Fixed Bugs:
Using libtorch CPU packages from F# Interactive required explicit native loads
#510 Module.Load throws Mismatched state_dict sizes exception on BatchNorm1d
API Changes:
NOTE: This release contains breaking changes.
'Module.named_parameters()', 'parameters()', 'named_modules()', 'named_children()' all return IEnumerable instances instead of arrays.
Adding weight and bias properties to the RNN modules.
Lower-cased names: Module.Train --> Module.train and Module.Eval --> Module.eval
Fixed Bugs:
#496 Wrong output shape of torch.nn.Conv2d with 2d stride overload
#499 Setting Linear.weight is not reflected in 'parameters()'
#500 BatchNorm1d throws exception during eval with batch size of 1
API Changes:
Added OneCycleLR and CyclicLR schedulers
Added DisposeScopeManager and torch.NewDisposeScope() to facilitate a new solution for managing disposing of tensors with fewer usings.
Added Tensor.set_()
Added 'copy' argument to Tensor.to()
NOTES:
The 'Weight' and 'Bias' properties on some modules have been renamed 'weight' and 'bias'.
The 'LRScheduler.LearningRate' property has been removed. To log the learning rate, get it from the optimizer that is in use.
Fixed Bugs:
#476 BatchNorm does not expose bias,weight,running_mean,running_var
#475 Loading Module that's on CUDA
#372 Module.save moves Module to CPU
#468 How to set Conv2d kernel_size=(2,300)
#450 Smoother disposing
API Changes:
The previously unused Tensor.free() method was renamed 'DecoupleFromNativeHandle()' and is meant to be used in native interop scenarios.
Tensor.Handle will now validate that the internal handle is not 'Zero', and throw an exception when it is. This will catch situations where a disposed tensor is accessed.
Fixed Bugs:
There were a number of functions in torchvision, as well as a number of optimizers, that did not properly dispose of temporary and intermediate tensor values, leading to "memory leaks" in the absence of explicit GC.Collect() calls.
A couple of randint() overloads caused infinite recursion, crashing the process.
API Changes:
Added a Sequential factory method to create Sequential from a list of anonymous submodules.
Added TotalCount and PeakCount static properties to Tensor, useful for diagnostic purposes.
Fixed Bugs:
#432 Sequential does not dispose of intermediary tensors.
This version integrates with LibTorch 1.10.0.
API Changes:
Added a 'strict' option to Module.load().
See tracking issue #416 for a list of new 1.10.0 APIs. dotnet#416
Fixed Bugs:
#414 LRScheduler -- not calling the optimizer to step() [The original, closing fix was actually incorrect, but was then fixed again.]
API Changes:
Added the NAdam and RAdam optimizers.
Added several missing and new learning rate schedulers.
Fixed Bugs:
#413 Random Distributions Should Take a Generator Argument
#414 LRScheduler -- not calling the optimizer to step()
API Changes:
Added Module.Create() to create a model and load weights.
Fixed Bugs:
#407 rand() and randn() must check that the data type is floating-point.
#410 Support for passing random number generators to rand(), randn(), and randint()
API Changes:
Added some overloads to make F# usage more convenient.
Added convenience overloads to a number of random distribution factories.
Added '_' to the torch.nn.init functions. They overwrite the input tensor, so they should have the in-place indicator.
Fixed Bugs:
#399 Data() returns span that must be indexed using strides.
This was a major bug, affecting any code that pulled data out of a tensor view.
API Changes:
Tensor.Data() -> Tensor.data()
Tensor.DataItem() -> Tensor.item()
Tensor.Bytes() -> Tensor.bytes
Tensor.SetBytes() -> Tensor.bytes
This release introduces a couple of new NuGet packages, which bundle the native libraries that you need:
TorchSharp-cpu
TorchSharp-cuda-linux
TorchSharp-cuda-windows
With this release, the native libtorch package version was updated to 1.9.0.11, and that required rebuilding this package.
With this release, releases will have explicit control over the patch version number.
Fixed Bugs:
Fixed incorrectly implemented Module APIs related to parameter / module registration.
Changed Module.state_dict() and Module.load() to 'virtual,' so that saving and restoring state may be customized.
#353 Missing torch.minimum (with an alternative raising exception)
#327 Tensor.Data should do a type check
#358 Implement ModuleList / ModuleDict / Parameter / ParameterList / ParameterDict
API Changes:
Removed the type-named tensor factories, such as 'Int32Tensor.rand(),' etc.
Documentation Changes:
Added an article on creating custom modules.
This was the first release since moving TorchSharp to the .NET Foundation organization. Most of the new functionality is related to continuing the API changes that were started in the previous release, and fixing some bugs.
Fixed Bugs:
#318 A few inconsistencies with the new naming
Added Features:
''' torch.nn.MultiHeadAttention torch.linalg.cond torch.linalg.cholesky_ex torch.linalg.inv_ex torch.amax/amin torch.matrix_exp torch.distributions.* (about half the namespace) '''
API Changes:
CustomModule removed, its APIs moved to Module.