Epic: Democratize Machine Learning for .NET developers

[briacht]

This is a draft/proposed Epic and not committed for .NET 6.0 yet.

Why

The ML.NET framework and associated Visual Studio and command-line tooling democratizes machine learning by enabling .NET developers to leverage ML in their .NET applications without having to leave the .NET ecosystem and without requiring a background in data science or ML.

With ML.NET, we aspire to keep developers in the .NET ecosystem, not entice data scientists to move away from Python or other languages and tech stacks commonly associated with machine learning. We must offer first-class support for ML in .NET so that .NET developers do not need to find alternative ways to integrate machine learning with other languages which brings about its own set of challenges and maintenance costs to users.

Summary

Currently, only some ML.NET scenarios have AutoML implementations to automate the machine learning process for developers. However, to truly complete the .NET developer-friendly ML experience, we must achieve scenario parity by adding AutoML support to the rest of the ML.NET scenarios so that developers are not left on their own to figure out how to build the best machine learning models for these scenarios.

User Stories

P0:

• Users can use ML.NET in all future .NET versions Users can use ML.NET in all future .NET versions #1070

P1:

• Users can use data from Azure when training in the cloud Users can use data from Azure when training in the cloud #556
• Users have all the ML scenarios they need through ML.NET and AutoML Additional Scenarios: Users have all the ML scenarios they need through ML.NET and AutoML #1073
• Users can do inference with ML.NET models anywhere that .NET runs Cross Platform: Users can do inference on ML.NET models anywhere that .NET runs #1068
• Internal users can easily migrate from TLC to ML.NET Internal users can easily migrate from TLC to ML.NET #1072
• All ML.NET transforms are exportable to ONNX
• All ONNX models consumed with ML.NET have comparable performance to inferencing with ONNX Runtime
• Users can easily deploy, productionize, and manage their models Users can easily deploy, productionize, and manage their models #1048
• Users can visualize model explainability after training their model so that developers can easily and responsibly interpret their models Model Explainability: Users can visualize model explainability after training their model so that developers can easily and responsibly interpret their models #615

P2:

• Users aren't dropped off a cliff after the getting started experience and have a guided path to more advanced data science tools and frameworks Users aren't dropped off a cliff after the getting started experience and have a guided path to more advanced data science tools and frameworks  #1071
• Users can leverage Torch models in their ML.NET pipelines for building neural networks Deep Learning: Users can leverage Torch models in their ML.NET pipelines for building neural networks #1069

Completed

• Users have options for more advanced data loading options in case tooling processes datasets incorrectly Users have options for more advanced data loading options in case tooling processes datasets incorrectly #642

[saint4eva]

With ML.NET, we aspire to keep developers in the .NET ecosystem, not entice data scientists to move away from Python or other languages and tech stacks commonly associated with machine learning

@briacht Since the purpose is to provide first class support to .NET developers first, it would be nice to stick to the C#/ .NET naming convention and idiosyncrasies.