MLflow 2.16.2

MLflow 2.16.2 includes several major features and improvements

Bug fixes:

• [Models] Revert "Update Dependency Extraction for Agents (#13105)" (#13155, @aravind-segu)

MLflow 2.16.1

MLflow 2.16.1 is a patch release that includes some minor feature improvements and addresses several bug fixes.

Features:

• [Tracing] Add Support for an Open Telemetry compatible exporter to configure external sinks for MLflow traces (#13118, @B-Step62)
• [Model Registry, AWS] Add support for utilizing AWS KMS-based encryption for the MLflow Model Registry (#12495, @artjen)
• [Model Registry] Add support for using the OSS Unity Catalog server as a Model Registry (#13034, #13065, #13066, @rohitarun-db)
• [Models] Introduce path-based transformers logging to reduce memory requirements for saving large transformers models (#13070, @B-Step62)

Bug fixes:

• [Tracking] Fix a data payload size issue with Model.get_tags_dict by eliminating the return of the internally-used config field (#13086, @harshilprajapati96)
• [Models] Fix an issue with LangChain Agents where sub-dependencies were not being properly extracted (#13105, @aravind-segu)
• [Tracking] Fix an issue where the wrong checkpoint for the current best model in auto checkpointing was being selected (#12981, @hareeen)
• [Tracking] Fix an issue where local timezones for trace initialization were not being taken into account in AutoGen tracing (#13047, @B-Step62)

Documentation updates:

• [Docs] Added RunLLM chat widget to MLflow's documentation site (#13123, @likawind)

For a comprehensive list of changes, see the release change log, and check out the latest documentation on mlflow.org.

MLflow 2.16.0

We are excited to announce the release of MLflow 2.16.0. This release includes many major features and improvements!

Major features:

• LlamaIndex Enhancements🦙 - to provide additional flexibility to the LlamaIndex integration, we now have support for the models-from-code functionality for logging, extended engine-based logging, and broadened support for external vector stores.

• LangGraph Support - We've expanded the LangChain integration to support the agent framework LangGraph. With tracing and support for logging using the models-from-code feature, creating and storing agent applications has never been easier!

• AutoGen Tracing - Full automatic support for tracing multi-turn agent applications built with Microsoft's AutoGen framework is now available in MLflow. Enabling autologging via mlflow.autogen.autolog() will instrument your agents built with AutoGen.

• Plugin support for AI Gateway - You can now define your own provider interfaces that will work with MLflow's AI Gateway (also known as the MLflow Deployments Server). Creating an installable provider definition will allow you to connect the Gateway server to any GenAI service of your choosing.

Features:

• [UI] Add updated deployment usage examples to the MLflow artifact viewer (#13024, @serena-ruan, @daniellok-db)
• [Models] Support logging LangGraph applications via the models-from-code feature (#12996, @B-Step62)
• [Models] Extend automatic authorization pass-through support for Langgraph agents (#13001, @aravind-segu)
• [Models] Expand the support for LangChain application logging to include UCFunctionToolkit dependencies (#12966, @aravind-segu)
• [Models] Support saving LlamaIndex engine directly via the models-from-code feature (#12978, @B-Step62)
• [Models] Support models-from-code within the LlamaIndex flavor (#12944, @B-Step62)
• [Models] Remove the data structure conversion of input examples to ensure enhanced compatibility with inference signatures (#12782, @serena-ruan)
• [Models] Add the ability to retrieve the underlying model object from within pyfunc model wrappers (#12814, @serena-ruan)
• [Models] Add spark vector UDT type support for model signatures (#12758, @WeichenXu123)
• [Tracing] Add tracing support for AutoGen (#12913, @B-Step62)
• [Tracing] Reduce the latency overhead for tracing (#12885, @B-Step62)
• [Tracing] Add Async support for the trace decorator (#12877, @MPKonst)
• [Deployments] Introduce a plugin provider system to the AI Gateway (Deployments Server) (#12611, @gabrielfu)
• [Projects] Add support for parameter submission to MLflow Projects run in Databricks (#12854, @WeichenXu123)
• [Model Registry] Introduce support for Open Source Unity Catalog as a model registry service (#12888, @artjen)

Bug fixes:

• [Tracking] Reduce the contents of the model-history tag to only essential fields (#12983, @harshilprajapati96)
• [Models] Fix the behavior of defining the device to utilize when loading transformers models (#12977, @serena-ruan)
• [Models] Fix evaluate behavior for LlamaIndex (#12976, @B-Step62)
• [Models] Replace pkg_resources with importlib.metadata due to package deprecation (#12853, @harupy)
• [Tracking] Fix error handling for OpenAI autolog tracing (#12841, @B-Step62)
• [Tracking] Fix a condition where a deadlock can occur when connecting to an SFTP artifact store (#12938, @WeichenXu123)
• [Tracking] Fix an issue where code_paths dependencies were not properly initialized within the system path for LangChain models (#12923, @harshilprajapati96)
• [Tracking] Fix a type error for metrics value logging (#12876, @beomsun0829)
• [Tracking] Properly catch NVML errors when collecting GPU metrics (#12903, @chenmoneygithub)
• [Deployments] Improve Gateway schema support for the OpenAI provider (#12781, @danilopeixoto)
• [Model Registry] Fix deletion of artifacts when downloading from a non-standard DBFS location during UC model registration (#12821, @smurching)

Documentation updates:

• [Docs] Add documentation guides for LangGraph support (#13025, @BenWilson2)
• [Docs] Add additional documentation for models from code feature (#12936, @BenWilson2)
• [Docs] Add documentation for model serving input payloads (#12848, @serena-ruan)

Small bug fixes and documentation updates:

#12987, #12991, #12974, #12975, #12932, #12893, #12851, #12793, @serena-ruan; #13019, #13013, @aravind-segu; #12943, @piyushdaftary; #12906, #12898, #12757, #12750, #12727, @daniellok-db; #12995, #12985, #12964, #12962, #12960, #12953, #12951, #12937, #12914, #12929, #12907, #12897, #12880, #12865, #12864, #12862, #12850, #12847, #12833, #12835, #12826, #12824, #12795, #12796, @harupy; #12592, @antbbn; #12993, #12984, #12899, #12745, @BenWilson2; #12965, @nojaf; #12968, @bbqiu; #12956, @mickvangelderen; #12939, #12950, #12915, #12931, #12919, #12889, #12849, #12794, #12779, #12836, #12823, #12737, @B-Step62; #12903, @chenmoneygithub; #12905, @Atry; #12884, #12858, #12807, #12800, #10874, @WeichenXu123; #12342, @kriscon-db; #12742, @edwardfeng-db

MLflow 2.15.1

MLflow 2.15.1 is a patch release that addresses several bug fixes.

Bug fixes:

• [Tracking] Fix silent disabling of LangChain autologging for LangChain >= 0.2.10. (#12779, @B-Step62)
• [Tracking] Fix mlflow.evaluate crash on binary classification with data subset only contains single class (#12825, @serena-ruan)
• [Tracking] Fix incompatibility of MLflow Tracing with LlamaIndex >= 0.10.61 (#12890, @B-Step62)
• [Tracking] Record exceptions in OpenAI autolog tracing (#12841, @B-Step62)
• [Tracking] Fix regression of connecting to MLflow tracking server on other Databricks workspace (#12861, @WeichenXu123)
• [UI] Fix refresh button for model metrics on Experiment and Run pages (#12869, @beomsun0829)

Documentation updates:

• [Docs] Update doc for Spark ML vector type (#12827, @WeichenXu123)
  Small bug fixes and documentation updates:

#12823, #12860, #12844, #12843, @B-Step62; #12863, #12828, @harupy; #12845, @djliden; #12820, @annzhang-db; #12831, #12873, @chenmoneygithub

MLflow 2.15.0

MLflow 2.15.0 includes many major features and improvements:

Major features:

• 🦙 LlamaIndex Flavor - MLflow now offers a native integration with LlamaIndex, one of the most popular libraries for building GenAI apps centered around custom data. This integration allows you to log LlamaIndex indices within MLflow, allowing for the loading and deployment of your indexed data for inference tasks with different engine types. MLflow also provides comprehensive tracing support for LlamaIndex operations, offering unprecedented transparency into complex queries. Check out the MLflow LlamaIndex documentation to get started! (#12633, @michael-berk, @B-Step62)
• 🔍 OpenAI Tracing - We've enhanced our OpenAI integration with a new tracing feature that works seamlessly with MLflow OpenAI autologging. You can now enable tracing of their OpenAI API usage with a single mlflow.openai.autolog() call, thereby MLflow will automatically log valuable metadata such as token usage and a history of your interactions, providing deeper insights into your OpenAI-powered applications. To start exploring this new capability, please check out the tracing documentation! (#12267, @gabrielfu)
• ✅ Enhanced Model Deployment Validation - To improve the reliability of model deployments, MLflow has added a new method to validate your model before deploying it to an inference endpoint. This feature helps to eliminate typical errors in input and output handling, streamlining the process of model deployment and increasing confidence in your deployed models. By catching potential issues early, you can ensure a smoother transition from development to production. (#12710, @serena-ruan)
• 📊 Custom Metrics Definition Recording for Eval - We've strengthened the flexibility of defining custom metrics for model evaluation by automatically logging and versioning metrics definitions, including models used as judges and prompt templates. With this new capability, you can ensure reproducibility of evaluations across different runs and easily reuse evaluation setups for consistency, facilitating more meaningful comparisons between different models or versions. (#12487, #12509, @xq-yin)
• 🔐 Databricks SDK Integration - MLflow's interaction with Databricks endpoints has been fully migrated to use the Databricks SDK. This change brings more robust and reliable connections between MLflow and Databricks, and access to the latest Databricks features and capabilities. We mark the legacy databricks-cli support as deprecated and will remove in the future release. (#12313, @WeichenXu123)
• 💥 Spark VectorUDT Support - MLflow's Model Signature framework now supports Spark Vector UDT (User Defined Type), enabling logging and deployment of models using Spark VectorUDT with robust type validation. (#12758, @WeichenXu123)

Other Notable Changes

Features:

• [Tracking] Add parent_id as a parameter to the start_run fluent API for alternative control flows (#12721, @Flametaa)
• [Tracking] Add U2M authentication support for connecting to Databricks from MLflow (#12713, @WeichenXu123)
• [Tracking] Support deleting remote artifacts with mlflow gc (#12451, @M4nouel)
• [Tracing] Traces can now be deleted conveniently via UI from the Traces tab in the experiments page (#12641, @daniellok-db)
• [Models] Introduce additional parameters for the ChatModel interface for GenAI flavors (#12612, @WeichenXu123)
• [Models] [Transformers] Support input images encoded with b64.encodebytes (#12087, @MadhuM02)
• [Models Registry] Add support for AWS KMS encryption for the Unity Catalog model registry integration (#12495, @artjen)
• [Models] Fix MLflow Dataset hashing logic for Pandas dataframe to use iloc for accessing rows (#12410, @julcsii)
• [Models Registry] Support presigned urls without headers for artifact location (#12349, @artjen)
• [UI] The experiments page in the MLflow UI has an updated look, and comes with some performance optimizations for line charts (#12641, @hubertzub-db)
• [UI] Line charts can now be configured to ignore outliers in the data (#12641, @daniellok-db)
• [UI] Creating compatibility with Kubeflow Dashboard UI (#12663, @cgilviadee)
• [UI] Add a new section to the artifact page in the Tracking UI, which shows code snippet to validate model input format before deployment (#12729, @serena-ruan)

Bug fixes:

• [Tracking] Fix the model construction bug in MLflow SHAP evaluation for scikit-learn model (#12599, @serena-ruan)
• [Tracking] File store get_experiment_by_name returns all stage experiments (#12788, @serena-ruan)
• [Tracking] Fix Langchain callback injection logic for async/streaming request (#12773, @B-Step62)
• [Tracing] [OpenAI] Fix stream tracing for OpenAI to record the correct chunk structure (#12629, @BenWilson2)
• [Tracing] [LangChain] Fix LangChain tracing bug for .batch call due to thread unsafety (#12701, @B-Step62)
• [Tracing] [LangChain] Fix nested trace issue in LangChain tracing. (#12705, @B-Step62)
• [Tracing] Prevent intervention between MLflow Tracing and other OpenTelemetry-based libraries (#12457, @B-Step62)
• [Models] Fix log_model issue in MLflow >= 2.13 that causes databricks DLT py4j service crashing (#12514, @WeichenXu123)
• [Models] [Transformers] Fix batch inference issue for Transformers Whisper model (#12575, @B-Step62)
• [Models] [LangChain] Fix the empty generator issue in predict_stream for AgentExecutor and other non-Runnable chains (#12518, @B-Step62)
• [Scoring] Fix Spark UDF permission denied issue in Databricks runtime (#12774, @WeichenXu123)

Documentation updates:

• Add documentation on authentication for Databricks UC Model Registry (#12552, @WeichenXu123)
• Adding model-from-code documentation for LangChain and Pyfunc (#12325, #12336, @sunishsheth2009)
• Add FAQ entry for viewing trace exceptions (#12309, @BenWilson2)
• Add note about fork vs spawn method when using multiprocessing for parallel runs (#12337, @B-Step62)
• Add example usage of extract_fields for mlflow.search_traces (#12319, @xq-yin)
• Replace GPT-3.5-turbo with GPT-4o-mini (#12740, #12746, @Acksout)

Small bug fixes and documentation updates:

#12727, #12709, #12685, #12667, #12673, #12602, #12601, #12655, #12641, #12635, #12634, #12584, #12428, #12388, #12352, #12298, #12750, #12727, #12757, @daniellok-db; #12726, #12733, #12691, #12622, #12579, #12581, #12285, #12311, #12357, #12339, #12338, #12705, #12797, #12787, #12784, #12771, #12737, @B-Step62; #12715, @hubertzub-db; #12722, #12804, @annzhang-db; #12676, #12680, #12665, #12664, #12671, #12651, #12649, #12647, #12637, #12632, #12603, #12343, #12328,...

MLflow 2.15.0rc0

MLflow 2.15.0rc0 is a release candidate to provide users with an early preview of the major features and changes coming in the stable release. We encourage you to try out these new features, and if you encounter any issues or have suggestions, please open an issue on our GitHub repository!

Try It Out

To install the release candidate, run the following command:

pip install mlflow==2.15.0rc0

Please note that the release candidate may contain bugs or incomplete features. We encourage you to test it in a non-production environment and provide feedback on any issues you encounter.

MLflow 2.14.3

MLflow 2.14.3 is a patch release that addresses bug fixes and additional documentation for released features

Features:

• [Model Registry] Add support for server-side encryption when uploading files to AWS S3 (#12495, @artjen)

Bug fixes:

• [Models] Fix stream trace logging with the OpenAI autologging implementation to record the correct chunk structure (#12629, @BenWilson2)
• [Models] Fix batch inference behavior for Whisper-based translation models to allow for multiple audio file inputs (#12575, @B-Step62)

Documentation updates:

• [Docs] Add documentation for OpenAI autologging (#12608, @BenWilson2)

Small bug fixes and documentation updates:

#12556, #12628, @B-Step62; #12582, #12560, @harupy; #12553, @nojaf

MLflow 2.14.2

MLflow 2.14.2 is a patch release that includes several important bug fixes and documentation enhancements.

Bug fixes:

• [Models] Fix an issue with requirements inference error handling when disabling the default warning-only behavior (#12547, @B-Step62)
• [Models] Fix dependency inference issues with Transformers models saved with the unified API llm/v1/xxx task definitions. (#12551, @B-Step62)
• [Models / Databricks] Fix an issue with MLlfow log_model introduced in MLflow 2.13.0 that causes Databricks DLT service to crash in some situations (#12514, @WeichenXu123)
• [Models] Fix an output data structure issue with the predict_stream implementation for LangChain AgentExecutor and other non-Runnable chains (#12518, @B-Step62)
• [Tracking] Fix an issue with the predict_proba inference method in the sklearn flavor when loading an sklearn pipeline object as pyfunc (#12554, @WeichenXu123)
• [Tracking] Fix an issue with the Tracing implementation where other services usage of OpenTelemetry would activate MLflow tracing and cause errors (#12457, @B-Step62)
• [Tracking / Databricks] Correct an issue when running dependency inference in Databricks that can cause duplicate dependency entries to be logged (#12493, @sunishsheth2009)

Documentation updates:

• [Docs] Add documentation and guides for the MLflow tracing schema (#12521, @BenWilson2)

Small bug fixes and documentation updates:

#12311, #12285, #12535, #12543, #12320, #12444, @B-Step62; #12310, #12340, @serena-ruan; #12409, #12432, #12471, #12497, #12499, @harupy; #12555, @nojaf; #12472, #12431, @xq-yin; #12530, #12529, #12528, #12527, #12526, #12524, #12531, #12523, #12525, #12522, @dbczumar; #12483, @jsuchome; #12465, #12441, @BenWilson2; #12450, @StarryZhang-whu

MLflow 2.14.1

MLflow 2.14.1 is a patch release that contains several bug fixes and documentation improvements

Bug fixes:

• [Models] Fix params and model_config handling for llm/v1/xxx Transformers model (#12401, @B-Step62)
• [UI] Fix dark mode user preference (#12386, @daniellok-db)
• [Docker] Fix docker image failing to build with install_mlflow=False (#12388, @daniellok-db)

Documentation updates:

• [Docs] Add link to langchain autologging page in doc (#12398, @xq-yin)
• [Docs] Add documentation for Models from Code (#12381, @BenWilson2)

Small bug fixes and documentation updates:

#12415, #12396, #12394, @harupy; #12403, #12382, @BenWilson2; #12397, @B-Step62

v2.14.0

2.14.0 (2024-06-17)

MLflow 2.14.0 includes several major features and improvements that we're very excited to announce!

Major features:

• MLflow Tracing: Tracing is powerful tool designed to enhance your ability to monitor, analyze, and debug GenAI applications by allowing you to inspect the intermediate outputs generated as your application handles a request. This update comes with an automatic LangChain integration to make it as easy as possible to get started, but we've also implemented high-level fluent APIs, and low-level client APIs for users who want more control over their trace instrumentation. For more information, check out the guide in our docs!
• Unity Catalog Integration: The MLflow Deployments server now has an integration with Unity Catalog, allowing you to leverage registered functions as tools for enhancing your chat application. For more information, check out this guide!
• OpenAI Autologging: Autologging support has now been added for the OpenAI model flavor. With this feature, MLflow will automatically log a model upon calling the OpenAI API. Each time a request is made, the inputs and outputs will be logged as artifacts. Check out the guide for more information!

Other Notable Features:

• [Models] Support input images encoded with b64.encodebytes (#12087, @MadhuM02)
• [Tracking] Support async logging per X seconds (#12324, @chenmoneygithub)
• [Tracking] Provide a way to set urllib's connection number and max size (#12227, @chenmoneygithub)
• [Projects] Make MLflow project runner supporting submit spark job to databricks runtime >= 13 (#12139, @WeichenXu123)
• [UI] Add the "description" column to the runs table (#11996, @zhouyou9505)

Bug fixes:

• [Model Registry] Handle no headers presigned url (#12349, @artjen)
• [Models] Fix docstring order for ChatResponse class and make object field immutable (#12305, @xq-yin)
• [Databricks] Fix root user checking in get_databricks_nfs_temp_dir and get_databricks_local_temp_dir (#12186, @WeichenXu123)
• [Tracking] fix _init_server process terminate hang (#12076, @zhouyou9505)
• [Scoring] Fix MLflow model container and slow test CI failure (#12042, @WeichenXu123)

Documentation updates:

• [Docs] Enhance documentation for autologging supported libraries (#12356, @xq-yin)
• [Tracking, Docs] Adding Langchain as a code example and doc string (#12325, @sunishsheth2009)
• [Tracking, Docs] Adding Pyfunc as a code example and doc string (#12336, @sunishsheth2009)
• [Docs] Add FAQ entry for viewing trace exceptions in Docs (#12309, @BenWilson2)
• [Docs] Add note about 'fork' vs 'spawn' method when using multiprocessing for parallel runs (#12337, @B-Step62)
• [Docs] Fix type error in tracing example for function wrapping (#12338, @B-Step62)
• [Docs] Add example usage of "extract_fields" for mlflow.search_traces in documentation (#12319, @xq-yin)
• [Docs] Update LangChain Autologging docs (#12306, @B-Step62)
• [Docs] Add Tracing documentation (#12191, @BenWilson2)

Small bug fixes and documentation updates:

#12359, #12308, #12350, #12284, #12345, #12316, #12287, #12303, #12291, #12288, #12265, #12170, #12248, #12263, #12249, #12251, #12239, #12241, #12240, #12235, #12242, #12172, #12215, #12228, #12216, #12164, #12225, #12203, #12181, #12198, #12195, #12192, #12146, #12171, #12163, #12166, #12124, #12106, #12113, #12112, #12074, #12077, #12058, @harupy; #12355, #12326, #12114, #12343, #12328, #12327, #12340, #12286, #12310, #12200, #12209, #12189, #12194, #12201, #12196, #12174, #12107, @serena-ruan; #12364, #12352, #12354, #12353, #12351, #12298, #12297, #12220, #12155, @daniellok-db; #12311, #12357, #12346, #12312, #12339, #12281, #12283, #12282, #12268, #12236, #12247, #12199, #12232, #12233, #12221, #12229, #12207, #12212, #12193, #12167, #12137, #12147, #12148, #12138, #12127, #12065, @B-Step62; #12289, #12253, #12330 @xq-yin; #11771, @lababidi; #12280, #12275, @BenWilson2; #12246, #12244, #12211, #12066, #12061, @WeichenXu123; #12278, @sunishsheth2009; #12136, @kriscon-db; #11911, @jessechancy; #12169, @hubertzub-db