Skip to content

Commit

Permalink
[DOCS] Add ml batch inference tutorials (#2057)
Browse files Browse the repository at this point in the history
Co-authored-by: Jay Chia <[email protected]@users.noreply.github.com>
  • Loading branch information
jaychia and Jay Chia authored Apr 3, 2024
1 parent 8023c2e commit 3ef6616
Show file tree
Hide file tree
Showing 5 changed files with 1,516 additions and 2 deletions.
2 changes: 0 additions & 2 deletions daft/table/table_io.py
Original file line number Diff line number Diff line change
Expand Up @@ -414,8 +414,6 @@ def write_tabular(
partition_null_fallback: str = "__HIVE_DEFAULT_PARTITION__",
) -> MicroPartition:

pass

[resolved_path], fs = _resolve_paths_and_filesystem(path, io_config=io_config)
if isinstance(path, pathlib.Path):
path_str = str(path)
Expand Down
12 changes: 12 additions & 0 deletions docs/source/user_guide/tutorials.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,18 @@ Generate images from text prompts using a deep learning model (Mini DALL-E) and
`Run this tutorial on Google Colab <https://colab.research.google.com/github/Eventual-Inc/Daft/blob/main/tutorials/text_to_image/text_to_image_generation.ipynb>`__


.. These can't be run because DeltaLake can't be accessed in anonymous mode from Google Colab
.. ML model batch inference/training on a Data Catalog
.. ---------------------------------------------------
.. Run ML models or train them on data in your data catalog (e.g. Apache Iceberg, DeltaLake or Hudi)
.. 1. `Local batch inference <https://colab.research.google.com/github/Eventual-Inc/Daft/blob/main/tutorials/delta_lake/1-local-image-batch-inference.ipynb>`__
.. 1. `Distributed batch inference <https://colab.research.google.com/github/Eventual-Inc/Daft/blob/main/tutorials/delta_lake/2-distributed-batch-inferece.ipynb>`__
.. 1. `Single-node Pytorch model training <https://colab.research.google.com/github/Eventual-Inc/Daft/blob/main/tutorials/delta_lake/3-pytorch-ray-single-node-training.ipynb>`__
.. Other ideas:
.. Scaling up in the cloud with Ray **[Coming Soon]**
.. Building a HTTP service **[Coming Soon]**
Expand Down
Loading

0 comments on commit 3ef6616

Please sign in to comment.