Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Checkpoint tfds data iterator #954

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Checkpoint tfds data iterator #954

wants to merge 7 commits into from
+102 −3

Commits on Oct 7, 2024

  1. Update checkpointing.py w/ tfds checkpoint handler

    @mattdonati
    mattdonati authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    a9b1f4d View commit details
    Browse the repository at this point in the history

  2. Update train.py w/ option to checkpoint tfds iterator

    @mattdonati
    mattdonati authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    5f288ea View commit details
    Browse the repository at this point in the history

  3. Update max_utils.py with option to load tfds iterator from ckpt

    @mattdonati
    mattdonati authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    83df17a View commit details
    Browse the repository at this point in the history

  4. Update base.yml by adding flag for tfds iter checkpointing 

    Decided to add a flag because checkpointing the iterator creates large checkpoints and may not be preferred by all users.
    @mattdonati
    mattdonati authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    2f76034 View commit details
    Browse the repository at this point in the history

  5. Update Data_Input_Pipeline.md with itfds iter ckpt notes

    @mattdonati
    mattdonati authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d187384 View commit details
    Browse the repository at this point in the history

  6. Update Data_Input_Pipeline.md

    @mattdonati
    mattdonati authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    c1ffcaa View commit details
    Browse the repository at this point in the history

  7. Update checkpointing.py

    @mattdonati
    mattdonati authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    aef8281 View commit details
    Browse the repository at this point in the history