Add Fine-Tune BERT LLM Example

[andreyvelich]

I added example to fine-tune BERT model with Yelp review dataset.
This example is based on this tutorial: https://huggingface.co/docs/transformers/en/training
I used split_dataset_by_node to distribute data across PyTorchJob workers.

Also, I moved our SDK examples to these folders: text-classification, image-classification, language-modeling.
Similar to HuggingFace transformers examples: https://github.com/huggingface/transformers/tree/main/examples/pytorch

I think, we should refactor our Training Operator examples directory in the future PRs to make it more ML engineers and data scientists friendly.
Regardless if users are using Kubeflow Python SDK or kubectl and YAMLs to submit jobs they should be able to find the appropriate AI/ML training use-cases.

From my perspective, for the long-term we should guide our users more towards Python SDK to submit Kubeflow Jobs.

Please let me know what do you think @johnugeorge @gaocegege @terrytangyuan @kuizhiqing @deepanker13 @Jeffwan @droctothorpe @tenzen-y @StefanoFioravanzo

/hold for review

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[coveralls]

Pull Request Test Coverage Report for Build 8237084497

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.02%) to 42.896%

---

Totals
Change from base Build 8214367884:	0.02%
Covered Lines:	3756
Relevant Lines:	8756

---

💛 - Coveralls

[PeterWrighten]

Is it fine to use space blank in filename?

[andreyvelich]

For Notebook files it should be normal, but if we want, we can name files without space blanks.

[kuizhiqing]

Happy to see it, thank @andreyvelich for driving this. Maybe we can make a roadmap for the examples and I'd like to contribute too.

One thing from my personal perspective is that if we dedicate to the notebook format or we can support pure python code with markdown or something like that?

[tenzen-y]

Happy to see it, thank @andreyvelich for driving this. Maybe we can make a roadmap for the examples and I'd like to contribute too.

+1
Let's create a dedicated issue.

[tenzen-y]

Line #8.        num_workers=3,  # Number of PyTorch workers to use.

Should we replace this with 1 aligned with the above comment:

At least 1 GPU on your Kubernetes cluster to fine-tune BERT model on 1 worker.

Or, should we update the above comment?

Reply via ReviewNB

[andreyvelich]

Sure, let me change it to 3.

[tenzen-y]

Line #9.    good_review = "This is one of the best restaurants. I've ever been to."

Reply via ReviewNB

[andreyvelich]

Nice catch, however LLM can easily recognize sentences without full stop 😃
Also, I believe this sentence is grammatically correct, isn't ?

This is one of the best restaurants I've ever been to

[tenzen-y]

It makes sense.
Also, Grammarly says this is fine as well :)

[andreyvelich]

Maybe we can make a roadmap for the examples and I'd like to contribute too.

That's good idea, do you want to start issue to identify examples that we want to add (e.g. text-generation, speech recognition, and more)?

One thing from my personal perspective is that if we dedicate to the notebook format or we can support pure python code with markdown or something like that?

I think, we have pros and cons of using Notebook files vs Python files. For example, it's easier to implement E2Es test for Python files, but Notebooks are usually more Data Scientists friendly. I an open to your suggestions @tenzen-y @kuizhiqing @johnugeorge.

[tenzen-y]

I think, we have pros and cons of using Notebook files vs Python files. For example, it's easier to implement E2Es test for Python files, but Notebooks are usually more Data Scientists friendly. I an open to your suggestions @tenzen-y @kuizhiqing @johnugeorge.

Anyway, I think that we should have a dedicated issue.

[tenzen-y]

/lgtm
/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich, tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [andreyvelich,tenzen-y]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[andreyvelich]

/assign @kuizhiqing @johnugeorge @deepanker13 For the final review

[kuizhiqing]

/lgtm

[johnugeorge]

We will add a similar one using Train API
/hold cancel