Skip to content

Commit

Permalink
Added suggesitions and fix format issue
Browse files Browse the repository at this point in the history
  • Loading branch information
@SurajBaloni
SurajBaloni committed Sep 20, 2024
2 parents 354f237 + a71cd45 commit d0888bd
Showing 1 changed file with 110 additions and 3 deletions.
113 changes: 110 additions & 3 deletions guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d048eaea-3ea5-45b7-9a93-191777d3dd82",
"metadata": {},
"source": [
"# Creating Custom NLP Model with Model Extensiblity"
]
},
{
"cell_type": "markdown",
"id": "168721af",
Expand Down Expand Up @@ -40,6 +48,7 @@
"id": "4941d9fb",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"## Custom Python NLP Function"
]
},
Expand All @@ -49,6 +58,9 @@
"metadata": {},
"source": [
"You can create an NLP function in Python to integrate third-party language models into a text processing pipeline. NLP functions handle text data and perform various text processing tasks."
=======
"## Writing Python NLP Function"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand All @@ -57,13 +69,41 @@
"metadata": {},
"source": [
"### Anatomy of a Python NLP Function"
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
=======
]
},
{
"cell_type": "markdown",
"id": "2d027e60",
"metadata": {},
"source": [
"This section explains how to create a new Natural Language Processing (NLP) function in Python. NLP functions work with text data and perform different text processing tasks. The aim is to learn how to fit your Python NLP function into a text processing pipeline, similar to how raster functions are used in image processing."
]
},
{
"cell_type": "markdown",
"id": "f5ce75f6",
"metadata": {},
"source": [
"### Core Methods"
]
},
{
"cell_type": "markdown",
"id": "0438547a",
"metadata": {},
"source": [
"Here's a breakdown of essential methods you might need to implement for an NLP function:"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
"cell_type": "markdown",
"id": "eb0c423a",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"#### `__init__`\n",
"\n",
"The `__init__` method initialize instance variables such as name, description, and other attributes essential for the NLP function.\n",
Expand All @@ -83,6 +123,27 @@
"#### `predict`\n",
"\n",
"The `predict` method responsible for converting input text to the desired output. It uses defined parameters and processing logic to produce the final result.\n"
=======
"* `__init__`\n",
"\n",
" This method initializes class variables such as `name`, `description`, and other attributes essential for your NLP function.\n",
"\n",
"* `initialize`\n",
"\n",
" This is called at the start of your Python NLP function. Use this method to load the NLP model and set up any initial configurations.\n",
"\n",
"* `getParameterInfo`\n",
"\n",
" This method specifies the parameters that your NLP function accepts. This includes text inputs and any configuration settings required for processing.\n",
"\n",
"* `getConfiguration`\n",
"\n",
" This method describes how the function should handle input processing and generate outputs. It includes details on any pre-processing or post-processing steps necessary for the function.\n",
"\n",
"* `predict`\n",
"\n",
" This method is the core function responsible for converting input text into the desired output. It uses the defined parameters and processing logic to produce the final result.\n"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand All @@ -98,15 +159,23 @@
"id": "17735a1e",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"#### `__init__`"
=======
"* `__init__()`"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
"cell_type": "markdown",
"id": "e30241ac",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"The `__init__` method is the constructor of the custom NLP function and initializes instance variables such as the name, description, and other essential attributes. This method sets the properties that will define the behavior and characteristics of the NLP function.\n",
=======
"This method initializes class variables such as name, description, and other attributes essential for your NLP function. This method sets up the foundational properties that will define the behavior and characteristics of your NLP component. \n",
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
"\n",
"The constructor creates an instance of a custom NLP class with all the attributes required for processing and analysis. When creating an instance of an NLP class, this method ensures that it has the required settings and default values. For example, if the NLP function needs specific settings such as paths to models, or special tokens, they can be set up in this method."
]
Expand Down Expand Up @@ -137,7 +206,11 @@
"id": "a9958400",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"#### `initialize`"
=======
"* `initialize()`"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand Down Expand Up @@ -177,7 +250,11 @@
"id": "11e3da28",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"#### `getParameterInfo`"
=======
"* `getParameterInfo()`"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand Down Expand Up @@ -244,7 +321,11 @@
"id": "b6d674aa",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"#### `getConfiguration`"
=======
"* `getConfiguration()`"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand Down Expand Up @@ -286,7 +367,13 @@
"id": "30cd88e7",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"In the example above, the custom NLP function is configured by doing the following:\n",
=======
"#### Explanation\n",
"\n",
"This method configures the NLP function by:\n",
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
"\n",
"- Setting the `class_names` parameter from the provided arguments.\n",
"- Setting the `prompt` parameter from the provided arguments.\n",
Expand All @@ -299,7 +386,11 @@
"id": "e6798934",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"#### `predict`"
=======
"* `predict()`"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand Down Expand Up @@ -358,7 +449,11 @@
"id": "62a2a40b",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"## Esri Model Definition (.emd) File"
=======
"## Refer to the custom NLP function in the Esri Model Definition (.emd) File"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand Down Expand Up @@ -403,7 +498,11 @@
"id": "a5490415",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"## Custom Deep Learning Package (.dlpk) Model File\n",
=======
"## Create Custom Deep Learning Package (.dlpk) Model File\n",
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
"\n",
"To complete a custom NLP setup, create a .dlpk model file. The .dlpk file allows you to use a model with the with the `arcgis.learn` Python API and the inference tools in the Text Analysis toolset.\n",
"\n",
Expand Down Expand Up @@ -467,6 +566,7 @@
"id": "da397fa7",
"metadata": {},
"source": [
<<<<<<< HEAD:guide/14-deep-learning/use_third_party_language_models_with_ArcGIS.ipynb
"The `arcgis.learn` Python API supports for third party custom models through various classes, including `TextClassifier`, `EntityRecognizer`, and `SequenceToSequence`. To utilize your custom `.dlpk` file for inference with these classes, follow the steps below. The example provided demonstrates how to use the `TextClassifier` class, but you can apply similar steps for the other supported classes."
]
},
Expand All @@ -476,6 +576,13 @@
"metadata": {},
"source": [
"### Example with TextClassifier"
=======
"## Using the Custom .dlpk with the arcgis.learn API\n",
"\n",
"The `arcgis.learn` Python API supports model extensibility through various classes, including `TextClassifier`, `EntityRecognizer`, and `SequenceToSequence`. To utilize your custom `.dlpk` file for inference with these classes, follow the steps below. The example provided demonstrates how to use the `TextClassifier` class, but you can apply similar steps for the other supported classes.\n",
"\n",
"### Example with TextClassifier\n"
>>>>>>> a71cd459e26888bbc4ab70134c95117d87398ae3:guide/14-deep-learning/add_custom_nlp_models_using_model_extension.ipynb
]
},
{
Expand Down Expand Up @@ -629,9 +736,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [conda env:conda-dlenv_1] *",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "conda-env-conda-dlenv_1-py"
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -643,7 +750,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
"version": "3.11.0"
}
},
"nbformat": 4,
Expand Down

0 comments on commit d0888bd

Please sign in to comment.