diff --git a/docs/source/guide/explanation/additional_features/models_optimization.rst b/docs/source/guide/explanation/additional_features/models_optimization.rst index 4b3c9ec4787..08715fb3165 100644 --- a/docs/source/guide/explanation/additional_features/models_optimization.rst +++ b/docs/source/guide/explanation/additional_features/models_optimization.rst @@ -1,17 +1,17 @@ Models Optimization =================== -OpenVINO™ Training Extensions provides two types of optimization algorithms: `Post-training Optimization Tool (POT) `_ and `Neural Network Compression Framework (NNCF) `_. +OpenVINO™ Training Extensions provides two types of optimization algorithms: `Post-Training Quantization tool (PTQ) `_ and `Neural Network Compression Framework (NNCF) `_. ******************************* -Post-training Optimization Tool +Post-Training Quantization Tool ******************************* -POT is designed to optimize the inference of models by applying post-training methods that do not require model retraining or fine-tuning. If you want to know more details about how POT works and to be more familiar with model optimization methods, please refer to `documentation `_. +PTQ is designed to optimize the inference of models by applying post-training methods that do not require model retraining or fine-tuning. If you want to know more details about how PTQ works and to be more familiar with model optimization methods, please refer to `documentation `_. -To run Post-training optimization it is required to convert the model to OpenVINO™ intermediate representation (IR) first. To perform fast and accurate quantization we use ``DefaultQuantization Algorithm`` for each task. Please, see the `DefaultQuantization Parameters `_ for further information about configuring the optimization. +To run Post-training quantization it is required to convert the model to OpenVINO™ intermediate representation (IR) first. To perform fast and accurate quantization we use ``DefaultQuantization Algorithm`` for each task. Please, refer to the `Tune quantization Parameters `_ for further information about configuring the optimization. -POT parameters can be found and configured in ``template.yaml`` and ``configuration.yaml`` for each task. For Anomaly and Semantic Segmentation tasks, we have separate configuration files for POT, that can be found in the same directory with ``template.yaml``, for example for `PaDiM `_, `OCR-Lite-HRNe-18-mod2 `_ model. +PTQ parameters can be found and configured in ``template.yaml`` and ``configuration.yaml`` for each task. For Anomaly and Semantic Segmentation tasks, we have separate configuration files for PTQ, that can be found in the same directory with ``template.yaml``, for example for `PaDiM `_, `OCR-Lite-HRNe-18-mod2 `_ model. ************************************ Neural Network Compression Framework @@ -25,9 +25,9 @@ You can refer to configuration files for default templates for each task accordi NNCF tends to provide better quality in terms of preserving accuracy as it uses training compression approaches. Compression results achievable with the NNCF can be found `here `_ . -Meanwhile, the POT is faster but can degrade accuracy more than the training-enabled approach. +Meanwhile, the PTQ is faster but can degrade accuracy more than the training-enabled approach. .. note:: The main recommendation is to start with post-training compression and use NNCF compression during training if you are not satisfied with the results. -Please, refer to our :doc:`dedicated tutorials <../../tutorials/base/how_to_train/index>` on how to optimize your model using POT or NNCF. \ No newline at end of file +Please, refer to our :doc:`dedicated tutorials <../../tutorials/base/how_to_train/index>` on how to optimize your model using PTQ or NNCF. \ No newline at end of file diff --git a/docs/source/guide/get_started/cli_commands.rst b/docs/source/guide/get_started/cli_commands.rst index 74d1e975729..a280da898f5 100644 --- a/docs/source/guide/get_started/cli_commands.rst +++ b/docs/source/guide/get_started/cli_commands.rst @@ -342,10 +342,10 @@ To use the exported model as an input for ``otx explain``, please dump additiona Optimization ************ -``otx optimize`` optimizes a model using `NNCF `_ or `POT `_ depending on the model format. +``otx optimize`` optimizes a model using `NNCF `_ or `PTQ `_ depending on the model and transforms it to ``INT8`` format. - NNCF optimization used for trained snapshots in a framework-specific format such as checkpoint (.pth) file from Pytorch -- POT optimization used for models exported in the OpenVINO™ IR format +- PTQ optimization used for models exported in the OpenVINO™ IR format With the ``--help`` command, you can list additional information: @@ -383,16 +383,16 @@ Command example for optimizing a PyTorch model (.pth) with OpenVINO™ NNCF: --output outputs/nncf -Command example for optimizing OpenVINO™ model (.xml) with OpenVINO™ POT: +Command example for optimizing OpenVINO™ model (.xml) with OpenVINO™ PTQ: .. code-block:: (otx) ...$ otx optimize SSD --load-weights \ --val-data-roots \ - --output outputs/pot + --output outputs/ptq -Thus, to use POT pass the path to exported IR (.xml) model, to use NNCF pass the path to the PyTorch (.pth) weights. +Thus, to use PTQ pass the path to exported IR (.xml) model, to use NNCF pass the path to the PyTorch (.pth) weights. *********** @@ -419,7 +419,7 @@ With the ``--help`` command, you can list additional information, such as its pa --test-data-roots TEST_DATA_ROOTS Comma-separated paths to test data folders. --load-weights LOAD_WEIGHTS - Load model weights from previously saved checkpoint.It could be a trained/optimized model (POT only) or exported model. + Load model weights from previously saved checkpoint. It could be a trained/optimized model (with PTQ only) or exported model. -o OUTPUT, --output OUTPUT Location where the intermediate output of the task will be stored. --workspace WORKSPACE Path to the workspace where the command will run. @@ -532,7 +532,7 @@ Demonstration -i INPUT, --input INPUT Source of input data: images folder, image, webcam and video. --load-weights LOAD_WEIGHTS - Load model weights from previously saved checkpoint.It could be a trained/optimized model (POT only) or exported model. + Load model weights from previously saved checkpoint.It could be a trained/optimized model (with PTQ only) or exported model. --fit-to-size FIT_TO_SIZE FIT_TO_SIZE Width and Height space-separated values. Fits displayed images to window with specified Width and Height. This options applies to result visualisation only. --loop Enable reading the input in a loop. diff --git a/docs/source/guide/tutorials/advanced/semi_sl.rst b/docs/source/guide/tutorials/advanced/semi_sl.rst index 236c72f1189..b4ce1142627 100644 --- a/docs/source/guide/tutorials/advanced/semi_sl.rst +++ b/docs/source/guide/tutorials/advanced/semi_sl.rst @@ -28,7 +28,7 @@ The process has been tested on the following configuration: To learn how to export the trained model, refer to `classification export <../base/how_to_train/classification.html#export>`__. - To learn how to optimize the trained model (.xml) with OpenVINO™ POT, refer to `classification optimization <../base/how_to_train/classification.html#optimization>`__. + To learn how to optimize the trained model (.xml) with OpenVINO™ PTQ, refer to `classification optimization <../base/how_to_train/classification.html#optimization>`__. Currently, OpenVINO™ NNCF optimization doesn't support a full Semi-SL training algorithm. The accuracy-aware optimization will be executed on labeled data only. So, the performance drop may be more noticeable than after ordinary supervised training. diff --git a/docs/source/guide/tutorials/base/deploy.rst b/docs/source/guide/tutorials/base/deploy.rst index 8fbbdac7d97..367597d414c 100644 --- a/docs/source/guide/tutorials/base/deploy.rst +++ b/docs/source/guide/tutorials/base/deploy.rst @@ -52,7 +52,7 @@ using the command below: 2023-01-20 09:30:41,737 | INFO : Deploying the model 2023-01-20 09:30:41,753 | INFO : Deploying completed -You can also deploy the quantized model, that was optimized with NNCF or POT, passing the path to this model in IR format to ``--load-weights`` parameter. +You can also deploy the quantized model, that was optimized with NNCF or PTQ, passing the path to this model in IR format to ``--load-weights`` parameter. After that, you can use the resulting ``openvino.zip`` archive in other application. diff --git a/docs/source/guide/tutorials/base/how_to_train/action_classification.rst b/docs/source/guide/tutorials/base/how_to_train/action_classification.rst index 12295845c8b..7304660dc15 100644 --- a/docs/source/guide/tutorials/base/how_to_train/action_classification.rst +++ b/docs/source/guide/tutorials/base/how_to_train/action_classification.rst @@ -212,7 +212,7 @@ Export ********* 1. ``otx export`` exports a trained Pytorch `.pth` model to the OpenVINO™ Intermediate Representation (IR) format. -It allows running the model on the Intel hardware much more efficiently, especially on the CPU. Also, the resulting IR model is required to run POT optimization. IR model consists of two files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. +It allows running the model on the Intel hardware much more efficiently, especially on the CPU. Also, the resulting IR model is required to run PTQ optimization. IR model consists of two files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. 2. Run the command line below to export the trained model and save the exported model to the ``openvino`` folder. @@ -235,7 +235,7 @@ and save the exported model to the ``openvino`` folder. 2023-02-21 22:54:35,424 - mmaction - INFO - Exporting completed -3. Check the accuracy of the IR model and the consistency between the exported model and the PyTorch model, +3. Check the accuracy of the IR optimimodel and the consistency between the exported model and the PyTorch model, using ``otx eval`` and passing the IR model path to the ``--load-weights`` parameter. .. code-block:: @@ -254,22 +254,24 @@ Optimization ************* 1. You can further optimize the model with ``otx optimize``. -Currently, quantization jobs that include POT is supported for X3D template. MoViNet will be supported in near future. +Currently, quantization jobs that include PTQ is supported for X3D template. MoViNet will be supported in near future. + +The optimized model will be quantized to ``INT8`` format. Refer to :doc:`optimization explanation <../../../explanation/additional_features/models_optimization>` section for more details on model optimization. 2. Example command for optimizing -OpenVINO™ model (.xml) with OpenVINO™ POT. +OpenVINO™ model (.xml) with OpenVINO™ PTQ. .. code-block:: (otx) ...$ otx optimize --load-weights openvino/openvino.xml \ - --output pot_model + --output ptq_model ... Performance(score: 0.6252587703095486, dashboard: (3 metric groups)) -Keep in mind that POT will take some time (generally less than NNCF optimization) without logging to optimize the model. +Keep in mind that PTQ will take some time (generally less than NNCF optimization) without logging to optimize the model. 3. Now, you have fully trained, optimized and exported an efficient model representation ready-to-use action classification model. diff --git a/docs/source/guide/tutorials/base/how_to_train/action_detection.rst b/docs/source/guide/tutorials/base/how_to_train/action_detection.rst index c1252ad011a..6103735beb0 100644 --- a/docs/source/guide/tutorials/base/how_to_train/action_detection.rst +++ b/docs/source/guide/tutorials/base/how_to_train/action_detection.rst @@ -201,6 +201,8 @@ Optimization 1. You can further optimize the model with ``otx optimize``. Currently, only PTQ is supported for action detection. NNCF will be supported in near future. + +The optimized model will be quantized to ``INT8`` format. Refer to :doc:`optimization explanation <../../../explanation/additional_features/models_optimization>` section for more details on model optimization. 2. Example command for optimizing @@ -209,7 +211,7 @@ OpenVINO™ model (.xml) with OpenVINO™ PTQ. .. code-block:: (otx) ...$ otx optimize --load-weights openvino/openvino.xml \ - --save-model-to pot_model + --save-model-to ptq_model ... diff --git a/docs/source/guide/tutorials/base/how_to_train/anomaly_detection.rst b/docs/source/guide/tutorials/base/how_to_train/anomaly_detection.rst index 31b51809c95..42058ffdf8a 100644 --- a/docs/source/guide/tutorials/base/how_to_train/anomaly_detection.rst +++ b/docs/source/guide/tutorials/base/how_to_train/anomaly_detection.rst @@ -156,7 +156,7 @@ Export ****** 1. ``otx export`` exports a trained Pytorch `.pth` model to the OpenVINO™ Intermediate Representation (IR) format. -It allows running the model on the Intel hardware much more efficient, especially on the CPU. Also, the resulting IR model is required to run POT optimization. IR model consists of 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. +It allows running the model on the Intel hardware much more efficient, especially on the CPU. Also, the resulting IR model is required to run PTQ optimization. IR model consists of 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. 2. We can run the below command line to export the trained model and save the exported model to the ``openvino`` folder: @@ -200,10 +200,11 @@ This gives the following results: Optimization ************ -Anomaly tasks can be optimized either in POT or NNCF format. For more information refer to the :doc:`optimization explanation <../../../explanation/additional_features/models_optimization>` section. +Anomaly tasks can be optimized either in PTQ or NNCF format. The model will be quantized to ``INT8`` format. +For more information refer to the :doc:`optimization explanation <../../../explanation/additional_features/models_optimization>` section. -1. Let's start with POT +1. Let's start with PTQ optimization. .. code-block:: @@ -211,7 +212,7 @@ optimization. otx optimize ote_anomaly_detection_padim \ --train-data-roots datasets/MVTec/bottle/train \ --load-weights otx-workspace-ANOMALY_DETECTION/openvino/openvino.xml \ - --output otx-workspace-ANOMALY_DETECTION/pot_model + --output otx-workspace-ANOMALY_DETECTION/ptq_model This command generates the following files that can be used to run :doc:`otx demo <../demo>`: @@ -233,7 +234,7 @@ weights to the ``opitmize`` command: --load-weights otx-workspace-ANOMALY_DETECTION/models/weights.pth \ --output otx-workspace-ANOMALY_DETECTION/nncf_model -Similar to POT optimization, it generates the following files: +Similar to PTQ optimization, it generates the following files: - image_threshold - pixel_threshold diff --git a/docs/source/guide/tutorials/base/how_to_train/classification.rst b/docs/source/guide/tutorials/base/how_to_train/classification.rst index c84448f4c7d..6569194cd3a 100644 --- a/docs/source/guide/tutorials/base/how_to_train/classification.rst +++ b/docs/source/guide/tutorials/base/how_to_train/classification.rst @@ -177,7 +177,7 @@ Export ********* 1. ``otx export`` exports a trained Pytorch `.pth` model to the OpenVINO™ Intermediate Representation (IR) format. -It allows running the model on the Intel hardware much more efficient, especially on the CPU. Also, the resulting IR model is required to run POT optimization. IR model consists of 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. +It allows running the model on the Intel hardware much more efficient, especially on the CPU. Also, the resulting IR model is required to run PTQ optimization. IR model consists of 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. 2. You can run the below command line to export the trained model and save the exported model to the ``openvino_model`` folder: @@ -212,7 +212,7 @@ Optimization ************* 1. You can further optimize the model with ``otx optimize``. -It uses NNCF or POT depending on the model format. +It uses NNCF or PTQ depending on the model and transforms it to ``INT8`` format. Please, refer to :doc:`optimization explanation <../../../explanation/additional_features/models_optimization>` section for more details on model optimization. @@ -235,18 +235,18 @@ a PyTorch model (`.pth`) with OpenVINO™ NNCF. The optimization time relies on the hardware characteristics, for example on 1 NVIDIA GeForce RTX 3090 and Intel(R) Core(TM) i9-10980XE it took about 10 minutes. 3. Command example for optimizing -OpenVINO™ model (.xml) with OpenVINO™ POT. +OpenVINO™ model (.xml) with OpenVINO™ PTQ. .. code-block:: (otx) ...$ otx optimize --load-weights openvino_model/openvino.xml \ - --output pot_model + --output ptq_model ... Performance(score: 0.9577656675749319, dashboard: (3 metric groups)) -Please note, that POT will take some time (generally less than NNCF optimization) without logging to optimize the model. +Please note, that PTQ will take some time (generally less than NNCF optimization) without logging to optimize the model. 4. Now you have fully trained, optimized and exported an efficient model representation ready-to-use classification model. diff --git a/docs/source/guide/tutorials/base/how_to_train/detection.rst b/docs/source/guide/tutorials/base/how_to_train/detection.rst index f6235e76d84..b2434830c45 100644 --- a/docs/source/guide/tutorials/base/how_to_train/detection.rst +++ b/docs/source/guide/tutorials/base/how_to_train/detection.rst @@ -330,7 +330,7 @@ Export 1. ``otx export`` exports a trained Pytorch `.pth` model to the OpenVINO™ Intermediate Representation (IR) format. It allows to efficiently run it on Intel hardware, especially on CPU, using OpenVINO™ runtime. -Also, the resulting IR model is required to run POT optimization in the section below. IR model contains 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. +Also, the resulting IR model is required to run PTQ optimization in the section below. IR model contains 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. 2. That's how we can export the trained model ``../outputs/weights.pth`` from the previous section and save the exported model to the ``../outputs/openvino/`` folder. @@ -384,11 +384,11 @@ Optimization ************* 1. We can further optimize the model with ``otx optimize``. -It uses NNCF or POT depending on the model format. +It uses NNCF or PTQ depending on the model and transforms it to ``INT8`` format. ``NNCF`` optimization is used for trained snapshots in a framework-specific format such as checkpoint (.pth) file from Pytorch. It starts accuracy-aware quantization based on the obtained weights from the training stage. Generally, we will see the same output as during training. -``POT`` optimization is used for models exported in the OpenVINO™ IR format. It decreases the floating-point precision to integer precision of the exported model by performing the post-training optimization. +``PTQ`` optimization is used for models exported in the OpenVINO™ IR format. It decreases the floating-point precision to integer precision of the exported model by performing the post-training optimization. The function results with the following files, which could be used to run :doc:`otx demo <../demo>` as well with PyTorch (`.pth`) and IR model (`.xml`): @@ -420,20 +420,20 @@ with OpenVINO NNCF. 3. Command example for optimizing OpenVINO™ model (.xml) -with OpenVINO™ POT. +with OpenVINO™ PTQ. .. code-block:: (otx) ...$ otx optimize --load-weights ../outputs/openvino/openvino.xml \ - --output ../outputs/pot \ - --output ../outputs/pot + --output ../outputs/ptq \ + --output ../outputs/ptq ... 2023-01-10 06:29:46,751 | INFO : Loading OpenVINO OTXDetectionTask 2023-01-10 06:29:47,685 | INFO : OpenVINO task initialization completed - 2023-01-10 06:29:47,685 | INFO : Start POT optimization - 2023-01-10 06:34:29,304 | INFO : POT optimization completed + 2023-01-10 06:29:47,685 | INFO : Start PTQ optimization + 2023-01-10 06:34:29,304 | INFO : PTQ optimization completed 2023-01-10 06:34:29,419 | INFO : Start OpenVINO inference 2023-01-10 06:34:33,275 | INFO : OpenVINO inference completed 2023-01-10 06:34:33,275 | INFO : Start OpenVINO metric evaluation @@ -441,7 +441,7 @@ with OpenVINO™ POT. Performance(score: 0.5389435989256938, dashboard: (1 metric groups)) The optimization time highly relies on the hardware characteristics, for example on 1 NVIDIA GeForce RTX 3090 it took about 10 minutes. -Please note, that POT will take some time without logging to optimize the model. +Please note, that PTQ will take some time without logging to optimize the model. 4. Finally, we can also evaluate the optimized model by passing it to the ``otx eval`` function. diff --git a/docs/source/guide/tutorials/base/how_to_train/instance_segmentation.rst b/docs/source/guide/tutorials/base/how_to_train/instance_segmentation.rst index 925bf9a257a..3d9526d92ab 100644 --- a/docs/source/guide/tutorials/base/how_to_train/instance_segmentation.rst +++ b/docs/source/guide/tutorials/base/how_to_train/instance_segmentation.rst @@ -327,7 +327,7 @@ Export 1. ``otx export`` exports a trained Pytorch `.pth` model to the OpenVINO™ Intermediate Representation (IR) format. -It allows running the model on the Intel hardware much more efficient, especially on the CPU. Also, the resulting IR model is required to run POT optimization. IR model consists of 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. +It allows running the model on the Intel hardware much more efficient, especially on the CPU. Also, the resulting IR model is required to run PTQ optimization. IR model consists of 2 files: ``openvino.xml`` for weights and ``openvino.bin`` for architecture. 2. We can run the below command line to export the trained model and save the exported model to the ``outputs/**_export/openvino`` folder. @@ -354,7 +354,7 @@ Optimization ************* 1. We can further optimize the model with ``otx optimize``. -It uses NNCF or POT depending on the model format. +It uses NNCF or PTQ depending on the model and transforms it to ``INT8`` format. Please, refer to :doc:`optimization explanation <../../../explanation/additional_features/models_optimization>` section to get the intuition of what we use under the hood for optimization purposes. @@ -371,13 +371,13 @@ a PyTorch model (`.pth`) with OpenVINO™ `NNCF ` section to get the intuition of what we use under the hood for optimization purposes. 2. Command example for optimizing @@ -234,18 +235,18 @@ a PyTorch model (`.pth`) with OpenVINO™ NNCF. The optimization time relies on the hardware characteristics, for example on 1 NVIDIA GeForce RTX 3090 and Intel(R) Core(TM) i9-10980XE it took about 15 minutes. 3. Command example for optimizing -OpenVINO™ model (.xml) with OpenVINO™ POT. +OpenVINO™ model (.xml) with OpenVINO™ PTQ. .. code-block:: (otx) ...$ otx optimize --load-weights openvino_model/openvino.xml \ - --output pot_model + --output ptq_model ... Performance(score: 0.9577656675749319, dashboard: (1 metric groups)) -Please note, that POT will take some time (generally less than NNCF optimization) without logging to optimize the model. +Please note, that PTQ will take some time (generally less than NNCF optimization) without logging to optimize the model. 4. Now we have fully trained, optimized and exported an efficient model representation ready-to-use semantic segmentation model. diff --git a/src/otx/cli/tools/eval.py b/src/otx/cli/tools/eval.py index 2ed3b22a477..f210ba4a395 100644 --- a/src/otx/cli/tools/eval.py +++ b/src/otx/cli/tools/eval.py @@ -49,8 +49,8 @@ def get_args(): ) parser.add_argument( "--load-weights", - help="Load model weights from previously saved checkpoint." - "It could be a trained/optimized model (POT only) or exported model.", + help="Load model weights from previously saved checkpoint. " + "It could be a trained/optimized model (with PTQ only) or exported model.", ) parser.add_argument( "-o",