Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update XPU install #3516

Merged
merged 15 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 49 additions & 31 deletions docs/source/guide/get_started/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ The current version of OpenVINO™ Training Extensions was tested in the followi
Install OpenVINO™ Training Extensions for users (CUDA/CPU)
**********************************************************

1. Install OpenVINO™ Training Extensions package:
1. Install OpenVINO™ Training Extensions
package:

* A local source in development mode

Expand All @@ -41,7 +42,8 @@ Install OpenVINO™ Training Extensions for users (CUDA/CPU)
pip install -e .
2. Install PyTorch & Requirements for training according to your system environment.
2. Install PyTorch & Requirements for training
according to your system environment.

.. tab-set::

Expand Down Expand Up @@ -76,49 +78,63 @@ OpenVINO™ Training Extensions command line functionality.
Install OpenVINO™ Training Extensions for users (XPU devices)
*************************************************************

1. Follow the first two steps from above instructions
on cloning the repository and creating a virtual environment.
1. Install OpenVINO™ Training Extensions
package:

2. Install Intel Extensions For Pytorch (IPEX).
Follow the `official documentation <https://intel.github.io/intel-extension-for-pytorch/index.html#installation?platform=gpu&version=v2.1.10%2Bxpu>`_ to install prerequisites such as OneAPI and proper drivers.
* A local source in development mode

.. code-block:: shell
.. tab-set::

python -m pip install torch==2.1.0.post0 torchvision==0.16.0.post0 torchaudio==2.1.0.post0 intel-extension-for-pytorch==2.1.20+xpu oneccl_bind_pt==2.1.200+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
.. tab-item:: PyPI

3. Install MMCV.
It is required to install mmcv from source to properly build it with IPEX.
.. code-block:: shell
.. code-block:: shell
pip install otx
git clone https://github.com/open-mmlab/mmcv
cd mmcv
git checkout v2.1.0
MMCV_WITH_OPS=1 pip install -e .
.. tab-item:: Source

4. Install OpenVINO™ Training Extensions
package from either:
.. code-block:: shell
* A local source in development mode
# Clone the training_extensions repository with the following command:
git clone https://github.com/openvinotoolkit/training_extensions.git
cd training_extensions
.. code-block:: shell
# Set up a virtual environment.
python -m venv .otx
source .otx/bin/activate
pip install -e .
pip install -e .
* PyPI
2. Install Intel Extensions for Pytorch & Requirements
for training according to your system environment.

.. code-block:: shell
.. tab-set::

.. tab-item:: Minimum requirements

pip install otx
.. code-block:: shell
5. Install requirements for training
excluding Pytorch.
pip install '.[xpu]' --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
.. code-block:: shell
.. note::

otx install -v --do-not-install-torch
Models from mmlab are not available for this environment. If you want to use mmlab models, you must install them with Full Requirements.
Also, some tasks may not be supported by minimum requirements.

.. tab-item:: Full Requirements
kprokofi marked this conversation as resolved.
Show resolved Hide resolved

6. Activate OneAPI environment
.. code-block:: shell
python -m pip install torch==2.1.0.post2 torchvision==0.16.0.post2 torchaudio==2.1.0.post2 intel-extension-for-pytorch==2.1.30+xpu oneccl_bind_pt==2.1.300+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
git clone https://github.com/open-mmlab/mmcv
cd mmcv
git checkout v2.1.0
MMCV_WITH_OPS=1 pip install -e .
cd ..
otx install -v --do-not-install-torch
[Optional] Refer to the `Intel® Extension for PyTorch documentation guide <https://intel.github.io/intel-extension-for-pytorch/index.html#installation?platform=gpu&version=v2.1.10%2bxpu&os=linux%2fwsl2&package=pip>`_

3. Activate OneAPI environment
and export required IPEX system variables

.. code-block:: shell
Expand All @@ -127,7 +143,7 @@ and export required IPEX system variables
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30
export IPEX_FP32_MATH_MODE=TF32
7. Once the package is installed in the virtual environment, you can use full
3. Once the package is installed in the virtual environment, you can use full
OpenVINO™ Training Extensions command line functionality.

.. code-block:: shell
Expand All @@ -153,15 +169,17 @@ Then you may change code, and all fixes will be directly applied to the editable
Install OpenVINO™ Training Extensions by using Docker
*****************************************************

1. By executing the following commands, it will build two Docker images: ``otx:${OTX_VERSION}-cuda`` and ``otx:${OTX_VERSION}-cuda-pretrained-ready``.
1. By executing the following commands, it will build two
Docker images: ``otx:${OTX_VERSION}-cuda`` and ``otx:${OTX_VERSION}-cuda-pretrained-ready``.

.. code-block:: shell
git clone https://github.com/openvinotoolkit/training_extensions.git
cd docker
./build.sh
2. After that, you can check whether the images are built correctly such as
2. After that, you can check whether the
images are built correctly such as

.. code-block:: shell
Expand Down
16 changes: 16 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,22 @@ docs = [
"sphinxemoji",
"nbsphinx",
]
xpu = [
"torch==2.1.0.post2",
"torchvision==0.16.0.post2",
"intel-extension-for-pytorch==2.1.30+xpu",
"oneccl_bind_pt==2.1.300+xpu",
"lightning==2.1.2",
"pytorchcv",
"timm",
"openvino==2024.0",
"openvino-dev==2024.0",
"openvino-model-api==0.2.0",
"onnx==1.16.0",
"onnxconverter-common==1.14.0",
"nncf==2.9.0",
"anomalib[core]==1.0.1",
]
base = [
"torch==2.1.1",
"lightning==2.1.2",
Expand Down
1 change: 1 addition & 0 deletions src/otx/cli/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ def otx_install(
requirements_dict = get_requirements("otx")
# Add base and openvino requirements.
requirements = requirements_dict["base"]
requirements_dict.pop("xpu", None)
if option == "full":
for extra in requirements_dict:
requirements.extend(requirements_dict[extra])
Expand Down
2 changes: 2 additions & 0 deletions tests/unit/cli/test_install.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ def setup(self, mocker: MockerFixture) -> None:
"openvino": [Requirement.parse("openvino")],
"mmlab": [Requirement.parse("mmpretrain")],
"api": [Requirement.parse("test1")],
"xpu": [Requirement.parse("torch==2.0.0"), Requirement.parse("ipex")],
}
mocker.patch("otx.cli.install.get_requirements", return_value=requirements_dict)

Expand Down Expand Up @@ -63,5 +64,6 @@ def test_install_full(self, mocker: MockerFixture, monkeypatch: MonkeyPatch) ->
assert "openvino" in argument_call_list
assert "pytorchcv" in argument_call_list
assert "mmpretrain" not in argument_call_list
assert "ipex" not in argument_call_list
mm_argument_call_list = mock_mim_installation.call_args_list[-1][0][-1]
assert "mmpretrain" in mm_argument_call_list
Loading