Move mpa.deploy to otx.algorithms.common

[harimkang]

Summary

Move mpa.deploy to otx.algorithms.common.adapters.mmdeploy

How to test

Checklist

• I submit my changes into the develop branch
• I have added description of my changes into CHANGELOG
• I have updated the documentation accordingly
• I have added tests to cover my changes
• I have linked related issues

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below)

# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

[harimkang]

@cih9088 Hi, I'm currently working on moving the files from mpa.module to otx. I'm getting the following unit-test error while moving the deploy related codes (tests/unit/algorithms/common/adapters/mmdeploy/test_deploy_apis.py::TestMMdeployExporter::test_partition), could you please see if I'm missing something? (Some of the modified code is to fix pylint errors, please let me know if you find anything wrong. I'll fix it right away.)

    
        with tempfile.TemporaryDirectory() as tempdir:
>           MMdeployExporter.export2openvino(
                tempdir,
                build_classifier,
                config,
                deploy_config,
            )

tests/unit/algorithms/common/adapters/mmdeploy/test_deploy_apis.py:209: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
otx/algorithms/common/adapters/mmdeploy/apis.py:244: in export2openvino
    MMdeployExporter.onnx2openvino(
otx/algorithms/common/adapters/mmdeploy/apis.py:330: in onnx2openvino
    raise e
otx/algorithms/common/adapters/mmdeploy/apis.py:317: in onnx2openvino
    openvino_api.from_onnx(onnx_ready_path, output_dir, input_info, output_names, mo_options)
venv/lib/python3.9/site-packages/mmdeploy/apis/core/pipeline_manager.py:356: in _wrap
    return self.call_function(func_name_, *args, **kwargs)
venv/lib/python3.9/site-packages/mmdeploy/apis/core/pipeline_manager.py:326: in call_function
    return self.call_function_local(func_name, *args, **kwargs)
venv/lib/python3.9/site-packages/mmdeploy/apis/core/pipeline_manager.py:275: in call_function_local
    return pipe_caller(*args, **kwargs)
venv/lib/python3.9/site-packages/mmdeploy/apis/core/pipeline_manager.py:107: in __call__
    ret = func(*args, **kwargs)
venv/lib/python3.9/site-packages/mmdeploy/backend/openvino/onnx2openvino.py:114: in from_onnx
    mo_output = run(command, stdout=PIPE, stderr=PIPE, shell=True, check=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = ('mo --input_model="/tmp/tmp81n5d140/partition_ready.onnx" --output_dir="/tmp/tmp81n5d140" --output="" --input="input" --input_shape="[1, 3, 50, 50]" --model_name "partition" ',), kwargs = {'shell': True, 'stderr': -1, 'stdout': -1}
process = <Popen: returncode: 1 args: 'mo --input_model="/tmp/tmp81n5d140/partition_re...>, stdout = b'', stderr = b'[ ERROR ]  No node with name input\n', retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command 'mo --input_model="/tmp/tmp81n5d140/partition_ready.onnx" --output_dir="/tmp/tmp81n5d140" --output="" --input="input" --input_shape="[1, 3, 50, 50]" --model_name "partition" ' returned non-zero exit status 1.

../../../anaconda3/lib/python3.9/subprocess.py:528: CalledProcessError
============================================================================================================= short test summary info ==============================================================================================================
FAILED tests/unit/algorithms/common/adapters/mmdeploy/test_deploy_apis.py::TestMMdeployExporter::test_partition - subprocess.CalledProcessError: Command 'mo --input_model="/tmp/tmp81n5d140/partition_ready.onnx" --output_dir="/tmp/tmp81n5d140" --output="" --input="input" --input_shape="[1, 3, 50, 50]" --model_name "partition" ' returned non-zero exit s...
========================================================================================================== 1 failed, 12 warnings in 6.18s ==========================================================================================================

[cih9088]

(tests/unit/algorithms/common/adapters/mmdeploy/test_deploy_apis.py::TestMMdeployExporter::test_partition), could you please see if I'm missing something? (Some of the modified code is to fix pylint errors, please let me know if you find anything wrong. I'll fix it right away.)

@harimkang Fixed.

[codecov-commenter]

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (128154f) 80.72% compared to head (e5b0bd0) 80.72%.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1903   +/-   ##
========================================
  Coverage    80.72%   80.72%           
========================================
  Files          489      489           
  Lines        33388    33388           
========================================
  Hits         26951    26951           
  Misses        6437     6437           

Impacted Files	Coverage Δ
.../algorithms/common/adapters/mmdeploy/utils/onnx.py	100.00% <ø> (ø)
...ommon/adapters/mmdeploy/utils/operations_domain.py	100.00% <ø> (ø)
...algorithms/common/adapters/mmdeploy/utils/utils.py	95.23% <ø> (ø)
otx/algorithms/common/adapters/mmdeploy/apis.py	90.71% <100.00%> (ø)
...orithms/common/adapters/mmdeploy/utils/mmdeploy.py	100.00% <100.00%> (ø)
...ers/mmdet/models/detectors/custom_atss_detector.py	63.01% <100.00%> (ø)
...mmdet/models/detectors/custom_maskrcnn_detector.py	36.00% <100.00%> (ø)
...t/models/detectors/custom_single_stage_detector.py	64.94% <100.00%> (ø)
...rs/mmdet/models/detectors/custom_yolox_detector.py	33.73% <100.00%> (ø)
...lgorithms/detection/adapters/mmdet/nncf/patches.py	98.80% <100.00%> (ø)
... and 6 more

... and 4 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[jaegukhyun]

I left some minor fix suggest, If you don't want to change, then just resolve them.

[harimkang]

@jaegukhyun @JihwanEom I think we can merge this, but can you double check?

[sungmanc]

Looks nice to me, it seems that you resolves plenty of pre-commit issues