[GPU/OpenCL] OpenCL pipeline for layer execution on GPU @open sesame 03/07 10:42 #2465

s-debadri · 2024-02-06T09:33:25Z

This PR includes the initial version of OpenCL pipeline for layers to run on GPU. Following wrappers have been added which uses OpenCL APIs internally:

opencl_loader: Loading required OpenCL functions.
opencl_context_manager: Getting device details and managing context creation globally.
opencl_command_queue_manager: Creation and deletion of global command queue, reading and writing buffers, dispatching kernels.
opencl_kernel: Managing kernels and setting arguments.
opencl_program: Building and managing OpenCL program.
opencl_buffer: Creating, reading and writing data on OpenCL buffer.

opencl_op_interface has been created to handle the workflow of kernel execution.

Tensor level changes:

Added cl_interface to handle tensor operations which will be executed on GPU.
Added experimental naïve OpenCL kernel for SGEMV which can be useful for testing the initial pipeline.
Modified sum and sum_by_batch function signatures to use boolean flag for choosing compute engine (CPU/GPU).

enable-opencl flag incorporated in meson.options

Update (23 Feb 2024):

ml::train::LayerComputeEngine enum added to handle compute engine information. Only FullyConnected layer API signature modified as of now.
createLayer API modified to propagate compute engine info via factory to layer_node.
cl_context created to handle global configuration of OpenCL environment which will create command queue and context for OpenCL and register layers to be run on GPU.
Getter and setter for compute engine added to layer_context along with utility to create kernel.
Modified build configurations as required.
Added UT for cl_context global instance creation with GPU command queue.

opencl_op_interface can be deprecated later. Restored sum and sum_by_batch function signatures as tensor level kernels are not required at this moment. Refactored experimental naïve OpenCL kernel for SGEMV.

Signed-off-by: Debadri Samaddar [email protected]

taos-ci · 2024-02-06T09:33:28Z

📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2465. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

taos-ci · 2024-02-06T10:07:27Z

cibot: @s-debadri, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2465-202402061833290.23417901992798-9fac4504a9442c86c07d1fc1e9dadfb201f8bdf6/.

jijoongmoon · 2024-02-07T00:04:00Z

nntrainer/opencl/meson.build

+
+foreach h : opencl_headers
+  nntrainer_headers += meson.current_source_dir() / h
+endforeach


please add new line at the end of the file.

Resolved in next commit

jijoongmoon · 2024-02-07T00:10:20Z

nntrainer/meson.build

@@ -42,7 +42,8 @@ nntrainer_elements = [
  'optimizers',
  'tensor',
  'utils',
-  'graph'
+  'graph',
+  'opencl'


This can be optional. how about adding option('enable-opencl', type: 'boolean', value: false) in meson_options.txt

Added related changes in next commit. Updated description.

taos-ci · 2024-02-07T10:41:49Z

cibot: @s-debadri, nntrainer/tensor/cl_operations/cl_sgemv.hpp does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

taos-ci · 2024-02-07T10:41:53Z

cibot: @s-debadri, nntrainer/opencl/opencl_buffer.cpp includes bug(s). Please fix incorrect coding constructs in your commit before entering a review process.

taos-ci

@s-debadri, 💯 All CI checkers are successfully verified. Thanks.

jijoongmoon · 2024-02-07T11:54:01Z

nntrainer/opencl/meson.build

+endforeach
+
+foreach h : opencl_headers
+  nntrainer_headers += meson.current_source_dir() / h


It might be better to use .h for header files.

jijoongmoon · 2024-02-07T11:55:28Z

nntrainer/opencl/opencl_buffer.cpp

+
+#include <nntrainer_log.h>
+
+namespace nntrainer::internal {


How about nntrainer::opencl namesapce rather than internal to make it clearer?

jijoongmoon · 2024-02-07T11:57:42Z

nntrainer/opencl/opencl_buffer.hpp

+#include "opencl_context_manager.hpp"
+#include "third_party/cl.h"
+
+namespace nntrainer::internal {


How about adding more comments following the oxygen to provide a better understanding?

jihochu · 2024-02-07T13:04:57Z

nntrainer/opencl/opencl_context_manager.cpp

+
+#include <vector>
+
+#include "opencl_loader.hpp"


please check order.

jihochu · 2024-02-07T13:10:04Z

nntrainer/opencl/opencl_kernel.cpp

+  cl_program prgm = program.GetProgram();
+  kernel_ = clCreateKernel(prgm, function_name.c_str(), &error_code);
+  if (!kernel_ || error_code != CL_SUCCESS) {
+    kernel_ = nullptr;


As a doc, clCreateKernel returns null with error code whenever it fails.

taos-ci

@s-debadri, 💯 All CI checkers are successfully verified. Thanks.

taos-ci

@s-debadri, 💯 All CI checkers are successfully verified. Thanks.

taos-ci · 2024-03-06T08:55:29Z

cibot: @s-debadri, nntrainer/opencl/opencl_buffer.cpp includes bug(s). Please fix incorrect coding constructs in your commit before entering a review process.

taos-ci

@s-debadri, 💯 All CI checkers are successfully verified. Thanks.

DonghakPark

Hi you got 3 formatting error

clang-format reports: 3 file(s) not formatted
nntrainer/opencl/opencl_buffer.cpp
nntrainer/opencl/third_party/cl.h
nntrainer/opencl/third_party/cl_platform.h

on here, now our TAOS CI and gitaction CI has different version
so please fomatting based on gitaction CI ?

taos-ci

@s-debadri, 💯 All CI checkers are successfully verified. Thanks.

taos-ci · 2024-03-07T04:57:21Z

cibot: @s-debadri, nntrainer/opencl/opencl_buffer.cpp includes bug(s). Please fix incorrect coding constructs in your commit before entering a review process.

taos-ci · 2024-03-07T05:24:44Z

cibot: @s-debadri, nntrainer/opencl/third_party/cl_platform.h does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

taos-ci · 2024-03-07T05:24:47Z

cibot: @s-debadri, nntrainer/opencl/opencl_buffer.cpp includes bug(s). Please fix incorrect coding constructs in your commit before entering a review process.

DonghakPark · 2024-03-07T05:36:30Z

@s-debadri
in case of third party code you can ignore doxygen ci error
And if you want to fix, cpp or clang ci error --> you can see details --> git action summary or cmd log

taos-ci

@s-debadri, 💯 All CI checkers are successfully verified. Thanks.

Fixed CI issues for the following: Third party files: clang Non third party files: clang, doxygen Signed-off-by: Debadri Samaddar <[email protected]>

taos-ci · 2024-03-07T07:15:13Z

cibot: @s-debadri, nntrainer/opencl/third_party/cl_platform.h does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

taos-ci · 2024-03-07T07:15:16Z

cibot: @s-debadri, nntrainer/opencl/opencl_buffer.cpp includes bug(s). Please fix incorrect coding constructs in your commit before entering a review process.

Handled conditions by reducting ifdef checks Signed-off-by: Debadri Samaddar <[email protected]>

taos-ci

@s-debadri, 💯 All CI checkers are successfully verified. Thanks.

skykongkong8 · 2024-03-11T00:04:58Z

I found this commit emits error when using ndk-build in the current main

ld: error: undefined symbol: nntrainer::ClContext::Global()
>>> referenced by layers_dependent_common_tests.cpp:38 (../unittest/layers/layers_dependent_common_tests.cpp:38)
>>>               /home/sungsik/nntrainer/test/obj/local/arm64-v8a/objs/unittest_layers/__/unittest/layers/layers_dependent_common_tests.o:(LayerSemantics_createFromClContext_pn_Test::TestBody())

ld: error: undefined symbol: int const nntrainer::ClContext::registerFactory<nntrainer::Layer>(std::__ndk1::function<std::__ndk1::unique_ptr<nntrainer::Layer, std::__ndk1::default_delete<nntrainer::Layer> > (std::__ndk1::vector<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >, std::__ndk1::allocator<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > > > const&)>, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, int)
>>> referenced by layers_dependent_common_tests.cpp:40 (../unittest/layers/layers_dependent_common_tests.cpp:40)
>>>               /home/sungsik/nntrainer/test/obj/local/arm64-v8a/objs/unittest_layers/__/unittest/layers/layers_dependent_common_tests.o:(LayerSemantics_createFromClContext_pn_Test::TestBody())

ld: error: undefined symbol: nntrainer::opencl::ContextManager::ReleaseContext()
>>> referenced by layer_context.h:829 (/home/sungsik/nntrainer/nntrainer/layers/layer_context.h:829)
>>>               /home/sungsik/nntrainer/test/obj/local/arm64-v8a/objs/unittest_layers/__/unittest/layers/layers_golden_tests.o:(nntrainer::RunLayerContext::~RunLayerContext())
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

s-debadri · 2024-03-11T04:18:09Z

I found this commit emits error when using ndk-build in the current main

ld: error: undefined symbol: nntrainer::ClContext::Global()
>>> referenced by layers_dependent_common_tests.cpp:38 (../unittest/layers/layers_dependent_common_tests.cpp:38)
>>>               /home/sungsik/nntrainer/test/obj/local/arm64-v8a/objs/unittest_layers/__/unittest/layers/layers_dependent_common_tests.o:(LayerSemantics_createFromClContext_pn_Test::TestBody())

ld: error: undefined symbol: int const nntrainer::ClContext::registerFactory<nntrainer::Layer>(std::__ndk1::function<std::__ndk1::unique_ptr<nntrainer::Layer, std::__ndk1::default_delete<nntrainer::Layer> > (std::__ndk1::vector<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >, std::__ndk1::allocator<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > > > const&)>, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, int)
>>> referenced by layers_dependent_common_tests.cpp:40 (../unittest/layers/layers_dependent_common_tests.cpp:40)
>>>               /home/sungsik/nntrainer/test/obj/local/arm64-v8a/objs/unittest_layers/__/unittest/layers/layers_dependent_common_tests.o:(LayerSemantics_createFromClContext_pn_Test::TestBody())

ld: error: undefined symbol: nntrainer::opencl::ContextManager::ReleaseContext()
>>> referenced by layer_context.h:829 (/home/sungsik/nntrainer/nntrainer/layers/layer_context.h:829)
>>>               /home/sungsik/nntrainer/test/obj/local/arm64-v8a/objs/unittest_layers/__/unittest/layers/layers_golden_tests.o:(nntrainer::RunLayerContext::~RunLayerContext())
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

@skykongkong8
This requires ENABLE_OPENCL to be true in meson_options.txt while building libnntrainer.so.
Or, nntrainer/test/jni/Android.mk can be modified as such:

Modify setup for unittest_layers and remove -DENABLE_OPENCL=1 from LOCAL_CFLAGS variable.
This will disable OpenCL dependency for unittest_layers.

skykongkong8 · 2024-03-11T05:03:26Z

@s-debadri
Right.. then could you make a PR for that issue?

github-actions bot added the Need Review label Feb 6, 2024

jijoongmoon reviewed Feb 7, 2024

View reviewed changes

taos-ci approved these changes Feb 7, 2024

View reviewed changes

jijoongmoon reviewed Feb 7, 2024

View reviewed changes

jihochu reviewed Feb 7, 2024

View reviewed changes

s-debadri changed the title ~~[WIP][GPU/OpenCL] OpenCL pipeline for tensor operations to be executed on GPU~~ [WIP][GPU/OpenCL] OpenCL pipeline for layer execution on GPU Feb 23, 2024

s-debadri force-pushed the gpu_pipeline_core branch from cb7ec72 to 295fb51 Compare February 23, 2024 14:46

taos-ci approved these changes Feb 23, 2024

View reviewed changes

s-debadri marked this pull request as ready for review February 27, 2024 07:36

s-debadri requested review from myungjoo, again4you, jaeyun-jung, leemgs, wooksong, helloahn, kparichay, gichan-jang, anyj0527, zhoonit, lhs8928, songgot, DonghakPark and SeoHyungjun as code owners February 27, 2024 07:36

s-debadri force-pushed the gpu_pipeline_core branch from 134cb47 to 61dedf3 Compare March 6, 2024 06:53

taos-ci approved these changes Mar 6, 2024

View reviewed changes

s-debadri force-pushed the gpu_pipeline_core branch 2 times, most recently from d24c2e5 to c4de26a Compare March 6, 2024 08:55

s-debadri force-pushed the gpu_pipeline_core branch from c4de26a to 7c8a025 Compare March 6, 2024 09:23

taos-ci approved these changes Mar 6, 2024

View reviewed changes

DonghakPark approved these changes Mar 7, 2024

View reviewed changes

github-actions bot added PR/READY2MERGE and removed Need Review labels Mar 7, 2024

DonghakPark changed the title ~~[GPU/OpenCL] OpenCL pipeline for layer execution on GPU~~ [GPU/OpenCL] OpenCL pipeline for layer execution on GPU @open sesame 03/07 10:42 Mar 7, 2024

taos-ci approved these changes Mar 7, 2024

View reviewed changes

s-debadri force-pushed the gpu_pipeline_core branch from 7c8a025 to 108a81f Compare March 7, 2024 04:57

s-debadri force-pushed the gpu_pipeline_core branch from 108a81f to bc32461 Compare March 7, 2024 05:18

taos-ci approved these changes Mar 7, 2024

View reviewed changes

[OpenCL] CI issues fixed for clang and doxygen

7403a02

Fixed CI issues for the following: Third party files: clang Non third party files: clang, doxygen Signed-off-by: Debadri Samaddar <[email protected]>

s-debadri force-pushed the gpu_pipeline_core branch from c3e642e to 7403a02 Compare March 7, 2024 07:15

[OpenCL] Reduced ifdef checks

4a529ed

Handled conditions by reducting ifdef checks Signed-off-by: Debadri Samaddar <[email protected]>

taos-ci approved these changes Mar 7, 2024

View reviewed changes

jijoongmoon merged commit 459810e into nnstreamer:main Mar 7, 2024
27 of 28 checks passed

s-debadri deleted the gpu_pipeline_core branch May 23, 2024 07:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GPU/OpenCL] OpenCL pipeline for layer execution on GPU @open sesame 03/07 10:42 #2465

[GPU/OpenCL] OpenCL pipeline for layer execution on GPU @open sesame 03/07 10:42 #2465

s-debadri commented Feb 6, 2024 •

edited

Loading

taos-ci commented Feb 6, 2024

taos-ci commented Feb 6, 2024

jijoongmoon Feb 7, 2024

s-debadri Feb 7, 2024

jijoongmoon Feb 7, 2024 •

edited

Loading

s-debadri Feb 7, 2024

taos-ci commented Feb 7, 2024

taos-ci commented Feb 7, 2024

taos-ci left a comment

jijoongmoon Feb 7, 2024

jijoongmoon Feb 7, 2024 •

edited

Loading

jijoongmoon Feb 7, 2024

jihochu Feb 7, 2024

jihochu Feb 7, 2024

taos-ci left a comment

taos-ci left a comment

taos-ci commented Mar 6, 2024

taos-ci left a comment

DonghakPark left a comment

taos-ci left a comment

taos-ci commented Mar 7, 2024

taos-ci commented Mar 7, 2024

taos-ci commented Mar 7, 2024

DonghakPark commented Mar 7, 2024

taos-ci left a comment

taos-ci commented Mar 7, 2024

taos-ci commented Mar 7, 2024

taos-ci left a comment

skykongkong8 commented Mar 11, 2024

s-debadri commented Mar 11, 2024

skykongkong8 commented Mar 11, 2024


		#include <nntrainer_log.h>

		namespace nntrainer::internal {


		#include <vector>

		#include "opencl_loader.hpp"

[GPU/OpenCL] OpenCL pipeline for layer execution on GPU @open sesame 03/07 10:42 #2465

[GPU/OpenCL] OpenCL pipeline for layer execution on GPU @open sesame 03/07 10:42 #2465

Conversation

s-debadri commented Feb 6, 2024 • edited Loading

taos-ci commented Feb 6, 2024

taos-ci commented Feb 6, 2024

jijoongmoon Feb 7, 2024

Choose a reason for hiding this comment

s-debadri Feb 7, 2024

Choose a reason for hiding this comment

jijoongmoon Feb 7, 2024 • edited Loading

Choose a reason for hiding this comment

s-debadri Feb 7, 2024

Choose a reason for hiding this comment

taos-ci commented Feb 7, 2024

taos-ci commented Feb 7, 2024

taos-ci left a comment

Choose a reason for hiding this comment

jijoongmoon Feb 7, 2024

Choose a reason for hiding this comment

jijoongmoon Feb 7, 2024 • edited Loading

Choose a reason for hiding this comment

jijoongmoon Feb 7, 2024

Choose a reason for hiding this comment

jihochu Feb 7, 2024

Choose a reason for hiding this comment

jihochu Feb 7, 2024

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

taos-ci commented Mar 6, 2024

taos-ci left a comment

Choose a reason for hiding this comment

DonghakPark left a comment

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

taos-ci commented Mar 7, 2024

taos-ci commented Mar 7, 2024

taos-ci commented Mar 7, 2024

DonghakPark commented Mar 7, 2024

taos-ci left a comment

Choose a reason for hiding this comment

taos-ci commented Mar 7, 2024

taos-ci commented Mar 7, 2024

taos-ci left a comment

Choose a reason for hiding this comment

skykongkong8 commented Mar 11, 2024

s-debadri commented Mar 11, 2024

skykongkong8 commented Mar 11, 2024

s-debadri commented Feb 6, 2024 •

edited

Loading

jijoongmoon Feb 7, 2024 •

edited

Loading

jijoongmoon Feb 7, 2024 •

edited

Loading